Gartner: Pageflakes is the “Cool Web 2.0” Personalized Homepage

It’s been an award-winning spring
season for Pageflakes! This time
Gartner, the highly respected
technology industry analysis and research firm, has named
Pageflakes the
“Cool Web 2.0” personalized homepage for 2007
. Many
Business and enterprise users have discovered our group
collaboration and page publishing features, and are using
Pageflakes for all kind of interesting applications at work such as
internal company intranets and managing teams and
projects. Plus, they’re doing this with no
technical skills, no IT department approvals, no waiting – and no
cost.

Gartner has taken notice, and in its “Cool Web 2.0 Vendors 2007”

research report ()
, analyst David Gootzit says
“Pageflakes differentiates itself by adding some community building
features” and “Pageflakes users can publish tabs from their own
start pages to the general Pageflakes community or select groups of
users. This feature enables Pageflakes users without programming
know-how to design their own Web sites.” The report goes on to say
that, “Enterprise users should examine Pageflakes as a
tactical collaboration tool” and that Pageflakes features
”are also easily leveraged by the average enterprise user and
require no monetary investment.”

Thanks to Gartner and to all of our users working that are
working hard at the office. We’re glad to hear that Pageflakes
is making life a little easier and more fun at work.

And the winner is … Pageflakes. Duh!

Two weeks back, Read Write Web started a poll on who’s the most
popular Start Page in the world now. The result is astonishing:

Pageflakes has got 30% vote, ahead of Google’s Personalized
Homepage (26%) and way ahead of Netvibes (21%). Many doubted
this poll and thought it must have been an arranged poll by
Pageflakes guys. But Read Write Web has declared Google to be the
winner and Pageflakes #2 for some reason. Well, no problem.
The mighty PC World has declared Pageflakes to be the #1 Start
Page!


Read the article here.

Let’s see now, how many awards we got so far. First there was
the Web 2.0 awards by
SEOMoz
where Pageflakes raked #1 Start Page ahead of
Google IG and Microsoft Live.com. Then there was the “Ajax King”
award by
InformationWeek
which said Google is no longer the Ajax
King, Pageflakes is. Then we got highest vote on ReadWriteWeb poll.
Recently we won #1 Start Page title in PC World. Soon we
hope to win the Emmy Awards, the Oscar, Miss Universe …

Cleanup inactive anonymous users from ASP.NET Membership Tables

ASP.NET 2.0 Websites that allow anonymous visit and anonymous
user profile have a unique challenge to cleanup unused data which
is generated by anonymous users who never come back. Every first
visit is creating one anonymous user, page setup, and other user
specific content. If the user never comes back, it still remains in
the database permanently. It is possible user might come back
within a day, or a week or a month. But there’s no guaranty
if user will ever come back or not. Generally sticky users are max
30% of the total users who come to most websites. So, you end up
with 70% unused data which are never needed. All these requires
cleanup, otherwise the database keeps growing uncontrollably and
gets slower and slower. This cleanup operation is humongous for
busy websites. Think about deleting millions of rows from several
tables, one after another while maintaining foreign key
constraints. Also the cleanup operation needs to run while the site
is running, without hampering site’s overall performance. The whole
operation results in heavily fragmented index and space in the MDF
file. The log file also becomes enormous in order to keep track of
the transactions. Hard drives get really hot and start sweating
furiously. While the CPU keeps smiling having nothing to do with
it, it’s really painful to watch SQL Server go through this
every day. Unless you clean up the database and maintain its size
under control; you can’t keep up with SQL Server’s RAM and
Disk IO requirement.

When a user visits the site, Asp.net Membership Provider updates
the LastActivityDate of aspnet_users table. From this field, I can
find out how long the user has been idle. The IsAnonymous bit field
tells me whether the user account is anonymous or registered. If it
is registered, no need to worry. But if it is anonymous and more
than 30 days old, I can be sure that the user will never come back
because the cookie has already expired. If you repeatedly logout
from your start page, all cookie related to the site gets cleared.
That means you are producing one new anonymous user record during
each log out. That anonymous record is never used because you will
soon log in to have your customized pages back and then you will
log out again. This will result in another anonymous user account
which again becomes useless as soon as you log in.

Here’s how the whole cleanup process works:

  • Find out the users which are old enough to be discarded
  • Find out the pages user has
  • Delete all the widget instances on those pages
  • Then delete those pages
  • Remove rows from child tables related to aspnet_users like
    aspnet_profile, aspnet_UsersInRoles, aspnet_PersonalizationPerUser.
    Remove rows for users to be deleted
  • Remove the users from aspnet_users
  • Pray that you did not accidentally remove any good user

Here’s the giant DB script which does it all. I have put
enough inline comment so that you can understand what the script is
doing:

   1: -- Number of days after which we give users 'bye bye'
   2: DECLARE @Days int
   3: SET @Days = 14
   4:
   5: -- No of users to delete per run. If it's too high, database will get stuck
   6: -- for a long time. If it's too low, you will end up having more trash than
   7: -- you can cleanup
   8: DECLARE @NoOfUsersToDelete int
   9: SET @NoOfUsersToDelete = 1000
  10:
  11: -- Create temporary tables which holds the users and pages to delete
  12: -- As the user and the page is used to find out other tables, instead
  13: -- of running SELECT ID FORM ... repeatedly, it's better to have them
  14: -- in a temp table
  15: IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[PagesToDelete]') AND type in (N'U'))
  16: DROP TABLE [dbo].[PagesToDelete]
  17: IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[aspnetUsersToDelete]') AND type in (N'U'))
  18: DROP TABLE [dbo].[AspnetUsersToDelete]
  19:
  20: create table PagesToDelete (PageID int NOT NULL PRIMARY KEY)
  21: create table AspnetUsersToDelete (UserID uniqueidentifier NOT NULL PRIMARY KEY)
  22:
  23: -- Find out inactive anonymous users and store the UserID in the temporary
  24: -- table
  25: insert into AspnetUsersToDelete
  26: select top(@NoOfUsersToDelete) UserID from aspnet_Users where
  27: (isAnonymous = 1) and (LastActivityDate < (getDate()-@Days))
  28: order by UserID -- Saves SQL Server from sorting in clustered index again
  29:
  30: print 'Users to delete: ' + convert(varchar(255),@@ROWCOUNT)
  31: GO
  32:
  33: -- Get the pages of the users which will be deleted
  34: insert into PagesToDelete
  35: select ID from Page where UserID in
  36: (
  37: select UserID from AspnetUsersToDelete
  38: )
  39:
  40: print 'Pages to delete: ' + convert(varchar(255),@@ROWCOUNT)
  41: GO
  42:
  43: -- Delete all Widget instances on the pages to be deleted
  44: delete from WidgetInstance where PageID IN
  45: ( SELECT PageID FROM PagesToDelete )
  46:
  47: print 'Widget Instances deleted: ' + convert(varchar(255), @@ROWCOUNT)
  48: GO
  49:
  50: -- Delete the pages
  51: delete from Page where ID IN
  52: ( SELECT PageID FROM PagesToDelete )
  53: GO
  54:
  55: -- Delete User Setting
  56: delete from UserSetting WHERE UserID IN
  57: ( SELECT UserID FROm AspnetUsersToDelete )
  58: GO
  59:
  60: -- Delete profile of users
  61: delete from aspnet_Profile WHERE UserID IN
  62: ( SELECT UserID FROm AspnetUsersToDelete )
  63: GO
  64:
  65: -- Delete from aspnet_UsersInRoles
  66: delete from aspnet_UsersInRoles WHERE UserID IN
  67: ( SELECT UserID FROm AspnetUsersToDelete )
  68: GO
  69:
  70: -- Delete from aspnet_PersonalizationPerUser
  71: delete from aspnet_PersonalizationPerUser WHERE UserID IN
  72: ( SELECT UserID FROm AspnetUsersToDelete )
  73: GO
  74:
  75: -- Delete the users
  76: delete from aspnet_users where userID IN
  77: ( SELECT UserID FROm AspnetUsersToDelete )
  78:
  79: PRINT 'Users deleted: ' + convert(varchar(255), @@ROWCOUNT)
  80: GO
  81:
  82:
  83: drop table PagesToDelete
  84: drop table AspnetUsersToDelete
  85: GO

Now the question comes, when can I run this script? It depends on
several factors:

  • The lowest traffic period. For example, USA
    midnight time when everyone in USA is sleeping if your majority
    users are from USA
  • The period when there’s no other
    maintenance tasks running like Index Defrag or Database Bakup. If
    by any chance any other maintenance task conflicts with this
    enormous delete operation, SQL Server is dead.
  • The operation will take from 10 mins to
    hours depending on the volume of trash to cleanup. So, consider the
    duration of running this script and plan other maintenance jobs
    accordingly.
  • It’s best to run 30 mins before INDEX
    DEFRAG jobs run. After the script completes, the tables will be
    heavily fragmented. So, you need to defrag the indexes.

Before running this script, there are some preparations to
take:

  • Make sure you have turned of AutoShrink from Database Property.
    Database size will reduce after the cleanup and if SQL Server tried
    to shrink the database, there will be a big IO activity. Turn off
    auto shrink because the database will grow again.
  • Make sure the LOG file’s initial size is big enough to
    hold such enormous transactions. You can specify 1/3rd of the MDF
    size as LDF’s Initial Size. Also make sure the log file is
    not shrunk. Let it occupy HD space. It saves SQL Server from
    expanding the file and shrinking the file. Both of these require
    high Disk IO.

Once the cleanup job runs and the INDEX DEFRAG runs, the
database performance will improve significantly. The tables are now
smaller. That means the indexes are now smaller. SQL Server need
not to run through large indexes anymore. Future INDEX DEFRAGs take
shorter time because there’s not much data left to optimize.
SQL Server also takes less RAM because it has to work with much
less amount of data. Database backup size also reduces because the
MDF size does not keep increasing indefinitely. As a result, the
significant overhead of this cleanup operation is quite acceptable
when compared to all the benefits.

Note: I will be posting some stuffs from my old blog to new blog.
Please ignore if you have read them before.

Prevent Denial of Service (DOS) attacks in your web application

Web services are the most attractive target for hackers because
even a pre-school hacker can bring down a server by repeatedly
calling a web service which does expensive work. Ajax Start Pages
like Pageflakes are the
best target for such DOS attack because if you just visit the
homepage repeatedly without preserving cookie, every hit is
producing a brand new user, new page setup, new widgets and what
not. The first visit experience is the most expensive one.
Nonetheless, it’s the easiest one to exploit and bring down
the site. You can try this yourself. Just write a simple code like
this:

   1: for( int i = 0; i < 100000; i ++ )
   2: {
   3:    WebClient client = new WebClient();
   4:    client.DownloadString("http://www.pageflakes.com/default.aspx");
   5: }

In your great surprise, you will notice that, after a couple of
call, you don't get valid response. It’s not that you have
succeeded in bringing down the server. It’s that your
requests are being rejected. You are happy that you no longer get
any service, thus you achieve Denial of Service (for yourself). I
am happy to Deny You of Service (DYOS).

The trick I have in my sleeve is an inexpensive way to remember
how many requests are coming from a particular IP. When the number
of request exceeds the threshold, deny further request for some
duartion. The idea is to remember caller’s IP in Asp.net
Cache and maintain a count of request per IP. When the count
exceeds a predefined limit, reject further request for some
specific duration like 10 mins. After 10 mins, again allow requests
from that IP.

I have a class named ActionValidator which maintains a count of
specific actions like First Visit, Revisit, Asynchrnous postbacks,
Add New widget, Add New Page etc. It checks whether the count for
such specific action for a specific IP exceeds the threshold value
or not.

   1: public static class ActionValidator
   2: {
3: private const int DURATION = 10; // 10 min period
   4:
   5:     public enum ActionTypeEnum
   6:     {
   7:         FirstVisit = 100, // The most expensive one, choose the valu wisely. 
   8:         ReVisit = 1000, // Welcome to revisit as many times as use likes
   9:         Postback = 5000,   // Not must of a problem for us
  10:         AddNewWidget = 100,
  11:         AddNewPage = 100,
  12:     }

The enumeration contains the type of actions to check for and
their threshold value for a specific duration – 10 mins.

A static method named IsValid does the check. It
returns true if the request limit is not passed, false if the
request needs to be denied. Once you get false, you can call
Request.End() and prevent Asp.net from proceeding further.
You can also switch to a page which shows “Congratulations!
You have succeeded in Denial of Service Attack.”

   1:
public static bool IsValid( ActionTypeEnum actionType )
   2: {
   3: HttpContext context  = HttpContext.Current;
   4:
if( context.Request.Browser.Crawler ) return false;
   5:
   6: string key = actionType.ToString()  + context.Request.UserHostAddress;
   7:
   8: HitInfo hit  = (HitInfo)(context.Cache[key] ?? new HitInfo());
   9:
  10: if( hit.Hits > (int)actionType ) return false;
  11: else hit.Hits ++;
  12:
  13: if( hit.Hits == 1 )
  14:     context.Cache.Add(key, hit, null, DateTime.Now.AddMinutes(DURATION),
  15:       System.Web.Caching.Cache.NoSlidingExpiration, System.Web.Caching.CacheItemPriority.Normal, null);
  16:
  17: return true;
  18: }

The cache key is built with a combination of action type and
client IP address. First it checks if there’s any entry for
the action and the client IP in Cache or not. If not, start the
count and store remember the count for the IP in cache for the
specific duration. The absolute expiration on cache item ensures
after the duration, the cache item will be cleared and the count
will restart. When there’s already an entry in the cache, get
the last hit count, and check if the limit is exceeded or not. If
not exceeded, increase the counter. There is no need to store the
updated value in the cache again by doing: Cache[url]=hit; because
the hit object is by reference and changing it means it gets
changed in the cache as well. In fact, if you do put it again in
the cache, the cache expiration counter will restart and fail the
logic of restarting count after specific duration.

The usage is very simple:

   1:
protected override void OnInit(EventArgs e)
   2: {
   3:   base.OnInit(e);
   4:
   5:   // Check if revisit is valid or not
   6:   if(  !base.IsPostBack )
   7:   {
   8:     // Block cookie less visit attempts
   9:     if( Profile.IsFirstVisit )
  10:     {
  11:        if( !ActionValidator.IsValid(ActionValidator.ActionTypeEnum.FirstVisit) Response.End();
  12:     }
  13:    else
  14:     {
  15:      if( !ActionValidator.IsValid(ActionValidator.ActionTypeEnum.ReVisit) ) Response.End();
  16:     }
  17:   }
  18:  else
  19:   {
  20:    // Limit number of postbacks
  21:     if( !ActionValidator.IsValid(ActionValidator.ActionTypeEnum.Postback)  Response.End();
  22:   }
  23: }

Here I am checking specific scenario like First Visit, re-visit,
postbacks etc.

Of course you can put in some Cisco firewall and prevent DOS
attack. You will get guaranty from your hosting provider that their
entire network is immune to DOS and DDOS (Distributed DOS) attacks.
What they guaranty is network level attack like TCP SYN attacks or
malformed packet floods etc. There is no way they can analyze the
packet and find out a particular IP is trying to load the site too
many times without supporting cookie or trying to add too many
widgets. These are called application level DOS attack which
hardware cannot prevent. It must be implemented in your own
code.

There are very few websites out their which take such precaution
for application level DOS attacks. Thus it’s quite easy to
make servers go mad by writing a simple loop and hitting expensive
pages or web services continuously from your home broadband
connection. I hope this small but effective class will help
you implement DOS attack in your own web applications.

Update

Here's the code of the full ActionValidator class:

// Copyright (c) Omar AL Zabir. All rights reserved.
// For continued development and updates, visit
http://msmvps.com/omar

using System;
using System.Data;
using System.Configuration;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;

///

/// Summary description for ActionValidator
///

namespace Dropthings.Web.Util
{
public static class ActionValidator
{
private const int DURATION = 10; // 10 min period

/*
* Type of actions and their maximum value per period
*
*/
public enum ActionTypeEnum
{
None = 0,
FirstVisit = 100, // The most expensive one, choose the
value wisely.
Revisit = 1000, // Welcome to revisit as many times as user
likes
Postback = 5000, // Not must of a problem for us
AddNewWidget = 100,
AddNewPage = 100,
}

private class HitInfo
{
public int Hits;
private DateTime _ExpiresAt =
DateTime.Now.AddMinutes(DURATION);
public DateTime ExpiresAt { get { return _ExpiresAt; } set {
_ExpiresAt = value; } }
}

public static bool IsValid( ActionTypeEnum actionType )
{
HttpContext context = HttpContext.Current;
if( context.Request.Browser.Crawler ) return false;

string key = actionType.ToString() +
context.Request.UserHostAddress;

HitInfo hit = (HitInfo)(context.Cache[key] ?? new
HitInfo());

if( hit.Hits > (int)actionType ) return false;
else hit.Hits ++;

if( hit.Hits == 1 )
context.Cache.Add(key, hit, null,
DateTime.Now.AddMinutes(DURATION),
System.Web.Caching.Cache.NoSlidingExpiration,
System.Web.Caching.CacheItemPriority.Normal, null);

return true;
}
}
}

ASP.NET Ajax Extender for multi-column widget drag & drop

I have discontinued this Ajax Extender. I now use jQuery plugins to offer drag & drop, which is much faster, smaller and easier than ASP.NET AJAX stuffs. I suggest you see the latest implementation of it from:

http://www.codeproject.com/KB/ajax/Web20Portal.aspx

My open source Ajax Start Page “http://dropthings.omaralzabir.com/”>dropthings.omaralzabir.com has an ASP.NET

Ajax Extender which provides multi-column drag & drop for
widgets. It allows reordering of widgets on the same column and
also drag & drop between column. It also supports client side
notification so that you can call web service and store the
position of the widgets behind the scene without (async)
postback.

“http://omar.mvps.org/images/ASP.NETAjaxExtenderformulticolumnwidgetd_12D5/image01.png”
width=”574″ alt=””>

I first thought of going for a plain vanilla Javascript based
solution for drag & drop. It requires less code, less
architectural complexity and provides better speed. Another reason
was the high learning curve for making Extenders in proper way in
Asp.net Ajax given that there’s hardly any documentation
available on the web (during the time of writing this blog).
However, writing a proper extender which pushes Asp.net Ajax to the
limit is a very good way to learn under-the-hood secrets of Asp.net
Ajax framework itself. So, the two extenders I will introduce here
will tell you almost everything you need to know about Asp.net Ajax
Extenders.

Ajax Control Toolkit comes with a DragPanel
extender which I could use to provide drag & drop support to
panels. It also has a ReorderList control which I could use to
provide reordering of items in a single list. Widgets are basically
panels that flow vertically in each column. So, it might be
possible that I could create reorder list in each column and use
the DragPanel to drag the widgets. But I could not use
ReorderList because:

  • ReorderList strictly uses Html Table to render its
    items in a column. I have no table inside the columns. Only one
    Panel is there inside an UpdatePanel.
  • ReorderList takes a Drag Handle template and
    creates a drag handle for each item at runtime. I already have drag
    handle created inside a Widget which is the widget header. So, I
    cannot allow ReorderList to create another drag
    handle.
  • I need client side callback on drag & drop so that I can
    make Ajax calls and persist the widget positions. The callback must
    give me the Panel where the widget is dropped, which is dropped and
    at what position.

Next challenge is with the DragPanel extender. The default
implement of Drag & Drop in Ajax Control Toolkit has some
problems:

  • When you start dragging, the item becomes absolutely
    positioned, but when you drop it, it does not become static
    positioned. There’s a small hack needed for restoring the original
    position to “static”.
  • It does not put the dragging item on top of all items. As a
    result, when you start dragging, you see the item being dragged
    below other items which makes the drag get stuck especially when
    there’s an IFRAME.

So, I have made a CustomDragDropExtender and a
CustomFloatingExtender.
CustomDragDropExtender is for the column containers
where widgets are placed. It provides the reordering support. You
can attach this extender to any Panel control.

Here’s how you can attach this extender to any
Panel and make that Panel support drag
& drop of Widgets:

   1:
<
asp:Panel
ID
="LeftPanel"
runat
="server"
class
="widget_holder"
columnNo
="0"
>
   2:
<
div
id
="DropCue1"
class
="widget_dropcue"
>
   3:
 
div
>
   4:
 
asp:Panel
>
   5:
   6:
<
cdd:CustomDragDropExtender
ID
="CustomDragDropExtender1"
   7:
runat
="server"
   8:
TargetControlID
="LeftPanel"
   9:
DragItemClass
="widget"
  10:
DragItemHandleClass
="widget_header"
  11:
DropCueID
="DropCue1"
  12:
OnClientDrop
="onDrop"
/>

offers the
following properties:

  • TargetControlID – ID of the Panel which becomes the Drop
    zone
  • DragItemClass – All child elements inside the Panel
    having this class will become draggable. E.g. Widget DIV has this
    class so that it can become draggable.
  • DragItemHandleClass – Any child element having this class
    inside the draggable elements will become the drag handle for the
    draggable element. E.g. Widget Header area has this class, so it
    acts as the drag handle for the Widget.
  • DropCueID – ID of an element inside the Panel which acts
    as Drop Cue.
  • OnClientDrop – Name of a Javascript function which is
    called when widget is dropped on the Panel.

LeftPanel becomes a widget container which allows widgets to be
dropped on it and reordered. The DragItemClass attribute on the
extender defines the items which can be ordered. This prevents from
non-widget Html Divs from getting ordered. Only the DIVs with the
class “widget” are ordered. Say there are 5 DIVs with the class
named “widget”. It will allow reordering of only these five
divs:

   1:
<
div
id
="LeftPanel"
class
="widget_holder"
>
   2:
<
div
class
="widget"
> ...
 
div
>
   3:
<
div
class
="widget"
> ...
 
div
>
   4:
   5:
<
div
class
="widget"
> ...
 
div
>
   6:
<
div
class
="widget"
> ...
 
div
>
   7:
<
div
class
="widget"
> ...
 
div
>
   8:
   9:
<
div
>This DIV will not move

div
>
  10:
<
div
id
="DropCue1"
class
="widget_dropcue"
>
div
>
  11:

div
>

When a widget is dropped on the panel, the extender fires the
function specified in OnClientDrop. It offers standard
Ajax Events. But unlike basic Ajax events where you have to
programmatically bind to events, you can define property and
specify the function name to call. So, instead of doing this:

   1:
function pageLoad( sender, e ) {
   2:
   3:
var extender1 =
get(‘CustomDragDropExtender1’);
   4:   extender1.add_onDrop( onDrop );
   5:
   6: }

You can do this:

   1:
<
cdd:CustomDragDropExtender
ID
="CustomDragDropExtender1"
   2:
runat
="server"
   3:
OnClientDrop
="onDrop"
/>

When the event is raised, the function named onDrop
gets fired. This is done with the help of some handy library
available in ACT project.

When the event is fired, it sends the container, the widget and
the position of the widget where the widget is dropped.

   1:
function onDrop( sender, e )
   2: {
   3:
var container = e.get_container();
   4:
var item = e.get_droppedItem();
   5:
var position = e.get_position();
   6:
   7:
//alert( String.format( "Container: {0}, Item:
{1}, Position: {2}", container.id, item.id, position ) );
   8:
   9:
var instanceId =
parseInt(item.getAttribute(
"InstanceId"));
  10:
var columnNo =
parseInt(container.getAttribute(
"columnNo"));
  11:
var row = position;
  12:
  13:
WidgetService.MoveWidgetInstance( instanceId, columnNo, row );
  14: }

The widget location is updated by calling the
WidgetService.MoveWidgetInstance.

CustomDragDropExtender has 3 files:

  • CustomDragDropExtender.cs – The server side extender
    implementation
  • CustomDragDropDesigner.cs – Designer class for the
    extender
  • CustomDragDropExtender.js – Client side scriptfor the
    extender

Server side class CustomDragDropExtender.cs has the following
code:

   1: [assembly:
System.Web.UI.WebResource(
"CustomDragDrop.CustomDragDropBehavior.js",

"text/javascript")]
   2:
   3:
namespace CustomDragDrop
   4: {
   5:     [Designer(
typeof(CustomDragDropDesigner))]
   6:     [ClientScriptResource(
"CustomDragDrop.CustomDragDropBehavior",
"CustomDragDrop.CustomDragDropBehavior.js")]
   7:     [TargetControlType(
typeof(WebControl))]
   8:     [RequiredScript(
typeof(CustomFloatingBehaviorScript))]
   9:     [RequiredScript(
typeof(DragDropScripts))]
  10:
public
class CustomDragDropExtender :
ExtenderControlBase
  11:     {
  12:
// TODO: Add your property accessors here.
  13:
//
  14:         [ExtenderControlProperty]
  15:
public
string DragItemClass
  16:         {
  17:             get
  18:             {
  19:
return GetPropertyValue(
"DragItemClass",
string.Empty);
  20:             }
  21:             set
  22:             {
  23:
SetPropertyValue(
"DragItemClass",
value);
  24:             }
  25:         }
  26:
  27:         [ExtenderControlProperty]
  28:
public
string DragItemHandleClass
  29:         {
  30:             get
  31:             {
  32:
return GetPropertyValue(
"DragItemHandleClass",
string.Empty);
  33:             }
  34:             set
  35:             {
  36:
SetPropertyValue(
"DragItemHandleClass",
value);
  37:             }
  38:         }
  39:
  40:         [ExtenderControlProperty]
  41:         [IDReferenceProperty(
typeof(WebControl))]
  42:
public
string DropCueID
  43:         {
  44:             get
  45:             {
  46:
return GetPropertyValue(
"DropCueID",
string.Empty);
  47:             }
  48:             set
  49:             {
  50:
SetPropertyValue(
"DropCueID",
value);
  51:             }
  52:         }
  53:
  54:         [ExtenderControlProperty()]
  55:         [DefaultValue(
"")]
  56:         [ClientPropertyName(
"onDrop")]
  57:
public
string OnClientDrop
  58:         {
  59:             get
  60:             {
  61:
return GetPropertyValue(
"OnClientDrop",
string.Empty);
  62:             }
  63:             set
  64:             {
  65:
SetPropertyValue(
"OnClientDrop",
value);
  66:             }
  67:         }
  68:
  69:     }
  70: }

Most of the code in the extender defines the property. The
important part is the declaration of the class:

   1: [assembly:
System.Web.UI.WebResource(
"CustomDragDrop.CustomDragDropBehavior.js",

"text/javascript")]
   2:
   3:
namespace CustomDragDrop
   4: {
   5:     [Designer(
typeof(CustomDragDropDesigner))]
   6:     [ClientScriptResource(
"CustomDragDrop.CustomDragDropBehavior",
"CustomDragDrop.CustomDragDropBehavior.js")]
   7:     [TargetControlType(
typeof(WebControl))]
   8:     [RequiredScript(
typeof(CustomFloatingBehaviorScript))]
   9:     [RequiredScript(
typeof(DragDropScripts))]
  10:
public
class CustomDragDropExtender :
ExtenderControlBase
  11:     {

The extender class inherits from
ExtenderControlBase defined in ACT project. This base
class has additional features over Ajax runtime provided Extender
base class. It allows you to define RequiredScript
attribute, which makes sure all the required scripts are downloaded
before the extender script is downloaded and initialized. This
extender has dependency over another extender named
CustomFloatingBehavior. It also depends on ACT’s
DragDropManager. So, the RequiredScript
attribute makes sure those are downloaded before this
extender’s script is downloaded. The
ExtenderControlBase is a pretty big class and does a
lot of work for us. It contains default implementations for
discovering all the script files for the extender and rendering
them properly.

The [assembly:System.Web.UI.WebResource] attribute
defines the script file containing the script for extender. The
script file is an embedded resource file.

[ClientScriptResource] attribute defines the
scripts required for the extender. This class is also defined in
ACT. ExtenderControlBase uses this attribute to find out which .js
files are working for the extender and renders them properly.

The challenge is to make the client side javascript for the
extender. On the js file, there’s a Javascript pseudo
class:

   1: Type.registerNamespace(
'CustomDragDrop');
   2:
   3:
CustomDragDrop.CustomDragDropBehavior =
function(element) {
   4:
   5:
CustomDragDrop.CustomDragDropBehavior.initializeBase(
this, [element]);
   6:
   7:
this._DragItemClassValue =
null;
   8:
this._DragItemHandleClassValue =
null;
   9:
this._DropCueIDValue =
null;
  10:
this._dropCue =
null;
  11:
this._floatingBehaviors = [];
  12: }

During initialize, it hooks on the Panel it is attached to and
the drop cue to show while drag & drop is going on over the
Panel:

   1:
CustomDragDrop.CustomDragDropBehavior.prototype = {
   2:
   3:     initialize :
function() {
   4:
// Register ourselves as a drop target.
   5:
AjaxControlToolkit.DragDropManager.registerDropTarget(
this);
   6:
//Sys.Preview.UI.DragDropManager.registerDropTarget(this);
   7:
   8:
// Initialize drag behavior after a while
   9:         window.setTimeout(
Function.createDelegate(
this,
this._initializeDraggableItems ), 3000 );
  10:
  11:
this._dropCue = get(
this.get_DropCueID());
  12:     },

After initializing the DragDropManager and marking the Panel as
a drop target, it starts a timer to discover the dragable items
inside the panel and create Floating behavior for them. Floating
behavior is the one which makes a DIV draggable.

FloatingBehavior makes one DIV freely draggable on the page. But
it does not offer drop functionality. DragDropBehavior offers the
drop functionality which allows a freely moving DIV to rest on a
fixed position.

Discovering and initializing floating behavior for the dragable
items is the challenging work:

   1:
// Find all items with the drag item class and
make each item
   2:
// draggable        
   3: _initializeDraggableItems :
function()
   4: {
   5:
this._clearFloatingBehaviors();
   6:
   7:
var el =
this.get_element();
   8:
   9:
var child = el.firstChild;
  10:
while( child !=
null )
  11:     {
  12:
if( child.className ==
this._DragItemClassValue && child
!=
this._dropCue)
  13:         {
  14:
var handle =
this._findChildByClass(child,
this._DragItemHandleClassValue);
  15:
if( handle )
  16:             {
  17:
var handleId = handle.id;
  18:
var behaviorId = child.id +
"_WidgetFloatingBehavior";
  19:
  20:
// make the item draggable by adding floating
behaviour to it                    
  21:
var floatingBehavior =
create(CustomDragDrop.CustomFloatingBehavior,
  22:                         {
"DragHandleID":handleId,
"id":behaviorId,
"name": behaviorId}, {}, {}, child);
  23:
  24:                 Array.add(
this._floatingBehaviors, floatingBehavior
);
  25:             }
  26:         }
  27:         child = child.nextSibling;
  28:     }
  29: },

Here’s the algorithm:

  • Run through all immediate child elements of the control where
    the extender is attached to
  • If the child item has the class for draggable item, then:
    • Find any element under the child item which has the class for
      Drag handle
    • If such item found, then attach a CustomFloatingBehavior with
      the child item

The _findChildByClass function recursively iterates
through all the child elements and looks for an element which has
the defined class. It’s an expensive function. So, it is
important that the drag handle is very close to the dragable
element.

   1: _findChildByClass :
function(item, className)
   2: {
   3:
// First check all immediate child items
   4:
var child = item.firstChild;
   5:
while( child !=
null )
   6:     {
   7:
if( child.className == className )
return child;
   8:         child = child.nextSibling;
   9:     }
  10:
  11:
// Not found, recursively check all child
items
  12:     child = item.firstChild;
  13:
while( child !=
null )
  14:     {
  15:
var found =
this._findChildByClass( child, className
);
  16:
if( found !=
null )
return found;
  17:         child = child.nextSibling;
  18:     }
  19: },

When user drags an item over the Panel where the extender is
attached to, DragDropManager fires the following
events:

   1: onDragEnterTarget :
function(dragMode, type, data) {
   2:
this._showDropCue(data);
   3: },
   4:
   5: onDragLeaveTarget :
function(dragMode, type, data) {
   6:
this._hideDropCue(data);
   7: },
   8:
   9: onDragInTarget :
function(dragMode, type, data) {
  10:
this._repositionDropCue(data);
  11: },

Here we deal with the drop cue. The challenging work is to find
out the right position for the drop cue.

“http://omar.mvps.org/images/ASP.NETAjaxExtenderformulticolumnwidgetd_12D5/image08.png”
width=”524″ alt=””>

We need to find out where we should show the drop cue based on
where user is dragging the item. The idea is to find out the widget
which is immediately below the dragged item. The item is pushed
down by one position and the drop cue takes its place. While
dragging, the position of the drag item can be found easily. Based
on that, I locate the widget below the drag item:

   1: _findItemAt :
function(x,y, item)
   2:     {
   3:
var el =
this.get_element();
   4:
   5:
var child = el.firstChild;
   6:
while( child !=
null )
   7:         {
   8:
if( child.className ==
this._DragItemClassValue && child
!=
this._dropCue && child != item )
   9:             {
  10:
var pos =
Sys.UI.DomElement.getLocation(child);
  11:
  12:
if( y <= pos.y )
  13:                 {
  14:
return child;
  15:                 }
  16:             }
  17:             child =
child.nextSibling;
  18:         }
  19:
  20:
return
null;
  21:     },

This function returns the widget which is immediately under the
dragged item. Now I add the drop cue immediately above the
widget:

   1: _repositionDropCue :
function(data)
   2:     {
   3:
var location =
Sys.UI.DomElement.getLocation(data.item);
   4:
var nearestChild =
this._findItemAt(location.x, location.y,
data.item);
   5:
   6:
var el =
this.get_element();
   7:
   8:
if(
null == nearestChild )
   9:         {
  10:
if( el.lastChild !=
this._dropCue )
  11:             {
  12:                 el.removeChild(
this._dropCue);
  13:                 el.appendChild(
this._dropCue);
  14:             }
  15:         }
  16:
else
  17:         {
  18:
if( nearestChild.previousSibling !=
this._dropCue )
  19:             {
  20:                 el.removeChild(
this._dropCue);
  21:                 el.insertBefore(
this._dropCue, nearestChild);
  22:             }
  23:         }
  24:     },

One exception to consider here is that there can be no widget
immediately below the dragged item. It happens when user is trying
to drop the widget at the bottom of column. In that case, the drop
cue is shown at the bottom of the column.

When user releases the widget, it drops right on top of drop cue
and the drop cue disappears. After the drop the onDrop
event is raised to notify where the widget is dropped.

   1: _placeItem :
function(data)
   2:     {
   3:
var el =
this.get_element();
   4:
   5:
data.item.parentNode.removeChild( data.item );
   6:         el.insertBefore( data.item,

this._dropCue );
   7:
   8:
// Find the position of the dropped item
   9:
var position = 0;
  10:
var item = el.firstChild;
  11:
while( item != data.item )
  12:         {
  13:
if( item.className ==
this._DragItemClassValue ) position++;
  14:             item =
item.nextSibling;
  15:         }
  16:
this._raiseDropEvent(
/* Container */ el,
/* droped item */ data.item,
/* position */ position );
  17:     }
Generally you can make events in extenders by adding two
functions in the extender:
   1: add_onDrop :
function(handler) {
   2:
this.get_events().addHandler(
"onDrop", handler);
   3: },
   4:
   5: remove_onDrop :
function(handler) {
   6:
this.get_events().removeHandler(
"onDrop", handler);
   7: },
But this does not give you the support for defining the event
listener name in the ASP.NET declaration:
   1:
<
cdd:CustomDragDropExtender
ID
="CustomDragDropExtender1"
   2:
runat
="server"
   3:
OnClientDrop
="onDrop"
/>

The declaration only allows properties. In order to support such
declarative assignment of events, we need to first introduce a
property named OnClientDrop in the extender. Then
during assignment of the property, we need to find out the function
specified there and attach event notification on that function. The
discovery of the function from its name is done by
CommonToolkitScripts.resolveFunction which is
available in ACT project.

   1:
// onDrop property maps to onDrop event
   2:     get_onDrop :
function() {
   3:
return
this.get_events().getHandler(
"onDrop");
   4:     },
   5:
   6:     set_onDrop :
function(value) {
   7:
if (value && (0 <
value.length)) {
   8:
var func =
CommonToolkitScripts.resolveFunction(value);
   9:
if (func) {
  10:
this.add_onDrop(func);
  11:             }
else {
  12:
throw Error.argumentType(
'value',
typeof(value),
'Function',
'resize handler not a function, function name, or
function text.');
  13:             }
  14:         }
  15:     },
Raising the event is same as basic Ajax events:
   1: _raiseEvent :
function( eventName, eventArgs ) {
   2:
var handler =
this.get_events().getHandler(eventName);
   3:
if( handler ) {
   4:
if( !eventArgs ) eventArgs =
Sys.EventArgs.Empty;
   5:             handler(
this, eventArgs);
   6:         }
   7:     },
This is all about the CustomDragDropExtender.
Next challenge is to make the CustomFloatingBehavior.
The server side class is declared as:
   1: [assembly:
System.Web.UI.WebResource(
"CustomDragDrop.CustomFloatingBehavior.js",

"text/javascript")]
   2:
   3:
namespace CustomDragDrop
   4: {
   5:     [Designer(
typeof(CustomFloatingBehaviorDesigner))]
   6:     [ClientScriptResource(
"CustomDragDrop.CustomFloatingBehavior",
"CustomDragDrop.CustomFloatingBehavior.js")]
   7:     [TargetControlType(
typeof(WebControl))]
   8:     [RequiredScript(
typeof(DragDropScripts))]
   9:
public
class CustomFloatingBehaviorExtender :
ExtenderControlBase
  10:     {
  11:         [ExtenderControlProperty]
  12:         [IDReferenceProperty(
typeof(WebControl))]
  13:
public
string DragHandleID
  14:         {
  15:             get
  16:             {
  17:
return GetPropertyValue(
"DragHandleID",
string.Empty);
  18:             }
  19:             set
  20:             {
  21:
SetPropertyValue(
"DragHandleID",
value);
  22:             }
  23:         }
  24:     }
  25: }

There’s only one property –
DragHandleID. Widget’s header works as the drag
handle. So, the header ID is specified here.

This extender has dependency on DragDropManager so
the [RequiredScript(typeof(DragDropScripts))]
attribute is there.

Besides the designer class, there’s one more class which
CustomDragDropExtender need in order to specify its
dependency over this floating behavior:

   1: [ClientScriptResource(
null,
"CustomDragDrop.CustomFloatingBehavior.js")]
   2:
public
static
class CustomFloatingBehaviorScript
   3:   {
   4:   }

This class can be used inside RequiredScript
attribute. It only defines which script file contains the client
side code for the extender.

The client side Javascript is same as
FloatingBehavior that comes with ACT. The only
difference is some hack when drag starts.
DragDropManager does not return the item being dragged
to static position once it makes it absolute. It also does not
increase the zIndex of the item. If the drag item does
not become the top most item, while dragging it goes below other
elements on the page. So, I have made some changes in the
mouseDownHandler of the behavior to add these features:

   1:
function mouseDownHandler(ev) {
   2:     window._event = ev;
   3:
var el =
this.get_element();
   4:
   5:
if (!
this.checkCanDrag(ev.target))
return;
   6:
   7:
// Get the location before making the element
absolute
   8:     _location =
Sys.UI.DomElement.getLocation(el);
   9:
  10:
// Make the element absolute 
  11:     el.style.width = el.offsetWidth
+
"px";
  12:     el.style.height =
el.offsetHeight +
"px";
  13:
Sys.UI.DomElement.setLocation(el, _location.x, _location.y);
  14:
  15:     _dragStartLocation =
Sys.UI.DomElement.getLocation(el);
  16:
  17:     ev.preventDefault();
  18:
  19:
this.startDragDrop(el);
  20:
  21:
// Hack for restoring position to static
  22:     el.originalPosition =
"static";
  23:     el.originalZIndex =
el.style.zIndex;
  24:     el.style.zIndex =
"60000";
  25: }

Setting el.originalPosition = static fixes the bug
in DragDropManager. It incorrectly stores absolute has
the originalPosition when startDragDrop
is called. So, after calling this function, I set it to correct
originalPosition which is “static”.

When drag completes, the original zIndex is restored and left,
top, width and height is cleared. DragDropManager makes the item
position static, but it does not clear the left, top, width and
height attributes. This moves the element away from the place where
it is dropped. This bug is fixed in the onDragEnd event:

   1:
this.onDragEnd =
function(canceled) {
   2:
if (!canceled) {
   3:
var handler =
this.get_events().getHandler(
'move');
   4:
if(handler) {
   5:
var cancelArgs =
new Sys.CancelEventArgs();
   6:             handler(
this, cancelArgs);
   7:             canceled =
cancelArgs.get_cancel();
   8:         }
   9:     }
  10:
  11:
var el =
this.get_element();
  12:     el.style.width =
el.style.height = el.style.left = el.style.top =
"";
  13:     el.style.zIndex =
el.originalZIndex;
  14: }

That’s all folks!

You can get the “http://omar.mvps.org/images/CustomDragDrop.zip”>code for the
extenders here.

ASP.NET Ajax in-depth performance analysis

Let’s do some ASP.NET runtime analysis on www.dropthings.com. Those who don’t
know what it is, it’s an open source start page I made using
ASP.NET Ajax, .NET 3.0 and Linq.

ASP.NET Ajax has a pretty big runtime which consists of Core
Framework, scripts for UpdatePanel, Preview Script required for
Drag & Drop. Additionally I need Ajax Control Toolkit
(ACT). All these add up to a staggering 564 KB of download in 12
script references on the page. The download size mostly depends on
the usage of extenders and Ajax features. A moderate use of
extender results in the following download:

Here’s a simulation of 256kbps internet speed on 200ms
network latency. I use a tool named Charles ( www.xk72.com/charles) to capture
traffic and simulate slow internet speed by throttling data
transfer speed. From the durations, we see almost 20 sec is spent
on downloading the runtime over a 256kbps line!

Let’s explain which script in the above list does what. I
will be identifying the scripts using their file size in order of
their appearance:

  • 21.64 KB – Handy script for Postbacks
  • 83.38 KB – Microsoft Ajax Core runtime
  • 30.16 KB – UpdatePanel, partial update, asynchronous postback
    scripts.
  • 136.38 KB – Preview version of Ajax which allows Drag
    & Drop script
  • 36.02 KB – The actual drag & drop script in Preview
    library
  • 45.25 KB – Ajax Control Toolkit
  • 4.08 KB – Timer script
  • 140.86 KB – ACT Animation framework
  • 18.05 KB – ACT Behavior base implementation. Required for
    Behaviors provided in the Ajax Control Toolkit project.
  • 16.48 KB – ACT Animation Behavior
  • 7.32 KB – My Custom Drag Drop behavior
  • 9.73 KB – My Custom Floating Behavior

The total payload for only the runtime is too high to accept
because we cannot make a user wait for 20 sec just to download Ajax
scripts before user can actually start interacting on the page. So,
I took several approaches to reduce the size of the
download:

  • Eliminate Preview version of Ajax completely and use ACT for
    Drag & Drop
  • Use IIS 6 compression to deliver compressed scripts from the
    client
  • Combine multiple script files into one file

ACT comes with its own DragDropManager which we need for Drag
& Drop. So far I thought of using
Sys.Preview.UI.DragDropManager in the ASP.NET Ajax January CTP. But
just for the DragDropManager, I need to add nearly 180 KB of
scripts for the entire Preview Library runtime. If I use
ACT’s DrgaDropManager, I can get rid of the Preview
runtime.

Without the preview scripts, the scripts downloaded are:

The following downloads are missing here:

  • 136.38 KB – Preview version of Ajax which allows Drag
    & Drop script
  • 36.02 KB – The actual drag & drop script in Preview
    library

I saved about 180 KB, nearly 7 sec from total download. So,
user gets to work on the page 7 secs earlier than before.

When I enabled IIS 6 compression, the situation improved
dramatically. Here’s the data transfers when compression is
enabled:

The total download comes down from 448 KB to 163 KB! Total 64%
download bytes reduction! This results in 3 times faster page
download.

The scripts are downloaded in two steps – first the core
runtimes download and then ACT and other scripts download. The
content is displayed after the core runtime is downloaded. So, the
time it takes to show the content on the browser reduces
significantly because now it takes only 50KB to download before
something is visible on the screen compared to 130 KB on
non-compressed mode.

ScriptManager control has
LoadScriptsBeforeUI property which you
can set to “False” in order to postpone several script
downloads after the content is downloaded. This adds the script
references end of the tag. As a result, you see the
content first and then the additional scripts, exteders, ACT
scripts get downloaded and initialized.


< asp:ScriptManager ID =”ScriptManager1″ runat =”server” EnablePartialRendering =”true” LoadScriptsBeforeUI =”false” ScriptMode =”Release” > asp:ScriptManager >

You can explicitly set ScriptMode=false in order to emit release
mode highly optimized Ajax runtime scripts during local
debugging.

Live OneCare – don’t try this at home

I honestly and sincerely tried to use Windows Live OneCare for a
month. It made my computer crawl to death, I still insisted on
using it. It made Visual Studio Build process 5x slower, I still
kept it on. It made my every day email check at least 40 mins
longer than before, I still stubbornly kept using it for my sincere
love for Microsoft Products. But no more. I had enough of it. This
is the most crap product Microsoft has ever released and will ever
release (I bet, otherwise I am switching to Linux/Java platform). I
am so shockingly surprised how Microsoft can produce such a low
quality mass consumer product, release it and keep shouting every
where about its praise. If you go to Microsoft forums, you will be
surprised how badly everyone has rejected this product. If
Microsoft is really honest about their products, they should have
withdrawn OneCare from the market after so much negative
feedback.

Here are some of the major problems I have faced:

  • I had to spend at least 30 to 40 mins more in Outlook 2007
    since I installed it. Every single click, swtching folders,
    downloading emails was so slow that it felt like I am using a 386
    PC.
  • Almost whole day whenever I look at CPU meter, it shows MsMpEng
    is taking 50% CPU. I have dual core, so it’s occupying one core at
    100%.
  • Visual Studio build was like hell. I can make a cup of tea,
    drink it, chat with my friends while my solution builds.
  • It will not cancel Virus scan no matter how many times I tell
    it to stop. It will again start within a minute. There is simply no
    way to return to workable condition until you disable the Virus
    & Spyware scanning feature.
  • Every morning, I tried to tolerate it for an hour or so, then I
    disable the Virus & Spyware scanning and go back to normal
    working mode. OneCare leaves no room for anyone to work on PC when
    it’s installed.

Every single person I have seen so far installing OneCare, has
gone back to some other product. There was not a single person who
could continue with it. So, unless Microsoft makes major fix in
this product and releases it under a different name, I will never
use it. I strongly suggest you also never use it.

Article of the month – vote for me

My article “Build Google
IG like Ajax Start Page in 7 days using ASP.NET Ajax and .NET
3.0
” is now on the nomination list of Codeproject. If you found
this article helpful, please give me a vote here:

http://www.codeproject.com/script/survey/survey.asp?survey=647

If you have not read it yet, I would request you to take a look
at it. You can learn about ASP.NET Ajax, ASP.NET 2.0 tricks,
Workflow Foundation, real use of WF from ASP.NET and finally how to
make real use of Dlinq, XLinq and Linq.

Believe it or not – Head of MyYahoo! joins Pageflakes!

I generally do not talk about Corporate stuffs and try to keep
my readers entertained with hot technologies. But this is too cool
to hold back.

Dan Cohen has joined Pageflakes as CEO. As a serial startup
entrepreneur and former head of MyYahoo!, Dan shares our passion
for creating cutting-edge products that are easy to use, and he has
vast expertise with personalized homepages, having also worked on
the Google personalized homepage. Above all, Dan is a great guy and
a lot of fun to work with. He’s also a pretty good drummer 🙂

Dan will be working out of our new U.S. headquarters in San
Francisco, but will of course be spending plenty of time working
with our teams in Europe and Asia. He’ll of course be staying on
top of his interests and hobbies using Pageflakes – check out his
public personalized homepage right here
.

P.S.: To read more about Dan’s joining, check out our
press release
.

Pageflakes is now becoming a really cool company to work for. We
are always looking for talented developers who have excellent skill
on HTML, CSS, JavaScript, ASP.NET 2.0 and Ajax. If you have the
right skill set, come on in. Join the elite team of developers who
are working behind the scene at Pageflakes bringing new innovations
to the Start Page market almost every month.

Starting from Bangladesh back at 2005, Pageflakes offices now
have spread over USA, Germany, London and Malaysia. The development
team is at Bangladesh and Malaysia right now. But contract
developers from all over the world work for Pageflakes. So, if you
are interested, please send me your resume and your cool works on
web at this email: omar at pageflakes dot com.