Cleanup inactive anonymous users from ASP.NET Membership Tables

ASP.NET 2.0 Websites that allow anonymous visit and anonymous
user profile have a unique challenge to cleanup unused data which
is generated by anonymous users who never come back. Every first
visit is creating one anonymous user, page setup, and other user
specific content. If the user never comes back, it still remains in
the database permanently. It is possible user might come back
within a day, or a week or a month. But there’s no guaranty
if user will ever come back or not. Generally sticky users are max
30% of the total users who come to most websites. So, you end up
with 70% unused data which are never needed. All these requires
cleanup, otherwise the database keeps growing uncontrollably and
gets slower and slower. This cleanup operation is humongous for
busy websites. Think about deleting millions of rows from several
tables, one after another while maintaining foreign key
constraints. Also the cleanup operation needs to run while the site
is running, without hampering site’s overall performance. The whole
operation results in heavily fragmented index and space in the MDF
file. The log file also becomes enormous in order to keep track of
the transactions. Hard drives get really hot and start sweating
furiously. While the CPU keeps smiling having nothing to do with
it, it’s really painful to watch SQL Server go through this
every day. Unless you clean up the database and maintain its size
under control; you can’t keep up with SQL Server’s RAM and
Disk IO requirement.

When a user visits the site, Asp.net Membership Provider updates
the LastActivityDate of aspnet_users table. From this field, I can
find out how long the user has been idle. The IsAnonymous bit field
tells me whether the user account is anonymous or registered. If it
is registered, no need to worry. But if it is anonymous and more
than 30 days old, I can be sure that the user will never come back
because the cookie has already expired. If you repeatedly logout
from your start page, all cookie related to the site gets cleared.
That means you are producing one new anonymous user record during
each log out. That anonymous record is never used because you will
soon log in to have your customized pages back and then you will
log out again. This will result in another anonymous user account
which again becomes useless as soon as you log in.

Here’s how the whole cleanup process works:

  • Find out the users which are old enough to be discarded
  • Find out the pages user has
  • Delete all the widget instances on those pages
  • Then delete those pages
  • Remove rows from child tables related to aspnet_users like
    aspnet_profile, aspnet_UsersInRoles, aspnet_PersonalizationPerUser.
    Remove rows for users to be deleted
  • Remove the users from aspnet_users
  • Pray that you did not accidentally remove any good user

Here’s the giant DB script which does it all. I have put
enough inline comment so that you can understand what the script is
doing:

   1: -- Number of days after which we give users 'bye bye'
   2: DECLARE @Days int
   3: SET @Days = 14
   4:
   5: -- No of users to delete per run. If it's too high, database will get stuck
   6: -- for a long time. If it's too low, you will end up having more trash than
   7: -- you can cleanup
   8: DECLARE @NoOfUsersToDelete int
   9: SET @NoOfUsersToDelete = 1000
  10:
  11: -- Create temporary tables which holds the users and pages to delete
  12: -- As the user and the page is used to find out other tables, instead
  13: -- of running SELECT ID FORM ... repeatedly, it's better to have them
  14: -- in a temp table
  15: IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[PagesToDelete]') AND type in (N'U'))
  16: DROP TABLE [dbo].[PagesToDelete]
  17: IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[aspnetUsersToDelete]') AND type in (N'U'))
  18: DROP TABLE [dbo].[AspnetUsersToDelete]
  19:
  20: create table PagesToDelete (PageID int NOT NULL PRIMARY KEY)
  21: create table AspnetUsersToDelete (UserID uniqueidentifier NOT NULL PRIMARY KEY)
  22:
  23: -- Find out inactive anonymous users and store the UserID in the temporary
  24: -- table
  25: insert into AspnetUsersToDelete
  26: select top(@NoOfUsersToDelete) UserID from aspnet_Users where
  27: (isAnonymous = 1) and (LastActivityDate < (getDate()-@Days))
  28: order by UserID -- Saves SQL Server from sorting in clustered index again
  29:
  30: print 'Users to delete: ' + convert(varchar(255),@@ROWCOUNT)
  31: GO
  32:
  33: -- Get the pages of the users which will be deleted
  34: insert into PagesToDelete
  35: select ID from Page where UserID in
  36: (
  37: select UserID from AspnetUsersToDelete
  38: )
  39:
  40: print 'Pages to delete: ' + convert(varchar(255),@@ROWCOUNT)
  41: GO
  42:
  43: -- Delete all Widget instances on the pages to be deleted
  44: delete from WidgetInstance where PageID IN
  45: ( SELECT PageID FROM PagesToDelete )
  46:
  47: print 'Widget Instances deleted: ' + convert(varchar(255), @@ROWCOUNT)
  48: GO
  49:
  50: -- Delete the pages
  51: delete from Page where ID IN
  52: ( SELECT PageID FROM PagesToDelete )
  53: GO
  54:
  55: -- Delete User Setting
  56: delete from UserSetting WHERE UserID IN
  57: ( SELECT UserID FROm AspnetUsersToDelete )
  58: GO
  59:
  60: -- Delete profile of users
  61: delete from aspnet_Profile WHERE UserID IN
  62: ( SELECT UserID FROm AspnetUsersToDelete )
  63: GO
  64:
  65: -- Delete from aspnet_UsersInRoles
  66: delete from aspnet_UsersInRoles WHERE UserID IN
  67: ( SELECT UserID FROm AspnetUsersToDelete )
  68: GO
  69:
  70: -- Delete from aspnet_PersonalizationPerUser
  71: delete from aspnet_PersonalizationPerUser WHERE UserID IN
  72: ( SELECT UserID FROm AspnetUsersToDelete )
  73: GO
  74:
  75: -- Delete the users
  76: delete from aspnet_users where userID IN
  77: ( SELECT UserID FROm AspnetUsersToDelete )
  78:
  79: PRINT 'Users deleted: ' + convert(varchar(255), @@ROWCOUNT)
  80: GO
  81:
  82:
  83: drop table PagesToDelete
  84: drop table AspnetUsersToDelete
  85: GO

Now the question comes, when can I run this script? It depends on
several factors:

  • The lowest traffic period. For example, USA
    midnight time when everyone in USA is sleeping if your majority
    users are from USA
  • The period when there’s no other
    maintenance tasks running like Index Defrag or Database Bakup. If
    by any chance any other maintenance task conflicts with this
    enormous delete operation, SQL Server is dead.
  • The operation will take from 10 mins to
    hours depending on the volume of trash to cleanup. So, consider the
    duration of running this script and plan other maintenance jobs
    accordingly.
  • It’s best to run 30 mins before INDEX
    DEFRAG jobs run. After the script completes, the tables will be
    heavily fragmented. So, you need to defrag the indexes.

Before running this script, there are some preparations to
take:

  • Make sure you have turned of AutoShrink from Database Property.
    Database size will reduce after the cleanup and if SQL Server tried
    to shrink the database, there will be a big IO activity. Turn off
    auto shrink because the database will grow again.
  • Make sure the LOG file’s initial size is big enough to
    hold such enormous transactions. You can specify 1/3rd of the MDF
    size as LDF’s Initial Size. Also make sure the log file is
    not shrunk. Let it occupy HD space. It saves SQL Server from
    expanding the file and shrinking the file. Both of these require
    high Disk IO.

Once the cleanup job runs and the INDEX DEFRAG runs, the
database performance will improve significantly. The tables are now
smaller. That means the indexes are now smaller. SQL Server need
not to run through large indexes anymore. Future INDEX DEFRAGs take
shorter time because there’s not much data left to optimize.
SQL Server also takes less RAM because it has to work with much
less amount of data. Database backup size also reduces because the
MDF size does not keep increasing indefinitely. As a result, the
significant overhead of this cleanup operation is quite acceptable
when compared to all the benefits.

Note: I will be posting some stuffs from my old blog to new blog.
Please ignore if you have read them before. "http://msmvps.com/aggbug.aspx?PostID=709188" width="1" height="1">

21 Comments

  1. its easy to clean membership just call

    Membership.DeleteUser(Request.AnonymousID, True)

  2. In response to Allan:

    Membership.DeleteUser(Request.AnonymousID, True)

    is fine for handling one user.

    But if you want to delete hundreds or thousands of users it would be very inefficient to do hundreds or thousands of individual database calls.

    Vikram’s method handles all users in one go.

  3. If someone would be so kind as to inform me, a simple asp user on how to use/run this DB Script. Is it something I load into a file then execute, or do I put it into a console? Any help given is much appreciated.

    :)

  4. I ran this on one of my DBs with which I use anonymous profiles, and it seems to work. It did get rid of a lot of rows in the users table!

    However, I'm not using widgets, pages, or UserSettings at all in this site, so the script threw several errors:

    Msg 208, Level 16, State 1, Line 3

    Invalid object name 'Page'.

    Msg 208, Level 16, State 1, Line 3

    Invalid object name 'WidgetInstance'.

    Msg 208, Level 16, State 1, Line 3

    Invalid object name 'Page'.

    Msg 208, Level 16, State 1, Line 3

    Invalid object name 'UserSetting'.

    Should I just comment out the lines that refer to these non-existent objects? Or is it all right to run the script as-is, and just let the errors occur?

  5. Thanks for the script; it works great. However, the disk space used by the tables is not reduced after the rows have been deleted. Is there something else I need to do?

  6. Thanks for the script, and thanks for all your aticles, actually just found 2 articles for you today when i was looking for information about member profiles… And they really helped me understand the concept and i did implement them on my projects…

    Thanks a lot :)

  7. I tried your approach vs. having cascading deletes and simply doing:
    delete top (10000) aspnet_Users where IsAnonymous = 1 and LastActivityDate <= GETDATE() – 30

    Your approach took 1:30 mins where the other approach took 3:20 minutes, so yours obviously taking less than half the time!

    Thanks!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>