Client side Page Fragment Output cache, reduce page download time significantly

When you have a page full of lots of HTML, it will be best if
the whole page can be cached on the browser. You can do this using
HTTP Response caching header either by injecting them manually or
by using @OutputCache tag directive on ASPX pages.



<%
@ OutputCache
Location
=”Client”
Duration
=”86400″
VaryByParam
=”*”
VaryByHeader
=”*”

%>

But if part of the page is dynamic and part of the page is
static (like header, left side menu, foother, right side menu,
bottom part) etc where static parts of the page occupy a
significant amount of html, then if you could cache those parts on
the browser, you could save a lot of bytes that gets downloaded
every time the page downloads. On most of the websites, you will
find the header, navigation menu, footer, bottom part,
advertisements are mostly static and thus easily cacheable. If the
whole page size is say 50KB, at least 20KB is static and 30KB might
be dynamic. If you can use client side caching of page fragments
(not ASP.NET’s server side page output cache), you can save
40% download time easily.

ASP.NET offers you page fragment caching using @Outputcache
which is good, but that caching is on server side. It cache the
output of user controls and serves them from server side cache. You
cannot eliminate the download of those costly bytes. It just saves
some CPU process. Nothing much for users in it.

The only way to cache part of page is by allowing browser to
download those parts separately and making those parts cacheable
just like images/CSS/javascripts get cached. So, we need to
download page fragments separately and make them cached on the
browser’s cache. IFRAME is an easy way to do this but IFRAME
makes the page heavy and thus not follow CSS of the page. There are
many reasons why IFRAME can’t work. So, we have a better way,
we will use Javascript to render content of the page and
javascript will get cached on the browser’s cache
.

So, here’s the idea:

  • We will split the whole page into multiple parts
  • We will generate page content using Javascript. Each cacheable
    part comes from javascript and javascript renders the HTML of
    it.
  • The parts which are cachable gets cached by the browser and
    thus never downloaded again (until you want them to be). But those
    parts which are non-cachable and highly dynamic, does not get
    cached by browser.

So, let’s think of a page setup like this:

Logo

Header

Left navigation Menu

Dynamic part of the page

Footer


Here only one part is dynamic and the rest is fully cacheable.
So, the Default.aspx which renders this whole page looks like
this:

<%@PageLanguage="VB"AutoEventWireup="false"%>
<%@OutputCacheNoStore="true"Location="None"%>
DOCTYPEhtmlPUBLIC"-//W3C//DTDXHTML 1.0 Transitional//EN""http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<htmlxmlns="http://www.w3.org/1999/xhtml">
<headrunat="server">
    <title>MyBig Fat Pagetitle>
head>
<body>

<formid="form1"runat="server">
<tablewidth="100%"border="1">   
<tr>
<td>Somelogo heretd>
<td><scriptid="Script1"src="Header.aspx"type="text/javascript">script>td>
tr>       
<tr>
<td><scriptid="LeftMenu"src="LeftMenu.aspx"type="text/javascript">script>td>
<tdbgcolor="lightgrey"><div>
This is the dynamic part which gets changed on every load. Checkout the time when
it was generated: <%=DateTime.Now %>div>td>
tr>       
<tr>
<tdcolspan="2"><scriptid="Footer"src="Footer.aspx"type="text/javascript">script>td>
tr>
table>
form>

body>
html>

The page looks like this:

You see, the cached parts are 30 mins older. Browser has not
downloaded those parts at all and thus saved a significant amount
of data transfer. The only part that was downloaded was the dynamic
part.

When you load the page first time, all 4 files are downloaded.
But the last 3 files get cached and never downloaded until
browser’s cache expires. So, on second visit, only one file
downloaded and thus saves a significant amount of data
transfer.

Let’s look at one of the files Header.aspx which gets
cached. Nothing fancy here, it’s a regular ASPX page:

The interesting thing here is the “ContentType”
which I have set to “text/html/javascript”. This is not
something built-in, I have introduced this type.

When you put an ASPX inside a Script tag, it surely does not
work because < script
id
=”Script1″
src
=”Header.aspx”
type
=”text/javascript”>
expects javascript output, not html output. If html output is
provided, browsers simply ignores it. So, we need to convert the
output of Header.aspx into Javascript which when downloaded and
executed by the browser, emits the original html that was generated
when ASP.NET executed the page.

We use HTTP Module to intercept all .aspx calls and when the
page is about to be written to the output, we check if the content
type is “text/html/javascript”. If it is, this is our
cue to convert the page output to javascript representation.

If you want to know details about HTTP Module and how to use
Response Filter to modify page output, please read this wonderful
article:

http://www.aspnetresources.com/articles/HttpFilters.aspx

It really explains all the things. I would recommend you read
this article first and then continue with the rest.

We have made a response filter named Html2JSPageFilter.js
(available in the code download), which overrides the Write method
of Stream and converts the entire HTML of the page to javascript
representation:

    publicoverridevoidWrite(byte[]buffer, intoffset, intcount)
    {
        stringstrBuffer = System.Text.UTF8Encoding.UTF8.GetString(buffer, offset, count);

        //---------------------------------
        //Wait for the closing  tag
        //---------------------------------
        Regexeof = newRegex("",RegexOptions.IgnoreCase);

        if(!eof.IsMatch (strBuffer))
        {
            responseHtml.Append (strBuffer);
        }
        else
        {
            responseHtml.Append (strBuffer);
            string finalHtml = responseHtml.ToString ();

            //extract only the content inside the form tag tag ASP.NET generatesin all .aspx
            intformTagStart = finalHtml.IndexOf(");
            intformTagStartEnd = finalHtml.IndexOf('>',formTagStart);
            intformTagEnd = finalHtml.LastIndexOf("");

            stringpageContentInsideFormTag = finalHtml.Substring(formTagStartEnd + 1,formTagEnd - formTagStartEnd - 1);

First we get the entire page output, then we get only what is
inside the

tag that ASP.NET generates for all .aspx
pages.

Next step is to remove the viewstate hidden field because this
will conflict with the view state on the default.aspx.

            //Remove the __VIEWSTATE tag because page fragments don't needviewstate
            //Note this will make all ASP.NET controls in the page fragments gomad which 
            //needs viewstate to do their work.
            Regexre = newRegex("()",RegexOptions.IgnoreCase);
            pageContentInsideFormTag =re.Replace(pageContentInsideFormTag, string.Empty);

Now we convert the entire html output to javascript string
format:

            ///Convert the HTML to javascript string representation
            stringjavascript2Html = 
                pageContentInsideFormTag.Replace("r","")
                .Replace("n","")
                .Replace("   ","")
                .Replace(" ","")
                .Replace("  ","")
                .Replace("","")
                .Replace("'","'");

Final touch is to put that javascript string inside a
“document.write(‘…’);” call. When you
call document.write to emit html, it gets part of the page
html:

            //Generate the document.write('...') which adds the content in thedocument
            stringpageOutput = "document.write('"+ javascript2Html + "');";

This is basically the trick. Use a Response filter to get the
.aspx output, and then convert it to Javascript representation.

For convenience, I have used a HttpModule to hook into ASP.NET
pipeline and wait for .aspx files which try to emit content type of
“text/html/javascript”. Again this content type is
nothing special, you could use “text/Omar Al
Zabir”.

    voidIHttpModule.Init(HttpApplicationcontext)
    {
        context.ReleaseRequestState += newEventHandler(InstallResponseFilter);
    }

    privatevoidInstallResponseFilter(objectsender, EventArgse) 
    {
     HttpResponseresponse = HttpContext.Current.Response;

     if(response.ContentType == "text/html/javascript")
     {
         response.ContentType = "text/javascript";
         response.Filter = newHtml2JSPageFilter(response.Filter);
     }
    }

And finally in web.config, we have to register the HttpModule so
that it gets called:

        <httpModules>
            <addname="Html2JSModule"type="Html2JavascriptModule"/>
        httpModules>

The entire source code is available in this URL:

Download Source
code of: Client side Page Fragment Output cache, reduce page
download time significantly

Enjoy. Use this approach in your aspx and html files and save
significant amount of download time on users end. Although it
slightly increases first time visit download time (200+ms for each
script tag), but it makes second time visit a breeze. See the
performance difference yourself. First visit www.pageflakes.com. Then close your
browser, open it again and enter www.pageflakes.com. See how fast it
loads. If you use a HTTP debugger to monitor how much data is
transferred, you will see it’s only 200 bytes!

IIS 6 Compression – quickest and effective way to do it for ASP.NET compression

IIS 6 has builtin gzip compression ability which can compress
output of dynamic webpages (.aspx) and webservices (.asmx). The
compression is really good and can easily reduce 60% download time.
You should always turn this feature on in your production server.
The CPU usage is not that high compared to the reduction of
download time for users. Your users will love the significant
download time reduction when you turn it on.

Now, on internet you will find a lot of solutions. I tried all
of them which appears in first 30 Google search results. But failed
to make any of them work properly. Finally I was able to make it
work, but I realized you have to do it in a very specific way and
in specific order. Here it goes:

  • Go to IIS Manager from Administrative Tools
  • Right click on your computer name (not on websites or Default
    Web Site)
  • Choose All Tasks-> Restart IIS
  • From the drop down, choose “Stop IIS” and click OK.
  • IIS is not stopped. Make sure it’s not still running
  • Now go to C:WINDOWSSYSTEM32INETSRV
  • Make a copy of the file Metabase.xml. This is a dangerous
    file, don’t play around with it. Make sure you have a backup before
    you do what I am going to tell you now.
  • Open the metabase.xml in Notepad. Don’t use any other editor
    besides Notepad, Notepad2 or Visual Studio.
  • Find using “IIsCompressionScheme”
  • You will find a match which looks like this:
    “<IIsCompressionScheme Location=”/LM/W3SVC/Filters/Compression/deflate”
    There are two nodes named IIsCompressionScheme and one node with plural IIsCompressionSchemes.
  • Delete these nodes.
  • Once you have deleted the 3 nodes, paste the text from this
    link in in their position:
    http://tinypaste.com/630fc
Now start IIS and hit your site once. When it runs for the
first time, it will send uncompressed output, but it will compress
it behind the scene. So, next hit will give you the compressed
output.

Go to www.pipeboost.com
and enter the URL to ensure you are getting compressed content.
Before you do so, make sure you have visited your site for a while
in your local browser so that the pages got the chance to get
themselves compressed.

Blogging from InfoPath

I’m trying to Blog from InfoPath. There’s a fantastic InfoPath
template available here:
I have tried Word 2007’s new blogging feature. It works
fine for posts without code. But if you insert code, the code looks
super horrible when the blog gets published. Even the “Copy as
HTML” can’t give you good output. Also no image upload
support.
I am thinking what if there was a nice addin for MS Word which
can make a document publish as blog by automatically uploading
pictures in it and nicely formatting code blocks, it could make us
so productive. Mankind has invented so many great tools, but a
really good Blogging tool that works for all is still
missing.

Send a message individually to all recipients using Outlook

Say you have written a message which has lots of users in the
recipient list. Now you don’t want to send the mail to all those
users who can see each others email address. You want to send the
message one by one. Here’s what you do:

  • After composing the message save it to make it go to Drafts
    folder.
  • Select the message
  • Press ALT+F11 to bring the Visual Basic Editor.
  • Now paste the following code:

Sub SendIndividually()

Dim objItem As MailItem
Set objItem = Application.ActiveExplorer.Selection.Item(1)

Dim objTo As Recipient
Dim objClonedMail As MailItem

On Local Error Resume Next

For Each objTo In objItem.Recipients

Set objClonedMail = objItem.Copy()
objClonedMail.To = “”
objClonedMail.Recipients.Add objTo
objClonedMail.Send

Next

End Sub

  • Hit F5 and wait for a while. You will see Outbox is full of
    copy of the message with individual recipient in the “To”

Read my previous post on how to generate a message with all the
sender’s email address from all the message in a folder. You can
combine both approach to send everyone important notice.

Get email address of all users from all mails in Outlook Folder

Sometimes you want to send some important notice to everyone who
has ever mailed you. Let’s say you have a folder named “Friends” in
Outlook where you store all the emails from your friends. Now you
want to get all of their email addresses. Pretty difficult work if
you have thousands of such mails. Here’s an easy way.

  • Select the folder in Outlook and press ALT+F11. It will open
    Visual Basic Editor.
  • Double click on ThisOutlookSession from the Project tree.
  • Paste the following function:

Sub GetALLEmailAddresses()

Dim objFolder As Folder
Set objFolder = Application.ActiveExplorer.Selection

Dim dic As New Dictionary
Dim strEmail As String
Dim strEmails As String

Dim objItem As MailItem
For Each objItem In objFolder.Items

strEmail = objItem.SenderEmailAddress
If Not dic.Exists(strEmail) Then
strEmails = strEmails + strEmail + “;”
dic.Add strEmail, “”
End If

Next

Debug.Print strEmails
End Sub

Hit F5 and it will run for a while. Then press Ctrl+G. You will
see the email addresses in the “Immediate
Window”.

Copy the whole string and you have all the email addresses from
all the emails in the selected Outlook folder. There will be no
duplicate address in the list.

Mac OSX vs. Windows Vista

Today we purchased an iMac (Intel Dual Core) in order to test
Safari 2.0 support for Pageflakes. My first impression, I
am totally blown away by its performance. The UI is unbelievable
well done and the visual effects you see are 10 years ahead of
time. I was comparing OS features of Mac OS X 10.4.X.X with Windows
Vista. To be honest, I am suffering from inferiority complex now.
Windows Vista is so lame compared to Mac OS X. Vista copied all OS
X’s concepts for which at least I feel ashamed whether Microsoft
feels or not. But it did not make anything better after copying.
Normally Microsoft copies others ideas and makes them really better
in all possible ways. But Mac OS X is so seriously better than any
other OS. The widgets you see with the OS X are mind
blowing. Vista side bar goes nowhere near to it. Also the
concept of switching to regular mode and widget mode is by far the
best idea. We normally don’t want to occupy desktop space with
widgets. That’s why in OS X, the whole screen switches to widget
view when you press middle button.

OSX kernel is made from Unix. So, it’s
seriously powerful and stable. The command line tools are very
feature rich. Those who love *nix command line, will feel at home
in OSX. Vista still offers those poor command line tools from
Windows XP/2003. Nothing improved much.

If somehow OSX can run Windows apps like Visual Studio, MS Word
etc, I will undoubtedly switch to OSX. Even Windows Vista won’t be
able to compromise me.

Best of all, you can buy an iMac with Intel Processor and use
dual boot to run both Windows and Mac OSX. So, when you work, boot
on Windows. When you want to have fun, go to OSX.

Cache-Control header cannot be set properly in ASP.NET 2.0 – A solution

Do you know that you cannot set Cache-Control: max-age header in
ASP.NET 2.0? No matter how many combinations you try, you will
always end up with “max-age=0” for sure.

I found solution to the SetMaxAge problem in HttpCachePolicy.
Here’s the code of the function in HttpCachePolicy
(decompiled):

public void SetMaxAge(TimeSpan
delta)
{

if (delta < TimeSpan.Zero)

{

throw new ArgumentOutOfRangeException(“delta”);

}

if (HttpCachePolicy.s_oneYear < delta)

{

delta = HttpCachePolicy.s_oneYear;

}

if (!this._isMaxAgeSet || (delta < this._maxAge))

{

this.Dirtied();

this._maxAge = delta;

this._isMaxAgeSet = true;

}
}

Someone is setting maxAge already to 0. So, if I assign a higher
value it does not get set.

I am not sure why the “(delta < this._maxAge)" condition is necessary. If I somehow set a lower maxAge, I can never increase it!

So, I have made an HttpModule which does this:

void context_EndRequest(object
sender, EventArgs e) {
if (HttpContext.Current.Items.Contains(“SetMaxAge”))
{

FieldInfo maxAge =
HttpContext.Current.Response.Cache.GetType().GetField(“_maxAge”,
BindingFlags.Instance | BindingFlags.NonPublic);

maxAge.SetValue(HttpContext.Current.Response.Cache,
(TimeSpan)HttpContext.Current.Items[“SetMaxAge”]);


// SetMaxAge does not
work.
//HttpContext.Current.Response.Cache.SetMaxAge((TimeSpan)HttpContext.Current.Items[“SetMaxAge”]);

HttpContext.Current.Response.Cache.SetCacheability(HttpCacheability.Public);
HttpContext.Current.Response.Cache.SetExpires(DateTime.Now.AddMinutes(5));
HttpContext.Current.Response.Cache.SetRevalidation(HttpCacheRevalidation.AllCaches);
}
}

If someone sets “SetMaxAge” to a Timespan in the Context.Items,
it will get set to the response and the end of the request. This
ensures, no matter who does what with cache object, maxAge will
definitely be set.

Although this is not a Atlas specific problem, it seems to be a
problem in ASP.NET 2.0 itself.

Now I can cache AJAX call responses on the browser and prevent
roundtrips.

Pageflakes #1 Start page in Web 2.0 awards

Pageflakes won the Web
2.0 Award for #1 Start Page. See the “Start Page” category
from:

http://web2.0awards.org

#2 is Google’s IG and #3 is Microsoft’s Windows Live.

Pageflakes is developed using ASP.NET 2.0 and Microsoft
Atlas.

Here’s what Pageflakes say:

Pageflakes is your personalized Internet. You can add what you
like and remove what you don’t like – and it’s totally simple. What
you see on the site right now is just a selection of a few standard
modules (“flakes”, as we call them) that allow you to read blogs,
do web searches, create a to-do list, check your Gmail account and
read the latest news. Normally you’d have to go to a different web
page for each of those things. With Pageflakes, you have it all on
one page! And it’s all free.

Note: You can now create public sites and share your page with
your friends! Setup a personal site very easily and collaboratively
work together