jChaart – Web Dashboard Framework

A Javascript-only, web based Dashboard library, that you can use to make eye-catchy Charts showing Transactions stats for your application, Server CPU, Memory, IO graphs etc. It offers libraries to convert different types of delimited text files into Charts. Those delimited files are generated from various sources, eg running SQL queries against a Database, or running shell scripts to collect system stats, or Powershell scripts to process IIS logs. End result is a nice-looking Twitter Bootstrap powered, responsive Web Dashboard, that you can get up and running in no time, on any platform. Since it is HTML and Javascript, you can customize it to show exactly what you want, how you want. Quite handy for earning brownie points and wooing your customers.

Here’s an example how a Transaction Dashboard may look like:

 

Screenshot

 

And an example of a System Monitoring dashboard:

ServerHealth

The GitHub project is here:

https://github.com/oazabir/jChaart

You are welcome to join and participate.

7 hard earned lessons from a Single Page Application

My talk at London AJAX User group on some performance optimization techniques that can give a javascript rich Single Page Application big boost in terms of page load performance.

MyPicAJAXTalk

See the lecture here:

http://skillsmatter.com/podcast/ajax-ria/7-real-life-lessons-learnt-from-a-single-page-application/mh-6922

Here are the slides.

image

Fast Streaming Ajax Proxy with GET PUT POST DELETE

I have enhanced my streaming Ajax Proxy with POST, PUT and
DELETE features. Previously it supported only GET. Now it supports
all 4 popular methods for complete REST support. Using this proxy,
you can call REST API on external domain directly from your
website’s javascript code. You can test the proxy from this
link:


labs.omaralzabir.com/ajaxstreamingproxy/GetPutDeleteTest.aspx

The latest source code for the Ajax Proxy is available here:

http://code.google.com/p/fastajaxproxy/

You can find a detail CodeProject article that explains how the
streaming asynchronous aspect of this proxy works:

Fast, Scalable,
Streaming AJAX Proxy – continuously deliver data from across
domains

Here’s how the test UI looks like where you can test POST,
PUT and DELETE:


image

If you want to run the sample source code on your local IIS,
make sure you allow the POST, PUT, and DELETE headers on .ashx
extension from IIS properties:


image

The sample project shows how you can use the proxy to make calls
to external domains. You can directly hit any external URL and
perform POST or DELETE from your javascript code:

var proxyUrl = "StreamingProxy.ashx";
function download(method, proxyUrl, contentUrl, isJson, bodyContent, completeCallback) { var request = new Sys.Net.WebRequest(); if (method == "POST" || method == "PUT") request.set_httpVerb("POST"); else request.set_httpVerb("GET"); var url = proxyUrl + "?m=" + method +
(
isJson ? "&t=" + escape("application/json") : "") + "&u=" + escape(contentUrl); request.set_url(url); if (bodyContent.length > 0) { request.set_body(bodyContent); request.get_headers()["Content-Length"] = bodyContent.length; } var startTime = new Date().getTime(); request.add_completed(function(executor) { if (executor.get_responseAvailable()) { var content = executor.get_responseData(); var endTime = new Date().getTime(); var statistics = "Duration: " + (endTime - startTime) + "ms" + 'n' + "Length: " + content.length + " bytes" + 'n' + "Status Code: " + executor.get_statusCode() + " " + 'n' + "Status: [" + executor.get_statusText() + "]" + 'n'; appendStat(statistics); get('resultContent').value = content; completeCallback(); } }); var executor = new Sys.Net.XMLHttpExecutor(); request.set_executor(executor); executor.executeRequest(); }

I am using MS AJAX here. You can use jQuery to perform the same
test as well. All you need to do is hit the URL of the
StreamingProxy.ashx and pass the actual URL in query string
parameter “u” and pass the type of the http method in
query string parameter “m”. That’s it!

7 tips for for loading Javascript rich Web 2.0-like sites significantly faster

Introduction

When you create rich Ajax application, you use external
JavaScript frameworks and you have your own homemade code that
drives your application. The problem with well known JavaScript
framework is, they offer rich set of features which are not always
necessary in its entirety. You may end up using only 30% of jQuery
but you still download the full jQuery framework. So, you are
downloading 70% unnecessary scripts. Similarly, you might have
written your own javascripts which are not always used. There might
be features which are not used when the site loads
for the first time, resulting in unnecessary download during
initial load. Initial loading time is crucial – it can make
or break your website. We did some analysis and found that every
500ms we added to initial loading, we lost approx 30% traffic who
never wait for the whole page to load and just close browser or go
away. So, saving initial loading time, even by couple of hundred
milliseconds, is crucial for survival of a startup, especially if
it’s a Rich AJAX website.

You must have noticed Microsoft’s new tool Doloto
which helps solve the following problem:

Modern Web 2.0 applications, such as GMail, Live Maps, Facebook
and many others, use a combination of Dynamic HTML, JavaScript and
other Web browser technologies commonly referred as AJAX to push
page generation and content manipulation to the client web browser.
This improves the responsiveness of these network-bound
applications, but the shift of application execution from a
back-end server to the client also often dramatically increases the
amount of code that must first be downloaded to the browser. This
creates an unfortunate Catch-22: to create responsive distributed
Web 2.0 applications developers move code to the client, but for an
application to be responsive, the code must first be transferred
there, which takes time.

Microsoft Research looked at this problem and published
this
research paper in 2008
, where they showed how much improvement
can be achieved on initial loading if there was a way to split the
javascripts frameworks into two parts – one primary part
which is absolutely essential for initial rendering of the page and
one auxiliary part which is not essential for initial load and can
be downloaded later or on-demand when user does some action. They
looked at my earlier startup Pageflakes and reported:

2.2.2 Dynamic Loading: Pageflakes
A contrast to Bunny Hunt is the Pageflakes application, an
industrial-strength mashup page providing portal-like
functionality.
While the download size for Pageflakes is over 1 MB, its
initial
execution time appears to be quite fast. Examining network
activity
reveals that Pageflakes downloads only a small stub of code
with the initial page, and loads the rest of its code dynamically
in
the background. As illustrated by Pageflakes, developers today
can
use dynamic code loading to improve their web application’s
performance.
However, designing an application architecture that is
amenable to dynamic code loading requires careful consideration
of JavaScript language issues such as function closures,
scoping,
etc. Moreover, an optimal decomposition of code into
dynamically
loaded components often requires developers to set aside the
semantic
groupings of code and instead primarily consider the execution
order of functions. Of course, evolving code and changing
user workloads make both of these issues a software maintenance
nightmare.

Back in 2007, I was looking at ways to improve the initial load
time and reduce user dropout. The number of users who would not
wait for the page to load and go away was growing day by day as we
introduced new and cool features. It was a surprise. We thought new
features will keep more users on our site but the opposite
happened. Analysis concluded it was the initial loading time that
caused more dropout than it retained users. So, all our hard work
was essentially going to drain and we had to come up with something
ground breaking to solve the problem. Of course we had already
tried all the basic stuffs –
IIS compression
,
browser caching
, on-demand loading of JavaScript,
css and html
when user does something, deferred
JavaScript execution
– but nothing helped. The frameworks
and our own hand coded framework was just too large. So, the idea
tricked me, what if we could load functions inside a class in two
steps. First step will load the class with absolutely essential
functions and second step will inject more functions to the
existing classes.

I published a codeproject article which shows you 7 tricks to
significantly improve page load time even if you have large amount
of Javascript used on the page.

7
Tips for Loading JavaScript Rich Web 2.0-like Sites Significantly
Faster

  1. Use Doloto
  2. Split a Class into Multiple JavaScript Files
  3. Stub the Functions Which Aren’t Called During Initial Load
  4. JavaScript Code in Text
  5. Break UI Loading into Multiple Stages
  6. Always Grow Content from Top to Bottom, Never Shrink or
    Jump
  7. Deliver Browser Specific Script from Server

If you like these tricks, please vote for me!

Web 2.0 AJAX Portal using jQuery, ASP.NET 3.5, Silverlight, Linq to SQL, WF and Unity

Dropthings
– my open
source
Web 2.0 Ajax Portal has gone through a technology
overhauling. Previously it was built using ASP.NET AJAX, a little
bit of Workflow Foundation and Linq to SQL. Now Dropthings boasts
full jQuery front-end combined with ASP.NET AJAX
UpdatePanel, Silverlight widget, full
Workflow Foundation implementation on the business
layer, 100% Linq to SQL Compiled Queries on the
data access layer, Dependency Injection and Inversion of Control
(IoC) using Microsoft Enterprise Library 4.1 and
Unity. It also has a ASP.NET AJAX Web Test
framework that makes it real easy to write Web Tests that simulates
real user actions on AJAX web pages. This article will walk you
through the challenges in getting these new technologies to work in
an ASP.NET website and how performance, scalability, extensibility
and maintainability has significantly improved by the new
technologies. Dropthings has been licensed for commercial use by
prominent companies including BT Business, Intel, Microsoft IS,
Denmark Government portal for Citizens; Startups like Limead and
many more. So, this is serious stuff! There’s a very cool
open source implementation of Dropthings framework available at
National
University of Singapore
portal.

Visit: http://dropthings.omaralzabir.com


Dropthings AJAX Portal

I have published a new article on this on CodeProject:

http://www.codeproject.com/KB/ajax/Web20Portal.aspx

Get the source code

Latest source code is hosted at Google code:

http://code.google.com/p/dropthings

There’s a CodePlex site for documentation and issue
tracking:

http://www.codeplex.com/dropthings

You will need Visual Studio 2008 Team Suite with Service Pack 1
and Silverlight 2 SDK in order to run all the projects. If you have
only Visual Studio 2008 Professional, then you will have to remove
the Dropthings.Test project.

New features introduced

Dropthings new release has the following features:

  • Template users – you can define a user who’s pages
    and widgets are used as a template for new users. Whatever you put
    in that template user’s pages, it will be copied for every
    new user. Thus this is an easier way to define the default pages
    and widgets for new users. Similarly you can do the same for a
    registered user. The template users can be defined in the
    web.config.
  • Widget-to-Widget communication – Widgets can send message
    to each other. Widgets can subscribe to an Event Broker and
    exchange messages using a Pub-Sub pattern.
  • WidgetZone – you can create any number of zones in any
    shape on the page. You can have widgets laid in horizontal layout,
    you can have zones on different places on the page and so on. With
    this zone model, you are no longer limited to the Page-Column model
    where you could only have N vertical columns.
  • Role based widgets – now widgets are mapped to roles so
    that you can allow different users to see different widget list
    using ManageWidgetPersmission.aspx.
  • Role based page setup – you can define page setup for
    different roles. For ex, Managers see different pages and widgets
    than Employees.
  • Widget maximize – you can maximize a widget to take full
    screen. Handy for widgets with lots of content.
  • Free form resize – you can freely resize widgets
    vertically.
  • Silverlight Widgets – You can now make widgets in
    Silverlight!

Why the technology overhauling

Performance, Scalability, Maintainability and Extensibility
– four key reasons for the overhauling. Each new technology
solved one of more of these problems.

First, jQuery was used to replace my personal hand-coded large
amount of Javascript code that offered the client side drag &
drop and other UI effects. jQuery already has a rich set of library
for Drag & Drop, Animations, Event handling, cross browser
javascript framework and so on. So, using jQuery means opening the
door to thousands of jQuery plugins to be offered on Dropthings.
This made Dropthings highly extensible on the client side.
Moreover, jQuery is very light. Unlike AJAX Control Toolkit jumbo
sized framework and heavy control extenders, jQuery is very lean.
So, total javascript size decreased significantly resulting in
improved page load time. In total, the jQuery framework, AJAX basic
framework, all my stuffs are total 395KB, sweet! Performance is
key; it makes or breaks a product.

Secondly, Linq to SQL queries are replaced with Compiled
Queries. Dropthings did not survive a load test when regular lambda
expressions were used to query database. I could only reach up to
12 Req/Sec using 20 concurrent users without burning up web server
CPU on a Quad Core DELL server.

Thirdly, Workflow Foundation is used to build operations that
require multiple Data Access Classes to perform together in a
single transaction. Instead of writing large functions with many
if…else conditions, for…loops, it’s better to
write them in a Workflow because you can visually see the flow of
execution and you can reuse Activities among different Workflows.
Best of all, architects can design workflows and developers can
fill-in code inside Activities. So, I could design a complex
operations in a workflow without writing the real code inside
Activities and then ask someone else to implement each Activity. It
is like handing over a design document to developers to implement
each unit module, only that here everything is strongly typed and
verified by compiler. If you strictly follow Single Responsibility
Principle for your Activities, which is a smart way of saying one
Activity does only one and very simple task, you end up with a
highly reusable and maintainable business layer and a very clean
code that’s easily extensible.

Fourthly, Unity
Dependency Injection (DI) framework is used to pave the path for
unit testing and dependency injection. It offers Inversion of
Control (IoC), which enables testing individual classes in
isolation. Moreover, it has a handy feature to control lifetime of
objects. Instead of creating instance of commonly used classes
several times within the same request, you can make instances
thread level, which means only one instance is created per thread
and subsequent calls reuse the same instance. Are these going over
your head? No worries, continue reading, I will explain later
on.

Fifthly, enabling API for Silverlight widgets allows more
interactive widgets to be built using Silverlight. HTML and
Javascripts still have limitations on smooth graphics and
continuous transmission of data from web server. Silverlight solves
all of these problems.

Read the article for details on how all these improvements were
done and how all these hot techs play together in a very useful
open source project for enterprises.

http://www.codeproject.com/KB/ajax/Web20Portal.aspx

Don’t forget to vote for me if you like it.

ensure – Ensure relevant Javascript and HTML are loaded before using them

ensure allows you to load Javascript, HTML and CSS
on-demand, whenever they are needed. It saves you from writing a
gigantic Javascript framework up front so that you can ensure all
functions are available whenever they are needed. It also saves you
from delivering all possible html on your default page (e.g.
default.aspx) hoping that they might some day be needed on some
user action. Delivering Javascript, html fragments, CSS during
initial loading that is not immediately used on first view makes
initial loading slow. Moreover, browser operations get slower as
there are lots of stuff on the browser DOM to deal with. So,
ensure saves you from delivering unnecessary
javascript, html and CSS up front, instead load them on-demand.
Javascripts, html and CSS loaded by ensure remain in
the browser and next time when ensure is called with
the same Javascript, CSS or HTML, it does not reload them and thus
saves from repeated downloads.

Ensure supports jQuery,
Microsoft ASP.NET AJAX and
Prototype framework. This
means you can use it on any html, ASP.NET, PHP, JSP page that uses
any of the above framework.

For example, you can use ensure to download
Javascript on demand:

ensure( { js: "Some.js" }, function()
{
SomeJS(); // The function SomeJS is available in Some.js only
});

The above code ensures Some.js is available before executing the
code. If the SomeJS.js has already been loaded, it executes the
function write away. Otherwise it downloads Some.js, waits until it
is properly loaded and only then it executes the function. Thus it
saves you from deliverying Some.js upfront when you only need it
upon some user action.

Similarly you can wait for some HTML fragment to be available,
say a popup dialog box. There’s no need for you to deliver HTML for
all possible popup boxes that you will ever show to user on your
default web page. You can fetch the HTML whenever you need
them.

ensure( {html: "Popup.html"}, function()
{
// The element "Popup" is available only in Popup.html
document.getElementById("Popup").style.display = "";
});

The above code downloads the html from “Popup.html” and adds it
into the body of the document and then fires the function. So, you
code can safely use the UI element from that html.

You can mix match Javascript, html and CSS altogether in one
ensure call. For example,

ensure( { js: "popup.js", html: "popup.html", css: "popup.css" }, function()
{
PopupManager.show();
});

You can also specify multiple Javascripts, html or CSS files to
ensure all of them are made available before executing the
code:

ensure( { js: ["blockUI.js","popup.js"], html: ["popup.html", "blockUI.html"], css: ["blockUI.css", "popup.css"] }, function()
{
BlockUI.show();
PopupManager.show();
});

You might think you are going to end up writing a lot of
ensure code all over your Javascript code and result
in a larger Javascript file than before. In order to save you
javascript size, you can define shorthands for commonly used
files:

var JQUERY = { js: "jquery.js" };
var POPUP = { js: ["blockUI.js","popup.js"], html: ["popup.html", "blockUI.html"], css: ["blockUI.css", "popup.css"] };
...
...
ensure( JQUERY, POPUP, function() {
("DeleteConfirmPopupDIV").show();
});
...
...
ensure( POPUP, function()
{
("SaveConfirmationDIV").show();
);

While loading html, you can specify a container element where
ensure can inject the loaded HTML. For example, you can say load
HtmlSnippet.html and then inject the content inside a DIV named
“exampleDiv”

ensure( { html: ["popup.html", "blockUI.html"], parent: "exampleDiv"}, function(){});

You can also specify Javascript and CSS that will be loaded
along with the html.

How ensure works

The following CodeProject article explains in detail how ensure
it built. Be prepared for a high dose of Javascript techniques:

http://www.codeproject.com/KB/ajax/ensure.aspx

If you find ensure useful, please vote for me.

Download Code

Download latest source code from CodePlex: www.codeplex.com/ensure

Share this post :

kick it on DotNetKicks.com

UFrame: goodness of UpdatePanel and IFRAME combined

UFrame combines
the goodness of UpdatePanel and IFRAME in a cross browser and
cross platform solution. It allows a DIV to behave like an
IFRAME loading
content from any page either static or dynamic. It can load pages
having both inline and external Javascript and CSS, just like an
IFRAME. But unlike IFRAME, it loads the content within the main
document and you can put any number of UFrame on your page without
slowing down the browser. It supports ASP.NET postback nicely and
you can have DataGrid or any other complex
ASP.NET control within a UFrame. UFrame works perfectly with
ASP.NET MVC making it an replacement for
UpdatePanel. Best
of all, UFrame is
implemented 100% in Javascript making it a cross platform solution.
As a result, you can use UFrame on ASP.NET, PHP,
JSP
or any other platform.

<div class="UFrame" id="UFrame1" src="SomePage.aspx?ID=UFrame1" >
  <p>This should get replaced with content from Somepage.aspxp>
div>

Response from SomePage.aspx is rendered
directly inside the UFrame. Here you see two
UFrame‘s are used
to load the same SomePage.aspx as if they are
loaded inside IFRAME. Another UFrame is used to load
AnotherPage.aspx
that shows photos from Flickr.


image

See it in action!

You can test UFrame from:

What is UFrame?

UFrame can load
and host a page (ASP.NET, PHP or regular html) inside a DIV. Unlike
IFRAME which loads the content inside a browser frame that has no
relation with the main document, UFrame loads the content within
the same document. Thus all the Javascripts, CSS on the main
document flows through the loaded content. It’s just like
UpdatePanel with
IFRAME’s src
attribute.

The above UFrames are declared like
this:

<div id="UFrame1" src="SomePage.aspx" >
    <p>This should get replaced with content from Somepage.aspxp>
div>

The features of UFrame are:

  • You can build regular ASP.NET/PHP/JSP/HTML page and make them
    behave as if they are fully AJAX enabled! Simple regular postback
    will work as if it’s an UpdatePanel, or simple
    hyperlinks will behave as if content is being loaded using
    AJAX.
  • Load any URL inside a DIV. It can be a PHP, ASP.NET, JSP or
    regular HTML page.
  • Just like IFRAME, you can set src property of DIVs and they
    are converted to UFrames when UFrame library loads.
  • Unlike IFRAME, it loads the content within the main document.
    So, main document’s CSS and Javascripts are available to the loaded
    content.
  • It allows you to build parts of a page as multiple fully
    independent pages.
  • Each page is built as standalone page. You can build, test and
    debug each small page independently and put them together on the
    main page using UFrames.
  • It loads and executes both inline and external scripts from
    loaded page. You can also render different scripts during
    UFrame
    postback.
  • All external scripts are loaded before the body content is set.
    And all inline scripts are executed when both external scripts and
    body has been loaded. This way the inline scripts execute when the
    body content is already available.
  • It loads both inline and external CSS.
  • It handles duplicates nicely. It does not load the same
    external Javascript or CSS twice.

Download the code

You can download latest version of UFrame along with the VS 2005
and VS 2008 (MVC) example projects from CodePlex:

www.codeplex.com/uframe

Please go to the “Source Code” tab for the latest version. You
are invited to join the project and improve it or fix bugs.

Read the article about UFrame

I have published an article about UFrame at CodeProject:

http://www.codeproject.com/KB/aspnet/uframe.aspx

The article explains in details how the UFrame is built. Be
prepared for a big dose of Javascript code.

If you find UFrame or the article useful, please vote for me at
CodeProject.


kick it on DotNetKicks.com

Fast ASP.NET web page loading by downloading multiple javascripts in batch

A web page can load a lot faster and feel faster if the
javascripts on the page can be loaded after the visible content has
been loaded and multiple javascripts can be batched into one
download. Browsers download one external script at a time and
sometimes pause rendering while a script is being downloaded and
executed. This makes web pages load and render slow when there are
multiple javascripts on the page. For every javascript reference,
browser stops downloading and processing of any other content on
the page and some browsers (like IE6) pause rendering while it
processes the javascript. This gives a slow loading experience and
the web page kind of gets ‘stuck’ frequently. As a result, a web
page can only load fast when there are small number of external
scripts on the page and the scripts are loaded after the visible
content of the page has loaded.

Here’s an example, when you visit
http://dropthings.omaralzabir.com, you see a lot of Javascripts
downloading. Majority of these are from the ASP.NET AJAX framework
and the ASP.NET AJAX Control Toolkit project.


Andysnap_003

Figure: Many scripts downloaded on a typical ASP.NET AJAX page
having ASP.NET AJAX Control Toolkit

As you see, browser gets stuck for 15 times as it downloads and
processes external scripts. This makes page loading “feel” slower.
The actual loading time is also pretty bad because these 15 http
requests waste 15*100ms = 1500ms on the network latency inside USA.
Outside USA, the latency is even higher. Asia gets about 270ms and
Australia gets about 380ms latency from any server in USA. So,
users outside USA wastes 4 to 6 seconds on network latency where no
data is being downloaded. This is an unacceptable performance for
any website.

You pay for such high number of script downloads only because
you use two extenders from AJAX Control Toolkit and the
UpdatePanel of
ASP.NET AJAX.

If we can batch the multiple individual script calls into one
call like Scripts.ashx as shown in the
picture below and download several scripts together in one shot
using an HTTP Handler, it saves us a lot of http connection which
could be spent doing other valuable work like downloading CSS for
the page to show content properly or downloading images on the page
that is visible to user.


Andysnap_002

Figure: Download several javascripts over one connection and save
call and latency

The Scripts.ashx
handler can not only download multiple scripts in one shot, but
also has a very short URL form. For example:

/scripts.ashx?initial=a,b,c,d,e&/

Compared to conventional ASP.NET ScriptResource URLs like:

/ScriptResource.axd?d=WzuUYZ-Ggi7-B0tkhjPDTmMmgb5FPLmciWEXQLdjNjt
bmek2jgmm3QETspZjKLvHue5em5kVYJGEuf4kofrcKNL9z6AiMhCe3SrJrcBel_c1
&t=633454272919375000

The benefits of downloading multiple Javascript over one http
call are:

  • Saves expensive network roundtrip latency where neither browser
    nor the origin server is doing anything, not even a single byte is
    being transmitted during the latency
  • Create less “pause” moments for the browser. So, browser can
    fluently render the content of the page and thus give user a fast
    loading feel
  • Give browser move time and free http connections to download
    visible artifacts of the page and thus give user a “something’s
    happening” feel
  • When IIS compression is enabled, the total size of individually
    compressed files is greater than multiple files compressed after
    they are combined. This is because each compressed byte stream has
    compression header in order to decompress the content.
  • This reduces the size of the page html as there are only a few
    handful of script tag. So, you can easily saves hundreds of bytes
    from the page html. Especially when ASP.NET AJAX produces gigantic
    WebResource.axd and
    ScriptResource.axd
    URLs that have very large query parameter

The solution is to dynamically parse the response of a page
before it is sent to the browser and find out what script
references are being sent to the browser. I have built an http
module which can parse the generated html of a page and find out
what are the script blocks being sent. It then parses those script
blocks and find the scripts that can be combined. Then it takes out
those individual script tags from the response and adds one script
tag that generates the combined response of multiple script
tags.

For example, the homepage of Dropthings.com produces the
following script tags:

< script type="text/javascript">
...
//]]>

< script src="/Dropthings/WebResource.axd?d=_w65Lg0FVE-htJvl4_zmXw2&t=633403939286875000" 
type="text/javascript"> ... < script src="Widgets/FastFlickrWidget.js" type="text/javascript"> < script src="Widgets/FastRssWidget.js" type="text/javascript"> < script src="/Dropthings/ScriptResource.axd?d=WzuUYZ-Ggi7-B0tkhjPDTmMmgb5FPLmciWEXQLdj
Njtbmek2jgmm3QETspZjKLvHue5em5kVYJGEuf4kofrcKNL9z6AiMhCe3SrJrcBel_c1
&t=633454272919375000"
type="text/javascript"> < script type="text/javascript"> // ... < script src="/Dropthings/ScriptResource.axd?d=WzuUYZ-Ggi7-B0tkhjPDTmMmgb5FPLmciWEXQLdjNjtbmek2j
gmm3QETspZjKLvHIbaYWwsewvr_eclXZRGNKzWlaVj44lDEdg9CT2tyH-Yo9jFoQij_XIWxZNETQkZ90
&t=633454272919375000"
type="text/javascript"> < script type="text/javascript"> ... < script type="text/javascript"> ... < script type="text/javascript" charset="utf-8"> ... < script src="Myframework.js" type="text/javascript"> < script type="text/javascript"> ... < script type="text/javascript">if( typeof Proxy == "undefined" ) Proxy = ProxyAsync; < script type="text/javascript"> ... < script src="/Dropthings/ScriptResource.axd?d=WzuUYZ-Ggi7-B0tkhjPDTmMmgb5FPLmciWEXQLdjN
jtbmek2jgmm3QETspZjKLvH-H5JQeA1OWzBaqnbKRQWwc2hxzZ5M8vtSrMhytbB-Oc1
&t=633454272919375000"
type="text/javascript"> < script src="/Dropthings/ScriptResource.axd?d=BXpG1T2rClCdn7QzWc-HrzQ2ECeqBhG6oiVakhRAk
RY6YSaFJsnzqttheoUJJXE4jMUal_1CAxRvbSZ_4_ikAw2
&t=633454540450468750"
type="text/javascript"> < script src="/Dropthings/ScriptResource.axd?d=BXpG1T2rClCdn7QzWc-HrzQ2ECeqBhG6oiVakhRA
kRYRhsy_ZxsfsH4NaPtFtpdDEJ8oZaV5wKE16ikC-hinpw2
&t=633454540450468750"
type="text/javascript"> < script src="/Dropthings/ScriptResource.axd?d=BXpG1T2rClCdn7QzWc-HrzQ2ECeqBhG6oiVakhRAk
RZbimFWogKpiYN4SVreNyf57osSvFc_f24oloxX4RTFfnfj5QsvJGQanl-pbbMbPf01
&t=633454540450468750"
type="text/javascript">
...

< script type="text/javascript"> ...

As you see, there are lots of large script tags, in total 15 of
them. The solution I will show here will combine the script links
and replace with two script links that download 13 of the
individual scripts. I have left two scripts out that are related to
ASP.NET AJAX Timer extender.


< script type="text/javascript"> ...


< script type="text/javascript" src="Scripts.ashx?initial=a,b,c,d,e,f&/dropthings/">
< script type="text/javascript"> ... < script type="text/javascript"> ... < script type="text/javascript"> ... < script type="text/javascript"> ... < script type="text/javascript">if( typeof Proxy == "undefined" ) Proxy = ProxyAsync; < script type="text/javascript"> ... < script src="/Dropthings/ScriptResource.axd?d=WzuUYZ-..." type="text/javascript"> < script src="/Dropthings/ScriptResource.axd?d=BXpG1T2..." type="text/javascript">
< script type="text/javascript" src="Scripts.ashx?post=C,D,E,F,G,H,I,J&/dropthings/"> < script type="text/javascript"> ...

As you see, 13 of the script links have been combined into two
script links. The URL is also smaller than majority of the script
references.

There are two steps involved here:

  1. Find out all the script tags being emitted
    inside generated response HTML and collect them in a buffer. Move
    them after the visible artifacts in the HTML, especially the

    tag
    that contains the generated output of all ASP.NET controls on the
    page
  2. Parse the buffer and see which script references can be
    combined into one set. The sets are defined in a configuration
    file. Replace the individual script references with the combined
    set reference.

The whole solution is explained in this CodeProject article:

Fast ASP.NET web page loading by downloading multiple
javascripts after visible content and in batch
http://www.codeproject.com/KB/aspnet/fastload.aspx

You should be able to use this approach in any ASP.NET (even
better if AJAX) application and give your site a big performance
boost.

If you like the idea, please vote for me.


kick it on DotNetKicks.com

Fast page loading by moving ASP.NET AJAX scripts after visible content

ASP.NET ScriptManager control has a property
LoadScriptsBeforeUI, when set to false, should
load all AJAX framework scripts after the content of the page. But
it does not effectively push down all scripts
after the content. Some framework scripts, extender scripts and
other scripts registered by Ajax Control Toolkit still load before
the page content loads. The following screen taken from www.dropthings.com shows several
script tags are still added at the beginning of

which
forces them to download first before the page content is loaded and
displayed on the page. Script tags pause rendering on several
browsers especially in IE until the scripts download and execute.
As a result, it gives user a slow loading impression as user stares
at a white screen for some time until the scripts before the
content download and execute completely. If browser could render
the html before it downloads any script, user would see the page
content immediately after visiting the site and not see a white
screen. This will give user an impression that the website is
blazingly fast (just like Google homepage) because user will
ideally see the page content, if it’s not too large, immediately
after hitting the URL.

image
Figure: Script blocks being delivered before the content

From the above screen shot you see there are some scripts from
ASP.NET AJAX framework and some scripts from Ajax Control Toolkit
that are added before the content of the page. Until these scripts
download, browser don’t see anything on the UI and thus you get a
pause in rendering giving user a slow load feeling. Each script to
external URL adds about 200ms avg network roundtrip delay outside
USA while it tries to fetch the script. So, user basically stares
at a white screen for at least 1.5 sec no matter how fast internet
connection he/she has.

These scripts are rendered at the beginning of form tag
because they are registered using
Page.ClientScript.RegisterClientScriptBlock. Inside
Page class of System.Web, there’s a method
BeginFormRender which renders the client script blocks
immediately after the form tag.

   1: internal void BeginFormRender(HtmlTextWriter writer, string formUniqueID)
   2: {
   3:     ...
   4:         this.ClientScript.RenderHiddenFields(writer);
   5:         this.RenderViewStateFields(writer);
   6:         ...
   7:         if (this.ClientSupportsJavaScript)
   8:         {
   9:             ...
  10:             if (this._fRequirePostBackScript)
  11:             {
  12:                 this.RenderPostBackScript(writer, formUniqueID);
  13:             }
  14:             if (this._fRequireWebFormsScript)
  15:             {
  16:                 this.RenderWebFormsScript(writer);
  17:             }
  18:         }
  19:         this.ClientScript.RenderClientScriptBlocks(writer);
  20: }

Figure: Decompiled code from System.Web.Page class

Here you see several script blocks including scripts registered
by calling ClientScript.RegisterClientScriptBlock are
rendered right after form tag starts.

There’s no easy work around to override the
BeginFormRender method and defer rendering of these scripts.
These rendering functions are buried inside System.Web and
none of these are overridable. So, the only solution seems to be
using a Response Filter to capture the html being written and
suppress rendering the script blocks until it’s the end of the
body tag. When the tag is about to be
rendered, we can safely assume page content has been successfully
delivered and now all suppressed script blocks can be rendered at
once.

In ASP.NET 2.0, you to create Response Filter which is an
implementation of a Stream. You can replace default
Response.Filter with your own stream and then ASP.NET will
use your filter to write the final rendered HTML. When
Response.Write is called or Page’s Render method
fires, the response is written to the output stream via the filter.
So, you can intercept every byte that’s going to be sent to the
client (browser) and modify it the way you like. Response Filters
can be used in variety ways to optimize Page output like stripping
off all white spaces or doing some formatting on the generated
content, or manipulating the characters being sent to the browser
and so on.

I have created a Response filter which captures all characters
being sent to the browser. It it finds that script blocks are being
rendered, instead of rendering it to the
Response.OutputStream, it will extract the script blocks out
of the buffer being written and render the rest of the content. It
stores all script blocks, both internal and external, in a string
buffer. When it detects tag is about to be
written to the response, it flushes all the captured script blocks
from the string buffer.

   1: public class ScriptDeferFilter : Stream
   2: {
   3:     Stream responseStream;
   4:     long position;
   5:
   6:     /// 
   7:     /// When this is true, script blocks are suppressed and captured for 
   8:     /// later rendering
   9:     /// 
  10:     bool captureScripts;
  11:
  12:     /// 
  13:     /// Holds all script blocks that are injected by the controls
  14:     /// The script blocks will be moved after the form tag renders
  15:     /// 
  16:     StringBuilder scriptBlocks;
  17:
  18:     Encoding encoding;
  19:
  20:     public ScriptDeferFilter(Stream inputStream, HttpResponse response)
  21:     {
  22:         this.encoding = response.Output.Encoding;
  23:         this.responseStream = response.Filter;
  24:
  25:         this.scriptBlocks = new StringBuilder(5000);
  26:         // When this is on, script blocks are captured and not written to output
  27:         this.captureScripts = true;
  28:     }

Here’s the beginning of the Filter class. When it initializes,
it takes the original Response Filter. Then it overrides the
Write method of the Stream so that it can capture the
buffers being written and do it’s own processing.

   1: public override void Write(byte[] buffer, int offset, int count)
   2: {
   3:     // If we are not capturing script blocks anymore, just redirect to response stream
   4:     if (!this.captureScripts)
   5:     {
   6:         this.responseStream.Write(buffer, offset, count);
   7:         return;
   8:     }
   9:
  10:     /* 
  11:      * Script and HTML can be in one of the following combinations in the specified buffer:          
  12:      * .....< script ....>..........
  13:      * < script ....>..........
  14:      * < script ....>.....
  15:      * < script ....>..... .....
  16:      * ....< script ....>..... 
  17:      * < script ....>..... 
  18:      * ..........
  19:      * .....
  20:      * < script>.....
  21:      * .... 
  22:      * ......
  23:      * Here, "...." means html content between and outside script tags
  24:     */
  25:
  26:     char[] content = this.encoding.GetChars(buffer, offset, count);
  27:
  28:     int scriptTagStart = 0;
  29:     int lastScriptTagEnd = 0;
  30:     bool scriptTagStarted = false;
  31:
  32:     for (int pos = 0; pos < content.Length; pos++)
  33:     {
  34:         // See if tag start
  35:         char c = content[pos];
  36:         if (c == '<')
  37:         {
  38:             int tagStart = pos;
  39:             // Check if it's a tag ending
  40:             if (content[pos+1] == '/')
  41:             {
  42:                 pos+=2; // go past the 
  43:
  44:                 // See if script tag is ending
  45:                 if (isScriptTag(content, pos))
  46:                 {
  47:                     /// Script tag just ended. Get the whole script
  48:                     /// and store in buffer
  49:                     pos = pos + "script>".Length;
  50:                     scriptBlocks.Append(content, scriptTagStart, pos - scriptTagStart);
  51:                     scriptBlocks.Append(Environment.NewLine);
  52:                     lastScriptTagEnd = pos;
  53:
  54:                     scriptTagStarted = false;
  55:                     continue;
  56:                 }
  57:                 else if (isBodyTag(content, pos))
  58:                 {
  59:                     /// body tag has just end. Time for rendering all the script
  60:                     /// blocks we have suppressed so far and stop capturing script blocks
  61:
  62:                     if (this.scriptBlocks.Length > 0)
  63:                     {
  64:                         // Render all pending html output till now
  65:                         this.WriteOutput(content, lastScriptTagEnd, tagStart - lastScriptTagEnd);
  66:
  67:                         // Render the script blocks
  68:                         byte[] scriptBytes = this.encoding.GetBytes(this.scriptBlocks.ToString());
  69:                         this.responseStream.Write(scriptBytes, 0, scriptBytes.Length);
  70:
  71:                         // Stop capturing for script blocks
  72:                         this.captureScripts = false;
  73:
  74:                         // Write from the body tag start to the end of the inut buffer and return
  75:                         // from the function. We are done.
  76:                         this.WriteOutput(content, tagStart, content.Length - tagStart);
  77:                         return;
  78:                     }
  79:                 }
  80:                 else
  81:                 {
  82:                     // some other tag's closing. safely skip one character as smallest
  83:                     // html tag is one character e.g. . just an optimization to save one loop
  84:                     pos++;
  85:                 }
  86:             }
  87:             else
  88:             {
  89:                 if (isScriptTag(content, pos+1))
  90:                 {
  91:                     /// Script tag started. Record the position as we will 
  92:                     /// capture the whole script tag including its content
  93:                     /// and store in an internal buffer.
  94:                     scriptTagStart = pos;
  95:
  96:                     // Write html content since last script tag closing upto this script tag 
  97:                     this.WriteOutput(content, lastScriptTagEnd, scriptTagStart - lastScriptTagEnd);
  98:
  99:                     // Skip the tag start to save some loops
 100:                     pos += "< script".Length;
 101:
 102:                     scriptTagStarted = true;
 103:                 }
 104:                 else
 105:                 {
 106:                     // some other tag started
 107:                     // safely skip 2 character because the smallest tag is one character e.g. 
 108:                     // just an optimization to eliminate one loop 
 109:                     pos++;
 110:                 }
 111:             }
 112:         }
 113:     }
 114:
 115:     // If a script tag is partially sent to buffer, then the remaining content
 116:     // is part of the last script block
 117:     if (scriptTagStarted)
 118:     {
 119:
 120:         this.scriptBlocks.Append(content, scriptTagStart, content.Length - scriptTagStart);
 121:     }
 122:     else
 123:     {
 124:         /// Render the characters since the last script tag ending
 125:         this.WriteOutput(content, lastScriptTagEnd, content.Length - lastScriptTagEnd);
 126:     }
 127: }

There are several situations to consider here. The Write
method is called several times during the Page render process
because the generated HTML can be quite big. So, it will contain
partial HTML. So, it's possible the first Write call contains a
start of a script block, but no ending script tag. The following
Write call may or may not have the ending script block. So, we need
to preserve state to make sure we don't overlook any script block.
Each Write call can have several script block in the buffer
as well. It can also have no script block and only page
content.

The idea here is to go through each character and see if there's
any starting script tag. If there is, remember the start position
of the script tag. If script end tag is found within the buffer,
then extract out the whole script block from the buffer and render
the remaining html. If there's no ending tag found but a script tag
did start within the buffer, then suppress output and capture the
remaining content within the script buffer so that next call to
Write method can grab the remaining script and extract it
out from the output.

There are two other private functions that are basically helper
functions and does not do anything interesting:

   1: private void WriteOutput(char[] content, int pos, int length)
   2: {
   3:     if (length == 0) return;
   4:
   5:     byte[] buffer = this.encoding.GetBytes(content, pos, length);
   6:     this.responseStream.Write(buffer, 0, buffer.Length);
   7: }
   8:
   9: private bool isScriptTag(char[] content, int pos)
  10: {
  11:     if (pos + 5 < content.Length)
  12:         return ((content[pos] == 's' || content[pos] == 'S')
  13:             && (content[pos + 1] == 'c' || content[pos + 1] == 'C')
  14:             && (content[pos + 2] == 'r' || content[pos + 2] == 'R')
  15:             && (content[pos + 3] == 'i' || content[pos + 3] == 'I')
  16:             && (content[pos + 4] == 'p' || content[pos + 4] == 'P')
  17:             && (content[pos + 5] == 't' || content[pos + 5] == 'T'));
  18:     else
  19:         return false;
  20:
  21: }
  22:
  23: private bool isBodyTag(char[] content, int pos)
  24: {
  25:     if (pos + 3 < content.Length)
  26:         return ((content[pos] == 'b' || content[pos] == 'B')
  27:             && (content[pos + 1] == 'o' || content[pos + 1] == 'O')
  28:             && (content[pos + 2] == 'd' || content[pos + 2] == 'D')
  29:             && (content[pos + 3] == 'y' || content[pos + 3] == 'Y'));
  30:     else
  31:         return false;
  32: }

The isScriptTag and isBodyTag functions may look
weird. The reason for such weird code is pure performance. Instead
of doing fancy checks like taking a part of the array out and doing
string comparison, this is the fastest way of doing the check. Best
thing about .NET IL is that it's optimized, if any of the condition
in the && pairs fail, it won't even execute the rest. So,
this is as best as it can get to check for certain
characters.

There are some corner cases that are not handled here. For
example, what if the buffer contains a partial script tag
declaration. For example, ".... and that's it. The
remaining characters did not finish in the buffer instead next
buffer is sent with the remaining characters like "ipt src="..."
>...... In such case, the script tag won't be
taken out. One way to handle this would be to make sure you always
have enough characters left in the buffer to do a complete tag name
check. If not found, store the half finished buffer somewhere and
on next call to Write, combine it with the new buffer sent and do
the processing.

In order to install the Filter, you need to hook it in in the
Global.asax BeginRequest or some other event that's fired before
the Response is generated.

   1: protected void Application_BeginRequest(object sender, EventArgs e)
   2: {
   3:     if (Request.HttpMethod == "GET")
   4:     {
   5:         if (Request.AppRelativeCurrentExecutionFilePath.EndsWith(".aspx"))
   6:         {
   7:             Response.Filter = new ScriptDeferFilter(Response);
   8:         }
   9:     }
  10: }

Here I am hooking the Filter only for GET calls to
.aspx pages. You can hook it to POST calls as well. But
asynchronous postbacks are regular POST and I do not want to
do any change in the generated JSON or html fragment. Another way
is to hook the filter only when ContentType is
text/html.

When this filter is installed, www.dropthings.com defers all
script loading after the

tag completes.

image
Figure: Script tags are moved after the

tag when the
filter is used

You can grab the Filter class from the
App_CodeScriptDeferFilter.cs of the Dropthings project. Go
to CodePlex site
and download the latest code for the latest filter.