Reduce web site build time

When you have a pretty large web site in Visual Studio 2005, the
site build time grows dramatically and it becomes very difficult to
change code and then hit F5 (or Ctrl+Shift+B) in order to see if
the project builds properly or not. In Pageflakes, we have a giant web
site. It takes so long to build the site that we generally issue
the build command, go to kitchen and make a cup of tea, drink it,
come back and see the site still building. Generally when we do
this 20 to 30 times per day, we start feeling sick (not
because of VS 2005 instead of too much tea).

Here’s a cool idea we tried. You can make
folders hidden by going to Windows Explorer, go to Folder
properties, check mark “Hidden” and click OK and then select “Apply
to current item only”. This will make the folder hidden and the
folder will disappear from Windows Explorer unless you have turned
on the option to show hidden files and folders. You will also see
the folders disappeared from Visual Studio. Now we use Total
Commander instead of Windows Explorer so hidden folders are not a
problem to us. Bascially none of us remember anymore how to use
Windows Explorer because we are so used to Total Commander now. Try it, it will
change your life.

When you make folders hidden, Visual Studio will not build them!
So you can make App_Data, App_Themes, images, stylesheets,
javascripts, html files hidden and Visual Studio will skip them.
Around 70% of our web site is non C# code. So, we gained dramatical
reduction in web site build time by trying this idea. Another idea
is to move all classes from App_Code to external DLL project. Thus
they will build once and VS will not build them again and again if
you make some changes in some codebehind file or some .aspx or
.ascx file. This will also give you significant build time
reduction.

Of course you can turn off Build Website and use Build Page
option. But that does not help. We want to ensure the web site code
is not broken due to any change. So, we need Build Website
option.

ASP.net Ajax under the hood secrets

I have published a new article in codeproject:

ASP.net Ajax under the hood secrets

You will find the following tricks on it:

  • Batch call advantages and disadvantages
  • Implement auto retry in all web method call by modifying Ajax
    core framework
  • Implement queue based web method call in order to prevent
    browser from getting stuck with too many Ajax calls
  • Caching web method response, fix bug in ASP.net 2.0 cache
    header problem
  • How slow is Http Post, capture data transfer over the wire and
    see

Some of the techniques may be known to you if you have read my
earlier posts. But I have converted all code to the latest ASP.net
Ajax Beta 2 version which is not available in the blog posts.

Please rate the article if you like it at codeproject.

Also please digg
this article
.

How to become a good developer overnight

I get a lot of request from a lot of people who see my works and
get inspired and ask me how can they do the same? The questions I
generally get are following:

  • How can I develop projects like you did?
  • What do I need to learn in C# to become like you?
  • Does Microsoft Certifications help? Should I go for the
    exams?
  • What did you do to become MVP? I want to become MVP too.
  • I am 23 (or 24), fresh graduate and I want to become like you. What do I need
    to do?

Generally the questions are like this. Everyone asks me for a
“shortcut” way to becoming a really good developer. So, here’s the
magical secret for becoming a really good developer and achieve
everything I have achieved:

Work 16 hours per day, 7 days a week, 360 days a year for
13 years.

Yes! That’s the secret. It’s pretty easy. The only thing you
need to do is “work” and do nothing else and you will achieve
everything that I have achieved. Pretty easy. I did that, so you
can do it too! Piece of cake.

If you want to go for the “long” way then here’re the things you
can try:

  • Take part in open source projects or make several yourselve.
    This is the best way to learn really useful things.
  • If you can, try setting up your own company. I have setup 3
    companies so far. 2 were not that succesful, 1 is very
    successful. It helped me learn so many things that I would have
    never learnt by working in other’s companies as an employee. The best lessons are learnt from the failures. I learnt a long list of “What not to do in a software development project”, which came very handy when I did my later projects. The more I failed in my earlier projects, the less mistakes I made in my later projects. The key is not to give up even if you fail to deliver 10 projects successfully in a row. Keep an open eye, do a retrospective and find out what made it fail, and especially what *you* did wrong.
  • Read articles everyday. There are thousands of articles to read
    from www.codeproject.com and
    msdn.microsoft.com. I use my Pageflakes page www.pageflakes.com/omar to keep in touch with the technology world everyday. I still read almost all the articles that get
    published in codeproject every week. If you read 10 articles per
    week and do it for a year weeks, you have the knowledge
    of 480 articles! Who can beat you then?
  • Not only read articles, but try out the attached source codes.
    Make similar projects yourselves and use the ideas presented in the
    articles in your own project. I spend everyday at least 1 hour in
    trying out new technologies. This not only increases my knowledge
    but also makes me more experienced in doing things better and helps
    me do my office work better and faster.
  • Get into companies which gives you exciting projects to work on
    and you get to do something in everything. For example, join a
    company which gives you the freedom to design your modules, develop
    it, test it, document it etc. The idea is to gain experience from
    all stages of development. Make sure the company has enough bright
    stars to learn from. If you just become another cow in a big dairy
    farm, no benefit.
  • Don’t leave a company if you are underpaid but you do a lot for
    the company. Have patience. Build yourself up and you will one day
    get what you deserve. I used to get $250/mo in my first
    company, which used to do outsourced projects for a really big
    company in US. I worked day and night in that company and worked in
    8 projects in 7 years. I did not leave the company because of
    the technologies I could learn and apply and the variety of things
    I could do there besides just coding. Best of all, I could work on many outsourced
    projects myself from various countries which exposed me to a wide
    variety of technologies, culture, and business. So, when I left the company and
    joined another one, with the vast experience I had gathered from my
    previous underpaid company, my salary became $200/day in the new
    company. See the difference. If I had left earlier seeking higher
    salary instead of technologies, I would not have learned all the
    cool things and I would not become so expensive as I am
    now.
  • This is very important for those who cost $200/day now. Don’t
    change yourself once you start earning this much. Be the same
    person as you were when you used to get $200/month. Remember, it
    was the attitude and the burning desire to learn and grow that
    made you become what you are now. If the burning sun inside you
    becomes a dying candle, you lose.
  • Don’t start your career in a company where you are given nice
    specs to read on, you have lead developers to decide all
    input/output/pseudocodes, you have a solid QA team to test
    your work, you have managers and administrators to take care of
    every management and administrative issues etc. In such a company,
    the only thing you become is a “smart typist”. You basically
    translate English to C#. The right side of your brain does not
    develop at all. Start your career with small companies which deal
    with lots of challenges and you get the chance to burn your brain
    and fingers out. The sweet smell of your roasted brain is far
    better than the sweet scent of your polished cubicle inside a
    decorated corporate office.
  • I have seen the following evolutionary cycle of developers and
    see where you fit in:
    • Beginner: Does not wear shoes, comes to office on sandles.
      Looks very sober. Shirt is outside pant. When you ask them,
      do you know .NET events and delegates? They say, “uh, ummm, no I
      don’t think so. Is it birthday events?”
    • Beginner+: Gives you “I know it all” look whenever you talk
      about programming. Wears shiny shoes, full sleve shirt is
      nicely put inside dockers pants. Back brushed hair wtih
      “Set Wet” gel and always on 300 sunglasses. When you ask them, “do
      you know .NET events and delegates?” They say, “Events and
      delegates are coooool man! You can do anything with them and mark
      my word man, “anything”. I haved used them in sooooo many
      projects. Did you just learn about .NET events and
      delegates?”
    • Intermediate: Clothing turns a bit pale. Sunglass is
      old-school. No hair gel. Anytime you speak about some terms like
      EJB, Spring, Design Patterns, their eyes sparkle like the brightest
      star in the November sky. They start doing a lot of off-the-record
      work inside office. They start going to online groups, start
      working with friends on open source projects, start reading MSDN
      Magazines etc. If you ask them, “Can you make it?” They always
      reply, “Sure, you will get it tomorrow.” But usually you get it
      after a month.
    • Intermediate+: Generally you get it within 1 or 2 weeks overdue
      schedule.
    • Advanced: They wear the same “I am a Geek”
      or “Microsoft Windows XP” logo T-shirts everyday (until it
      stinks and you can smell it as soon as they enter the
      office) and shiny sports shoes. They start talking about
      software development processes, RUP, Extreme Programming,
      Agile Development etc. If you ask them to do something, they
      reply “Give me a functional specification, a technical
      specification, test plans, milestones, release plans, mockups and N
      number of developers and I will get it done.”
    • Very advanced: Does not wear shoes, comes to office on sandles.
      Looks very sober. Shirt is outside pant. When you ask them,
      do you know .NET events and delegates? They say, “They seem to
      suffer from bi-directional strong reference problem which
      prevents garbage collectors from collecting the listener properly
      and the only way to release the reference is to bring down the app
      domain”
  • Do take Microsoft Certifications without cheating. You will
    learn a lot.
  • Write articles & blogs. Share everything you learn. Someone
    out there will benefit from it someday. Don’t hesitate thinking
    that you don’t know much to write about.

MVP Award

I have been awarded MVP Award 2007 on Visual C# again. This is
my second year. This time the gifts are realy nice. This program is
getting better and better every year. The level of support and
appreciation MVPs receive from Microsoft by this award is
phenomenal. Free MSDN Universal Susbcription, Visual Studio Team
System, TechEd 2006 DVDs, woohoo!

You might be thinking, you did nothing for the community! How
come you get this award?

Here’s what I did:

Ajax Series

ASP.NET 2.0 Series

You say, “Huh, big deal, I can write such simple artlcles
everyday”. Then read this:


StickOut – .NET 2.0, VSTS, Outlook Addin, MS Word/Excel
integration

My university life

When I was in univerisity, from day 1 at my first semester, I
looked for oppotunities not to attend classes or do homeworks
instead go and work at my office more. Attending classes, taking
class tests, collecting lecture notes, paying dues in long queues,
doing course registration all seemed like a waste of my valuable
time. I could do so many things if I were not doing those things.
So, I was thinking about automating all these so that I can do them
over the web from my office or home and thus save going to
university. This resulted in a state-of-the-art web based, smart
client powered univerisity automation and collaboration system
which now runs at my univeristy web site www.aiub.edu.

It created a lot of buzz in the country because it was the first
ever complete implementation of such a system not only in
Bangladesh but also in many other countries. You will hardly find
such feature rich automated system in Universites in US, Europe or
Australia. We got so many proposals from famous universities in US
who wanted to buy this project. This project was also
praised in many daily newspapers. There’s a recent review published
in a daily which suddenly brought back old memories and made it to
this post:

http://www.thedailystar.net/campus/2006/09/04/acad…

It was also the first ever .NET project completed in my
country.

Here’s what a student sees when logged in:

Here’s details of a course:

Atlas 2: HTTP POST is slower and it’s default in Atlas

Atlas by default makes HTTP POST for all AJAX calls. Http POST
is more expensive than Http GET. It transmits more bytes over the
wire, thus taking precious network time and it also makes ASP.NET
do extra processing on the server end. So, you should use Http Get
as much as possible. However, Http Get does not allow you to pass
objects as parameters. You can pass numeric, string and date only.
When you make a Http Get call, Atlas builds an encoded url and
makes a hit to that url. So, you must not pass too much content
which makes the url become larger than 2048 chars. As far as I
know, that’s what is the max length of any url.

Another evil thing about http post is, it’s actually 2 calls.
First browser sends the http post headers and server replies with
“HTTP 100 Continue”. When browser receives this, it sends the
actual body. Here’s what the headers look like:

Request:

POST / Atlas1 / Default . aspx
HTTP
/ 1.1 __serviceMethodName = Timeout&__serviceMethodParams
= {}&__VIEWSTATE =/ wEPDwUJMTY3NTY1MjM2ZGSlWDXIdYj44hhTUd0z8yyp1q
%2 bUtw %3
d %3 d Response:
HTTP
/ 1.1 100
Continue Server: ASP
. NET Development Server / 8.0
. 0.0 Date :
Mon
, 11 Sep
2006 15 :
04 : 13
GMT Content-Length:
0

After getting this clearance from the server, the actual body is
sent. This is done in order to prevent long posts which are not
going to succeed anyway if the server cannot accept it. But it
comes at the cost of network latency.

So, Http Get should be at least twice faster than Http Post.

Here’s how you make Http GET calls in Atlas:

Step 1: Decorate web method with attribute

First you need to put a special attribute on the web method:

[WebMethod] [WebOperation(
true )] public string HelloWorld() { return Hello
World
; }

Step 2: Set useGetMethod=true while making the call

Here’s how you call the web method using Http Get:

WebService.HelloWorld( {
useGetMethod:
true
, onMethodComplete:
function (result) { debug.dump(result); } } );

Note: It needs to be a web service (.asmx). It will not work if
you use it on page methods.

Please see this page
in Atlas Quickstart
for further info.

Atlas 1: Try not to use page methods

One of the easiest thing in Atlas is to use the
Page Method
feature. If you use Atlas on your web page say
Default.aspx, you can directly call public methods on Default.aspx
from javascript. Just put a [WebMethod] attribute on a public
method in Default.aspx and then you can them from Javascript using
PageMethods.MethodName().

public partial class _Default
: System.Web.UI.Page {
protected void Page_Load( object sender,
EventArgs e) { } [WebMethod]
public string DoSomething( string param)
{
return param; } }

On the client side you can call this method like this:

PageMethods.DoSomething(
Hi
, function (result) { alert(result); } );

Here’s the catch, Page method calls are always HTTP POST calls.
You can never make HTTP GET call to the Page methods but you can
make HTTP GET calls to Web service method. So, on later stage of
your project when you will need Http response caching in order to
save roundtrips, you will have to refactor all page methods to web
service methods and for this you will have to move all public
methods and related code from default.aspx to some web service. So,
try not to use Page methods from Day 1.

Atlas 7: Caching web service response on browser and save bandwidth significantly

Browser can cache images, javascripts, css files on users hard
drive and it can also cache Xml Http calls if the call is a Http
Get. The cache is based on Url. If it’s the same Url and it’s
cached on the computer then the response is loaded from cache, not
from the server when it is requested again. Basically, browser can
cache any Http Get call and return cached data based on Url. If you
make a Xml Http call as Http GET and server returns some special
header which informs the browser to cache the response; on future
calls, the response will be immediately returned from the cache and
thus save the delay of network roundtrip and download time.

At Pageflakes, we cache
user’s state so that when user visits again the following day, user
gets a cached page which loads instantly from browser cache, not
from the server. Thus second time load becomes very fast. We also
cache several small parts of the page which appears on users
action. When user does the same action again, a cached result is
loaded immediately from local cache and thus saves the network
roundtrip time. User gets a fast loading site and a very responsive
site. The perceived speed increases dramatically.

The idea is to make Http Get calls while making Atlas web
service calls and return some specific Http Response headers which
tells the browser to cache the response for some specific duration.
If you return “Expires” header during the response, browser will
cache the Xml Http response. There are 2 headers that you need to
return with response which will instruct browser to cache the
response:

HTTP/ 1.1 200
OK Expires: Fri , 1
Jan 2030 Cache-Control: public

This will instruct browser to cache the response till Jan 2030.
As long as you make the same Xml Http call with the same
parameters, you will get cached response from the computer and no
call will go to the server. There are more advanced ways to get
further control over response caching. For example, here is a
header which will instruct browser to cache for 60 seconds but do
contact server and get a fresh response after 60 seconds. It will
also prevent proxies from returning cached response when browser
local cache expires after 60 seconds.

HTTP/ 1.1 200
OK Cache-Control: private
, must-revalidate , proxy-revalidate , max-age = 60

Let’s try to produce such response headers from ASP.NET web
service call:

This will result in the following response headers:

Expires header is set properly. But the problem is with
Cache-Control. It is showing “max-age” is set to zero which will
prevent browser from doing any kind of caching. If you seriously
want to prevent caching, you should emit such cache-control header.
Looks like exactly the opposite thing happened.

There’s a bug in ASP.NET 2.0 that you cannot change “max-age”
header. As max-age is set to zero, ASP.NET 2.0 sets Cache-Control
to private because max-age = 0 means no cache needed. So, there’s
no way you can make ASP.NET 2.0 return proper headers which caches
the response.

Time for a hack. After decompiling the code of HttpCachePolicy
class (Context.Response.Cache object’s class), I found the
following code:

Somehow, this._maxAge is getting set to zero and the check: “if
(!this._isMaxAgeSet || (delta < this._maxAge))" is preventing it from getting set to a bigger value. Due to this problem, we need to bypass the SetMaxAge function and set the value of the _maxAge field directly using Reflection.

This will return the following headers:

Now max-age is set to 60 and thus browser will cache the
response for 60 seconds. If you make the same call again within 60
seconds, it will return the same response. Here’s a test output
which shows the date time returned from the server:

The client side code is like this:

function cachedHttpGet() { WebService.CachedGet( {
useGetMethod:
true
, onMethodComplete:
function (result) { debug.dump(result); } } );
}

Here you see, the response is cached for 60 seconds and after
the time elapsed, there was a server call made and new date was
returned. That response was again cached for 60 seconds.

Atlas 6: When ‘this’ is not really ‘this’

Atlas callbacks are not executed on the same context where they
are called. For ex, if you are making a Page method call from a
javascript class like this:

function SampleClass() { this .id =
1 ; this
.call = function () {
PageMethods.DoSomething(
Hi
, function (result) { debug.dump( this .id ); }
); } }

What happens when you call the “call” method? Do you get “1” on
the debug console? No, you get “null” on the debug console because
“this” is no longer the instance of the class. This is a common
mistake everyone makes. As this is not yet documented in Atlas
documentations, I have seen many developers spend time finding out
what’s wrong.

Here’s the reason. We know whenever Javascript events are raised
“this” refers to the html element which produced the event. So, if
you do this:

function SampleClass() { this .id =
1 ; this
.call = function () {
PageMethods.DoSomething(
Hi
, function (result) { debug.dump( this .id ); }
); } }

If you click the button, you see “ButtonID” instead of “1”. The
reason is that, the button is making the call. So, the call is made
within button object’s context and thus “this” maps to the button
object.

Similarly, when Xml Http raises the event onreadystatechanged
which Atlas traps and fires the callback, the code execution is
still on the Xml Http’s context. It’s Xml Http object which raises
the event. As a result, “this” refer to the Xml Http object, not to
your own class where the callback is declared.

In order to make the callback fire on the context of the
instance of the class so that “this” refers to the instance of the
class, you need to make the following change:

function SampleClass() { this .id =
1 ; this
.call = function () {
PageMethods.DoSomething(
Hi
, Function.createDelegate( this ,
function (result) { debug.dump( this .id ); }
) ); } }

Here, the Function.createDelegate is used to create a delegate
which calls the given function under the “this” context.
Function.createDelegate is defined in AtlasRuntime:

Function.createDelegate
= function (instance, method) { return function ()
{
return method.apply(instance, arguments); }
}