Blog Posts – Page 11 – Omar AL Zabir Blog

Safe COM: Managed Disposable Strongly Typed safe wrapper to late bound COM

There are several problems using COM from
.NET:

You cannot implement the Dispose pattern
by utilizing the “using” block in order to safely dispose
COM references.
You cannot guaranty COM references are finalized. There’s no
way to implement “~Destructor()” for COM
references.
COM reference is not released when a call to
Marshal.ReleaseComObject is skipped due to an
Exception.
When you “Add Reference…” to a COM library, the reference is
version specific. So, if you add a reference to the Office 2003 COM
library, it does not work properly when deployed to Office
2000.
The only solution to version independent COM is to use Late
Bound operations but you miss all the features of a strongly typed
language.

Let’s solve all these problems. We want to use a Managed
strongly typed approach to Late Bound COM operations and
also utilize the Dispose pattern on COM objects. The
solution proposed here
works for any COM object which can be Microsoft Office COM
libraries, IE & DHTML objects and even your own COM objects.
You should use this approach whenever you are dealing with any type
of COM library.

If you like this solution, please vote for me.
http://www.codeproject.com/csharp/safecomwrapper.asp

Ajax Portal using VS 2008 and .NET 3.5

I have built an Ajax Web Portal (just like Sharepoint or Google
IG) using Linq to SQL, Linq to XML, Workflow Foundation and ASP.NET
AJAX 3.5. Although it’s no where close to any real Portal, but it
works as a great example to show how web applications can implement
a widget enabled architecture just like Sharepoint. It also
showcases how these hot technologies can play together in a
complete web application.

Full source code is in Visual Studio 2008 using .NET 3.0 and
.NET 3.5.

CodePlex site
http://www.codeplex.com/dropthings

Production site
http://www.dropthings.com

Code Project article that explains the architecture and
development steps
http://www.codeproject.com/Ajax/MakingGoogleIG.asp

Technologies

ASP.NET 3.5
ASP.NET AJAX (.NET 3.5)
Linq to Sql
Linq to Xml
Workflow Foundation (.NET 3.0)
Visual Studio 2008 and SQL Server 2005

Technology Stack

TechnologyStack

What is an AJAX Portal
A portal refers to a page that allows users to customize their own
homepage by dragging and dropping widgets onto the page.
This approach gives users complete control over what content they
see on their Start Page, where they want to see it, and how they
want to interact with it.
A widget is a discrete piece on a Web page that performs a
particular function and comes with its own UI and set of features.
Examples of widgets include a to-do-list, an address book, a
contact list, an RSS feed, or even a clock, calendar, playlist,
stock ticker, weather report, traffic report, dictionary, game, or
almost anything you can imagine that can be packaged up and dropped
on a Web page. In a corporate environment, widgets can connect to
internal systems, such as an Expense Tracker widget that interacts
directly with the internal Accounting System. If you are familiar
with Sharepoint Portal, then you already know about Widgets. They
are called Web parts in Sharepoint’s term and also in ASP.NET
2.0.
Portals are powerful RSS aggregation platform. You can put as many
RSS widgets as you like on your page and get fresh content
delivered to you as soon as it is published.
An Ajax-powered portal is specifically a portal that uses Ajax
technologies to create richer experiences for its users. It is one
step ahead of previous generation portals like My Yahoo or MSN.com,
because it gives you state-of-the-art UI that behaves more like a
Windows client application — with widgets, animations, popups,
client side data grids, and other effects not usually found on a
non-Ajax Web portal.

How to run the project

Install Visual Studio 2008 (VS 2005 won’t work)
Install SQL Server 2005
Download latest code from CodePlex site.
Restore the database from databasedashboard.zip
Update web.config from the web project and set proper
connection string
Build and Run

How is ASP.NET AJAX used in this project?
It is an N-tier application, with a user interface (UI) layer, a
business layer, and a data access layer. I have used ASP.NET AJAX
to implement the UI layer of the portal application which includes
the homepage and the widgets’ UI. ASP.NET AJAX provides the
framework for loading widgets onto the Start page, updating widgets
without doing any postbacks (via UpdatePanel), and changing
page layout by dragging and dropping widgets on the page. It also
provides a rich collection of Control Extenders, that add cool
effects like fade in/fade out, smooth transitions, and client side
animations . You can add to the rich client-side experience by
providing auto-completion behavior on text boxes, asynchronous data
loading via webservice calls, and client-side paging, sorting and
many more.

How is .NET 3.5 used in this project
The business layer of the application is built with the Workflow
Foundation in .NET 3.0 . Major operations like a first-time user
visit, a subsequent user visit, adding a new widget, and creating a
new page are all orchestrated using workflow . The workflows
contain all the business rules and activities needed to complete
each operation. For example, the “New User Visit” workflow creates
the user account, populates the user profile with default values,
creates some default pages, populates them with specific widgets,
etc. Such compound operations are very easy to build with Workflows
, which enables you to break the complete workflow operation into
smaller chunks named Activities. Each Activity does a very small
amount of work. It talks to the data access layer and performs the
task. The data access layer is built with .NET 3.5 , utilizing LINQ
to SQL .
The web project and the widgets make good use of .NET 3.5 by
utilizing lambda expressions , LINQ to SQL, and LINQ to XML. You
will use Linq queries to work with collections and database rows.
Widgets make good use of Linq to Xml in order to consume XML from
external data sources.

NewUserVisitWorkflow

The above figure shows an workflow that creates the default
homepage for a new user.

You are invited to participate in continued development of the
project. If you want to develop widgets, you can download the code,
develop the widget offline and then send me the widget related
files. I will put them up on the production site. If you want to
participate in core development, let me know and I will make you a
developer on the codeplex project. Then you can check out code,
work on it and then check in when you are done.

Warning: Dropthings.com is a very simple, open-source example of
what can be done with AJAX and Microsoft technologies. It is
intended for educational purposes only. Dropthings.com has
absolutely nothing to do with pageflakes.com, and has no code or
technology in common. Trust me, you could not possibly build
a product as advanced, scalable and complex as Pageflakes using the
examples here, and you’d be insane to try 🙂 But this
project does a good job to show you how all these technologies work
together in a working web application.

10 cool web development related articles in 2007

Here’s a list of 10 cool ASP.NET, AJAX and web development
related articles and blog posts that I have written this year that
you might want to take a look:

13
disasters for production website and their solutions
Talks about 13 production disasters that can happen to any website
any time and bring down your business.

Build
Google IG like Ajax Start Page in 7 days using ASP.NET Ajax and
.NET 3.0
This block buster article shows how ASP.NET AJAX, Linq to XML, Linq
to SQL and Workflow Foundation can be used to create a Google IG
like start page in just 7 nights. Learn how to put together such
hot technologies into one project and make a production site out of
it.

Serve extensionless URL from ASP.NET without using ISAPI module or
IIS 6 Wildcard mapping
Currently there are only two ways to service extentionless URL like
www.pageflakes.com/omar that
hits something besides the default document – use a custom ISAPI
module or use IIS 6 wildcard mapping. Both has performance and
scalability problems because both intercepts each and every hit.
Learn how you can solve it by using a custom 404 handler.

Request format is unrecognized for URL unexpectedly ending in
/SomeWebServiceMethod
Since ASP.NET AJAX 1.0 release, Microsoft prevented JSON hijacking
by adding a special content type header. But this caused us some
trouble.

Cleanup inactive anonymous users from ASP.NET Membership
Tables
When you store anonymous user profile using ASP.NET Membership
provider and Anonymous Identification provider, you soon end up
with lots of idle anonymous user data where those users never come
back. We (Pageflakes) went through a lot of difficulty keeping our
database size down as we allow anonymous users to do almost
everything that a registered user can do. This introduces
scalability challenge. See how we solved this problem.

Prevent Denial of Service (DOS) attacks in your web
application
Web applications can be brought down to its knees by hitting the
site repeatedly or by calling expensive webservices randomly.
Anyone can write a simple loop that hits a webserver very
frequently from a high bandwidth connectivity and bring your
production server down. See how to prevent such application level
DOS attacks.

ASP.NET Ajax Extender for multi-column widget drag &
drop
It’s an ASP.NET AJAX extender that allows Pageflakes style drag
& drop functionality between columns and rows.

ASP.NET Ajax in-depth performance analysis
While building an open source start page using ASP.NET AJAX, I have
done a lot of performance analysis on AJAX framework in order to
improve first time load and perceived speed of javascript rich
pages. Check out my analysis.

Think you know how to write UPDATE statement? Think again.
Learn how to optimize common UPDATE statements

Make a surveillance application which captures desktop and then
emails you as attachment
Some time back I needed to capture a certain computers desktop in
order to find out what that user is doing every day. So, I made a
.NET 2.0 Winforms Application which stays on system tray (optional)
and capture the desktop in given time interval (say every 60 secs)
and emailed the captured images to me as message attachment (say
every 30 mins).

Today I received MVP award for the 3rd time on Visual C#. Thanks
to Microsoft for the award and setting up my new blog. I will continue
both my MVPS Blog and this
blog from now on.

Make your website faster using Content Delivery Network

Every request from a browser goes to your server traveling
through the Internet backbones that spans the world. The number of
countries, continents, oceans a request has to go through to reach
your server, the slower it is. For example, if you have your
servers in USA and someone from Australia is browsing your site,
each request is literary crossing the planet from one end to the
other in order to reach your server and then come back again to the
browser. If your site has large number of static files like images,
css, javascript; sending request for each of them and downloading
them across the world takes significant amount of time. If you
could setup a server in Australia and redirect users to your
Australian server, then each request would take fraction of the
time it takes to reach USA. Not only the network latency will be
lower but also the data transfer rate will be faster and thus
static content will download a lot faster. This will give
significant performance improvement on the user’s end if your
website is rich in static content. Moreover, ISPs provide far
greater speed for country wide network compared to the internet
because each country generally has handful of connectivity to the
Internet backbone that are shared by all ISPs within the country.
As a result, users having 4mbps broadband connection will get the
full 4mbps speed from servers that are within the same country. But
they will get as low as 512kbps from servers which are outside the
country. Thus having a server in the same country significantly
improves site download speed and responsiveness.

clip_image002[4]

Above figure shows average response time for www.pageflakes.com from
Washington, DC, where the servers are in Dallas, Texas. The average
response time is around 0.4 seconds. This response includes server
side execution time as well. Generally it takes around 0.3 to 0.35
seconds to execute the page on the server. So, time spent on
network is around 0.05 seconds or 50ms. This is a really fast
connectivity as there are only 4 to 6 hops to reach Dallas from
Washington DC.

clip_image004[4]

This figure shows average response time from Sydney,
Australia. The average response time is 1.5 seconds which is
significantly higher than Washington DC. It’s almost 4 times
compared to USA. There’s almost 1.2 seconds overhead on network
only. Moreover, there are around 17 to 23 hops from Sydney to
Dallas. So, the site downloads at least 4 times slower in Australia
than it is from anywhere in USA.

A content delivery network (CDN) is a system of computers
networked together across the Internet. The computers cooperate
transparently to deliver content (especially large media content)
to end users. CDN nodes (cluster of servers in a specific location)
are deployed in multiple locations, often over multiple backbones.
These nodes cooperate with each other to serve requests for content
by end users. They also transparently move content behind the
scenes to optimize the delivery process. CDN serves request by
intelligently choosing the nearest server. It looks for the fastest
connectivity between your computer to a nearest node that has the
content you are looking for. The number of nodes in different
countries and the number of redundant backbone connectivity a CDN
has measures its strength. Some of the most popular CDNs are
Akamai, Limelight, EdgeCast. Akamai is used by large companies like
Microsoft, Yahoo, AOL. It’s comparatively expensive solution.
However, Akamai has the best performance throughout the world
because they have servers in almost every prominent city in the
world. However, Akamai is very expensive and they only accept a
customer who can spend minimum 5K on CDN per month. For smaller
companies, Edgecast is a more affordable solution.

clip_image006[4]

This figure shows CDN Nodes that is closest to the browser
intercepts traffic and serves response. If it does not have the
response in cache, it fetches it from origin server using a faster
route and much more optimized connectivity that the browser’s
ISP can provide. If the content is already cached, then it’s
served directly from the node

There are generally two types of CDN. One is where you upload
content to CDN’s servers via FTP and you get a subdomain in their
domain like dropthings.somecdn.net. You change all the URL of
static content throughout your site to download content from the
CDN domain instead of relative URL to your own domain. So, a URL
like /logo.gif will be renamed to
http://dropthings.somecdn.net/logo.gif. This is easy to configure
but has maintenance problems. You will have to keep CDN’s
store synchronized with the files all the time. Deployment becomes
complicated because you need to update both your website and the
CDN store at the same time. Example of such a CDN (which is very
cheap) is Cachefly.

A more convenient approach is to store static content on your
own site but use domain aliasing. You can store your content in a
subdomain that points to your own domain like
static.dropthings.com. Then you use CNAME to map that subdomain to
a CDN’s nameserver like cache.somecdn. net. When a
browser tries to resolve static.dropthigns.com, the DNS lookup
request goes to the CDN nameserver. The nameserver then returns IP
of a CDN node which is closest to you and can give you the best
download performance. Browser then sends requests for files to that
CDN node. When CDN node sees the request, it checks whether it has
the content already cached. If it is cached, it delivers the
content directly from its local store. If not, it makes a request
to your server and then looks at the cache header generated in
response. Based on the cache header it decides how long to cache
the response in its own cache. In the meantime, browser does not
wait for CDN node to get content and return to it. CDN does
interesting trick on the internet backbone to actually route
request to the origin server so that the browser gets the response
directly served from origin server while CDN is updating its
cache. Sometimes CDN act as a proxy, intercepting each
request and then fetching uncached content from origin using a
faster route and optimized connectivity to the origin server.

Request format is unrecognized for URL unexpectedly ending in /SomeWebServiceMethod

At Pageflakes, our
webservers; event logs are being flooded with this error. In
ASP.NET AJAX 1.0 version, Microsoft added a check for all web
service calls to have Content-Type: application/json in the
request headers. Unless this request header is present, ASMX
handler fires an exception. This exception is raised directly from
the ScriptHandler which handles all web service calls made
via ASP.NET AJAX.

This is done for security reason. This prevents anyone from
feeding off from your webservices. For example, you might have a
webservice that returns some useful information that others might
be interested to consume. So, any one could just add a
< script> tag pointing to that web service URL and get the
json. If that webservice is a very expensive webservice in terms of
I/O and/or CPU, then other websites feeding off from your
webservice could easily bog down your server.

Now, this back fires when you have HTTP GET supported webservice
calls that produce response headers to cache the response. For
example, you might have a webmethod that returns Stock Quotes. You
have used response caching so that browser caches the response of
that webmethod and repeated visit do not produce repeated calls to
that I/O costly webservice. What will happen is proxy gateways or
proxy servers will see that their client users are making this
request frequently and it can be cached. So, they will make
periodic calls to that webservice and try to precache them on
behalf of their client users. However, during precache, they won’t
send the Content-Type: application/json header. That’s what
I have seen for several proxy servers. As a result, it produces
exception and you get your event log flooded with this
exception.

The reason why MS might have not seen this or anyone else is
because there’s no way to make HTTP GET response cacheable on
browser from webservice calls unless you do the hack
I mentioned in earlier blog post.

However, one thing that puzzles me is we get this error so
frequently and the request headers look so legitimate that I am not
100% sure whether it’s the proxies not producing the content-type
header properly all the time. Somehow the error patterns look like
an registered user is trying to use the site and failing. It is
possible that some firewall might see this content-type as invalid
content type and filter out the header.

Windows Vista – built-in disk partition resize utility

When I tried to install windows vista on my computer, the first
decision I had to take was to partition the hard disk. I started
looking for software on the Internet that can resize existing disk
partition. I found some free software but none of them looked
convincing enough to install and try on my computer. So I just
decided to go on and install Windows Vista anyway. I had two
partitions on my computer. The first partition was the C drive
which had windows XP installed on it and the other partition was D
drive which had all my data. There was no way I can install Windows
Vista on C drive. So, I installed Windows Vista on D drive.
After installing Windows Vista I went to the computer manager by
right cliking on the “computer” icon and selecting and “Manage”.
There you will find the Disk Management option. From there, select
the D drive and choose “Shrink”. It will shrink the parition to all
available space. This will take a long time to complete without any
notification. So, don’t think Vista has hung. Let it finish and you
will find D has been shrunk and there’s a new parition.

Now move all your data from D drive to the new drive, say E
drive.

If is better to do a disk defragmentation on D drive after doing
this.

Note: I found this half written post in my draft box after 7
months. So, this may not be a new information. Just to let you
know, for my regular development, I went back to Windows XP because
there’s significant difference between IIS 6 and IIS 7 and I need
to see how IIS 6 behaves with my applications. Moreover, Visual
Studio 2005 UI does not mix well with Vista. Since I went back,
somehow XP actually feels quite light and faster than Vista.
Finally, Vista burns out laptop battery within 40 mins where XP can
withstand up to 1.5 hours. This can be because of Sony Advance
Power Management drivers not working with Vista.

A significant part of sql server process memory has been paged out. This may result in performance degradation

If you are using SQL Sever Server standard edition 64 bit on a
Windows 2003 64bit, you will frequently encounter this problem
where SQL Server says:

A significant part of sql server process memory has been paged
out. This may result in performance degradation. Duration 0
seconds. Working set (KB) 25432, committed (KB) 11296912, memory
utilization 0%

The number in working set and duration will vary. What happens
here is SQL Server is forced to release memory to operating system
because some other application or OS itself needs to allocate
RAM.

We went through many support articles like:

918483:
How to reduce paging of buffer pool memory in the 64-bit version of
SQL Server 2005
905865:
The sizes of the working sets of all the processes in a console
session may be trimmed when you use Terminal Services to log on to
or log off from a computer that is running Windows Server 2003
920739:
You may experience a decrease in overall system performance when
you are copying files that are larger than approximately 500 MB in
Windows Server 2003 Service Pack 1

But nothing solved the problem. We still have the page out
problem happening every day.

The server has 16 GB RAM where 12 GB is maximum limit allocated
to SQL Server. 4 GB is left to OS and and other application. We
have also turned off antivirus and any large backup job. 12 GB RAM
should be plenty because there’s no other app running on the
dedicated SQL Server box. But the page out still happens. When this
happens, SQL Server becomes very slow. Queries timeout, website
throws error, transactions abort. Sometimes this problems goes on
for 30 to 40 minutes and website becomes slow/unresponsive during
that time.

I have found what causes SQL Server to page out. File System
cache somehow gets really high and forces SQL Server to trim
down.

clip_image002

You see the System cache resident bytes are very high. During
this time SQL Server gets much less RAM than it needs. Queries
timeout at very high rate like 15 per sec. Moreover, there’s high
SQL Lock Timeout/sec (around 15/sec not captured in screen
shot).

clip_image004

SQL Server max memory is configured 12 GB. But here it shows
it’s getting less than 8 GB.

While the file system cache is really high, there’s no
process that’s taking significant RAM.

clip_image006

After I used SysInternal’s
CacheSet to reset file system cache and set around 500 MB as
max limit, memory started to free up.

clip_image008

SQL Server started to see more RAM free:

clip_image010

Then I hit the “Clear” button to clear file system
cache and it came down dramatically.

clip_image012

Paging stopped. System cache was around 175 MB only. SQL Server
lock timeout came back to zero. Everything went back to normal.

So, I believe there’s either some faulty driver or the OS itself
is leaking file system cache in 64bit environment.

What we have done is, we have a dedicated person who goes to
production database servers every hour, runs the CacheSet program
and clicks “Clear” button. This clears the file system cache and
prevents it from growing too high.

There are lots of articles written about this problem. However,
the most informative one I have found is from the SQL Server PSS
team:

http://blogs.msdn.com/psssql/archive/2007/05/31/the-sql-server-working-set-message.aspx

UPDATE – THE FINAL SOLUTION!

The final solution is to run this program on Windows
Startup:

SetSystemFileCacheSize 128 256

This sets the lower and higher limit for the System Cache. You
need to run this on every windows startup because a restart will
undo the cache setting to unlimited.

You can run the program without any parameter to see what is the
current setting.

Download the program from this page:

http://www.uwe-sieber.de/ntcacheset_e.html

Go to the end and you will get the link to the
SetSystemFileCacheSize.zip

13 disasters for production website and their solutions

When we first went live with Pageflakes back in the year 2005,
most of us did not have experience with running a mass consumer
high volume web application on the Internet. In our first year of
operation, we went through all types of difficulties a web
application can face as it grows. Frequent problems with software,
hardware, and network were part of our daily life. We have overcome
a lot of obstacles and established ourselves as one of the top most
Web 2.0 applications in the world. From a thousand user website, we
have grown to a million user website over the years. We have learnt
how to architect a product that can withstand more than 2 million
hits per day and sudden spikes like 7 million hits on a day. We
have discovered under the hood secrets of ASP.NET 2.0 that solves
many scalability and maintainability problems. We have also gained
enough experience in choosing the right hardware and Internet
infrastructure which can make or break a high volume web
application. In this article, you will learn about 13 disasters
than can happen to any production website anytime. These real world
stories will help you prepare yourself well enough so that you do
not go through the same problems as we did. Being prepared for
these disasters upfront will save you a lot of time and money as
well as build credibility with your users.

We have gone through many disasters over the years. Some of them
are:

Hard drive crashed, burned, got corrupted several times
Controller malfunctions and corrupts all disks in the same
controller
RAID malfunction
CPU overheated and burned out
Firewall went down
Remote Desktop stopped working after a patch installation
Remote Desktop max connection exceeded. Cannot login anymore to
servers
Database got corrupted while we were moving the production
database from one server to another over the network
One developer deleted the production database accidentally
while doing routine work
Support crew at hosting service formatted our running
production server instead of a corrupted server that we asked to
format
Windows got corrupted and was not working until we
reinstalled
DNS goes down
Internet backbone goes down in different parts of the
world

This article of mine explains all these disasters and gives you
the solutions:

http://www.codeproject.com/install/13disasters.asp

Please vote for the article if you like it.

Webcast + Podcast + Blog + Widgets = Pagecast

You have seen webcasts and heard Podcasts. Skype introduced
Skypecast. Pageflakes now gives you Pagecast. Using the Page
publishing feature in Pageflakes, you can make your pages public
and share with the world and thus make Pagecast! On your page, you
can put almost anything you want. Put news feeds, photos, audio,
podcasts, movies etc. You can put cool widgets like Calendar,
To-do-list, Blog, Notes. You are no longer limited to providing
video via Webcast or audio via Podcasts. You can deliver rich
interactive content via Pagecast. Pagecast is a lot more powerful
than blogging or vblogging because you can deliver blog content
along with video, audio, and hundreds of rich interactive
components (flakes).