Real-time Oracle Database Monitoring Dashboard in ASP.NET

Oracle Performance Dashboard (OPD) is a small ASP.NET website that shows you performance & problems of one or more Oracle instances in near real-time. It uses the Dynamic Performance Views (DPV) and runs some popular DBA scripts in order to get meaningful, easy to understand information out of the server. You can use it to quickly spot blocking queries, who is blocking who, expensive query that are consuming high CPU or disk, see if there’s unusual locks, very high disk activity and so on.

Dashboard - Full

Demo

You can see a live demo of this from here:
http://odp.omaralzabir.com

Get the code

The binaries are here, which you can just extract into an IIS folder, put the connection strings in the web.config file, and you are ready to roll. No need to install any Oracle client software on the server.
GitHub Project Binaries

You can get the source code from the GitHub project site:
https://github.com/oazabir/OraclePerformanceDashboard

Feature walkthrough

OPD comes with the following features in V1:

  • Summary of all your instances in one screen, showing important indicators on each instance. Quick way to check if all your databases are doing fine or not.
  • Instance Dashboard showing details of an instance:
    • CPU usage on the OS.
    • CPU consumed by each session
    • Important System Statistics like Buffer Gets, Parse to execute ratio which would indicate some common problems on the server.
    • Sessions and what the sessions are doing, how much resource they are consuming, whether they are hogging the disk or not.
    • Waits, Blocks, Locks, deadlocks that make database suffer.
    • Historical analysis on the databse showing you some very useful stuff:
      • Most expensive queries in terms of CPU and Buffer Get, which are immediate convern for your devs to sort out.
      • IO usage on data files. You can see if some data file is unusually loaded and getting hammered by physical IO.
      • Tablespace usage. Alerts you if some tablespace is over 85% full.
      • Stale stats on tables. You should always keep this clean.
      • Killer Indexes that will actually blow up your system and confuse Oracle query optimizer. You need to get rid of those indexes and rewrite queries that you thought will hit those indexes for better performance. They won’t. They will kill your database.

7 hard earned lessons from a Single Page Application

My talk at London AJAX User group on some performance optimization techniques that can give a javascript rich Single Page Application big boost in terms of page load performance.

MyPicAJAXTalk

See the lecture here:

http://skillsmatter.com/podcast/ajax-ria/7-real-life-lessons-learnt-from-a-single-page-application/mh-6922

Here are the slides.

image

Using custom font without slowing down page load

Custom fonts are widely used nowadays to give your websites trendy look, but at the great expense of page loading speed. Each custom font has to be downloaded, initialized and applied throughout the page before browser can render anything. If you have a custom font in any CSS, anywhere on the page, most browsers (like IE) will not render anything (remain white) until the custom font is downloaded, initialized and applied. Usually custom fonts are over 300KB. As a result, they significantly slow down page load speed. See this timeline of a website that uses custom font:

image

The gray bar is the .eot font. The green vertical bar is when browser renders the page. Until that green bar, user is staring at a white page with absolutely nothing for 5.5 seconds. The .eot is the very last thing browser downloads before it can render anything on the page. Usually 5.5 second page load time is unacceptable for most websites because on an average 40% users abandon your site if it does not load in 3 seconds!

image

The above graph shows you that nearly 80% change in bounce rate for pages that have over 4 second page load time!

It is absolutely key that you maintain less than 4 sec home page load time. Browser cannot remain blank for more than 4 sec. Ideally the page should be fully loaded within 4 sec if you want to encourage users to come back to your site.

Let’s learn a trick that you can use to download custom fonts behind the scene, without preventing browser from rendering anything and then apply the font as soon as the font is downloaded. Once you apply this trick, browser will render the page before the custom font is downloaded:

image

As you see here, browser has rendered the page on 4th second before it has downloaded the custom font file. The font takes a long time to downloading. During this whole time, browser is free, content is rendered, user can scroll around. Once the font is downloaded, it is applied. It freezes from 8.5 sec till 9.5 second while it initializes the font and applies on the page.

Step 1: Create a CSS that loads the custom font

First create a css that will load the custom font and apply to the html elements. First create a IE version of the CSS:

@font-face{
  font-family:'Nikosh';
  src: url('nikosh2.eot'); /* here you go, IE */
}

p, li, input, select, div, span, em, i, b, u, strong {
    font-family: 'Nikosh', Arial, Helvetica, sans-serif;
}

IE only supports Embedded True Type font (.eot). It does not support TTF, OTF, WOFF.

Other browsers do not support .eot. So, for them, you need to use some other open format. Say you are using OTF:

@font-face{
  font-family:'Nikosh';
  src: url('nikosh2.otf') format('opentype');
}

p, li, input, select, div, span, em, i, b, u, strong {
    font-family: 'Nikosh', Arial, Helvetica, sans-serif;
}

This one IE won’t understand. So, you need two different CSS files. I understand there’s this bullet-proof font face hack which is claimed to work in all browsers. But I have tried it and it does not work in Chrome. Chrome does not understand the url(://) hack.

Step 2: Javascript to pre-cache the font file and download the CSS after font is loaded

Next step is to pre-cache the font file behind the scene so that browser is not frozen while the large custom font file downloads. You can use jQuery to download the font file.

var fontFile = jQuery.browser.msie ? '/wp-content/uploads/font/nikosh2.eot' 
    : '/wp-content/uploads/font/nikosh2.otf';
var cssFile = jQuery.browser.msie ? '/wp-content/uploads/font/nikosh2ie.css' 
    : '/wp-content/uploads/font/nikosh2nonie.css';

jQuery(document).ready(function(){
	jQuery.ajax({
	  url: fontFile,
	  beforeSend: function ( xhr ) {
		xhr.overrideMimeType("application/octet-stream");
	  }
	}).done(function (data) {
		jQuery("	")
		  .appendTo(jQuery('head'))
		  .attr({type : 'text/css', rel : 'stylesheet'})
		  .attr('href', cssFile);
	});
});

First it pre-caches the right font file based on browser version. Make sure you define the correct font file name here as you have defined in the browser specific css file. Once the file is downloaded, jQuery done will fire and then it will inject a link that tag will fetch the css. The css will refer the exact same font file, from the exact same location. So, browser will not download it again as it has just downloaded the file and cached in local cache. Thus the css will be applied immediately.

Remember, as soon as the <link> tag is added, browser will freeze and become white until the font is initialized and applied. So, you will some freezing here. As it is freezing after the page has loaded completely, user is at least looking at the page and thinks that the page has finished loading. So, the user experience is better.

Step 3: Enable static content expiration

For this to work, you must have static content expiration in the webserver. If you don’t have it then browser will not cache the font file and it will download the file again as soon as you add the <link> tag. In order to enable static content expiration, follow these steps:

  • Click on the website in IIS.
  • Click on “HTT Response Headers” icon.
  • Click on “Set Common headers” from the right side menu.
  • Turn on “Expire Web Content” and set a far future date.

image

This will turn on browser caching for all static files, not just font file.

Step 4: Set MIME type for fonts

IIS 7 or 8 does not come with MIME settings for .otf, .ttf etc font extensions. Only .eot is there. So, you need to set the MIME type for these extensions, otherwise it won’t deliver the font and browser will get a 404.3.

  • Click on the website on IIS.
  • Select “MIME Types” icons.
  • Click Add…
  • Add the .otf and .ttf extensions.
  • Set Content Type as: application/octet-stream

That should now enable custom fonts on your webserver.

Caching WCF javascript proxy on browser

When you use WCF services from Javascript, you have to generate the Javascript proxies by hitting the Service.svc/js. If you have five WCF services, then it means five javascripts to download. As browsers download javascripts synchronously, one after another, it adds latency to page load and slows down page rendering performance. Moreover, the same WCF service proxy is downloaded from every page, because the generated javascript file is not cached on browser. Here is a solution that will ensure the generated Javascript proxies are cached on browser and when there is a hit on the service, it will respond with HTTP 304 if the Service.svc file has not changed.

Here’s a Fiddler trace of a page that uses two WCF services.

image

You can see there are two /js hits and they are sequential. Every visit to the same page, even with the same browser session results in making those two hits to /js. Second time when the same page is browsed:

image

You can see everything else is cached, except the WCF javascript proxies. They are never cached because the WCF javascript proxy generator does not produce the necessary caching headers to cache the files on browser.

Here’s an HttpModule for IIS and IIS Express which will intercept calls to WCF service proxy. It first checks if the service is changed since the cached version on the browser. If it has not changed then it will return HTTP 304 and not go through the service proxy generation process. Thus it saves some CPU on server. But if the request is for the first time and there’s no cached copy on browser, it will deliver the proxy and also emit the proper cache headers to cache the response on browser.

http://www.codeproject.com/Articles/360437/Caching-WCF-javascript-proxy-on-browser

Don’t forget to vote.

Memory Stream Multiplexer–write and read from many threads simultaneously

Here’s an implementation of MemoryStream like buffer manager where one thread can write and many threads can read simultaneously. Each reading thread gets its own reader and can read from the shared stream on its own without blocking write operation or other parallel read operations. It supports blocking Read call so that reader threads can call Read(…) and wait until some data is available, exactly the same way you would expect a Stream to behave. You can use this to read content from network or file in one thread and then get it read by one or more threads simultaneously. Readers do not block writing. As a result, both read and write happens concurrently. Handy for building http proxy where you are downloading a certain file and you have multiple clients asking for the same file at the same time. You can download it in one thread and let one or more client threads read from the same buffer exactly at the same time. You can also use this to read same file on disk from multiple clients at the same time. You can also use this to implement a server side cache where the same buffer is read by multiple clients at the same time.

image

See the detail implementation here:

Memory Stream Multiplexer–write and read from many threads simultaneously

Don’t forget to vote.

Scaling ASP.NET websites from thousands to millions–LIDNUG

Here’s the recent presentation made on LIDNUG on scaling ASP.NET websites from thousands to millions of users. The action starts at 0:02:05.

Scaling ASP.NET websites from thousands to millions of users by Omar AL Zabir

Here’re the slides.

Browse internet faster and save power using a smart HOSTS file

Internet is full of flash ads nowadays that make page load slower, render slower and consumes more CPU, thus power. If you can browse without having any flash ads or in fact any ads loaded and without any of the tracking scripts – you can browse much faster, scroll through pages much smoother and have more hours from your battery. Nowadays, most websites use scripts from various analytics sites that track your browsing habit, use IFRAME to load tracking and social networking widgets. All of these add considerable delay to page loading and make browser consume more CPU and bandwidth. If you can turn all of them off, browsing internet feels a lot smoother, faster and you get more work hours while running on battery.

Moreover, you don’t get distracted by the flashy ads and save your children and young family members from looking at foul things.

If we could get 10% of the total internet users (2bn as of Jan 2011) to save 10% CPU, power and bandwidth while browsing everyday, we could save mega watts of power everyday throughout the world!

Using this solution, you can prevent ads and tracking scripts, prevent malicious and porn websites.

How bad is it?

Let’s take an example on a popular website. The red boxes are Flash Ads (read power suckers).

image

Once we disable all ads and tracking scripts, here’s how it looks:

image

Statistics:

  Before After
Total Requests 111 100
Total Download Size 1.2 MB 0.98 MB
Page load time 4.34s 3.64

 

Not just during page loading, while you are on the page, doing nothing, just reading, browser continuously consumes CPU.

Before:

image

After:

image

Before disabling the ads and tracking scripts, CPU is always around 20-25%. After disabling it is around 8-10%. The more CPU works, the more power it consumes. If you are running on battery, you can get at least 20% more time from your battery. If you have many tabs open all the time, you can save more.

Here’s how to save CPU, bandwidth and power

Go to this website and download the HOSTS file:

http://winhelp2002.mvps.org/hosts.htm

Follow the instruction to put the HOSTS file in your C:WindowsSystem32Driversetc folder.

Now go to Start Menu, type Notepad but do not hit enter. Right click on Notepad and select Run As Administrator.

image

Go to File menu and click Open.

Copy and paste this into the File Name and click OK.

c:windowssystem32driversetcHOSTS

image

Now go to Edit menu and select Replace. Put 127.0.0.1 in Find box and put 255.255.255.0 in Replace box. Click Replace All.

image

Once done, you need to type back 127.0.0.1 for the first entry localhost.

image

Remember, localhost cannot be 255.255.255.0.

When you have done this correctly, it will look like this.

image

Save the file and exit Notepad.

Then go to Start menu and type: services.msc

From the service list, double click on “DNS Client”.

image

First click “Stop” to stop the service.

Then from the Startup Type dropdown, select Disabled.

Click OK.

image

Close all your browsers and reopen them. I highly recommend restarting your PC.

You are ready to browse faster, smarter, cheaper! 

I also highly recommend everyone to use OpenDNS. You can save yourself from getting into malicious sites and being ripped off your bank balance, property, spouse and children. Just go to www.opendns.com and follow the instruction. It is the best thing happened on the internet after Wikipedia!

How does the HOSTS file trick work?

Here’s how internet works. You type www.something.com and it goes and finds out what is the IP address for this domain. First Windows checks a file called HOSTS. If it is not defined there, it will ask the DNS Server configured for your network to give it the IP for the domain so that it can connect to the webserver. If you put fake IP in HOSTS file, Windows will hand over fake IP to the browser and browser will connect to the fake IP. Thus by putting an invalid IP, we prevent browser or any application running on your PC from reaching the ads, tracker, malicious and porn websites.

Don’t forget to share this with your friends and families!

MVP Open Day 2011 at Cambridge

Microsoft Research arranged MVP Open Day 2011 at Cambridge on Oct 14, 2011. Beautiful university, made me feel like giving up my job and going back to study. Amazing research work going there, highly thought provoking. The session on DNA programming was out of the world. The most surprising thing I learnt that a 10cm long DNA strand can hold 10TB worth digitally encoded data and cells are thousand times more robust computing system than silicon based chips. Moreover, cells are self-powered, super energy efficient micro processors, hundred years ahead of Intel processors.

Can’t wait for the day when we will be able to use C# to program DNA:

protected void CancerCell_Found(object body, CellEventArgs e) { this.Attack(e.TargetCell); }

Here’s my presentation slide. Nothing NDA or DNA in this, feel free to distribute.