Tag Archive for tools

Build, deploy, anonymize config, zip package, git commit, push from a single command

gitautomate

gitautomateWhile working on open source projects, you have to frequently build your code, clean up all temporary files, remove your own passwords, connections strings from web.config, then create a binary deployment package in a zip format and then commit and git push to GitHub. Let’s automate all these using a configurable powershell script.

Here’s the full script.

First step, let’s define the parameters for the script with some default values:

param (
    [string]$solution = "OracleDashboard.sln",
    [string]$zipname = "OracleDashboard.zip",
    [string]$compressor = "c:\Program Files\7-Zip\7z.exe",
    [string]$folder = "OracleDashboard",
    [string]$deployPath = "..\Binary",
    [string]$commitFrom = "..",
    [Parameter(Mandatory=$true)][string]$comment
)

Some description of these parameters:

  • $solution = path of the solution file relative to the script location
  • $zipname = name of the zip file.
  • $compressor = 7-zip’s 7z.exe file path.
  • $folder = the folder that contains the code, which is zipped.
  • $deployPath = relative path where the zip file will be moved to.
  • $commitFrom = relative path where the script will run git commit and git push from.
  • $comment = A comment for the git commit.

The first thing the script does is look for the solution open in Visual Studio and close it. You can comment this section out if you want. But if Visual Studio is open, then /obj folder cannot be deleted.

# If visual studio has the solution open, close VS, as we can't delete obj folder while it is open
$windowTitle = $solution.Replace(".sln", "")
$vsProcess = get-process | where {$_.mainwindowtitle -match $windowTitle -and $_.ProcessName -eq "devenv"} 
if ($vsProcess.Length -gt 0) {
    Write-Host "Visual Studio has this solution open. Closing..."
    $vsProcess | ForEach-Object { $_.CloseMainWindow(); }
    Sleep 5
    Read-Host "Press ENTER to proceed if Visual Studio is closed"
    $vsProcess = get-process | where {$_.mainwindowtitle -match $windowTitle -and $_.ProcessName -eq "devenv"} 
    if ($vsProcess.Length -gt 0) {
        Write-Host "Visual Studio still has the solution open. Aborting."
        Return
    }
}

Next step is to do some spring cleaning:

Push-Location

if (Test-Path $zipname) { rm $zipname; }

# Clean up deploy folder 
rm $deployPath\*.* -Force -Recurse

First remember the current path. We have to come back to this path after we are done. Then remove the zip file if it already exists. Then cleanup the $deployPath. You can remove this if you want to keep old deployment packages. Then you have to handle generation of unique file names for the packages.

Now, let’s build and remove /obj folder:

# Build new version
msbuild /verbosity:minimal $solution

# Delete obj
if (Test-Path $folder\obj) { rm $folder\obj -Force -Recurse }

Next step: remove all sensitive information from the web.config, which includes connection strings, authorization block, appSettings entries etc. This is all up to you to customize:

# backup the web.config and remove sensitive entries before pushing to git, eg connectionString
[string]$filename = gi $folder\web.config 
[string]$backup = [System.IO.File]::ReadAllText($filename)
$xml =
$backup $xml.PreserveWhitespace = $true foreach($n in $xml.configuration.connectionStrings.add) { $n.ParentNode.RemoveChild($n); } # Anonymize any sensitive appSettings entry foreach($n in $xml.configuration.appSettings.add) { switch($n.key) { "Password" { $n.value = "Password" } } } # Remove authorization blocks $xml.configuration.'system.web'.authorization.RemoveAll() $xml.Save($filename)

Finally, let’s run some regular expression check to ensure the web.config does not have any sensitive information left accidentally. Again, this is all up to you to customize.

# verify if web.config still contains any sensitive info
[string]$config = gc $folder\web.config
if ( ($config -match 'connectionString="\w+') -or ($config -match 'users="\w+') ) {
    Write-Host "Configuration file is not cleaned."
    # Restore web.config
    [System.IO.File]::WriteAllText($filename, $backup)
    Exit
}

Now time to compress the source folder and create a zip file using 7-zip.

# Compress the solution folder and copy to deploy folder
cmd /c $compressor a -tzip $zipname $folder -r 
cmd /c copy $zipname $deployPath /Y
cmd /c del $zipname

Finally git commit and push:

# Commit and push to GitHub
cd $commitFrom
git pull
git add -A *.*
git commit -a -m $comment
git push 
Pop-Location

And last step is to restore your own web.config, that was anonymized:

# Restore web.config
[System.IO.File]::WriteAllText($filename, $backup)

That’s it. Now all you have to do is, just hit ./gitpush.ps1 from Powershell command line and you are done!

Real-time Oracle Database Monitoring Dashboard in ASP.NET

Dashboard - Full

Oracle Performance Dashboard (OPD) is a small ASP.NET website that shows you performance & problems of one or more Oracle instances in near real-time. It uses the Dynamic Performance Views (DPV) and runs some popular DBA scripts in order to get meaningful, easy to understand information out of the server. You can use it to quickly spot blocking queries, who is blocking who, expensive query that are consuming high CPU or disk, see if there’s unusual locks, very high disk activity and so on.

Dashboard - Full

Demo

You can see a live demo of this from here:
http://odp.omaralzabir.com

Get the code

The binaries are here, which you can just extract into an IIS folder, put the connection strings in the web.config file, and you are ready to roll. No need to install any Oracle client software on the server.
GitHub Project Binaries

You can get the source code from the GitHub project site:
https://github.com/oazabir/OraclePerformanceDashboard

Feature walkthrough

OPD comes with the following features in V1:

  • Summary of all your instances in one screen, showing important indicators on each instance. Quick way to check if all your databases are doing fine or not.
  • Instance Dashboard showing details of an instance:
    • CPU usage on the OS.
    • CPU consumed by each session
    • Important System Statistics like Buffer Gets, Parse to execute ratio which would indicate some common problems on the server.
    • Sessions and what the sessions are doing, how much resource they are consuming, whether they are hogging the disk or not.
    • Waits, Blocks, Locks, deadlocks that make database suffer.
    • Historical analysis on the databse showing you some very useful stuff:
      • Most expensive queries in terms of CPU and Buffer Get, which are immediate convern for your devs to sort out.
      • IO usage on data files. You can see if some data file is unusually loaded and getting hammered by physical IO.
      • Tablespace usage. Alerts you if some tablespace is over 85% full.
      • Stale stats on tables. You should always keep this clean.
      • Killer Indexes that will actually blow up your system and confuse Oracle query optimizer. You need to get rid of those indexes and rewrite queries that you thought will hit those indexes for better performance. They won’t. They will kill your database.

Utility to make important windows remain always on top

AlwaysOnTop

Do you sometimes fail to notice Outlook reminder window? Do you wish a chat window would remain always on top of other windows so that you never miss a message? Do you want to have a notepad always on top so that you can take quick notes anytime, while working on other apps?

We have an app for that.

It runs quietly on system tray and monitors open windows. Whenever it finds a window that you want to make always on top, it does that:

You can use this AlwaysOnTop utility to make any window remain always on top of other windows.

Get the tool and details here:

http://www.codeproject.com/Articles/794407/Utility-to-make-important-windows-remain-always-on

 

Sit back and relax, let Sharepoint remind and chase your team

EmailExample

Sharepoint Task List is a great place to record tasks for your team members. However, once you have recorded and assigned the tasks, then the fun begins. You have to remind your team members repeatedly about what they need to do today, what tasks are overdue, what’s coming this week and so on. Here’s an app that takes that chore away, and you can sit back and relax, while it will reminds your team members as many times as you want. Your team members will get an email like this, reminding them of their assigned tasks:

The email templates are fully customizable. You can define exactly how the email should look like. You can also include custom fields from sharepoint taks items to appear on the email.

Read the CodeProject article here for details:

http://www.codeproject.com/Articles/789418/Sit-back-and-relax-let-Sharepoint-remind-and-chase

Codeuml–design UML diagrams as fast as you can code

CodeumlPicture

Codeuml.com is a web based UML designer where you code the diagram using a special language and it generates the diagram on the fly. It is faster than using any visual designer where you have to drag & drop diagram elements and use mouse to connect them. Codeuml uses the open source plantuml engine to produce diagram from text. You can produce UML diagrams as fast as you can code.

This web application shows some interesting design and coding challenges. First, it shows you how to build a web based IDE like environment that mimics Windows 8 Metro UI. Second it shows how you can periodically collect data from the website, send to the server in the background asynchronously and get the result generated on the fly. Third and the most important, it shows how you can maintain a server side pool of very expensive resource that you cannot just create on every hit to the server and must have a finite pool that is shared by all your web users.

Read details about this from this CodeProject article:

Codeuml – design UML diagrams as fast as you can code

How to make screencasts in optimized animated GIF for free

I have been using animated GIFs to show short screencasts in my blogs and articles. Animated GIF is supported by all browsers and supports virtually any website in the world where even Flash is blocked. A picture is worth a thousand words, and an animation is worth a thousand multiplied by [frames in animation] words. So, I have been looking for a complete free solution to capturing screencasts and then converting it to animated GIF and then heavily compressing it.

First use CamStudio to capture the screenshot into an AVI. Before you capture, you need to set the CamStudio video recording setting to one frame per second, otherwise there will be too many frames in your animated GIF. You can set it so 2 or more frames per second if you are recording some frequent changes on the screen.

image

This will put one frame in animated GIF per second. Since animated GIF gets pretty large due to its lossless primitive compression, you need to put as little frames on it as possible.

Now you can record screenshot using CamStudio and save it in an AVI file.

Once you have the AVI file, you need to open the AVI using Microsoft GIF animator.

image

Then you need to click the “Select all” button and go to Image tab and put 100 on the Duration. This will set each frame delay to 1 second, exactly what you have set in the CamStudio Video Options. If you have set 2 frames per second in CamStudio, then you need to set 50 in Microsoft GIF Animator.

Now you can save the file as an animated GIF and use it wherever you like.

I would highly recommend you further optimize the animated GIF and eliminate duplicate frames and use some advanced compression. For this you can use the ImageMagick utility. You will find various ways to optimize animated GIF on this page. I just use the following command line and it gives me pretty good output:

c:Program Files (x86)ImageMagick-6.6.3-Q16>convert SourceImage.gif -layers OptimizePlus DestImage.gif

This optimizes animated GIFs pretty well. I have seen average 60% reduction on screen captures having white background and when there’s no translucent areas (eg Windows Vista/7 title bars).

Synchronize File Date Time in multiple servers, solve IIS ETag problem

When you deploy the same website on multiple webservers, you end up having each file getting different last modified date. As a result, each IIS produces different ETag for the static files. If user is hitting different servers for the same file (due to load balancing), each IIS is responding with different ETag and thus browser downloading the same file over and over again. if you have 3 servers, same user has most likely downloaded the same file thrice. This gives poor page load performance.

Moreover, if you want to mirror two or more locations using one location as a base, not only you need to copy the same files but you need to set the same Create Date and Last Modified Date on the files. Otherwise they aren’t true mirror. There are various use cases where you need a complete mirror not only at file content level but also at file date time level.

Here’s a powershell script that will do the job for you:

# Path of the base folder. File date times in this folder is used as the base.
$SourceFolder = ".Folder1"
# Put all the other locations here. These locations must have the same folder structure as the base
$DestFolders = @('.Folder2', '.Folder3')

function sync($sourcePath, $destinationPath)
{
$sourceFiles = [System.IO.Directory]::GetFiles($sourcePath);
foreach ($sourceFileName in $sourceFiles)
{
$sourceFile = Get-Item $sourceFileName
$destFilePath = Join-Path -Path $destinationPath -ChildPath $sourceFile.Name
$destFile = Get-Item $destFilePath
if ($destFile.Length -eq $sourceFile.Length)
{
$destFile.LastWriteTime = $sourceFile.LastWriteTime;
$destFile.CreationTime = $sourceFile.CreationTime;

Write-Host ("SYNCED: " + $sourceFileName + " -> " + $destinationPath)
}
else
{
Write-Host ("SIZE DOES NOT MATCH: " + $sourceFileName + " -> " + $destinationPath)
}
}

$childFolders = [System.IO.Directory]::GetDirectories($sourcePath);
foreach ($childFolderName in $childFolders)
{
$childFolder = Get-Item $childFolderName
$destFolderPath = Join-Path -Path $destinationPath -ChildPath $childFolder.Name
$destFolder = Get-Item $destFolderPath
sync $childFolder.FullName $destFolder.FullName
}
}

$Source = Get-Item $SourceFolder
foreach ($destFolderName in $DestFolders)
{
$destFolder = Get-Item $destFolderName
sync $Source.FullName $destFolder.FullName
}

Open Source WPF UML Design tool

"http://code.google.com/p/plantumleditor/">PlantUmlEditor is my
new free open source UML designer project built using WPF and .NET
3.5. If you have used "Plantuml open source free UML generator" href=
"http://plantuml.sourceforge.net">plantuml
before, you know
that you can quickly create sophisitcated UML diagrams without
struggling with a designer. Especially those who use Visio to draw
UML diagrams (God forbid!), you will be at heaven. This is a super
fast way to get your diagrams up and ready for show. You can
*write* UML diagrams in plain English, following a simple syntax
and get diagrams generated on-the-fly.

This editor really saves time designing UML diagrams. I have to
produce quick diagrams to convey ideas quickly to Architects,
Designers and Developers everyday. So, I use this tool to write
some quick diagrams at the speed of coding, and the diagrams get
generated on the fly. Instead of writing a long mail explaining
some complex operation or some business process in English, I can
quickly write it in the editor in almost plain English, and get a
nice looking activity/sequence diagram generated instantly. Making
major changes is also as easy as doing search-replace and
copy-pasting blocks here and there. You don’t get such agility in
any conventional mouse-based UML designers.

"http://plantumleditor.googlecode.com/files/PlantUmlScreencast3.gif"
alt="PlantUML editor screencast">

I have submited a full codeproject article to give you a detail
walkthrough how I have built this. Please read this article and
vote for me if you like it.

PlantUML Editor: A fast and simple UML editor using WPF

"http://www.codeproject.com/KB/smart/plantumleditor.aspx" href=
"http://www.codeproject.com/KB/smart/plantumleditor.aspx">http://www.codeproject.com/KB/smart/plantumleditor.aspx

You can download the project from here:

"http://code.google.com/p/plantumleditor/">http://code.google.com/p/plantumleditor/

Is your computer running slow, battery running out quickly?

If your computer is running hot or battery running out quickly
then it is most likely due to some application or process consuming
high CPU or memory. If you keep running applications for a long
time, for example, Outlook, then it continues to grow in memory
consumption and does not free up memory efficiently. As a result,
your computer runs out of physical memory and other applications
run slower. Sometimes Outlook, browser, image editing applications
or some other application start taking full CPU as they get into
some heavy internal processing and make your CPU hot and other
applications perform slower.

My new "Alert when CPU and Memory usage is too high for any applicatino"
href="http://code.google.com/p/cpualert">CPUAlert
is an
application that monitors CPU and memory consumption of
applications and alerts you if some application is consistently
taking high CPU or high memory. It not only saves your CPU and
Battery’s lifetime but also makes your computer run smooth
and let your active applications run as fast as they can be.

While it is running, if some process is consuming more than 200
MB memory, it will show you an alert:

"/wp-content/images/B0495B29F0DA5A2626B7FF3FFFE1EA18.png">
"border-right-width:0px;display:inline;border-top-width:0px;border-bottom-width:0px;border-left-width:0px;"
title="image" border="0" alt="image" src=
"/wp-content/images/C57106A41F1BAAA437C38BB21238946C.png"
width="458" height="225">

Here you can see my Outlook is taking 244 MB of physical
RAM.

You can either postpone the alert for 5 mins (just press ESC),
or ignore the process permanently so that you no longer receive
alert for the process anymore, or you can close it and reclaim
memory.

The handy feature is “Restart” which closes the
application and starts again. This generally frees up memory that
clogs up in the process.

Same alert will come if some process is consuming more than 30%
CPU for over 5 mins.

You can configure all these settings like what’s the
tolerable limit for CPU and memory, how frequently to show alert,
how long to wait before closing application etc by right clicking
on the Task bar icon and choosing Settings.

"/wp-content/images/5D3B3D25BCAE8C4842CF0A1504BE77BC.png">
"border-right-width:0px;display:inline;border-top-width:0px;border-bottom-width:0px;border-left-width:0px;"
title="image" border="0" alt="image" src=
"/wp-content/images/ABE0D2CEDF50E819251AB5B576FA223B.png"
width="198" height="182">

"/wp-content/images/A968FAD776BF82BD2C04053B96F442F2.png">
"border-right-width:0px;display:inline;border-top-width:0px;border-bottom-width:0px;border-left-width:0px;"
title="image" border="0" alt="image" src=
"/wp-content/images/C78B7F942E5847893C3414D1583DCF6B.png"
width="601" height="306">

Source code of the project is available at:

"http://code.google.com/p/cpualert/">http://code.google.com/p/cpualert/

The installer can also be downloaded from there.

Warning: The code is not in a good shape. I was frustrated at
some process taking high CPU and memory and I wrote this app within
hours to get the job done for me.

If you like the application, spread the word!