Engineering Enterprise Level Applications using Microsoft .NET Framework

If you have been into engineering medium sized business applications using Microsoft .NET Framework (or even any other framework/language), you must be familiar with ‘n-tier application’ and ‘service oriented architecture’ but the terms ‘dynamic clustering’, ‘horizontal scaling’, ‘fail over clusters’ and ‘interoperability’ might sound a bit fishy. These are the technologies and techniques involved in the engineering of fairly sophisticated, enterprise level, high performance, fault tolerant and horizontally scalable applications.

Tech geeks at Microsoft .NET Framework Developer Center have come up with a great end-to-end sample application in the name of ‘.NET StockTrader’ in order to demonstrate how to engineer enterprise level applications using Microsoft .NET Framework. Some of the distinguishing features of this sample application include

• Load Balancing and Failover clustering
  
• Centralized configuration management of clustered service nodes 
  
• Horizontally scalable via dynamic clustering 
  
• Interoperability with Java and PHP services 
  
• Service host failure detection and automatic restarts 
  
• Full interoperability with J2EE and IBM WebSphere's Trade 6.1 
  

The ‘.NET StockTrader’ sample application has two parts; the core ‘.NET StockTrader’ application and the ‘Configuration Service’ implemented by the ‘.NET StockTrader’ components (web app; business services, and order processing service). The ‘Configuration Service’ is engineered to provide dynamic clustering of service nodes, fail over and load balancing, replication of updates in configuration to allow adding new service nodes on the fly as well as providing the centralized management of configuration data through a single web based interface for the core ‘.NET Stock Trader’ application.

You can download the complete sample application with source code and technical documentation from the official website of ‘.NET Stock Trader’ Sample Application. Although it’s a fairly complex application and does need quite some time to completely absorb and understand it but it’s a great learning experience. I would highly recommend you to start experimenting with it and read and learn more and more about it in order to broaden your tech vision and have insight into engineering sophisticated applications using .NET Framework.

HTTP Compression?

Almost every reader of this blog must be familiar with the term ‘Compression’, but the term ‘HTTP Compression’, might sound a bit strange for quite a few of you.

Today, with the ever increasing tendency of companies providing more and more services over the web; the page load time and the overall responsiveness of the web application, are becoming more and more critical in the adaptability of that web application.

All the web engineers out there must have wondered that despite of their strive to improve the response time of their web application, using techniques like data caching, output caching, minimizing number of web requests, moving JavaScript and CSS in external files, placing JavaScript at bottom and CSS at top and what not; but still there is some thing missing and that’s hindering the overall load time of their web application and affecting the end user’s experience.

Why your web application’s load time is not comparable to the load time of Yahoo and MSN home pages, even though their home pages are showing a lot of dynamic contents? It’s true that their solution is exploiting the full capabilities of hardware too, along with the software but still, there is some thing which even you can configure at the software level to improve the overall page load time of your application in order to improve not only the performance of your web application but to have an improved end user’s experience as well. That software level tweak is called the ‘HTTP Compression’; something similar to zip or rar, but for HTTP responses.

‘HTTP Compression’ is a web standard in which GZip (GNU Zip) or Deflate encoding is applied to the overall payload of the HTTP response by the web server, and at the client end i.e. web browser, the HTTP response is decompressed and rendered to the end user. This result in a significant decrease in the overall network traffic and improves the over all load time of your web application to a very significant extent, with an additional advantage of saving the costs associated with the network bandwidth consumption.

‘HTTP Compression’ is supported by all the modern web browsers and web servers. You can not only compress the static contents (css, js …) but the dynamic contents (.aspx, .asp, .php …) as well. Here is a walkthrough on How to enable HTTP Compression on IIS 6.0 for ASP.NET based web applications. You can definitely Google for enabling this very feature on other web servers as well.

Here are links to the analysis reports, showing the percentage decrease in payload and the improvements in the overall load time for MSN, Yahoo, Facebook and trg tech @ blogspot. Do have a look at these in order to actually appreciate the overall performance gain achieved because of ‘HTTP Compression’ by these web giants.

Interop Forms 2.0

Did you know that Microsoft released the Interop Forms Toolkit 2.0. This toolkit allows developers to create Activex controls that can be hosted in VB as well as forms that can be called from VB. But this toolkit is not limited to VB alone. It can easily be used to create Forms and controls for use in VFP

To download the kit, please visit: Microsoft Interop Forms Toolkit 2.0

For a walkthrough of using Interop 2.0 with VFP9, please visit: Using the Interop Forms Toolkit 2.0 in VFP9 - A Walkthrough


Submitted by Kashif Pervaiz

Extension Methods in VS 2008 (C#)

Extension methods allow adding new methods to the public contract of an existing CLR type, without having to sub-class it or recompiling the original type. VB also supports extension methods.

A simple example would be to check for valid email address. We would probably create a static function and pass the input string

using System.Text.RegularExpressions;

namespace Master
{
public class GenericFunctions
{
public static boolean IsValidEmail(string strIn)
{
return Regex.IsMatch(strIn, "^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$");
}
}
}

//Add the reference
using Master.GenericFunctions;
...

string strEmail = Request.QueryString[“email”];

if (!GenericFunctions.IsValidEmail(strEmail))
MessageBox.Show(“Invalid Email Address”)

Now with the help of extension methods, we can do this…

string strEmail = Request.QueryString[“email”];

if (strEmail.IsValidEmail())
MessageBox.Show(“Invalid Email Address”)

By make the following changes in the GenericFunctions class

using System.Text.RegularExpressions;

public static class GenericFunctions
{
public static boolean IsValidEmail(this string strIn)
{
return Regex.IsMatch(strIn, "^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$");
}
}

The static method has a "this" keyword before the first parameter argument of type string. This tells the compiler that this particular Extension Method should be added to objects of type "string". Within the IsValidEmail () method, we can access all of the public properties/methods/events of the actual string instance that the method is being called on, and return true/false depending on whether it is a valid email or not. Next time we would explore extension methods provided within the System.Linq namespace. Happy journey!


Submitted by Kashif Pervaiz

Meet Windows Presentation Foundation (WPF)

"A new and elegant look for the applications."

Formerly known as Avalon, Windows Presentation Foundation (WPF) is the new graphical subsystem in Windows Vista that provides a holistic means for combining user interface, 2D and 3D graphics, documents, and digital media. Built on the .NET Framework, WPF provides a managed environment for development with the Windows operating system. This takes advantage of the existing investment made by Microsoft in the .NET Framework, and allows developers familiar with .NET technologies to rapidly begin developing applications that leverage WPF.

WPF introduces a new XML-based language to represent UI and user interaction, known as XAML (eXtensible Application Markup Language—pronounced “zammel”). Similar to Macromedia’s MXML specification, within XAML elements from the UI are represented as XML tags. Thus, XAML allows applications to dynamically parse and manipulate UI elements at either compile-time or runtime, providing a flexible model for UI composition.

WPF applications can be deployed as standalone applications or as web-based applications hosted in Internet Explorer. As with smart client applications, web-based WPF applications operate in a partial trust sandbox, which protects the client computer against applications with malicious purpose.
Furthermore, WPF applications hosted in Internet Explorer can exploit the capabilities of local client hardware, providing a rich web experience with 3D, digital media, and more, which is the best argument for web-based applications available today.

Submitted by Mohammed Atif

Have you heard about 'NHibernate'?

NHibernate is an Object-relational mapping (ORM) solution for the Microsoft .NET platform: it provides an easy to use framework for mapping an object-oriented domain model to a traditional relational database. Its purpose is to relieve the developer from a significant amount of relational data persistence-related programming tasks.

NHibernate's primary feature is mapping from .NET classes to database tables (and from CLR data types to SQL data types). NHibernate also provides data query and retrieval facilities. NHibernate generates the SQL commands and relieves the developer from manual data set handling and object conversion, keeping the application portable to most SQL databases, with database portability delivered at very little performance overhead.


NHibernate provides transparent persistence for Plain Old CLR Objects (POCOs). The only strict requirement for a persistent class is a no-argument constructor, which does not have to be public. (Proper behavior in some applications also requires special attention to the Equals() and GetHashCode() methods.)

NHibernate is free as open source software that is distributed under the GNU Lesser General Public License.

What's new in Sql Server 2008 (A string of articles by Irfan AQ)

Table Value Parameters

Another exciting feature of Sql Server 2008 is passing a parameter of table type to a stored procedure or function. Note that it’s parameter of table type, which means that first you have to create the table type and then pass it to the Sproc or function.

Below is a simple example which will create a table type for dventureWorks.HumanResources.Department, and we will create a Sproc which accepts only one parameter of table type and then execute it by passing a variable of new table type:


/* Creating reusable table type */

CREATE TYPE DepartmentType AS TABLE
(
Name VARCHAR (50) NOT NULL,
GroupName VARCHAR (50) NOT NULL
)
GO

/* Creating s Sproc to accept table valued parameter. This will print all the data in the parameter table, but many other operations can also be performed. However, do note that the parameter table cannot be modified */

CREATE PROCEDURE usp_TestTableType
@TableType DepartmentType READONLY
AS
BEGIN
SELECT * FROM @TableType;
END
GO

/* Creating a table variable and inserting four records in it */

DECLARE @TableVariable AS DepartmentType
INSERT INTO @TableVariable (Name, GroupName)
VALUES
('Dept1', 'Group ABC'),
('Dept2', 'Group ABC'),
('Dept3', 'Group XYZ'),
('Dept4', 'Group ABC')

/* Executing stored procedure */

EXEC usp_TestTableType @TableVariable

/* Droping the newly created objects */

DROP PROC usp_TestTableType
DROP TYPE departmenttype


Note that in this Sproc, there are some limitations; first we cannot modify the data in the parameter table henceforth it has to be created as READONLY every time. Secondly, table valued parameters cannot be of output type. Also, these table valued parameters are destroyed as soon as the flow goes out of current scope, therefore, all the insertion or updates on table-valued parameters have to be made in a single batch before passing it to the stored procedure or function.
A good thing is that these table types parameters are not handled in memory. Rather, these are created (materialized) in TEMPDB. This enables us to pass large amount of data as table valued parameters.

What's new in Sql Server 2008 (A string of articles by Irfan AQ)

Declaring and Initializing Multiple Variables

Initializing a variable at declaration time was not possible in SQL 2k5. But this functionality is now available in Sql Server 2k8. We can declare and initialize variables with any constant value or with any expression;

DECLARE
@a int = 777,
@b bit = 0,
@c VARCHAR (10) = 'trg tech',
@d CHAR (3) = Substring('trg tech', 1, 3)

PRINT @a
PRINT @b
PRINT @c
PRINT @d

They indeed are making our lives easy!

Web service VS Remoting

A Web service is equivalent to a static function library. If a client creates an instance of Web service and a member variable of the object with one method call, then attempts to read the value from another method call, the value will be the default value for the variable, not the new value. That's because a Web service doesn't maintain session state without considerable hacking outside the "SOAP" standard.

Remoting feature of .Net however, works with remote objects. If you need the ability to use the remote objects rather than just remote methods, Remoting is the way to go!

Can you identify a pro for Web service when measured against .Net remoting?

Re: What are we aiming to accomplish with this code?

Usama did it guys! We indeed are removing duplicate entries from a sorted array of integers!

Well done Monsiuer :-)

Pivot Table

This seems to be a popular question and quite a few chaps have recently asked me about this.

A Pivot Table can automatically sort, count, and total the data stored in one table or spreadsheet and create a second table displaying the summarized data. The PIVOT operator turns the values of a specified column into column names, effectively rotating a table. Here is an example;

USE AdventureWorks
GO
SELECT [CA], [AZ], [TX]
FROM
(
SELECT sp.StateProvinceCode
FROM Person.Address a
INNER JOIN Person.StateProvince sp
ON a.StateProvinceID = sp.StateProvinceID
) p
PIVOT
(
COUNT (StateProvinceCode)
FOR StateProvinceCode
IN ([CA], [AZ], [TX])
) AS Pvt

What's new in Sql Server 2008? (A string of articles by Irfan AQ)

Filtered Indexes (My Favourite!)

As the name shows, filtered indexes are criteria-based indexes. We specify the criteria as a WHERE clause similar to the WHERE clause in a simple SELECT statement. In fact a filtered index is an optimized non-clustered index. Also, clustered indexes cannot have a filter. If properly designed, filtered indexes enhance query performance to great extent. Randy Dyess (Introducing Filtered Indexes) has some interesting demonstration on how filtered indexes can make database developer’s life easy. I was reading another article on filtered indexes in which author was saying that with filtered indexes, days of over-indexing will be over.

Here is a simple example on creating filtered index:

USE AdventureWorks

CREATE NONCLUSTERED INDEX MyFilteredIndex
ON Sales.SalesOrderDetail
(
SalesOrderID ASC,
ProductID ASC
)
WHERE (ModifiedDate < '03/01/2004')

What's new in Sql Server 2008? (A string of articles by Irfan AQ)

Trusted, Productive, and Intelligent; this is what Microsoft claims about SQL Server 2008. The latest release of SQL Server 2008 was in February 2008, which is still a CTP (Community Technology Preview) and it has most certainly caught the attention of database developers. There are a lot of promising features in 2008 release of SQL Server and in this string of articles, I would discuss them briefly to introduce you to this fascinating upgrade!

Inserting Multiple Rows in Single Insert Statement!

With SQL Server 2005, you could not insert multiple rows in a table in a single insert statement (very annoying!). If there are 5 rows to be inserted, for example, you had to write 5 insert statements. This is not a restriction with 2008. We can insert multiple rows in a single statement. The multiple rows are enclosed in parenthesis and separated by commas. This is the same syntax which is available in MySQL. Below is the example:

INSERT INTO
AdventureWorks.HumanResources.Department
(Name
,GroupName
,ModifiedDate)
VALUES
('Software Development', 'Research and Development', GETDATE()),
('trg tech', 'Research and Development', GETDATE()),
('Software Marketing', 'Sales and Marketing', GETDATE())

Re: What are we aiming to accomplish with this code?

Hey Technotrons!

You still haven't been able to figure this one out yet :-)

int remove_duplicates(int * p, int size)
{
int current, insert = 1;
for (current=1; current < size; current++)
if (p[current] != p[insert-1])
{
p[insert] = p[current];
current++; insert++;
}
else current++;
return insert;
}

Re: You might need to scratch your head

Whoa Imran!

Atif is right. It seems like your query did skip out one year! But an interesting attempt, really.

Atif,

It seems like techies weren't able to crack it. Here is your solution;

SELECT A.*
FROM(SELECT ((SELECT MAX(Hire_t) FROM ABC) + 1) - ROW_NUMBER() OVER
(ORDER BY GETDATE() DESC) AS DT
FROM sysobjects WITH (NOLOCK)) A
WHERE A.DT BETWEEN
(SELECT MIN(Hire_t) FROM ABC)
AND (SELECT MAX(Hire_t) FROM ABC)
AND A.DT NOT IN (SELECT HIRE_T FROM ABC)

It ain't as simple as it sound!

What are we aiming to accomplish with this code?

int remove_duplicates(int * p, int size)
{
int current, insert = 1;
for (current=1; current < size; current++)
if (p[current] != p[insert-1])
{
p[insert] = p[current];
current++; insert++;
}
else current++;
return insert;
}

Re: You might need to scratch your head

Atif, it's seems like Imran has cracked this one!

Here we are with the solution;

SELECT Hire_T + 1
FROM ABC AS AC
WHERE NOT EXISTS
(
SELECT *
FROM ABC AS H
WHERE H.Hire_T = (AC.Hire_T + 1)
AND H.Number_Of_Employees_Hired > 0
)
AND Hire_T + 1 < (SELECT MAX(Hire_T) from ABC)


Can anyone else achieve the desired output in an alternate way?

You might need to scratch your head!

We have Table ABC;

Year_T         Number_Of_Employees_Hired
1995              7
1997              9
1999              2
2000              14
2003              5
2005              6

This table contains data of years and the number of employees hired in a company in the respective year.


Problem is to find out in which years the company did not hire any one.
(Means; we need years, 1996, 1998,2001,2002 and 2004).

Conditions;

We have to use a in-line SQL query only. No stored procedures are allowed. Also, we cannot alter this table or view. FYI, both column types are integers

Submitted by Atif Fasihi

Problematic Functions!

We have unique dilemma at hand. Two SQL Server 2k5 functions are giving different results;

SP_Columns and Information_Schema.Columns

What could be the reason and how to resolve it? And I thought I was adept with SQL Server! :-D

Submitted by Atif Fasihi

Simple but Tricky :-)

Does the following run for infinity or finite time;

for (uint I = 1 ; I >-1 ; I ++);

Reason your answer


Submitted by Imran Younus

Re: What is wrong with this query?

I received a lot of replies about this query but one man was dead on! There is a reason he is known as the "Babe Jee" of Databases! Abdullah Idris cracked this query in 60 secs flat!

Syntactically, there is nothing wrong with this query however it is very unefficient and the logic does not make any sense. Here is how this query should be written;

SELECT
C.CustomerID, C.CustomerName,
C.CustomerType, C.Address1, C.City,
C.State, S.TotalSales
FROM
Customers C
INNER JOIN
(SELECT
CustomerID, SUM(Sales) as TotalSales
FROM
Sales
GROUP BY
CustomerID) S
ON
C.CustomerID = S.CustomerID

What is wrong with this query?

SELECT
C.CustomerID, C.CustomerName,
C.CustomerType, C.Address1,
C.City, C.State, SUM(S.Sales) as TotalSales
FROM
Customers C
INNER JOIN Sales S
ON C.CustomerID = S.CustomerID
GROUP BY
C.CustomerID, C.CustomerName,
C.CustomerType, C.Address1, C.City, C.State

Re: To Index or Not to Index

Whoa! This really heated up people. It seems there are a lot of people out with comments about this. But the recommendation by Ali Asghar is right on with our in-house experiment.

Microsoft recommends an index on a BIT column, especially in SQL 2K5. Our tables where we have opted for BIT columns, the data distribution ratio is 97:3. We created a non-clustered index and studying the execution plan, we can see a performance gain of 20% VS table scan. Ergo, index that column!

Thank you Ali and Adeel for your detailed insight :-)

If anyone wants the script for the test environment, feel free to contact us.

To Index or Not to Index

Our DBA team has been discussing a lot recently about whether to index a BIT column in our tables or not.

Tell us what do you think. Do you think that a table scan would work just as fast as a non-clustered index seek?

Come on people, show us your DBA skills :-)

Re: Please, enlighten me!

No - pages will not migrate between files during an index reorganization!

Submitted by Muhammed Faheem

Quality!

A software developer is a person who should do QUALITY work. That is my counsel to all young developers starting off in IT. Programming is no kid’s workshop and it requires discipline, time and lots of hard work. Always keep up to date with new technologies. Keep your head straight, focus on task at hand, be responsible, pay attention to detail (very IMPORTANT) and avoid HACKS! (What we boastingly call, “Chay-pees”)!

A message by Zeeshan Mirza, Director of Software Services, trg tech

Please, enlighten me!

During an index reorganization operation, if the index spans multiple files, will pages be allowed to migrate between files?

Deadlock!

A deadlock occurs when two or more tasks permanently block each other by each task having a lock on a resource which the other tasks are trying to lock. In a deadlock situation, both transactions in the deadlock will wait forever unless the deadlock is broken by an external process – in a standard blocking scenario, the blocked task will simply wait until the blocking task releases the conflicting lock scenario.

A little humour doesn't hurt

Definition of Programmer

A person who passes as an exacting expert on the basis of being able to turn out, after innumberable poundings, an infinite series of incomprehensive answers calculated with micrometric precisions from vague assumptions based on debatable figures from inconclusive documents and carried out on instruments of problematical accuracy by persons of dubious reliability and questionable mentality for the avowed purpose of annoying and confounding a hopelessly defenseless department that was unfortunate enough to ask for the information in the first place.

Technology has the shelf life of a banana!

Keep yourself updated! Download the new .NET framework 3.5 and Sql Server 2008 (Developer Edition) and start exploring

Get Time Part of the DateTime in .net

Imran Younus, a software engineer and a relational algebra expert (believe me!) has an interesting tip for people who wish to get time part of datetime, in .net, for comparison purposes;

dMyDateTime.Ticks - DateTime.Parse(dMyDateTime.ToShortDateString).Ticks

This will give least value for 12:00 AM and max value for 11:59:59 PM

Well done Zain!

Zain, our brilliant analyst software engineer from Karachi has done his homework!

Answer to yesterday's post is;

"DCM"(Differential-Changed-Map) pages.

Differential backups read just the DCM pages to determine which extents have been modified. This greatly reduces the number of pages that a differential backup must scan. The length of time that a differential backup runs is proportional to the number of extents modified since the last BACKUP DATABASE statement and not the overall size of the database. The interval between DCM pages and BCM pages is the same as the interval between GAM and SGAM page, 64,000 extents. The DCM and BCM pages are located behind the GAM and SGAM pages in a physical file.

You think you've got what it takes to be a DBA?

If yes, I've got one for ya;

What special type of page within SQL Server that is responsible for tracking (via a simple bitmap) extents that have been modified since the last "BACKUP DATABASE" statement?

Answer will be posted tomorrow


email your answers to trgtechmaster@gmail.com

Cool Tip of the Day!

Did you know that when copy/pasting your results from Management Studio, SQL Server 2005(Grid view) into excel, you can copy the column headers too?

Go to Query>Query Options>Results>Grid and check (Include column headers when copying or saving results)

Submitted by Irfan Ali Qureshi

trg tech; where we encourage the creative geniuses to pursue their dreams

I am very pleased to post the very first trg tech blog. On this blog, we will share ideas from our developers, DBA and system architects about all aspects of technology.