Overview of windows workflow foundation architecture

Introduction:

Whenever I came to know about Windows Workflow Foundation (WWF) a very basic question comes up in my mind, "Why Windows Workflow foundation?"

If you look at different products of Microsoft they have more or less workflow capabilities or at least support. For example Share point portal server, BizTalk Server etc. Especially BizTalk Server includes sophisticated workflow capabilities.

Anyhow later on I have concluded that all these workflow solutions have deficiencies. For example BizTalk Server does not have runtime workflow definition support which is normally required in BPM or even in System to System workflow solutions. Rather than enhancing all these products again and again Microsoft has decided to come up a highly flexible Workflow solution so that they can meet versatile workflow development requirements.

What are the common workflow requirements?

Following are the major workflow requirements which are striking in my mind right now.

• Application workflows for example leave approval.
• Document Life cycle management.
  
• Business process management.
  
• Web Application page flows.
• Integrating different systems.
  

Now what is an effective workflow solution?

I think an effective workflow solution involves not only process modeling and business rules, but also monitoring and analytics in order to respond to exceptions and other note worthy events. It should also enable the business user to define new workflows and business rules on the fly and also editing existing ones without the involvement of software developers.

Windows workflow foundation is an attempt to entertain all these real time rich workflow requirements through one common framework. So that organizations and individuals can enhance and reuse their workflow solutions in different applications.

WWF is a framework not a specific product. It is part of .net framework and it provides a foundation for developing machine to machine workflows to highly people centric workflow applications.WWF addresses all above common requirements. The idea behind WWF is component based development. A component in WWF is Activity which needs to be inherited from the base activity class. In the world of WWF, activity is execution unit which can be composite or standalone.

WWF is based on highly flexible layered architecture. WWF is comprises of following three layers. See figure 1 for reference.

1. Workflow Design.
   
2. Workflow Runtime.
   
3. Workflow Hosting.
   
   
   

   

Workflow Design

At this layer workflows are developed using activities. You can define sequential, state machine and flow chart types workflow. I will further iterate all these in next posts. You can also write business rules policies and can use these policies with in your workflows. Custom workflow models can also be developed using the base framework if required.
In sequential workflow actions are executed through predefined order. For example for getting register for a course you will follow a sequence of steps. In State machine workflow order of actions is not predefined. It is implemented through set of states and transition between states is made based on events. For example in a banking application messages between banks may be send through different mediums for instance Swift, Fax, Signed email etc.

Workflow Runtime

The runtime layer is heart of windows workflow foundation. It contains critical services from workflow framework point of view. The execution service is responsible for scheduling activities, Event handling, exception, tracking etc. Tracking service creates tracking events and passes these events to hosting layer. State management manages states that can also be persisted. Scheduler service schedules execution of activities. Rules service for business policy execution.

Workflow Hosting

Workflow hosting includes default implementation of following services, custom services can also be developed both to replace existing services and to add new ones if required.

1. Persistence
2. Tracking
3. Transaction
4. Communication
5. Threading
   
6. Timer

All above services are obvious requirement of a workflow solution. Persistence is required for long running workflows. If a workflow process takes days or weeks to complete for example student's admission process, a workflow engine can't retain instances of workflows for such longer period in memory. Tracking is required for the visibility of system at runtime. Communication is required to communicate with different components and services within application and with external world as well. Likewise Timer for calculating event triggering time, Threading for performance and responsiveness and Transactions for reliability and integrity. All these services are pluggable you can develop your own for your specific scenario and requirements.

From execution point of view WWF runtime is responsible for execution of workflow instances. It creates and raises events those are further handle through appropriate interfaces at host layer. This is a brief introduction of WWF architecture, in my next post I will try to further excavate in WWF and create an example of sequential workflow.

A very good question answer session related to Architecture

We would appreciate any guidance you may have to offer on the following

• From experience are there any particular areas that need attention up-front that might be overlooked?

Long discussion; will need a separate session for this.

• Security – what measures have you taken to ensure that the application is secure?

Application security is not a short term task. Like other software development activities it should be practiced through out the SDLC.
In our mode of development, application security is a continuous process. Our process includes…

• Application Threat modelling: At the beginning of project, we did application threat modelling. During this process we have identified and quantified the application threats and have established a security document. We encourage our developers to go through the check list on periodic basis


• Establishment of secure development standards and team training


• Verification of code security during peer reviews

• What is authentication method is used?

We are using our own custom authentication mechanism. We have developed a generic portal application in which we can host our ASPX files. This portal provides mechanisms for application level user authentication and per resource authorisation. This portal provides role based security mechanism. Administrator can define different application roles and respective functions (web forms) associated with this particular role along with rights (can read, can edit etc). Then users can be associated with one or many application roles.
All passwords are stored in database after they are properly hashed with a salt.

• How do you handle security - re: different levels of access to areas of the system?

Explained above.

• Is a single user id used to access the main database?

Yes, during development phase. For testing phase we have proposed to create different users at database level with different sets of rights based upon high level user role (like FX trader should have right to access FX related tables).
In beginning we decided to create db users against every user in our application. But this dramatically reduces the possibility of connection pooling. So we dropped this idea. Other way around can be, create db users for particular role (there can be many users attached to a role). Authenticate a user using db user with minimum rights and then switch to db user who has enough rights to access data according to his rights.
There can be many possibilities. Please search for Microsoft Patterns and Practices.

• If a single user is used, does it have restricted database access?

Explained above.

• How is auditing handled?

None yet. We are currently evaluating procedures for it.

• Portal Layout and Site navigation – what methods do you use to control site-layout?

We are just using CSS based application design.
I would highly recommend you to use Master pages and themes/skin features provided by ASP .NET 2.0. I wish we had this feature in ASP .NET 1.1.

• Scalability – the application should scale to 500 users. What Load testing do you do and what tools do you use?

We are practicing standard routines to make the application scalable. Currently we haven’t done any load testing. The option that we have debated is to involve third party services who have the infrastructure and experience in this domain.

• Testing of components / web application

Internal procedures are being followed like in case of QA, developing test cases, test case execution, bug reporting and rectification cycle. In case of developers we conduct peer reviews and unit testing.
We are not currently deploying any Test driven development approaches.

• How do you achieve database independence? We have a large number (~ 100) of Oracle PL/SQL stored procedures which would have to be rewritten in Transact SQL.

We have developed our own database access layer making use of the design of ADO .NET. On database level, we strongly discourage placing business logic. We only create stored procedures for Querying and DML operations using standard SQL. There are scenarios where we needed to impart business logic in stored procedures, but this is very rare.
You will be using ADO .NET 2.0 which provides built in mechanism to achieve database independence. It provides creation of different objects (like connections etc) based upon factory pattern. It has some other exciting features which can be used to make an application more scalable. These features may include asynchronous reads and promote-able transactions etc

• How do you achieve Multilanguage, multi locale, multicurrency support?

Currently our application is not multilingual and multi locale. However it is multi currency. However these features have been initially thought through and require a separate discussion thread.

• How do you control paging?

Currently we are not, but when in future we will need it, we will do it at stored procedure level instead of at presentation tier. We have decided this while considering scalability issues.

• Is anyone at Lahore using Visual Studio 2005 or planning to use it in the future?

At Lahore office, nobody is using it officially. At personal level, some team members of inBanking including myself have been using Visual Studio 2005 since its CTP releases. About future plans, I can’t answer this. You have to ask higher management/architects who are responsible for such decisions.

• With Visual Studio 2005 Microsoft offers either the Team Suite or Visual Source Safe 2005 which of these will you be using and why?

Not a question for me, but if I am ever asked, I will vote for Team System. I have advocated this above.

• Do you run the application under IIS 5.0 or IIS 6.0?

We are developing and testing our application using IIS 5.0.

• Does the application only run under Internet Explorer or is it Browser independent?

We have recently run a cycle to make our application available in latest versions of four major browsers (MS IE, Fire Fox, Opera and Netscape).
We have also established a document explaining areas to focus on to make a web application (such as ours) browser independent. This document is currently in construction phase.

• What were the main difficulties that you had to overcome in the development of the inBanking and ePOS products?

There were many issues and I think it require a separate discussion thread.

• Are there any pitfalls that you can forewarn us of?

Should be carried out in separate thread.

• We do not have much information on the ePOS product can you let us know aht it entails


• We have copies of the TRAPEZE design document (2004) and TRAPEZE functional specification for the inBanking product. Are there more recent versions of these documents? Is the user manual for the product available.

No major revisions has been conducted as yet

• How do you handle errors within the application?

We have developed our own exception handling and logging framework.
I would recommend you to visit Patterns and Practices section and check out Exception Handling and logging block. I recommend using this application block. We are evaluating switching to it as well.

• GUI Design/Layout - have you used CSS? How difficult has this been? Should we consider using the new master template system within .NET 2005?

I strongly recommend you to use Master pages and skins/themes.

• What tool is used to create layout? e.g. Dreamweaver, GoLive etc…

We use Dream weaver MX

• What points should be borne in mind re: Scalability / Multiple users?

There can be many issues and I think it require a separate discussion thread. We have not documented any practices.
I personally think that such practices should be documented and should be made available at organisation level.

Here I would like to throw some light on what we are keeping in mind to achieve scalability…

• Open connection only when it is needed


• Open connection as late as you can


• Close it as early as possible


• Identify data which is rarely modified in database and establish data caching policy


• In ASP .NET 2.0 you can even cache at web controls level


• Do not place heavy objects in View State


• Where ever it is possible, use typed datasets


• Recommended deployment strategies


• Establishment of session server


• Establishment of web forms

• How have you performed stress testing?

We are not currently performing any application stress testing.

• When bolting onto an existing system that contains business rules logic, and new rules are required (e.g. governing user access in the web system), have these been separated from the main business rules?

Can’t get what you are trying to ask.

• Does ePOS use the business rules of the underlying system or is it standalone?


• Is security testing of the application done using automated tools or using 3rd party?

No automated tools are being currently used other then FX Cop. FX Cop is provided as built in feature in Visual Studio 2005. It not only provides mechanism to check the application code for known security issues but also tests the application for scalability issues, naming conventions etc.

• Was the design of the GUI in-house or passed to 3rd party? Was a static prototype created initially for look and feel?

It is In-house production and there is no third party involvement.

• How have you handled the control of web interface navigation


• Turn off browser buttons?


• Stop right clicking?


• Expire pages to force a refresh?

Turning off browser buttons: can be done if you start your application in full screen and No buttons mode.
Stop right clicking: can be done quite easily using some java scripts freely available on internet.
Expiring pages: It is a very complex issue. Pages are cached at different levels in their life cycle and these levels may involve IIS itself, ISP, proxy and then at browser. If we set page expiry date in past, page is not cached at most of the levels. This is the only measure we are currently taking. For more information, please google this topic. I would love to hear feed back from you as well.

• How are your ASP.NET projects organised? What file structure is used in the web app / project / source control?

Application level: All resource files are placed in logical grouping (using folders)
Project: Logical grouping
Source Control: Same as file structure but following the rules asserted by our QE department.

• Are documents produced via the browser? If so how (e.g. PDF / HTML)?

None yet

• Are Reports produced via the browser, if so how?

None yet

• Does any data persist from session to session that is not stored in the main database (e.g. incomplete records)? If so how is this handled?

None

• Is client side validation (i.e javascript) used or avoided?

I do not advocate the idea of avoiding client side validation. In our application we have used client side validation extensively. But since security rule number one is not to trust any input coming in, hence, we also validate it on server. In ASP .NET we can validate all the inputs by simply calling this.Page.IsValid.
I would highly recommend usage of Regular expressions to validate all the input fields. You should establish a separate infrastructure tier which should not only provide regular expressions to validate integers, doubles etc but also complex inputs where business rules are to be followed (like standards runs 3m, 6m and money short cuts 10m for 10 million etc)


Other pointers of interest:

• Consider ATLAS (alias of AJAX, for asynchronous XML calls.), updating application UI without page refresh

.NET Data Access Architecture Guide

http://msdn.microsoft.com/vbasic/using/arch/default.aspx?pull=/library/en-us/dnbda/html/daag.asp

Architecture and Design

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/e2k3/e2k3/ita_TheITAVisualStudioNETProject.asp

Enterprise Application Architecture Perspectives

Information in the enterprise architecture can be viewed from many perspectives and it can satisfy many needs. But four general perspectives are important and are commonly used. These are the business, application, information, and technology perspectives.

The business perspective
The business perspective describes how a business works. It includes broad business strategies along with plans for moving the organization from its current state to an envisaged future state. It will typically include the following:

• The enterprise's high-level objectives and goals.
• The business processes carried out by the entire enterprise, or a significant portion of the enterprise.
• The business functions performed.
• Major organizational structures.
• The relationships between these elements.

The application perspective

The application perspective defines the enterprise's application portfolio and is application-centered. This view will typically include:

• Descriptions of automated services that support the business processes.
• Descriptions of the interaction and interdependencies (interfaces) of the organization's application systems.
• Plans for developing new applications and revising old applications based on the enterprises objectives, goals, and evolving technology platforms.

The information perspective

The information perspective describes what the organization needs to know to run its business processes and operations. It includes:

• Standard data models.
• Data management policies.
• Descriptions of the patterns of information production and consumption in the organization.

The technology perspective

The technology perspective lays out the hardware and software supporting the organization. It includes, but is not limited to:

• Desktop and server hardware.
• Operating systems.
• Network connectivity components.
• Printers.
• Modems.

For detailed Information

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnea/html/eaarchover.asp

Things to do before starting to develop a project

Coding Standards
Commenting Standards
Threat Modeling
Exception/Logging
Authorization, Authentication and Auditing
concurrency avoid policy
Fesibility to existing components
External interface requirements
Knowedle sharing platform
Interface establishment code
Deployment Stretegies
Components Defination (Library Infrastructure)
Code Structure
Caching Policy
Recursive code paterns
caching policy
Assumptions related to Hardware, Software etc
GUI Controls
Scaling stratigy
Technology selection
secondary technologies Ajax etc
GUI Expectations
Reporting
Architecture Tiers etc...
Data Container for traveling data between tiers
Data Base independence
Front end independence
Development Language

How to roll back transaction completely

If a run-time statement error (such as a constraint violation) occurs in a batch, the default behavior in SQL Server is to roll back only the statement that generated the error. You can change this behavior using the SET xact_abort statement. After SET xact_abort ON is executed, any run-time statement error causes an automatic rollback of the current transaction. Compile errors, such as syntax errors, are not affected by SET

web application performance tips(asp.net)

I am a student of computer science at this time i am trying to write down some tips for web application performance boast, For this i have collected some data from microsoft articles, wrox professional Asp.net performance book etc and from my personal experiences.I must highly appreciate your comments, corrections and additions.------------------------------------------------------------------------------------------------
Tips
1-Review your database code to see if you have request paths that go to the database more than once. Each of those round-trips decreases the number of requests per second your application can serve. By returning multiple resultsets in a single database request, you can cut the total time spent communicating with the database. You'll be making your system more scalable, too, as you'll cut down on the work the database server is doing managing requests.

2-The ASP.NET DataGrid exposes a wonderful capability: data paging support. When paging is enabled in the DataGrid, a fixed number of records is shown at a time. Additionally, paging UI is also shown at the bottom of the DataGrid for navigating through the records. The paging UI allows you to navigate backwards and forwards through displayed data, displaying a fixed number of records at a time.one that i want to mention in this respect is, in this case your datagrid will be bound to all result set what if client needs only 10% data of this result set??? so in this case i would like to recomend little bit different approach that can be done with the help of stored procedure. i just want to suggest that you must return the actual number of records that are needed and size and there correct calculation beacuse it can be displayed correctly on client side.

3- In ideal scanario of application dployment whenver your data layer is seprately working on data server, effective connection pooling can increase your performance.

4- Effective caching can greatly increase your application performance but if it is not done intelligently than it is so harmful for your application like out of memory exception etc. There are several rules for caching data. First, if data can be used more than once it's a good candidate for caching. Second, if data is general rather than specific to a given request or user, it's a great candidate for the cache. If the data is user- or request-specific, but is long lived, it can still be cached, but may not be used as frequently. Third, an often overlooked rule is that sometimes you can cache too much that can create an out-of-memory error. Therefore, caching should be bounded. There are a several great features of the Cache that you need to know. The first is that the Cache implements a least-recently-used algorithm, allowing ASP.NET to force a Cache purge—automatically removing unused items from the Cache—if memory is running low. Secondly, the Cache supports expiration dependencies that can force invalidation. These include time, key, and file. Time is often used, but with ASP.NET 2.0 a new and more powerful invalidation type is being introduced: database cache invalidation. This refers to the automatic removal of entries in the cache when data in the database changes.

5- Per request cache is also very good option in some cases, small improvements to frequently traversed code paths can lead to big, overall performance gains. This can be done in asp.net by using HttpContext.Items.

6- you can achive performance gains by using worker threads in case of heavy processing.

7- If you have an ASP.NET page that generates output, whether HTML, XML, images, or any other data, and you run this code on each request and it generates the same output, you have a great candidate for page output caching. By simply adding this line to the top of your page . At this point i do agree that page out put cache does not make your application more efficient but it can reduce burden on servers.

8- There are a number of drawbacks to the use of view state, however. First of all, it increases the total payload of the page both when served and when requested. There is also an additional overhead incurred when serializing or deserializing view state data that is posted back to the server. Lastly, view state increases the memory allocations on the server. Several server controls, the most well known of which is the DataGrid, tend to make excessive use of view state, even in cases where it is not needed. The default behavior of the ViewState property is enabled, but if you don't need it, you can turn it off at the control or page level. Within a control, you simply set the EnableViewState property to false, or you can set it globally within the page using this setting: If you are not doing postbacks in a page or are always regenerating the controls on a page on each request, you should disable view state at the page level.

Business Rules Framework

Business Rules Composer:
This application enables you to define vocabularies and build and test rules. The rule-building experience is as simple as dragging and dropping facts and setting properties. You can also publish and deploy policies from here.
Rules Engine Deployment Wizard:
Vocabularies and policies are stored in a SQL Server rule store database. To help you move a vocabulary or policy to another computer, this wizard provides import/export functionality. You can also deploy or undeploy a policy.
Rules Engine:
This is the runtime engine that processes your policies. The engine evaluates rules, based on their facts, and decides which actions need to be executed. It also supports complexities like dealing with forward chaining. Forward chaining refers to the situation where your rule conditions modify the facts used in your policy, as the underlying facts change, the engine determines when it needs to re-evaluate your rule conditions

configuration of iis with asp.net and other related issues

aspnet_regiis.exe -i
http://support.microsoft.com/default.aspx?scid=kb;en-us;306005http://support.microsoft.com/default.aspx?scid=kb;en-us;318465

Understanding the .NET Framework - The .NET Architecture

http://www.aspfree.com/c/a/.NET/Understanding-the-.NET-Framework/1/

Specifying the tab order in ASP.NET controls

In the body tag write this code to set the focus on the first controlonload="javascript:document.Form1.TextBox1.focus();
Every control has a property, TAB INDEXstart numbering the tab index of the controls starting from 1The controls that you do not intend to specify tab order, assign them -1 tab indexNote: if 0 'zero' is assigned to the controls then the tab order is in the order the controls were added to the form

Implementation of Factory pattern, Smart Client and Refletion.

all these tricks can be achived throug reflection. An amazing thing that i thinks alot in my early programming days. And in end i got all these functionalities through reflectin.a smart client is an windows based application that is dynamically reffering some assemblies, those resides somewhere else with the help of iis.


SmartClient objSmartClient = new SmartClient();
//display splash screenSplash splash = new Splash();
splash.Show();
Application.DoEvents();
// Set the URL to load the Assembly fromstring
strURL = @"C:\Documents and Settings\alif\My Documents\dotnetprojects\SmartServer\bin\Debug\Smart.exe";
// Set the class to callstring
sClassName = "MySmartClient.SmartForm";Assembly assemblyContent = null;
assemblyContent = Assembly.LoadFrom(strURL);splash.Close();
// Create a object for the ClassType
typeContent = assemblyContent.GetType(sClassName);
// Invoke the method. Here we are invoking the
Main method.typeContent.InvokeMember ("Main", BindingFlags.Public BindingFlags.InvokeMethod BindingFlags.Static, null, null, null);

Biztalk Server 2004 Business Rule Engine Basic concepts

Basic concepts BRE
Facts are the entities that your rules consume and manipulate – examples include an element that appears in an XML document, a column in a database table, or even a method exposed by a .NET component.
A vocabulary is a collection of facts, and its purpose is both to apply meaningful, user-defined definition names to those facts and to provide a unit of versioning
A rule is a statement that defines the behavior of one aspect of a business process. rules are made up of facts, conditions and actions.
A policy is a set of rules, packaged and versioned as a unit. Policies are what you develop, test, and deploy when working with the Business Rules Framework

Single Sign-On (SSO)

SSO
Enterprise Single Sign-On (SSO) provides services to store and transmit encrypted user credentials across local and network boundaries, including domain boundaries. SSO stores the credentials in the Credential database. Because SSO provides a generic single sign-on solution, middleware applications and custom adapters can leverage SSO to securely store and transmit user credentials across the environment.The sub services of the Enterprise Single Sign-On (SSO) service are:
Mapping. This component maps the user account in the Windows system to the user accounts in the back-end systems (affiliate applications).
Lookup. This component looks up the user credentials in the Credential database in the back-end system. This is the SSO runtime component.
Administration. This component manages the affiliate applications and the mappings for each affiliate application.
Secret. This component generates the master secret and distributes it to the other SSO servers in the system. It is only active on the Single Sign-On server that is acting as the master secret server.

Calling Unmanaged Dll's from Managed Code

Calling Unmanaged Dll's from Managed CodeThis is a very comprehensive Tutorial on the underlying PInvoke that is responsile for calling the C++ Dll's from the Code of C#.http://msdn.microsoft.com/library/default.asp?url=/library/en-us/csref/html/vcwlkplatforminvoketutorial.asp