Hi!

A quick note that I must say in regards to hosting BizTalk in our virtual environment that Sun Virtual Box has really out did them. They have a really stable virtual machine product. BizTalk, SQL and MOSS have been running really smooth on my development box which runs Sun Virtual Box.

Another nice thing is that they continually update the product.

This is my strategy for templates.

I will build a Windows 2003/2008 Virtual Box base template.

1. Install Visual Studio 2008
2. Install SQL 2005/2008
3. Install BizTalk (DO NOT CONFIGURE)
4. Adapter Packs e.g. WCF LOB, SQL
5. DebugView
6. Resharper (if You like it)
7. Notepad++
8. Orchestration Profiler
9. BizTalk Environment Configuration: http://biztalkconfigloader.codeplex.com/

Rudolf Henning on CodePlex updated my original BizTalk Environment Configuration Tooland made some cool enhancements.

Then you can just close the template above for new developers or for new projects, of course you need a new computer name and IP address. The only trick is for SQL. When you close it, you just run the following command:

sp_dropserver <old_name>

GO

sp_addserver <new_name>, local

GO

A really stable product!

Sun Virtual Box is much better than VMware, hands down. Another aspect I like about Virtual Box is that you can use the console to work on the machine, and the mouse pointers etc feel as if the machine was real, no VMware tools or mouse pointer acceleration to deal with, or even remote desktop as Virtual Box has it’s own built in addin which is a must to install.

HTH

Hi Folks,

Once you have gone live in production with your BizTalk system, you will certainly need to tune the system in order to get a high throughput.

There were two things causing the throughput of our system to be slow.

1. Limitation of connections by the windows server to SOAP/HTTP servers. Limit is 2
2. A bug in the SQL query optimizer when doing implicit data type conversions

We were processing around 10 000 messages per hour when the two settings above were incorrect. After making some adjustments we bumped up the processing to 100 000 messages per hour. Each messages does the following:

1. Imported into the database
2. Sent to a GEO Coder using SOAP
3. Sent to XSLT engine using SOAP
4. Sent to a SQL Algorithm system using SOAP
5. Send to a CRM system using SQL

In all the steps, a transaction log and custom database is used to keep track of the information and also for a custom human workflow application to edit records that are not well formed.

The following is what was needed to get the throughput higher.

SOAP

Edit the BTSNTSvc.exe.config

Add the following section:

    <system.net>
    <connectionManagement>
        <add address = "*" maxconnection = "50"/>
    </connectionManagement>   
    </system.net>

What this does is ensures the server can have more than 2 connections (by default) simultaneously when calling a web service or http service. The setting above should be 12 x (Number of processors). We have a quad core processor, so 48-50 should be a good number for us.

You can read more here:  http://msdn.microsoft.com/en-us/library/aa545389.aspx

One way of knowing if this is a solution, is checking the event log in BizTalk. If you see allot of web service timeouts and you have set high timeout settings in your configuration files, proxy class and message context properties in orchestrations, then it must be the machine limiting the service calls or the web service needs some tuning. In our case, the server was limiting the number of calls.

What you will see in the Group Hub Page, is many send ports with this status:

READY TO RUN

Once we made the change in the BTSNTSVc.exe.config and restart the host instances, there SOAP send ports no longer had a ready to run status.

After the change, the timeouts in the event log disappeared and the query page looks much better, with ACTIVE status for all SOAP send port.

SQL

For the SQL side, we had many receive locations calling the same stored procedure, but using different values for the parameters. The parameters are the BATCH SIZE and STAGE.

CREATE  PROCEDURE [dbo].[GetWorkflowRecord]
@BatchSize int,
@Stage nvarchar(3) = null
AS
BEGIN

In the actual SQL table the Stage column has a data type of varchar(3). If you look above, the the data type is not the same, it is nvarchar(3), this was causing the SQL optimizer to not successfully calculate how the query should run and caused table scans. TO solve the problem we just changed it to varchar(3), and this would stop the stored procedure from doing an implicit conversion when querying the table, since the @stage is in a where clause:

wfr_wfs_code = (‘SUC’)
            AND        wfr_Batch is null
            AND        wfr_stg_code = @Stage

Here is a nice blog about it:

http://statisticsio.com/tabid/36/articleType/CategoryView/categoryId/10/query-optimizer.aspx

CREATE  PROCEDURE [dbo].[GetWorkflowRecord]
@BatchSize int,
@Stage varchar(3) = null
AS
BEGIN

Once this change was made, I was even able to change the 8 receive locations to call SQL every 1 second and increase the batch size from 150 to 500-1000.

Conclusion

Sometimes small trivial changes can make a huge different in performance, here, we were able to increase the throughout of the system from 240 000 records per day to 1.4 million records per day, with an average rate of 900-1000 messages per minute where each message was executing 10-12 transactions and calling 5-6 web services. Two factors limited the system, SOAP calls and a stored procedure causing table scans since implicit conversion of data in the procedure could not be optimised and detect the correct non-clustered index to use. Two simple change with major positive results! What more can I ask for before the Easter break.

Chronicles of South Africa

Here is a picture of two giraffes in a defensive pose. All the animals on this day were very frisky!

Nothing beats a good boerwors!

I love spiders!

Hi Folks,

I was away on holiday in South Africa and on my return, a colleague of mine mentioned that the BizTalk receive locations were unexpectedly shutting down. It would always be a random receive location.

We had just reached 50 receive locations and growing. In the Event Log BizTalk would complain:

Event Type: Error
Event Source: BizTalk Server 2006
Event Category: (1)
Event ID: 5649
Date: Date
Time: Time
User: N/A
Computer: ComputerName
Description:
The receive location "FileReceive" with URL "\\ServerName\FileDrop\FileName.FileNameExtension" is shutting down. Details:"The FILE receive location \\ServerName\FileDrop\FileName.FileNameExtension exhausted the network retry attempts. ".

The problem here is due to the application level protocol management system within windows. This all has to do with the LanMan server and LanMan workstation service within windows. Since these services are used to manage UNC connections to folder shares and the tried and trusted Service Message Block application protocol is used here.

To resolve this problem, increase the MaxCmds and MaxMpxCt registry values on the BizTalk Server computer and on the remote UNC Share computer. To do this, follow the steps that are described in Microsoft Knowledge Base article 810886.
For more information, click the following article number to view the article in the Microsoft Knowledge Base:

This issue may occur if the following conditions are true:

• This issue may occur if the client computer submits simultaneous, long-term requests against a file server that uses the Server Message Block (SMB) protocol. An example of a long-term request is when a client computer uses the FindFirstChangeNotification function to monitor a server share for changes.
• This issue may occur if the MaxCmds registry value setting on the client is less than 50, or the MaxMpxCtregistry value setting on the server is less than 50.
  Note The MaxMpxCt registry value setting may have a different name on other SMB/CIFS implementations. The Common Internet File System (CIFS) specification refers to it as MaxMpxCount.

In our case it was the former that caused it, since we had 50 receive locations. So we need to increase the value from 50 to let says 200.

To resolve this issue, verify that the MaxCmds and MaxMpxCt registry values are set to 50 or more. To do this, follow these steps:

1. Click Start, click Run, type regedit, and then click OK.
2. Locate and then click the following key in the registry:

   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanworkstation\parameters

3. In the right pane, double-click the MaxCmds value.
4. In the Value data box, verify that the value is 50 or more.
   Note In a Microsoft Windows Terminal Services environment, you may have to set the MaxCmds registry value to 500 or more. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

   232476 Terminal Server client connections and logon limited by MaxWorkItem and MaxMpxCt values

5. Locate and then click the following key in the registry:

   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters

   Note Make sure that you make this change to the lanmanserver\parameters registry key and not to thelanmanworkstation\parameters registry key mentioned in step 2.

6. In the right-pane, double-click the MaxMpxCt value.
   Note On SMB servers that are running a version of Windows earlier than Windows 2000, the MaxMpxCt registry entry is named MaxMpxCount, but has the same function.
7. In the Value data box, verify that the value is 50 or more.
   Note The MaxMpxCt value determines the maximum number of simultaneous, active requests that the server permits from an individual client.
8. Quit Registry Editor.

Note The MaxCmds and MaxMpxCt registry entries are REG_DWORD decimal entries. If they do not exist on your computer, you can create them as new REG_DWORD values. The range of values for these registry entries is between 0 and 65535.

It is important that you make this registry change on:

1. All BizTalk Servers
2. All File Share Servers being monitored

Happy BizTalking!

Here is a recent picture of South Africa where I am visiting a herd of elephants at the Thula Thula private game reserve:

Yes the elephants were not happy with us. On this day, we actually got mocked charged by a female elephant which is very rare, since the matriarch is usually responsible for this, maybe he was on sick leave! So with everything in life, and as Nature teaches us, we all have our limits, and with these elephants, it was overstaying our invitation.  The female in the background was within 4 metres from us by the time we put the peddle to the metal!

Hi Folks,

Sometimes you are running a rather large SQL stored procedure, and would like to know which part of the code is running and executing.

Below is a query you can use to see what code is executing at the current moment. I find this very useful, not only for detecting code blocks that are running, but also for identifying long running transaction and so forth.

Hope this helps with your queries and code re-factoring.

Hi Folks,

This blog will discuss the high level overview for developing a proxy on BizTalk to interface with composite web services. What we want to do is create a multi-part message on the fly when sending the data to the web service. BizTalk deals with data, but the workflow system deals with another message type, specify the stage and status of a message. This is where a SOAP proxy can be of use, by leveraging serializing etc.

The hardest part about making this work is supplying the correct message to the SOAP adapter at run time. If the specified proxy class method takes more than one parameter, BizTalk expects you to supply a multi-part message where each part corresponds to a parameter in the signature. BizTalk will deserialize each message part into the corresponding .NET Framework type before invoking the Web service proxy method, which then serializes everything back into a SOAP message.

Imagine we have a web service that manipulates data. It calls an XSLT engine and then does some formatting to the data. In our case, we use Altova XSLT engine, since users can develop maps and not need to be concerned with the BizTalk Mapper.

On the BizTalk side we would call the web service and parse in as parameters the XML from the BizTalk Message Store that subscribed to this particular send port.

You basically create a standard Send Port and then link the custom dll you have to represent your web proxy in the Web Service tab:

So how do you create a proxy dll. First I always use a dedicated proxy Visual Studio project. So in this case I call it MMIT.Workflow.Common.Proxies.<SubProxyName>

Sp the SubProxyName would be DataManipulation for a web service we need to call via BizTalk.

Proxies are cool, since you can do some formatting and extra logic to the data before sending it to the web service.

I assume in the blog, you all familiar with developing web services, if not, you can always read my other blogs about developing web services.

So here is the template code for calling a DataManipulation proxy service.

Here is how you make a proxy class, then just LINK the method name in BizTalk (see picture above) to the method in the proxy dll.

In the above code the Execute is a method call to from a web service reference, which accepts an Xml document.

That is all there is to it, whenever BizTalk calls the send port, it will execute the above code and send the data to the web service. In the above, we converting a XML message to a workflow message and then sending it to another system for processing. The main reason we have this feature is to create multi-part messages.

You can read more about this stuff here:

http://msdn.microsoft.com/en-us/magazine/cc163464.aspx

So I hope this blog gets you started.

Hi Folks,

Scenario

You have a web form where you need to do the following:

• Retrieve XML data from a SQL Data Store
• Provide an interface for Viewing and editing the data depending on the status of the data being viewed.
• The form view for the XML data should be easy to generate when the XSD changes
• The template used to publish the elements and attributes of the XML will be designed using InfoPath
• InfoPath will use Web Service data connections, which are published as Data Connection Libraries on a
  SharePoint 2007 Data Connection Library
• InfoPath will use a Edit and View mode in one template, so data can be controlled and secured for viewers and authors of data.
• InfoPath templates will be published on SharePoint 2007 InfoPath Forms library
• A custom ASPX page will leverage the XmlFormView control to embed the web based Info Path form which is hosted on SharePoint.

Design the Web Services

The first stage is to have the web services ready to Submit and Receive Data. All Info Path forms are designed with data connections to
Submit and Receive Data.

So the first step is to have the Data Layer built.

Design the InfoPath Form

This is divided into two stages

Stage 1

This is where you will create a Data Connection Library and publish it to the MOSS 2007 Server. The Data Connection library will have
all the properties to Submit and Receive data from the web service. When InfoPath forms are loaded it will retrieve the data connections|
from the SharePoint 2007 Data connection library. Below is a sample Library:

Notice above I have two libraries:

DCL for the data connections and WorkflowForms for the InfoPath Template.

Also notice I have no files saved in the WorkflowForms, the template is there, just no need to have a XML based file. The XmlFormView
can still use the template J

Notice the two types:

Main Query (Receive) and Submit.

If we click on of these published DCL’s we have this information:

MainQuery:

  <?xml version="1.0" encoding="UTF-8" ?>

  <?MicrosoftWindowsSharePointServices ContentTypeID="0x010100B4CBD48E029A4ad8B62CB0E41868F2B0"?>

– <udc:DataSource MajorVersion="2" MinorVersion="0" xmlns:udc="http://schemas.microsoft.com/office/infopath/2006/udc">

  <udc:Name>Main query</udc:Name>

  <udc:Description>Format: UDC V2; Connection Type: WebService; Purpose: ReadOnly; Generated by Microsoft Office InfoPath 2007 on 2009-05-08 at 13:25:16 by romiko.</udc:Description>

– <udc:Type MajorVersion="2" MinorVersion="0" Type="WebService">

  <udc:SubType MajorVersion="0" MinorVersion="0" Type="" />

  </udc:Type>

– <udc:ConnectionInfo Purpose="ReadOnly" AltDataSource="">

  <udc:WsdlUrl>http://Romiko.com/Workflow/WorkflowService.asmx?WSDL</udc:WsdlUrl>

– <udc:SelectCommand>

  <udc:ListId />

  <udc:WebUrl />

  <udc:ConnectionString />

  <udc:ServiceUrl UseFormsServiceProxy="false">http://Romiko.com/Workflow/WorkflowService.asmx</udc:ServiceUrl>

  <udc:SoapAction>http://Romiko.WebServices/GetWorkflowRecordByTransactionId</udc:SoapAction>

  <udc:Query />

  </udc:SelectCommand>

– <udc:UpdateCommand>

  <udc:ServiceUrl UseFormsServiceProxy="false" />

  <udc:SoapAction />

  <udc:Submit />

  <udc:FileName>Specify a filename or formula</udc:FileName>

  <udc:FolderName AllowOverwrite="" />

  </udc:UpdateCommand>

– <!–

udc:Authentication><udc:SSO AppId=” CredentialType=” /></udc:Authentication

  –>

  </udc:ConnectionInfo>

  </udc:DataSource>

Submit

  <?xml version="1.0" encoding="UTF-8" ?>

  <?MicrosoftWindowsSharePointServices ContentTypeID="0x010100B4CBD48E029A4ad8B62CB0E41868F2B0"?>

– <udc:DataSource MajorVersion="2" MinorVersion="0" xmlns:udc="http://schemas.microsoft.com/office/infopath/2006/udc">

  <udc:Name>Main submit</udc:Name>

  <udc:Description>Format: UDC V2; Connection Type: WebService; Purpose: WriteOnly; Generated by Microsoft Office InfoPath 2007 on 2009-05-08 at 13:24:00 by romiko.</udc:Description>

– <udc:Type MajorVersion="2" MinorVersion="0" Type="WebService">

  <udc:SubType MajorVersion="0" MinorVersion="0" Type="" />

  </udc:Type>

– <udc:ConnectionInfo Purpose="WriteOnly" AltDataSource="">

  <udc:WsdlUrl>http://Romiko.com/Workflow/WorkflowService.asmx?WSDL</udc:WsdlUrl>

– <udc:SelectCommand>

  <udc:ListId />

  <udc:WebUrl />

  <udc:ConnectionString />

  <udc:ServiceUrl UseFormsServiceProxy="false" />

  <udc:SoapAction />

  <udc:Query />

  </udc:SelectCommand>

– <udc:UpdateCommand>

  <udc:ServiceUrl UseFormsServiceProxy="false">http://Romiko.com/Workflow/WorkflowService.asmx</udc:ServiceUrl>

  <udc:SoapAction>http://Romiko.WebServices/SetWorkflowRecord</udc:SoapAction>

  <udc:Submit />

  <udc:FileName>Specify a filename or formula</udc:FileName>

  <udc:FolderName AllowOverwrite="" />

  </udc:UpdateCommand>

– <!–

udc:Authentication><udc:SSO AppId=” CredentialType=” /></udc:Authentication

  –>

  </udc:ConnectionInfo>

  </udc:DataSource>

This is all done in the InfoPath form designer and published via the designer as well.
These are basically udcx files published on SharePoint.

Stage 2

Now that you have defined the Data Connection Libraries, you can now design the look and feel.

A nice way to control access to the form, is to use views in the SAME template. Why?

SharePoint ALLOWS only ONE type of template per InfoPath Form Library. So if you need different views on the same data,
use ONE template with multiple views:

We can then access these views programmatically, based on business rules in the ASPX page we build J

In my case we have three views:

Default View

Read Only

Write

Programmatically Access Fields in View

You can also programtically access these fileds in your view, for example. When you retrieve the data, you
may need the primary key (TransactionID), so when you call the web service to update, you can then
send the XML data back in the ViewState and also have the TransactionID sent to the webservice, so it
knows what to update.

Another nice thing is you can make the textboxes readonly and programmatically insert the key in the box,
where the data is retrieved, so users cannot hack in and try edit records they do not have permission to!

Web Browser Compatible

Ensure that the form is Web Browser Compatible, in the Form Options set the options:

Notice to that the submit options are for a web service:

WSDL

InfoPath will automatically show all fields in the DataSource pane for the WSDL data, so it is easy to drag
the fields onto the form and then design!

Rules for switching views

You can switch views by linking buttons in the form to a custom rule, so need for custom code for the views.

If you are not familiar with InfoPath forms, there is more than enough material on the internet to get affiliated
with this.

Here are some links I liked:

http://msdn.microsoft.com/en-us/library/aa701078.aspx

ASPX Development

Before we get cracking you can read about the XmlFormView control here:

http://msdn.microsoft.com/en-us/library/microsoft.office.infopath.server.controls.xmlformview.aspx

Now, all you need to do is create a basic aspx page that will host the XmlFormView. Notice the xsnlocation is the
InfoPath forms document library on SharePoint!

Link Custom Page to XmlFormView Page

All you need to do now, is link your page where users can select data from a view (gridview, repeater etc) and then
the corresponding data is shown in a user friendly manner where they can edit/view the xml J

You can then add the XmlFormview you to your page in a panel or so:

You will need a way to reset the XmlFormView, this can be called from GridView_RowCommand event, when
you want to select data based on a key (transactionid in my case) from a row in a gridview.

Here is an overview of the form integrated into a aspx page.

So basically you can select a record, which triggers the RowCommand and then do the magic!

Here is a sample gridview row command that’s writes the transactionid to the form, so the user can then
click the button to get the data:

When users click the button to view the form, you can do a response.redirect:

You can also choose which view to write to, based on the status of the message, users can get a different
view:

WriteToInfoPathForm("false", "/dfs:myFields/my:FormReadOnly");

So we can now click the view button, and voila, the InfoPath on SharePoint gets the data:

SharePoint Server

The SharePoint server must have Info Path forms services enabled, and the feature must be activated.

Also you MUST enable the FormView is the SharePoint Central Administration pages of the MOSS server:

You need to enable Enterprise Features or have InfoPath Form Services Installed.

Also, you must ENABLE form view, this is needed to store the data from the form back to the server,
on post backs.

Summary

So basically your InfoPath form which is embedded in a Panel, will be seamlessly integrated into your
custom web application.

The infrastructure is:

• SharePoint Server with Info Path Forms Server
• IIS Server with custom web application
• XmlFormView.aspx page
• A Panel is a custom web form where you link the XmlFormView
• Some events to loads the Form and run rules
• Views are used to control access to the different forms options
• Very little logic in the code

So, this article should give you some ideas on how to integrate InfoPath forms with existing web applications.

So if XSD/WSDL changes, you can just upload/publish a new template to SharePoint and your users will get the
latest form with new fields etc J

Thanks to a colleague, Leon Droog who did allot of the prototyping and development of this solution. It
can sometimes be a real pain to get things going, but once they going it is allot easier than anticipiated!

Hi Folks,

Just finished the BizTalk 2006 Exam with a score of 985, I got one question wrong in the Enable Business Activity Monitoring section, I would like to take the opportunity to explain what you need to do to pass this exam from a practical perspective and get the best out of the experience!

1. Develop a Custom Send Pipeline
   1. Write a custom Disassemble Component
   2. Configure XSD Validation
2. Develop a Custom Receive Pipeline
   1. Write a custom Assemble Component

While we are on the subjects of pipelines, I think one of the biggest grey areas is debugging these buggers, I will provide an article at a later stage on how to effectively debug a Custom Pipeline Component.

1. Buy the book BizTalk 2006 Recipes and do all the exercises pertaining to BAM
2. Understand how to configure a Schema for Tag Identifiers, so that you can exclude Header and Footer Info or repeating fields
   1. Read about Tag Identifiers in the properties of Schema’s
3. Buy the book BizTalk 2006 Recipes and do all the exercises pertaining to BRE
4. Understand Assert and Retract built in functions in BRE and how custom fact retrievers work
5. Write your own Custom fact Retriever in C# and assert it into the Rule Engine at runtime
6. Understand how to priorities rules
7. Be familiar with the basics of the BizTalk Mapper (Lovely tool for small schemas, a death trap for large and complicated schema’s)
8. Orchestrations
   1. Learn about exception handling
   2. Learn about Persistence points and what shapes cause them!
   3. Understand Correlation inside out
9. Do not underestimate the usefulness of Dynamic Ports and Role Links!
10. The HAT tool was cool for school in BizTalk 2004, but in 2006 the HAT tool is there for those dev’s that were used to using it in 2004, so please focus on BizTalk Administration Tool and HUB Page.
11. Write at least one batch file to deploy an MSI package automatically with BTSTASK (BTSDEPLOY sucks, never use it!)
12. Know how to export and import MSI packages
13. Know how to export and import Rules
14. Have a good understanding of how you can use host instances to organize where receive and send locations run on
15. Remember the golden rule; always put common artifacts in one application. This makes deployment much easier
16. Read the BizTalk 2006 book by Wrox Press
17. Read the BizTalk 2006 book by Wrox Press again

I cannot delve to deep about the exam, but I think the above list is a good place to start, I never used any of the MOC material for the exam, the MOC material is great to get an idea, but to really understand the details, much better to get the BizTalk 2006 Recipes book and develop hands-on!

Good Luck

Hi Folks,

Sometimes in Orchestrations you would like to throw a custom exception. Before we begin, remember this rule:

• Never throw a general exception outside the catch block in an Orchestration

The rule above is a bit funny, because when you drag a default exception shape in a try block within an orchestration, it will default to General Exception and you cannot choose any other exception type.

Imagine we need to call a web service and then handle the response from it within an orchestration. However, sometimes I may get a timeout from the web service or I would only like to wait a 1 minute or so to get the response. What I usually do in these cases is create a listen shape after I have called the web service and then I use a delay shape to wait a specified amount of time for the web service response, if it does not respond within the time span , I simple throw a Timeout Exception.

However, as mentioned when you drag a default throw shape, the only option is to throw a general exception, to solve the problem, create a variable of type .NET Class

Then, navigate to the mscrolib library:

And choose the exception type as illustrated below.

Give the variable a friendly name in the Orchestration View

Add a throw exception under the Delay shape and assign the type of exception to the variable name you created (Weird I know, but this is how it is done)

Below are the properties of the Throw Exception Shape

As you can see above, I have a listen shape after sending a message to a web service and am now waiting for a response.

And you nearly done, what you then do is add a Catch Exception by right clicking the scope and clicking add exception handler:

• Remember all transaction should be in a scope, so your catch blocks will automatically be appended to it. Now it is time to customize the catch block.

Right click the new Catch Block and set the type of exception to catch, in my case, a timeout exception.

There are two properties to set here:

• Exception Object Type
• Exception Object Name

In my case this will be

Exception Object Type = System.TimeoutException (I had to choose .NET Class and navigate through the mscrolib again)

Exception Object Name = e ( I chose e, as this is what we use in coding most of the time, you can choose anything you like)

So the Exception Object Name is similar to doing this:

catch (Exception ex)

{

}

This is how looks is now.

Don’t forget to add custom code to handle the exception in the Catch Block, maybe to update a log table or send a message somewhere, like I have done above, so now when a web service times out, the orchestration will throw a timeout exception, this is much more intuitive than seeing a general exception!

SOAP exceptions from a web service

In order to catch soap exception from a web service, what you do is add a reference to the BizTalk project for:

System.Web.Services

Then add an exception handle block:

You can then access the information of the exception message like this:

e.Message

e.g.

Or you can create a Soap Exception message and send it to the message box for failure routing, here is an example from:

http://www.digitaldeposit.net/blog/2007/05/orchestration-handle-soap-exception-and.html, accessed 26 May, 2008

Source: Saravana Kumar, http://www.digitaldeposit.net/blog/2007/05/orchestration-handle-soap-exception-and.html

A word of warning, if a web service throws a soap exception, the port will retry by default 3 times at intervals of 5 minutes, I changed the retry here to 0 for my example, so that the soap exception is caught and a log in made in SQL, however if you using a delay shape make sure the amount in the delay is greater than the total interval in the send point, else you will never catch the soap exception and will just catch timeout exceptions!

So below, works fine with a delays of 2.5 minutes, however if I made retry count = 1, then the delay should be greater than 5 minutes! You can really get confused when a soap exception occurs and the orchestration does not catch it, it will only catch it after the retry count is done in the send port of the web service!

Here is the exception in SQL

To recap:

1. Create a VARIABLE in the orchestration view of the exception type you want to catch, use mscorlib to assign the variable type to an exception
2. Drag a throw shape and set the type of exception to throw to the variable name (yes it is automatically instantiated)
3. Add a custom Catch block to the SCOPE where the throw shape is located and once again, set the properties of Exception Object Name and Exception Object Type, where type will come from mscorlib and name is anything you like!

I would appreciate any comments you have or if you need any assistance with a BizTalk project. In later blogs I will be discussing Windows Communication Foundation and BizTalk R2.

Tip of the Day: Orchestrations should be used scarcely! The BizTalk books out their do not do any justice about this, since their main content is orchestrations and less on pure Content Based Routing using the core messaging engine i.e. Send Ports, Receive Ports, Adapters and Pipelines.

Secondly remember to always utilize serializable components if calling external components from within an orchestration, if you cannot encapsulate them by calling a web service, this ensures the calling code is dehydration friendly!

Cheers

Hi Folks,

This article is going to discuss how to use Enterprise Library and integrate it with BizTalk Components. The examples in this blog are based on a Pipeline component that I use to import flat files, you can easily convert the pipeline component to a file adapter, but the performance difference is negligible and is just a matter of preference. It is nice to make really robust application, but more importantly efficient caching, logging and exception handling is often missed! I assume the reader has in depth knowledge of Microsoft BizTalk 2006

First I create folders for Caching, Logging and Exception Handling in my current Pipeline Project.

The Decoders folder has the decoder components for a Pipeline

The Disassemblers folder has the disassemblers components for a Pipeline

The FileTranslator is a custom component that a disassemble component can call to transform flat files into XML using a custom configuration, this is not in scope of this blog and will be treated black box, in future blogs, I will discuss how to write such a component and it is far more powerful than using the BizTalk Mapper, since the BizTalk Mapper is only good for small schema’s when trandforms become complicated and many rules need to be applied, I recommend a custom file translator which you can implement as a File Adapter or Pipeline, does not really matter as long as you handle exceptions correctly and catch them and always treat you files in streams and use a recordline processing model so you never load the entire file in memory.

Requisites

Install Microsoft Enterprise Library 3.1 May 2007

http://www.microsoft.com/downloads/details.aspx?FamilyID=4c557c63-708f-4280-8f0c-637481c31718&displaylang=en

Logging

We are going to start setting up the Logging mechanism first.

• Add a reference to the Project to Microsoft.Practices.EnterpriseLibrary.Logging, I like to create an External Assemblies folders, so when deployment occurs, I remember to add it to the GAC. I add the dll from the install folder e.g. D:\Program Files\Microsoft Enterprise Library 3.1 – May 2007\Bin to my own project, that way, external dll’s are never lost.

• Add a using statement for Microsoft.Practices.EnterpriseLibrary.Logging
• Create a public, serializable class in the Logging folder and add a method that accepts two strings.

• The next step is to implement the configuration file for this. Things to note, usually when you have a Windows form application, the app.config file is automatically read, and if you use a web form then the web.config file is used. Now think about it, if we calling an Enterprise Library Component from a pipeline component, neither the app.config or web.config is going to be called, because this is excuted in the App Domain for the BizTalk process, we will need to configure the config file to use the ‘BTSNTSvc.exe.config’, if you ar efamilair with BizTalk, this is the place that BizTalk server reads for configuration as well as custom configuration. This file is located in C:\Program Files\Microsoft BizTalk Server 2006 by default. You can add many things here for example a custom timeout for HTTP connections:

   

  </system.runtime.remoting> <system.net> <connectionManagement> <add address="*" maxconnection="5000"/> </connectionManagement> </system.net>   </configuration>

  So, what we going to do is create a new logging application configuration file for Enterprise Library to read, then, we will modify the BTSNTSvc.exe.config to add a reference to the configuration file. So that at runtime the configuration file can still be found. There are many ways to do this, you could even use the SSO database, but for this article I am sticking to simple configuration files. We will store these configuration files in the C:\Program Files (x86)\Microsoft BizTalk Server 2006 folder, excuse my path, I am on a 64 bit OS.

• Open the Enterprise Library Configuration tool, located at Start\Program Files\Microsoft patterns & practices\ Enterprise Library 3.1 – May 2007 \Enterprise Library Configuration.
• Open the ‘BTSNTSvc.exe.config’ by clicking File > New Application
• Right click on the Application Configuration name and add a new Logging Application Block by clicking the file name and select New > Logging Application Block.

• Select the Trace Listeners and right click on it to select the New > Rolling File Trace Listener.

• Select Formatter on the right hand pane and select Text Formatter

I like to put all my log files in %windir%\system32\logfiles\<application name>\filename<TimeStampPattern>.log, since many MS application do this, why not make our application do the same to keep log files centrally located. We going to implement a rolling flat file logger that increments

• Change the FileName of the log file to %windir%\system32\logfiles\BizTalk\Pipeline.log (NOTE: DO NOT USE %WINDIR% use the path C:\…. etc, else EL will append the current execution folder to %windir%)
• Change the attribute for the RoleFileExistsBehavior to Increment
• Change the RoleInterval to Day, this will create a new log file per day, similar to IIS
• Change the TraceOutputOptions to DateTime, so we can get a nice row prefix for the logging

The options set here can be overridden when you hook the listener to a Special Source, which we see later.

This is what my configuration looks like:

• The next stage is to assign the listener. To do so you select Formatted EventLog TraceListener from the Special Sources.
• On the drop down many assign it the Rolling Flat File Trace Listener

So what we have done is created a listener and then told the configuration to log errors and warnings to this listener. If you click the plus sign you can override the default settings we defined earlier on.

• Set the Rolling Flat File Trace Listener in the Category Sources\General\Formatted EventLog TraceListener. Select the node and select the Referenced TraceListener to Rolling Flat File Trace Listener from the dropdown.

From a configuration perspective we are complete, we configured the Rolling Flat File as the location for logging.

We hooked the listener to be called when Logging Errors and Warnings and when the category is General.

The configuration tool from Enterprise Library is extremely customizable, for example we could add a new listener for the event log and create a new category called "Critical" and then hook the event log listener to this category, so that if you call the logger, with a different category, it will log to a different source, based on the configuration. i.e.

Logger.Write(message, category);

What is left is that we need to save the configuration to file, in my case, I will save it to:

C:\Program Files\Microsoft BizTalk Server 2006\ MMIT.BizTalk.Components.Logging.config

What we now need to do is add a section to the BTSNTSvc.exe.config that points to the configuration file we created.

• Open the BTSNTSvc.exe.config file

Be careful adding custom conifgSection to BTSNTSvc.exe.config file: it must be the first child of the root element.

You will get an unfriendly message in the event log:

• Add the following section to the config file

• So the entire configuration file for the BTSNTSvc.exe.config looks like this (assuming you don’t have any other custom stuff in it, you might do!)

Notice I have PublicKeyToken in the reference here above, so that it forces to look in the GAC!

The option in read is other customizing not needed here, but put in for additional info.

So after making these configuration files, you can quickly review them to have a look, I sorted my BizTalk folder my date modified to get easy access to them.

I now add some log calls in my custom pipeline component, so I open my disassemble component and add some logging e.g.

This particular disassemble component is called by a custom pipeline, in a separate BizTalk project, you will notice that my component project is just standard c# project, so the Pipeline will look in the GAC for the custom component, here it is:

So I hope this brings back a memory, the Pipeline is executed in a Receive Location like this:

I now drag my component DLL to the GAC, restart BizTalk services etc and the pipeline will now execute with the updated component J

Now, what I do is compile my custom pipeline component and deploy the component to the GAC, and restart the BizTalk application and Host Instance that uses this custom Pipeline component, here is a picture of my pipeline that calls this custom component. You do not need to recompile BizTalk projects, just the dll for the pipeline component!

I can now set breakpoints in the custom pipeline component, so that when the pipeline executes it will hit the breakpoint, what you do is attached to the BTSNTSvc.exe services, Im too lazy to find out which one here below, you may have other host instances running:

NOTE: Make sure you attach to Biztalk After you deployed the dll to the GAC and then restarted the host instances in that order (Only instances using the pipeline, in my case only one service to restart)

If you BizTalk service cannot restart you probably got an error in the BTSNTSvc.exe.config

To see if my logger works (You can create a test project for this, but then your configuration will need to be in an app.config file, we will do this later, since I like test driven development.

So now, we drop a file in the receive location that is configured to run the pipeline.

Above is the file receive location, with the custom pipeline.

Ok, I now drop a file at the receive location and then it hits the break point.

I get an error!

What you need to do here is ensure this assembly is in the GAC

• Microsoft.Practices.EnterpriseLibrary.Logging.dll

So drag the dll into the GAC or use your tool GACUTIL and then RESTART the host instance, since it is on host start up that the Microsoft.Practices.EnterpriseLibrary.Logging.dll will be loaded in memory and cached

After doing this, my step into works

However, I now get a new error:

The inner exception has this:

So we need to add this to the GAC to:

Ok, restart instance, attached to process and off we go again, fun debugging pipelines!

To cut a long story short, you can debug the problems with Enterprise Library and figure out the problem, in all these assemblies need to be in the GAC:

• Microsoft.Practices.ObjectBuilder.dll
• Microsoft.Practices.EnterpriseLibrary.Logging.dll
• Microsoft.Practices.EnterpriseLibrary.Common.dll

But before we get ahead of ourselves, I get another exception.

If you get this, ensure the BizTalk account has permission to the folder and create the folder BizTalk at the location of your log file or whatever you chose to use!

Now I get my log file!

And if we open it:

That’s it really, so you can add logging calls to your program and can even create other logging categories instead of general etc, so above I might as well add the file import end log and call it again when the file import is complete, totally up to you!

There is one problem left, you notice that a timestamp was not created and appended to the file!!!

However, I waited an entire day (just kidding, changed the time on the computer on my DEV machine) and get this when I dropped another file in the receive location:

Ok, that’s it for now. Don’t forget to add the other assemblies to the referenced assemblies, so when you have them in source safe, you remember to deploy them to the GAC!

• Microsoft.Practices.ObjectBuilder.dll
• Microsoft.Practices.EnterpriseLibrary.Logging.dll
• Microsoft.Practices.EnterpriseLibrary.Common.dll

In later blogs we will have a look at how we can leverage caching to optimize BizTalk custom assemblies.

Hi Folks,

I am going to show how to query data from a SQL Table based on values in an XML Data Column.

Here is the SQL Script:

As we can see, we always start off with the XML NameSpace and then where use the .value notation on the XML Data column name to run the Xpath on. Notice I queries both an element and attribute, where attributes are prefixed with the ‘@’ sign.

Results:

Activites                      CapitalCity                    Activites                      CapitalCity
—————————— —————————— —————————— ——————————
Surfing                        Pietermaritzburg               Partying                       Johannesburg

Hope this gets you started with querying XML in SQL, it is relatively straight forward. I could not think of any exciting XML Data, so South Africa provinces did the trick, some geography lessons for free 🙂