NServiceBus-ServiceMatrix Saga To Saga Request/Response Pattern

Summary

This document explains how to setup a SAGA to SAGA request Response pattern. Bus.Reply is used, as ReplyToOriginator is not supported. We will simulate a service receiving an order, then sending it to an Order Service, which then has a request/response pattern to process payment.

  • We will create 3 endpoints, OrderSender , OrderService, PaymentService
  • We will configure an order to send an initial Order from OrderSender to OrderSaga
  • We will then configure OrderSaga to send a Request/Response to PaymentSaga
  • Note, I have message correlation as well. This is needed for ReplyToOriginator to work between Saga’s from a timeout.

Saga to Saga Request/Response supports Bus.Reply, however do not use it in TimeOut handlers, as it will try reply to the timeout queue.

ReplyToOriginator also works, when you need to call the originating Saga, however there were issues on the bug report https://github.com/Particular/NServiceBus/issues/2018. However you can get it working by doing two things:

  1. Ensure the Calling Saga outlives the Called Saga (Create a Long TimeOut that marks Saga completed in Calling, Create a shorter timeout in the Called Saga that MarksItComplete)
  2. Add this code to the ProcessOrderHandlerConfigureHowToFindSaga.cs

You can download the source code at:

git@bitbucket.org:romiko/sagatosagarequestresponse.git

or

https://romiko@bitbucket.org/romiko/sagatosagarequestresponse.git

Just enabled NUGET package restore J

TimeOuts and responding to the original calling (originator) saga

Never use Bus.Reply within a TimeOut handle in the Saga, as it will try to reply to the timeout queue, as Bus.Reply will always respond to the source of the LAST incoming message.

To get ReplyToOriginator working between Saga’s, you need to:

  1. Ensure the called Saga (ProcessOrder) lives LONGER than the called (Payment), by using timeouts in both sagas
  2. You need to add a correlation

 

 

This is the message pattern with timeouts and a polling pattern, which you can run indefinitely if ever needed.

 

Create three endpoints

  1. Click New endpoint
  2. Create an OrderReceiver, as an NServiceBus Host. Do the same for OrderSaga and PaymentSaga


  3. Your canvas will look like this

 

Send a message from OrderReceiver to OrderSaga

So now we will simulate a service (OrderReceiver) that receives orders on a backend system and then sends them to our OrderSaga for long running transaction processing

  1. Click the OrderReceiver and click “Send Command”
  2. Set the Service name to Orders (Domain) and the command ProcessOrder

    Your canvas should look like this

  3. Click the undeployed component and select Deploy
  4. Select OrderSaga as the destination and click Done

    Your canvas should look like this, with a bit of interior design J

  5. Edit the ProcessOrder message and add the following properties
    OrderId
    Amount

  6. Open the ProcessOrderSender.cs file under the Orders folder, we will configure it to send 3 orders. We will implement IWantToRunWhenBusStartsAndStops

 

Note that I am not in the infrastructure folder, as this is generated code.

  1. Build the solution

 

Configure the OrderSaga as a Saga and Message Correlation

Great, so now we have the minimum need to set the OrderSaga endpoint to a real Saga, as a SAGA MUST have a message to handle. In this case ProcessOrder.

  1. Click the ProcessOrderHandler and click “Convert To Saga”
  2. This will open the ProcessOrderHandlerConfigureHowToFindSaga.cs file. Build the solution, so that partial classes are generated.
  3. We want to correlate order messages based on the orderId to the correct Saga instance. So here we will set the properties on how to find it. Add the following code:
  4. Open the file ProcessOrderHandlerSagaData.cs and add the OrderId, set the property to Unique, as this is how the Saga will correlate messages to the correct instance.

Excellent, so now we have correlation established between the OrderReceiver and the OrderSaga. So if ever the Saga receives order updates for the same order, the infrastructure will no which instance to send the processorder command to.

 

 

Configure Saga To Saga Command

Here we will configure the OrderSaga to send a message to the PaymentSaga, then we will update the PaymentSaga to become a Saga.

  1. Click the ProcessOrderHandler and click SendCommand
  2. Name the command ProcessOrderPayment

  3. Click Copy to Clipboard. This will then open the ProcessOrderHandler.cs file. Paste the code.
  4. Open the Canvas, it should look like this
  5. Click the ProcessOrderPaymentHandler, and Click Deploy.
  6. Select the Payment Saga, as this will handle the ProcessOrderPayment Request.
  7. Your canvas will look like this. BUILD SOLUTION
  8. Let’s CONVERT PaymentSaga endpoint to a Saga, as we have the minimum needed to do this!
    WARNING: NEVER convert an endpoint to a saga unless it has at least one message handler, else it cannot implement IAmStartedByMessages interface. You would have to wire it up manually, since the Infrastructure code generator will not know how.
  9. Click ProcessOrderPaymentHandler and click Convert to Saga…
  10. This will open the ProcessOrderPaymentHandlerConfigureHowToFindSaga
  11. Build the solution, to auto generate the Saga partial classes and infrastructure
  12. We want the payment instance to correlate to the correct order Id, so add this:


     

     

    Build the solution! We added properties so ConfigureHowToFindSaga will compile J

 

Configure Saga To Saga Response and Bus.Reply

  1. Open the ServiceMAtrix Canvas, confirm your canvas looks like this

    notice the icon for saga’s has a circle in it with a square.
  2. Click the ProcessOrderPaymentHandler in the payment Saga and click Reply with Message…
  3. Click Ok
  4. Copy the code to Clipboard
  5. Click the Copy To Clipboard, note the mouse pointer will show as busy, however you can still click copy to clipboard.
  6. This will open the ProcessOrderPaymentHandler.cs, paste the code here. Put in a Thread.Sleep to simulate a credit card payment.

  7. Your canvas will look like this now
  8. Add the following code to ProcessOrderHandler.cs file
  9. Build the solution

 

Testing the solution

Following the following in Order, so the msmq’s are created in the correct order, to avoid race conditions on the first time is starts.

  1. Start the SagaToSagaRequestResponse.PaymentSaga
  2. Start the SagaToSagaRequestResponse.OrderSaga
  3. Start the SagaToSagaRequestResponse.OrderReceiver
    You should see

And

In ServiceInsight we see:

Source Code

You can download the source code at:

git@bitbucket.org:romiko/sagatosagarequestresponse.git

or

https://romiko@bitbucket.org/romiko/sagatosagarequestresponse.git

Just enabled NUGET package restore J

Kill/Terminate process for current logged on user

Below is code that you can use to terminate processes that belong to the currently logged on user. This is using WMI and will work with all authenticated users, even non administrators.

You can use ExcludeMe to exclude a process, e.g. if you running a program and want to guarantee one instance on the machine, but it must not kill the current program.
KillProcesses(Process.GetCurrentProcess().ProcessName + “.exe”, false, Process.GetCurrentProcess());
The above will kill all other processes with same name, except the calling program.
e.g.

Download Source Code

e.g. KillProcesses(“chrome.exe”, true, null);

 public static void KillProcesses(string processName, bool currentUserOnly, Process excludeMe = null)
        {           
            var processes = new ManagementObjectSearcher(string.Format("SELECT * FROM Win32_Process WHERE  Name='{0}'", processName)).Get();
            foreach (var o in processes)
            {
                var process = (ManagementObject)o;
                var processId = int.Parse(process["ProcessId"].ToString());
                if (process["ExecutablePath"] == null) continue;
                if (excludeMe != null && processId == excludeMe.Id) continue;
                var ownerInfo = new object[2];
                process.InvokeMethod("GetOwner", ownerInfo);
                var owner = (string)ownerInfo[0];

                if (currentUserOnly)
                {
                    var windowsIdentity = WindowsIdentity.GetCurrent();
                    if (windowsIdentity == null) return;
                    var currentUser = windowsIdentity.Name;
                    if (currentUser.Contains(owner))
                        process.InvokeMethod("Terminate", null);
                }
                else
                    process.InvokeMethod("Terminate", null);
            }
        }

Download Source Code

Unit Testing PowerScribe 360 – RadWhereCOM SDK

Hi,

Of course no developer wants to write software that is not Unit Testable, else you going to spend 1 months building a project and 2 month debugging it! Why not get your Unit Tests started up front, then write all your use cases as unit tests and bang them one by one.

PowerScribe360 has allot of new features for Radiologists and we want to test when events get published or received, as well as fake all the internal dependencies of the COM e.g. Terminate require PowerScribe to be open, and of course when unit testing, this is a no no, as we want NO external dependencies.

So we will create a wrapper, here is the code, and you can get it from here as well:
Source Code

Note that PowerScribe360 is a class I created to wrap up the RadWhereCom into a public property e.g.

 public partial class Powerscribe360
 {
        public IMyRadWhereCom radWhereCom;
        public Powerscribe360(IMyRadWhereCom myRadWhereCom)
        {
            radWhereCom = myRadWhereCom;
            WireUpEvents();
         }

        private void WireUpEvents()
        {
            radWhereCom.UserLoggedIn += UserLoggedIn;
            radWhereCom.UserLoggedOut += UserLoggedOut;
            radWhereCom.AudioTranscribed += AudioTranscribed;
            radWhereCom.AccessionNumbersChanged += AccessionNumbersChanged;
            radWhereCom.ReportFinished += ReportFinished;
            radWhereCom.ReportClosed += ReportClosed;
            radWhereCom.ReportOpened += ReportOpened;
            radWhereCom.ReportChanged += ReportChanged;
            radWhereCom.DictationStarted += DictationStarted;
            radWhereCom.DictationStopped += DictationStopped;
            radWhereCom.Terminated += Terminated;
        }

        public void UserLoggedIn(string userName)
        {
                Hub.Publish(HubEvents.DictationSystem.LoginCompleted);
        }
 
        public void UserLoggedOut(string userName)
        {
            radWhereCom.Terminate();
            Hub.Publish(HubEvents.DictationSystem.PSInterop.LoggedOff);
        }
  }

e.g.
var ps = new Powerscribe360(radWhereCom);
ps.radWhereCom.UserLoggedOut += Raise.Event(“Foo”);

Source Code

Notice below, that if we did not MOCK RadWhereCOM, the call to radWhereCom.Terminate() in our event subscribe would fail.

The benefit is now that any “Cause” happening in PS360, we can fake the “effect” e.g.
radWhereCom.Terminate().ReturnsForAnyArgs(true);

        [Test]
        public void ShouldRaiseUserLoggedInEvent()
        {
            // arrange
            var wasCalled = false;
            Hub.Subscribe(HubEvents.DictationSystem.LoginCompleted, () =>
            {
                wasCalled = true;
            }, "Foo");
            var radWhereCom = Substitute.For<IMyRadWhereCom>();
            var ps = new Powerscribe360(radWhereCom);  

            // act
            ps.radWhereCom.UserLoggedIn += Raise.Event<RWC_UserLoggedInHandler>("Foo");

            // Assert
            Assert.IsTrue(wasCalled);
        }

        [Test]
        public void ShouldRaiseUserLoggedOffEvent()
        {
            // arrange
            var wasCalled = false;
            Hub.Subscribe(HubEvents.DictationSystem.PSInterop.LoggedOff, () =>
            {
                wasCalled = true;
            }, "Foo");

            var radWhereCom = Substitute.For<IMyRadWhereCom>();
            radWhereCom.Terminate().ReturnsForAnyArgs(true);
            var ps = new Powerscribe360(radWhereCom);

            // act 
            ps.radWhereCom.UserLoggedOut += Raise.Event<RWC_UserLoggedOutHandler>("Foo");

            // assert
            Assert.IsTrue(wasCalled);
        }

Suppose this is in the constructor for our Powerscribe360 class:
radWhereCom.UserLoggedIn += UserLoggedIn;

Then the test above will fail if the line below is removed.
Now the code will never throw an exception when Terminate or any other PowerScribe 360 dependency is called:

Happy Coding!

Source Code

Medical Connections – DICOM and PACS (DICOMObjects)

Hi,

After spending some time working with Medical DICOM Images, I have found that it can be a rather complex process to try get Image Counts off a PACS server, as the way to get it, is different when running a DICOM query at the STUDY or IMAGE level.

When CT Scans are being uploaded into your RIS server, it is very important to know what the number of images, especially for large studies with over 1000 images. So that you can ensure all images are received, so that Radiologists can start reporting immediately.

We leverage NService Bus to handle all the Study orders coming in, and the SAGA needs to know when to end, and part of the dependency is based on the number of images arrived during the import phase, akin to when a purchase order and all it’s items have been received by an order processor.

Basically, when you query a MASTER PACS server, the query level must be set to Study, and the way you get your image counts is based on a dataset field (0x0020, 0x1208).

When you query a Modality PACS server (Preferred), you use the Image Level query, and then can get the image count directly off the DicomDataSetCollection.

I have created a DICOM query Wrapper, that you can now use. Which can support backup servers to query, in the event the current server you are querying is not available. This can happen, if the PACS server is used allot, and it on the 50 connection limit.

Download Source Code

Below is how you would use the wrapper:

var preferredQuery =
    new DicomQueryManager(servers.DefaultPreferred, servers, workstation, port, o.AccessionNumber)
        .BuildPreferred();

var masterQuery =
    new DicomQueryManager(servers.DefaultMaster, servers, workstation, port, o.AccessionNumber)
        .BuildMaster();
        
if (preferredQuery.Found)
{
    //Do SOmething
    var imageCount = preferredQuery.ImageCount;
    preferredQuery.Move();
}

Notice above, you execute a BuildPreferred() or BuildMaster(), this logic encapsulates how the query will be built, if it is a Modality server, it will set the query level to Image, and if it is a Master server, will set it to Study level. The logic then to find the ImageCount is automatically done for you.

You can get the source code for this here:

Download Source Code

NServiceBus – Some Benefits

Hi,

I am not sure why, but in many organisation, there is allot of unnecessary complexity when looking at source code. From Programmers using try/catch blocks all over the show to unnecessary synchronous service calls with multiple threads, causing deadlocks on SQL Server.

I would like to give a simple example, consider this code

static ProcessOrder()
{
    const int workerCount = 4;
    WorkQueue = new ConcurrentQueue();
    Workers = new List();

    for (var index = 0; index; workerCount; index++)
    {
        Worker = new BackgroundWorker();
        Workers.Add(Worker);
        Worker.DoWork += Worker_DoWork;
        Worker.RunWorkerAsync();
    }
}

The above code is extremely dangerous. You have no control over the load it will cause to backend calls, especially when it is calling Stored Procs etc. Secondly, it is not durable.

The above code, can easily be replaced by a NServiceBus Saga or handler, in the above, a Saga is appropriate, as this message is a Root Aggregate, we have an order to process. Sagas will provide an environment to preserve state and all the threading and correlation is all handled for you.

partial void HandleImplementation(ProcessOrder message)
{
    Logger.Info("ProcessOrder received, Saga Instance: {0} for OrderId: {1}", Data.Id, Data.OrderId);
    RunOnlyOncePerInstance(message);         
    AlwaysRunForEveryMessage(message); //counts and extracts new order items.

    if (Data.OrderCount == 1)
    {
        SendSaveOrder(message);
        return;
    }

    if (Data.InitialDelayPassed) //Expired workflow
        SendUpdateOrderItem();
}

From the above, you can see state is preserved for each unique OrderId. This Saga processes multiple order items for an OrderId. Which can be submitted at any time during the day. We do not have to worry about multiple threads, correlation is dealt with automatically, well we set the correlation id to the OrderId, so order items coming in can be matched to the correct Saga instance.

We can now get rid of the unreliable issues with in memory worker threads, and uncontrolled pounding of the database.

By using a MSMQ infrastructure and leveraging NServiceBus, these sort of issues, you find in code, can be easily mitigated.

Install NServiceBus as a service with Powershell

Hi,

I have just completed a script that can be used to install a service bus host as a windows service. It is nice to use, when delegating the installs to the support team, this is always done via NServiceBus.Host.exe.

It can always be tricky parsing arguments in PowerShell. The trick is to use the Start-Process command, and isolate the arguments into it’s own variable.

Here is the two scripts.

Download the script

The one will install the license and the other installs your endpoints as a windows service. All you now need to do, is just type in the password.

It relies on a directory structure, where.

Root\powershellscripts.ps1
Root\ServiceNameA\Debug or Root\ServiceNameA\Release
Root\ServiceNameB\Debug or Root\ServiceNameB\Release

I hope this makes installing all your endpoints easy.

Download the script

Now, remember, if you install Subscriber endpoints before publishers, you will get a race condition, so with this script, add the endpoints in the array argument in the correct dependent order. First install all the publisher endpoints first, then the subscriber endpoints.

Romiko

C# Binary Search Tree

Hi Folks,

I thought I would post source code for a Binary Search Tree Algorithm that I recently worked on.

https://github.com/Romiko/Dev2

In a future post, I will post source code for a Balanced Binary Search Tree (Red Black Tree RBTree).

Remember with an unbalanced Binary Search Tree, if you insert data in order, it will actually resemble a linked list and not a binary tree, hence why it is important to use a Balanced Binary Search Tree.

Of course, you can use built in .NET framework types like SortedSet and SortedDictionary, but for those of you that want to write your own implementation, then this can be a good starting point.

Remember, the rules for a Binary Search Tree, which will also return data in a sorted order, using your IComparer implementation.

I also wrote a method to delete nodes from a Binary Search Tree, which will treats leaf nodes, nodes with one child and nodes with two children differently.

There is allot of articles out there on BST’s, but it is important, that if you want to understand them, that you must understand the rules governing the Insertion, Deletion and Retrieval algorithms.

You can find the source code, at my repository here:
https://github.com/Romiko/Dev2

You will notice that there is also numerous unit tests for the solution as well.

You will notice that the way I delete nodes, uses a Random dice, so not to Degenerate the Binary Tree, if it was balanced, this would not be necessary.

One more thing, I have avoided recursive functions, why? Well, with large binary trees, if you use recursive functions, the stack is going to run out of memory, so avoid them all together.

Here is some code snippets of important algorithms, if you have any questions, or want to contribute to the code, you are welcome to do so.


public void Add(TKey key, TValue value)
        {
            var newNode = new BinaryTreeNode { KeyValue = new KeyValuePair(key, value) };

            if (Root == null)
                Root = newNode;
            else
            {
                var current = Root;
                while (true)
                {
                    var compareResult = Comparer.Compare(key, current.KeyValue.Key);
                    if (compareResult == 0)
                        throw new ArgumentException(&quot;Duplicate key found.&quot;);

                    if (compareResult  0)
                        if (current.Right != null)
                            current = current.Right;
                        else
                        {
                            current.Right = newNode;
                            newNode.Parent = current;
                            break;
                        }
                }
            }
        }

private void Delete(BinaryTreeNode node)
        {
            var nodeType = GetNodeType(node);

            if (nodeType == NodeType.LeafeNode)
            {
                DeleteLeafNode(node);
                return;
            }
            if (nodeType == NodeType.HasOneChild)
            {
                DeleteNodeWithOneChild(node);
                return;
            }
            DeleteNodeWithTwoChildren(node);
        }

        private void DeleteNodeWithTwoChildren(BinaryTreeNode node)
        {
            // Use random delete method, to avoid degenerate tree structure.
            var replaceInOrderType = RollDiceForSuccessorOrPredecessor();

            if (replaceInOrderType == InOrderNode.Successor)
                UseSuccessor(node);
            else
                UsePredecessor(node);
        }

        private static void DeleteNodeWithOneChild(BinaryTreeNode node)
        {
            var theChild = node.Left ?? node.Right;
            theChild.Parent = node.Parent;
            switch (NodeLinkedToParentAs(node))
            {
                case NodeLinkToParentAs.Right:
                    node.Parent.Right = theChild;
                    break;
                case NodeLinkToParentAs.Left:
                    node.Parent.Left = theChild;
                    break;
                case NodeLinkToParentAs.Root:
                    node.Parent = null;
                    break;
            }
        }

        private static void DeleteLeafNode(BinaryTreeNode node)
        {
            if (NodeLinkedToParentAs(node) == NodeLinkToParentAs.Right)
                node.Parent.Right = null;
            else
                node.Parent.Left = null;
        }

        private void UsePredecessor(BinaryTreeNode node)
        {
            var replaceWith = ReadLastRightNode(node.Left);
            node.KeyValue = replaceWith.KeyValue;
            Delete(replaceWith);
        }

        private void UseSuccessor(BinaryTreeNode node)
        {
            var replaceWith = ReadLastLeftNode(node.Right);
            node.KeyValue = replaceWith.KeyValue;
            Delete(replaceWith);
        }

        private InOrderNode RollDiceForSuccessorOrPredecessor()
        {
            if (ForceDeleteType.HasValue)
                return ForceDeleteType.Value;

            var random = new Random();
            var choice = random.Next(0, 2);
            return choice == 0 ? InOrderNode.Predecessor : InOrderNode.Successor;
        }

https://github.com/Romiko/Dev2

Learning Field Guiding in Southern Africa

Hi!

I have been spending the last 2 and half months living in the bush between Selati and Karongwe game reserve in Southern Africa, within the Limpopo province.

I am currently studying my FGASA level 1 with Ecotraining

We have been working on Tracking, Navigation, Survival, 4×4 Drives and learning loads about Mammals, Big 5, Insects, Trees and much more!

I was very privileged to encounter a Cheetah on foot, and here are some pictures that I would love to share with you.

Ecotraining is an awesome training provider, and the quality of training is superb, everyday, we go on drives and walks. This week, we have ben walking in the bush with a rifle for 4 hours in the morning and 4 hours in the afternoon, carrying a rifle as well, to learn rifle handling techniques and safety.

Typical day is:

  • Wake up at 4am
  • Coffee and a rusk
  • 5am Walk
  • 10:30am Breakfast
  • Theory
  • Exercise (Gay Time Gym)
  • Lunch
  • Afternoon walk/drive
  • Campfire and chilling
  • Get lucky with elephants next to your tent at night!

It can be very easy to over eat in the bush and exercise is a must. I started a gym group called Gay Time gym, where we do Yoga, sit-ups, pushups and many other core exercises. We also play ultimate Frisbee to keep fit as a fiddle!

It is awesome taking time out to live a simple and rewarding life style out in the bush.

I have produced a Game called Ranger Rom. You can download it for Tablet and Phone on both Android and Apple. Please download and have a play, it will teach you about wildlife.

Google Play

Apple Store

I will be off to Botswana for December, and hope to get some awesome experiences out there. Until then, Succulent Karoo!

selati

MG 7068

MG 7100

MG 7156

MG 7289

MG 7317

MG 7467

MG 7490

MG 7854

MG 7821

MG 7819

MG 7781

MG 7648

IMG 8262

IMG 8502

IMG 8571

IMG 8609

Off to Africa – a life long dream

Hi Everyone!

It has been a while since I last posted any cool technical articles and so on. Well I am off to Africa!

I was surfing in Indonesia in April this year and that is when the penny dropped for me.

Romiko surfing indonesia

I realised that:

We make a Living
by what we Get
We make a Life
by what we Give

This was my answer. I realised that, I need to follow my passions, and it is possible to work and live life, in harmony!

This is where Ranger Rom was born.

Romiko thinking about Ranger Rom

I love technology. I take pride in the work that I believe in. Ranger Rom is proof of this. It uses state of the art technology from Amazon EC2, S3, CloudFront, Ruby On Rails, ImageMagick, Jekyll and all sort of cool stuff that could take 10 blogs to write on, seriously! Since working on Ranger Rom, I enjoy developing products, because it enriches peoples lives, but most of all, it’s fun.

I realised that I had a choice in life, and now is the time for change. Now is the time to step out from the sheep, pull the wool over the eyes of the wolf; and make a stab at something new and creative.

I might fail, I might fall hard, but at least I will have no regrets when I am 113 years of age, sitting on my bed, going – I wish I did that when I was young.

I encourage you all to live simple, keep the luxuries in life low. A lower carbon footprint and more money to go travel. You do not need to work 9 to 5, year in and year out. You can make a CHANGE and still WORK. You can do the same job but only for half the year, instead of the entire year. There is a will, there is a way!

But this has now changed, and I encourage you all to follow my New Adventures, but more important to follow your dreams and your heart. Today I have posted the first chapter, and yes, it is based on a true story. I am going to the bush for a long time. In between breaks, I will be doing crazy things to, like Paragliding from Mozambique to Madagascar and surfing enormous waves. However, my primary goal is to give back to nature. There will be breaks in between, to earn an income, but ultimately my goals is a balance between my career and what I do outside of it. I believe I can do both a 50/50 split.

Below is Ranger Rom’s first news report to read, so please feel free to join the world of Ranger Rom!

Ranger Rom – Off to Africa

Romiko and Ranger Rom in nature

Ranger Rom is all about promoting wildlife awareness by creating cool games for kids and adults to play. Scheduled for release on the IPhone and Android at the end of this month! IPad and Tablets will soon follow over the next couple of months.