Chaining with Service Bus part 2

In the previous post I demonstrated how to send messages to a topic and how to add subscriptions receiving messages based on a region. The post will demonstrate how to scale out your system by using chaining and enabling messages to be forwarded from the region subscription to a specific country subscription. That post described how messages send to a topic are picked up by subscriptions based on a property of the message (in this case a region like USA or EMEA).

This post will demonstrate how to further distribute messages by another property called Country. To enable this we will use a technique called auto-forwarding. Consider the following:

- different systems insert purchase orders in our system. The purchase order is reflected by the following simplified class:


    [DataContract]
    public class PurchaseOrder
    {
        [DataMember]
        public string Region;
        [DataMember]
        public decimal Amount;
        [DataMember]
        public string Article;
        [DataMember]
        public string Country;
    }


Auto-forwarding enables a subscription to be "chained" to another topic or queue. The scenario we are realizing here is to have messages that are added to our EMEA subscription or forwarded to topics based on country e.g. Holland and Germany. Messages that are forwarded are automatically removed from the subscription (in our case, the EMEA subscription) and placed in the designated topic. See the code snippet below.


            //FORWARDING
            SubscriptionDescription description = new SubscriptionDescription(poTopic.Path, "Holland");
            description.ForwardTo = "Holland";
            SqlFilter filter = new SqlFilter("Country = 'Holland'");
            if (!namespaceClient.SubscriptionExists(poTopic.Path, "Holland"))
            {
                namespaceClient.CreateSubscription(description, filter);
            }
            //FORWARDING


This piece of code enables messages being send to the EMEA topic (as before) with another property called  Country with the value 'Holland' to be forwarded to a specific topic called 'Holland' for further processing.

By using forwarding you can scale out your load on the initial topic and distribute messages based on some property. In the figure below you can see that no messages appear anymore in the EMEA topic but they all show up in the Holland Topic which is accomplished by the code snippet above.

Chaining Topics and Subscriptions

With the release of Windows Azure SDK 1.8 a new useful way of chaining queues or topics together is available. The ForwardTo method allows you to "connect" queues with eachother and implement scale out scenarios.

Consider a single Service Bus topic receiving purchase orders. After a while you notice that processing the orders by consumers takes more time than desired. To enable scale out scenarios where more consumers can take messages at a time you can choose to distribute messages to different topics based on e.g. region.

The code snippet below demonstrates how to use the initial topic scenario where messages are pushed to a subscription based on region. This snippet sends only one message to the EMEA region and then some sort of random based on Ticks. This snippet can be used to populate the Topic and its subscriptions.


            string ServiceNamespace = "yournamespace";
            TokenProvider credentials = TokenProvider.CreateSharedSecretTokenProvider("issuer",
                "yourkey");
            // Create namespace client
            NamespaceManager namespaceClient = new NamespaceManager(
                ServiceBusEnvironment.CreateServiceUri("sb", ServiceNamespace, string.Empty), credentials);
            TopicDescription poTopic = null;

            if (!namespaceClient.TopicExists("PurchaseOrderTopic"))
            {
                poTopic = namespaceClient.CreateTopic("PurchaseOrderTopic");
            }
            else
            {
                poTopic = namespaceClient.GetTopic("PurchaseOrderTopic");
            }

            MessagingFactory factory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb",
                ServiceNamespace, string.Empty), credentials);

            TopicClient myTopicClient = factory.CreateTopicClient(poTopic.Path);


            SqlFilter EmeaFilter = new SqlFilter("Region = 'EMEA'");
            SqlFilter UsaFilter = new SqlFilter("Region = 'USA'");


            if (!namespaceClient.SubscriptionExists(poTopic.Path, "EMEA"))
            {
                SubscriptionDescription myAgentSubscription = namespaceClient.CreateSubscription(poTopic.Path, "EMEA",
                    EmeaFilter);
            }

            if (!namespaceClient.SubscriptionExists(poTopic.Path, "USA"))
            {
                SubscriptionDescription myAuditSubscription = namespaceClient.CreateSubscription(poTopic.Path, "USA",
                    UsaFilter);
            }

            PurchaseOrder order = new PurchaseOrder()
                {
                     Amount = 100,
                     Article = "Car",
                     Region = "EMEA"
                };

            BrokeredMessage POMessage = new BrokeredMessage(order);
            POMessage.Properties["Region"] = order.Region;

            Console.WriteLine("Sending initial message to EMEA");
            myTopicClient.Send(POMessage);


            while (Console.ReadLine() != "q")
            {
                POMessage = new BrokeredMessage(order);

                if (DateTime.Now.Ticks % 2 == 0)
                {
                    Console.WriteLine("Sending another message to EMEA");
                    POMessage.Properties["Region"] = "EMEA";
                }
                else
                {
                    Console.WriteLine("Sending another message to USA");
                    POMessage.Properties["Region"] = "USA";
                }
                myTopicClient.Send(POMessage);
            }



Pressing several times will add random messages.

The next blog post demonstrates how to redistribute messages based on their country by using Auto-forwarding feature of the Service Bus.

Win A Free Copy of Packt's Windows Azure programming patterns for Start-ups e-book

We are pleased to announce that we have teamed up with Packt Publishing and are organizing a give away especially for you. All you need to do is just comment below the post and win a free copy of Windows Azure programming patterns for Start-ups. Two lucky winners stand a chance to win an e-copy of the book. Keep reading to find out how you can be one of the Lucky One.

Explore the book on Packt Publishing

Overview of Windows Azure programming patterns for Start-ups eBook

·         Explore the different features of Windows Azure and its unique concepts.

·         Get to know the Windows Azure platform by code snippets and samples by a single start-up scenario throughout the whole book.

·         A clean example scenario demonstrates the different Windows Azure features.

How to Enter?

Simply post your expectations from this book in comments section below. You could be one of the 2 lucky participants to win the e-copy.

DeadLine:

The contest will close on 21/12/2012 . Winners will be contacted by email, so be sure to use your real email address when you comment!

AppFabric Service Bus Topics part I

Available in the May CTP but release today on the 9th of August, an update of the AppFabric Service Bus containing the wonderful Queueus and Topics.

How to implement the great pub/sub mechanism by using Topics. Topics enable us to implement a 1-to-many messaging solution where the rules of filtering are NOT in your application (or database or whatsoever) but just like ACS, configuration on the Service Bus. It enables you to abstract messaging logic from your app to the bus instead of implementing complex rules in your application logic.

Imagine your application running on Azure (a workerrole) that is being diagnosed by using Performance Counters. These are collected locally and flushed to storage every configurable period. Now imagine your system administrators, responsible for monitoring your cloud assets, being at home but still need to be notified on events. Imagine 2 system administrators both having WP7 being the subscribers to your messaging solution. The first admin is responsible for scaling up and down your Windows Azure App (a workerrole doing lots of calculations) while the second system admin (who is the manager) is only to be notified on the actual scaling up and scaling down events. The first system admin will receive messages showing him averages of CPU utilization and free memory per hour. Based on this, he can decide to scale up or down. The example in the next blogpost below show how to setup the Service Bus and how to create this pub/sub mechanism.

I'll finish the next one in a few days.

Playing with IoC, Enterprise Library 5.0 and Azure

After some discussion with a fellow tweeter (thanks to Amit Bahree @bahree) I decided to write a bit on IoC, DI combined with the full force of Azure. Recently i wrote about the principle of a "Generic Worker", being a worker role on Azure that is able to dynamically load and unload assemblies and fully utilize every dollar you pay for Azure. The solution was pretty straightforward.

The next step in the Generic Worker is to use IoC and DI and fully decouple workerrole plumbing from the actual functionality. Using IoC also makes it easy to configure your workerrole(s) and e.g. dynamically add/remove aspects (AOP) to your applications. The power of AOP is weaving the mixed behaviors together. Apply different aspects to change behavior and functionality of classes without using techniques like inheritance.

The first step i take is to extend the basic Unity behaviour and write my own Resolve method to resolve types not loaded in my appdomain but actual types that reside in my assembly blob storage. Follow the next steps to accomplish completely uncoupled software that makes use of Blob Storage and Unity.

1. Create a classlibrary that contains the interfaces for your classes to be loosely coupled.

public interface ICalculation
{
int Addition(int a, int b);
}

2. Create a classlibary that has a concrete implementation of this interface. This class implements one or more of the interfaces you defined in the classlibrary you created in step 1.

public class DefaultCalculation : ICalculation
{
public int Addition(int a, int b)
{
return a + b;
}
}

3. Build your classlibrary containing the implementation. Take the assembly and upload it somewhere in your Azure Blob-o-sphere Storage. See this screenshot.



You can see the assembly is in my assemblyline storage account and assemblies container.

4. Extent the Unity container with your own method that Resolves in a different way. Not trying to find implementations somewhere in current appdomain but actually take assemblies from Blobstorage and load them. This code runs in my workerrole that's supposed to be awfully generic.

using (IUnityContainer container = new UnityContainer())
{
container.ResolveFromBlobStorage();
}

I will update my next code with a fancy LINQ query but no time right now.

        public static void ResolveFromBlobStorage<T>(this IUnityContainer container) where T : class
        {
            CloudStorageAccount csa = new CloudStorageAccount(
                            new StorageCredentialsAccountAndKey("assemblyline", "here goes your key"),
                            true); 

            //take the assemblies from Blob Storage
            CloudBlobContainer cbc = csa.CreateCloudBlobClient().GetContainerReference("assemblies");
            var assemblies = (from blobs  in  cbc.ListBlobs()
                              select blobs);
           
            foreach (IListBlobItem assembly in assemblies)
            {
                byte[] byteStream = cbc.GetBlobReference(assembly.Uri.AbsoluteUri).DownloadByteArray();
                //load the assembly from blob into currentdomain.
                AppDomain.CurrentDomain.Load(byteStream);

                foreach (Assembly currentAssembly in AppDomain.CurrentDomain.GetAssemblies())
                {
                    foreach (var type in currentAssembly.GetTypes())
                    {
                        if (!typeof(T).IsAssignableFrom(type) || type.IsInterface)
                            continue;

                        container.RegisterType(typeof(T), type, new ContainerControlledLifetimeManager());
                    }
                }
            }

After this code the Unity container is extended with the method ResolveFromBlobStorage.


Step 5 and final:

            using (IUnityContainer container = new UnityContainer())
            {
                container.ResolveFromBlobStorage<ICalculation>();
                ICalculation math = container.Resolve<ICalculation>();
                Console.WriteLine(String.Format("adding 2 and 3 makes : {0}", math.Addition(2 , 3).ToString()));
            }

The ResolveFromBlobStorage method makes it possible to have concrete implementations outside of my solution somewhere and stuffed away in blobstorage. I only need the interface that's it!

Happy programming!

Manage Windows Azure AppFabric Cache and some other considerations

The Windows Azure AppFabric Caching is a very powerful and easy-to-use mechanims that can speed up your applications and enhance performance and user experience.

It's Windows Server Cache but different
The Azure Caching contains a subset of features from the Server Appfabric. Developing for both requires the Microsoft.ApplicationServer.Caching namespace. You can use the same API but with some differences (isn't that a shame! because without this it would be a matter of deployment instead of an architectural decision). Differences are e.g. anything with regions, notifications and tags are not available (yet). The maximum size for a serialized object in Azure Cache is 8Mb. Furthermore, since it's cloud you don't manage or influence the cache directly So if you want to develop multiplatform for both azure & onpremise you need to differentiate on these issues and design for it. Always design for missing items in cache since you are not in charge (but the Azure Overlord is) and items might be gone for one reason or another especially in cases when you go beyound your cache limit.

Expiration of Windows Azure cache is not default behaviour so least used items are ousted when cache reaches it's limit. Remember that you can add items with a expiration date/time to overrule this default behaviour.

cache.Add(key, data, TimeSpan.FromHours(1));

It's obvious that this statement will cause my "data" to expire after one hour.

Keep in mind that using Windows Azure Caching you have caching on the tap and keeps you away from plumbing your own cache. Keeps you focused on the application itself while you just 'enable' caching in Azure and start using it. Fast access, massive scalability especially compared to SQL (Azure), one layer that provices cache access and a very easy, understable pricing structure.

A good alternative even for onpremise applications!

Windows Azure AppFabric Cache next steps

A very straightforward of using Windows Azure Appfabric is to store records from a SQL Azure table (or another source of course).

Get access to your data cache (assuming your config settings are fine, see previous post).

List lookUpItems= null;

DataCache myDataCache = CacheFactory.GetDefaultCache();
lookUpItems = myDataCache.Get("MyLookUpItems") as List;

if (lookUpItems != null) //there is something in cache obviously
{
lookUpItems.Add("got these lookups from myDataCache, don't pick me");
}
else //get my items from my datasource and save it in cache.
{
LookUpEntities myContext = new LookUpEntitites(); //EF
var lookupItems = from lookupitem in myContext.LookUpItems
select lookupitem.ID, lookupitem.Value;
lookUpItems = lookupItems.Tolist();

/* assuming my static table with lookupitems might chance only once a day or so.Therefore, set the expiration to 1 day. This means that after one day after setting the cache item, the cache will expire and will return null */
myDataCache.Add("myLookupItems", lookUpItems , TimeSpan.FromDays(1));
}

Easy to use and very effective. The more complex and timeconsuming your query to your datasource (wherever and whatever it is) the more your performance will benefit from this approach. But, still consider the price you have to pay! The natural attitude of developing for Azure is always: consider the costs of your approach and try to minimiza bandwidth and storage transactions.

Use local caching for speed
You can use local client caching to truely speed up lookups. Remember that changing local cache actually changes the items and changes the items in your comboboxes e.g.