A Generic Worker beats all

Windows Azure will charge you for the amount of time your application is up (and maybe running). In order to fully utilize the resources that are at your disposal, you better be efficient with your worker roles in general. This blogpost will be the first in a row showing you how to gain maximum efficiency and still have your workerrole scalable.

The magic behind this is: a generic worker that can handle different tasks.

Consider a workerrole as a program running and in the beginning doing absolutely nothing but listening at a queue in your storage environment. This queue is the broker for your workerrole and will be fed by the outside world (your own apps of apps of others you will serve). What will be in the message is description of a task that the generic worker has to fulfill. The description also contains the location of an uploaded assembly in BLOB somewhere, parameters and other information that is needed for the specific task.

After noticing the message on the queue the workerrole will look for the assembly in blob, load it in an appdomain and start executing the task that is described in the message. It can be a longrunning calculation or a task that listens to another queue where it will be fed task specific messages. It can be a single task being executed just once or a task that will run forever (as long as the app is deployed of course). The workerrole loads the different assemblies and starts executing the different tasks on configurable intervals or when a new message arrives.

Remember, this is a rough description of a generic worker that can be utilized up to the 100%. That's what you need, after all you are paying for it. Don't worry about the CPU getting hot!

To keep this workerrole scalable new instances of the role will need to preload the assemblies already available in the first instance. This requires some administration but hey, that's why we have storage at our disposal. Imagine a generic worker role that has 10's of tasks running. Once task is to provide elasticity to itself! When overrunning a certain limit (CPU, max number of threads, a steep growing number of messages in queue(s)) it will scale itself up! Now that's what i call magic.

Next blog post will show you how the bare generic worker will look like.

VM role the sequel

After playing some time with the VM Role beta and stumbling upon strange problems, i found out that VM beta was activated on my CTP08 subscription and not on my regular one. In the Windows Azure portal, having the information uncollapsed, it looks like it's active :-)

Anyway, testing with the VM role on a small instance now. Using remote desktop and testing if using VM role as a replacement for my own local VM images running in our own datacenter is appropriate. So far, it's looking good. The only thing is: we are running stateless. This means that information that needs to be stored should be stored in a cloudway and not to disk or other local options. Use Azure Drive, TFS hosted somewhere, skydrive, dropbox or other cloudservices that let you save information in a reliable way. Saving your work, while running a VM role, on the C: drive might cause a serious loss of the role gets recycled or crashes and it brought up somewhere else (with yet another c: drive). Although the VM role was never invented for being pure IaaS, it's still a nice alternative that can be very usefull in some scenarios.

We'll continue and make some nice differencing disks with specific tools for specific users (developers, testers, desktop workers etc.) and see how it will work. Developing using VS2010 on a 8 core cloudy thing with 14 gig of internal memory is a blessing. Having your sources stored on Azure drive or alternatives and directly connect to your TFS environment by using Azure Connect combines the best of all worlds and gives you a flexible, cost effective but most of all quick way of setting up images and also tearing them down fast.....