A Generic Worker beats all

Windows Azure will charge you for the amount of time your application is up (and maybe running). In order to fully utilize the resources that are at your disposal, you better be efficient with your worker roles in general. This blogpost will be the first in a row showing you how to gain maximum efficiency and still have your workerrole scalable.

The magic behind this is: a generic worker that can handle different tasks.

Consider a workerrole as a program running and in the beginning doing absolutely nothing but listening at a queue in your storage environment. This queue is the broker for your workerrole and will be fed by the outside world (your own apps of apps of others you will serve). What will be in the message is description of a task that the generic worker has to fulfill. The description also contains the location of an uploaded assembly in BLOB somewhere, parameters and other information that is needed for the specific task.

After noticing the message on the queue the workerrole will look for the assembly in blob, load it in an appdomain and start executing the task that is described in the message. It can be a longrunning calculation or a task that listens to another queue where it will be fed task specific messages. It can be a single task being executed just once or a task that will run forever (as long as the app is deployed of course). The workerrole loads the different assemblies and starts executing the different tasks on configurable intervals or when a new message arrives.

Remember, this is a rough description of a generic worker that can be utilized up to the 100%. That's what you need, after all you are paying for it. Don't worry about the CPU getting hot!

To keep this workerrole scalable new instances of the role will need to preload the assemblies already available in the first instance. This requires some administration but hey, that's why we have storage at our disposal. Imagine a generic worker role that has 10's of tasks running. Once task is to provide elasticity to itself! When overrunning a certain limit (CPU, max number of threads, a steep growing number of messages in queue(s)) it will scale itself up! Now that's what i call magic.

Next blog post will show you how the bare generic worker will look like.