Azure Tables, Entities and Partition Diagrams

It's always nice to explain things with diagrams, no matter how bad the diagrams are :)

A picture of a Table

The following diagram summarizes how a table of NxtGenUG (a UK user group) Coordinators might look in a Table in Windows Azure Storage Services.

This diagram shows that Chris Hay and Allister Frost are coordinators for the Cambridge region of NxtGenUG, and that Dave McMahon and Geff Lombardi are coordinators for the Birmingham region of NxtGenUG.

I have scribbled with my crayons how this relates to tables / partitions and entities in Windows Azure Tables.

The key thing to note about this table is that regions are partitions (birmingham, cambridge). The row keys are unique within a partition (but not outside of the partition, Chris and Dave have the same row key). Combinations of Partition Key and Row Key are unique.

Scaled Partitions

The following diagram shows how this table might be scaled in Windows Azure using Table partitioning.

The key thing to note is that entities within a partition always remain together (i.e. reside physically on the same server).

However partitions may not necessarily reside together, hence why querying across partitions is slower than querying exclusively within a partition

Silverlight and Tables

So this is how the tables look in the back end of Azure. This is a great structure and I can see this is something that Silverlight Applications could heavily utilize.

A lot of Silverlight applications don't necessarily need the full blown power of SQL Servers, and really want to save simple data in the backend, and be able to retrieve.

I can see this table type structure really being utilized within Silverlight. The downside is that Silverlight can't access it directly (you need to go via a ASMX / WCF Service). As I have said before I'm not sure you want to be able to give direct access, however perhaps some sort of delegated authentication scheme could work?

I think this will require more thought???

Azure Presentation

So I’ve just done my first version of the “Introduction to Windows Azure” presentation (the same one I am doing at DDD, called “Welcome to the Cloud”). This was an internal presentation to a local company in Cambridge.

The feedback was superb (one chap said, it was a better presentation than I had seen at TechEd, which was very nice).

From my point of view, there is a couple of things that I want to change for Saturday (DDD7) however nothing particularly major.

One thing that disappoints me about Saturdays is that there is no Silverlight whatsoever in the day, I am now considering changing one of my demos last minute to have a some Silverlight :) , I'm not sure I can cope with a full day without Silverlight

Anyways, I look forward to seeing folks on Saturday (Silverlight or no Silverlight)

First presentation of "Welcome to the Cloud" tomorrow

If you are going to DDD7 and you plan to attend my talk entitled "Welcome to the Cloud", which is a talk on Windows Azure, you will be pleased to hear that you won't be the Guinea pigs for this talk.

Very luckily I get to do a run through tomorrow for a local Cambridge company called Sagentia.

This should allow me iron out any glitches hopefully in time for Saturday :)

This has been a pretty tough presentation to put together, learning a new technology + presenting on it in 2.5 weeks (evenings + weekends to prepare only).  I wish i was one of those lucky folks who got to see it before PDC but I didn't, so I really have had 2.5 weeks to prepare this (i.e. when i got back from LA).

It's nice to be finally be presenting it for the first time :)

Silverlight: and Windows Azure

In my last article I was discussing Silverlight and Amazon's new CloudFront service.

I wanted now to discuss what the options with Azure are?

Hosting your Silverlight application in Azure

You can obviously host your Silverlight Application by hosting your website (which includes your Silverlight app) in Windows Azure, but I thought I'd explore avoidin the website, similar to what i was suggesting doing with Amazons Cloud Front.

The good news is that you can actually host your html file, your silverlight .xap file (set the mime type correctly), you will even be able to interact with web / wcf services via clientaccesspolicy / crossdomain files. You can host your Silverlight application completely in Windows Azure today (subject to the terms of service).

There are some caveats however at the moment:

• Your silverlight application won't be able to access private blob storage (can't sign the http request)
• Your silverlight application won't be able to access table or queues
• Haven't figured how to control cache yet (not saying it's not possible, I just haven't spent enough time to figure if it is possible)

To be honest I'm not sure I want my silverlight applications to access these services directly. Certainly not at the moment (wouldn't want my shared key in the wild). May'be with a good authentication method this would be attractive.

The big difference between CloudFront and Azure for this type of thing is:

• Content Expiry / Caching
• Terms of Service / Pricing
• Low Latency due to multiple datacenters and good routing

As I said, I think when Azure is out this will be a non issue.

Silverlight: Amazon CloudFront

I’ve been looking at Amazon CloudFront today, and it seems to me that Amazon is aggressively going for the CDN market.

This really is a CDN (Content Delivery Network) solution (and it’s cheap). All it is a method of serving up assets (images, documents, html, even Silverlight) with low latency by using a delivery network (so if your user is in hong kong, the asset will be served from the hong kong server).

The good news is that it is ridiculously cheap. 17 cents per GB data transfer, and 1 cent per 10,000 requests.

Silverlight

It also seems to me (although I haven’t tried it yet), that if you wanted to host your Silverlight application within Amazon CloudFront you could do so. You could have an HTML page (hosted in CloudFront), assets (hosted in cloudfront), silverlight XAP file (hosted in CloudFront), and then you could even allow calls to a back end web service (hosted in azure, or your normal boring old web server) via cross domain policies.

UPDATE: This is confirmed as working, be careful with setting your Content Types, see Tim Heur's comments in the comment section of this post. Thanks Tim.

Azure

I think once Microsoft have setup their many data centers, I would expect a similar type of offering from Azure also. To be honest I don’t think Azure is too far away from this. All Microsoft need to do is provide multiple datacenters, allow us to control the cache for Blob Storage Services, and use the closest data center to server up public url’s. To offer this a separate offering from Azure’s point of view would be wrong (i think).

I think that the competition between Amazon and Microsoft in the cloud space is going to be really great, and will drive down the cost of hosting / scaling.

Azure: Fabric + Fabric Controller

So I've been preparing for my session at DDD7 next week and since I couldn't find any suitable readymade diagrams of the Fabric and the Fabric Controller, I had to create my own.

I thought I would share it with the community at large (feel free to steal it, I happily give up any rights that I would be too ashamed to assert).

• Purple Blob - Fabric Controller
• Blue Blob - Server
• Orange Blob - Virtual Machine
• Yellow Blob - Role
• Green Blob - Agent

Azure: Hotmail should move to Azure

Maybe it’s time Hotmail moved to Azure.

This is the error message I got when I tried to login from messenger

In Azure they would be able to just spin up some more instances to cope with the load :)

Silverlight: (and non Silverlight) PDC Podcast and Interviews

So I was at the PDC this year, and we (myself and John McLoughlin) recorded a podcast and a bunch of interviews for NxtGenUG

We have some of the biggest names being interviewed:

• Brian Keller
• Dan Fernandez
• Martin Gudge
• Chris Anderson
• Don Box

Go check out the interviews:
http://www.nxtgenug.net/Podcasts.aspx?PodcastID=59

There is still some more to be made available in the podcasts, outstanding are (hopefully they will be available soon)

• Daniel Moth
• Shawn Burke (Silverlight Controls Fame)
• Mike Swanson
• Tim Sneath
• Pablo Castro
• The entire Workflow Team (Kenny Wolf included)

Azure: Sometimes I think too literally

In my last post, I complained that I couldn't set the cache in blob storage

It has just occurred to me that the prefix to meta data is occurring in the storage client sample layer.

I wonder if the cache control was set manually in the request if it would be applied to the blob.

I will investigate and let you know on the progress.

Azure: Unable to control cache for Blog Storage

So I've been messing around with the blob service, which is part of Storage Services of Windows Azure.

No Cache

One of the cool things about this service allows you to publish a file (image, document, video) to the service and expose a public uri which you can view in a browser.

The following is an example of a url for an image that i have published to my development fabric blob storage in the images container.

http://127.0.0.1:10000/devstoreaccount1/images/4d963b77-f855-47f3-ab80-593cb4d632ac

Although you can do really cool things such as set the content type, unfortunately you cannot control the cache.

There is a Metadata property bag exposed (NameValueCollection), unfortunately any meta tags that you specify are prefixed with (x-ms-meta-), which means that you cannot set the cache-control manually.

Please allow us to do this as a feature, it would be really useful.

Azure: Reading + Configuring, errgh Configuration

So there is a couple of things you need to do in order to correctly setup configuration in Azure.

Any configuration that you plan to use must be defined in the Service Definition and configured in the Service Configuration.

Service Definition

The following service definition defines that I will I have a configuration setting named "myWebRoleSetting", notice that I don't define the value at this point, just the fact that I will be configuring this value for the service.  This is defined in my Service Definition (ServiceDefinition.csdef)

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="HelloWorldWeb" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
  <WebRole name="WebRole">
    <InputEndpoints>
      <!-- Must use port 80 for http and port 443 for https when running in the cloud -->
      <InputEndpoint name="HttpIn" protocol="http" port="80" />
    </InputEndpoints>
    <ConfigurationSettings>
      <Setting name="myWebRoleSetting"/>
    </ConfigurationSettings>
  </WebRole>
</ServiceDefinition>

Service Configuration

Once I have defined my configuration setting, I need to specify the value to use at runtime.  This is configured in my Serive Configuration file (ServiceConfiguration.cscfg)

<?xml version="1.0"?>
<ServiceConfiguration serviceName="HelloWorldWeb" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
  <Role name="WebRole">
    <Instances count="1"/>
    <ConfigurationSettings>
      <Setting name="myWebRoleSetting" value="Hello World 2"/>
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

Accessing the configuration setting from my application

So once i have defined and configured my configuration setting.  I will obviously want to access within my application.  This is accessed via the RoleManager.

The following sample shows how you access the previously defined configuration setting.

RoleManager.GetConfigurationSetting("myWebRoleSetting")

As said in previous posts, that if you want your application to run both in and out of the Fabric, then you will need to abstract configuration into a common class.

Azure: Why can't I use appSettings

I have to admit I didn't get the point of this at all, until this afternoon.  It's one of these experience things.

So during the PDC (obviously didn't pay enough attention), I took note that in the world of Azure, we were to using the new ConfigurationSettings section instead of appSettings.  Ok I thought, but Why??

Why appSettings are no good?

I thought I'd try a sample that used appSettings, to see if threw up an exception.  I discovered that both in the "Development Fabric" and in "The Fabric", I had no problem using appSettings, so why was I supposed to use this new ConfigurationSettings section, and use the RoleManager to access it?

The answer is pretty simple it turns out, it comes down to changing the appSettings at runtime.

Guess what, you have no access to your web.config / appSettings at run time, and therefore if you want to change the value, then you must deploy a new version of the application.

However at runtime you have access to the the ServiceConfiguration.cscfg file and therefore can make on the fly modifications (as shown below)

This is one of those situations, that if you plan to run your application both in and out of the Fabric, then you should have a Configuration Manager of some sort to allow you to switch between the Azure ConfigurationSettings version and your normal appSettings version.

Azure: Logging in and out of the fabric

This article really ties two articles together.  In this article I talked about discovering whether you are running normally or in the Fabric, and in this second article I talked about logging in Windows Azure

The following line of code is how you write to a log within the Fabric (in azure)

RoleManager.WriteToLog("Critical", "I am a critical error");

If however you try to run this line of code when you are not in the Fabric (e.g. normal asp.net web application), you will get an Object Reference exception.

Therefore having a common logging class which checks whether you are running in the Fabric or running normally would be a pretty useful thing.

And you can achieve that by combining this post and the other 2 posts.

Azure: Invalid application domain name

So I got my invitation code for Windows Azure last night, so I thought I'd deploy my first service to the cloud.

I got a little stuck for a while, I kept getting the error message "Invalid application domain name".  It turns out that "silverlightuk" is not a valid name for my hosted service account.

In the end I went for chrishayuk.

So if you get this error message, you know you have picked a special name, and you need to pick another

Azure: Writing to Logs

So I've been messing around with logging of data when running web applications in the Fabric.

So it is fairly easy to write a message to the log

RoleManager.WriteToLog("Verbose", "I am verbose");

This fairly obviously writes a verbose message to the log.

Types of Logs

The WriteToLog method supports 5 types of logging.

• Critical



• Error



• Warning



• Information



• Verbose

I think I would have preferred however if an enum was used rather than passing it through as a string.

If you pass an invalid log name, then it raise an exception

Default Level of Logging

The default level of logging for a service seems to be Information.  This means that all types of logging messages will be logged except Verbose.

If you want to switch your level of logging to include Verbose, then within the Development Fabric you can just set the level of logging via Tools -> Logging Level Menu

Screenshot of Logs in the Development Fabric

Critical Logs

It turns out that messages written to the critical logs will be notified to you via your preferred notification method.

At the moment my notification method is messenger + email.  I must try that out later

Azure: Am I running in the fabric

So one of the cool things about Windows Azure is that you can take an existing asp.net application, and just run it in the cloud (by providing the ServiceConfiguration and ServiceDefinition config files).

So we know that working in the cloud and working out of the cloud, can require different behaviours from our application.

RoleManager.IsRoleManagerRunning (namespace Microsoft.ServiceHosting.ServiceRuntime) tells us whether our application (role) is in running in the Fabric.

Azure: Running Visual Studio in Administrator Mode

So this is a little annoying, in order to use Visual Studio 2008 and Windows Azure, you need to run Visual Studio 2008 in Administrator Mode.

The reason it needs to do this, is it needs the elevate privileges to start the development fabric.

However if I already have the development fabric running, I would have thought it would be okay to not use administrator mode, nope not the case.

Ho hum, I hope this is something resolved in the future, as it's a little annoying

Azure: Received my Invitation Token

Yay, thank you Microsoft,

My invitation toke arrived for Windows Azure.

Tomorrow night, I will have trying out deployments :)

Azure: Configuring SSL in the local development fabric

If you are wanting to test your website in the cloud against an https endpoint, then there is quite a lot of messing around to get it working.

This is a very cool article which goes through this process with you.

Azure: Spinning up new instances

So I want to test my cloud application and how it will react if I scale my application out.

So by default we run one instance of the service, so if I wish to I can test the number of instances in my development fabric, so I don't get nasty surprises.

To do this all I need to do is modify my Service Configuration file.

ServiceConfiguration.cscfg

To spin up more instances all i need to do is modify the Instances element's count attribute to the number of instances I require.

So in the config below, I have configured the number of instances I am using to 3.

<?xml version="1.0"?>
<ServiceConfiguration serviceName="HelloWorldWeb" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
  <Role name="WebRole">
    <Instances count="3"/>
    <ConfigurationSettings>
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

The really cool thing is that can see (and debug) the multiple instances in my development fragment (and look at the logging).

I've pasted a screenshot below: