I've been very pleased with the positive energy circling around Microsoft recently. I wanted to take a moment to share some information about the great work done by the Azure SDK team in releasing v1.0 of the Microsoft Azure Management Libraries for .NET.

The Microsoft Azure Management Libraries (MAML) are a collection of REST wrapper assemblies that compose together with a common library and shared lightweight dependencies to offer service automation, management and software-as-a-service capabilities to Azure developers.

With this release, REST management libraries are now available for a majority of today's Azure services. The libraries are straightforward, easy to use, and approachable: they are great building blocks.

Shipped via NuGet, automating Azure is now as easy as installing the package(s) for the services you want to work with, hooking up the necessary credentials, and creating clients using the centralized CloudContext type.

The .NET libraries embrace the expectations of modern developers and are fully async/await, portable class libraries with platform light-up, use Semantice Versioning (SemVer), have very few dependencies (HttpClient and Json.NET), are open source on GitHub, and ship with useful IntelliSense documentation and symbol/source packages for easy debugging.


I'd strongly recommend that you take a moment to open Brady Gaster's post about the release in another tab: Announcing the General Availability of the Microsoft Azure Management Libraries for .NET. He also gave a one hour talk on Azure automation at Build 2014 that is worth a watch.

Please join me in congratulating the SDK engineering team in this important 1.0 milestone. Do share your feedback with the team and go build something awesome!

A battle-tested investment in the future

The underlying technology developed as part of this effort has great potential for keeping Azure libraries much more current across a large swatch of supported cloud development languages.

The management libraries are built from a service definition specification using a sophisticated code generator that takes into account each target platform's ideal libraries and patterns for asynchronous coding, REST communication, JSON and XML serialization and deserialization, tracing and pipelining, and more.

The team has already released libraries for Node.js, is working hard on a Java solution, and some day will be able to simultaneously release updates across languages and services very rapidly.

A small surprise: although the 1.0 General Availability (GA) has just happened, MAML has been powering the Azure PowerShell experience since October 2013's v0.7.0 release. We did call the libraries WAML back then, because Microsoft's public cloud had a different name at the time.

The old management code was replaced with the new MAML code and has shipped in 7 releases since then with minimal regressions.

To make an automotive comparison, the team did a great job of essentially ripping out the engine of a running diesel car and replacing it with a sweet new electric motor.

Building the Azure PowerShell Cmdlets on top of MAML has been a great success. I still remember the joy the team had when we were approving a commit that removed over 125,000 lines of code with the move to MAML.


Node.js (JavaScript) libraries are also being created from the same specifications and power much of the azure cross-platform command line interface for Mac, Windows and Linux. Using our codegen technology and the same service specs helps to drive down defects and improve the team's engineering efficiency.

Azure's REST Management APIs

Automating Azure has been possible for years but never very fun. The MSDN Service Management REST documentation is rather complete, but as more services and releases come online across the huge organization, the docs have become less easy to grok. We're aware of times when new functionality has shipped but the documentation has lagged. We have to do better.

I still keep the MSDN page in my favorites while using MAML, but it is a pleasure to no longer have to hand-roll tedious REST calls. We've incorporated a lot of the MSDN docs into the inline IntelliSense documentation in the .NET libraries.

When you look at an Azure REST API on the wire you can identify where it lives in today's world:

• a front-end management APIs that flow through the red dog front-end (RDFE)
• a separate service endpoints

MAML focuses on the RDFE and a few service endpoints for specific clients.

RDFE: Service Management endpoints

RDFE is the front end to Azure's services and powers authentication, service provisioning, deprovisioning, configuration, and other operations related to billing operations.

Today these APIs live at https://management.core.windows.net for public Azure. RDFE is also a part of the on-premise Azure Pack and other clouds such as Windows Azure in China with different endpoints.

Authentication with RDFE is performed using client management certificates for security, although a bearer-token alternative is now available thanks to the power of Azure Active Directory.

Service-specific endpoints

Separately, there are service APIs, and they often live off of slightly different URLs: the storage service is a good example, where you'll find that you can provision storage accounts within a subscription using management APIs, but then actually dealing with blobs, tables and queues, you interact with different endpoints.

When you use Azure's Storage Service, for example to store a blob, you actually use a set of storage-specific credentials and then communicate with the blob service through the https://MYACCOUNT.blob.core.windows.net, where MYACCOUNT is the name of your storage account.

Resource Management APIs: Tomorrow's Service Management endpoints

Announced at Build, there are new automation and management endpoints coming online for Azure services this year. Available in preview form now and accessible through the Azure Preview Portal, the APIs will be much more consistent. A great deal of effort has gone into defining a more consistent, friendly set of REST APIs for the future, and we'll have plenty more to share here in the future as we look to offer newer experiences beyond RDFE's management surface area.

Azure Management wherever you want it

I am excited by the emphasis on allowing for management wherever you like:

• the Azure portal,
• REST APIs,
• PowerShell
• cross-platform command line tools,
• Visual Studio's Azure tooling,
• the new preview Azure management and automation services,
• and tools, apps and services you build with MAML or the REST APIs!

Several teams within Azure have had service management implementations for .NET for many years now, powering internal test infrastructure, Azure services and tools, even parts of the developer tooling, but for this project the teams came together across Azure to work to deliver something more polished, focusing on newer APIs, programming practices and design.

Let's look at the technology, decisions and features that make the Microsoft Azure Management Libraries great.

Best practices - modern .NET design

.NET development in 2014 is very different and the management libraries fully embrace the more modern developer ecosystem:

• fully embracing NuGet
• portable class libraries
• semantic versioning
• async/await
• only dependent on BCL packages, BCL HttpClient and Json.NET
• open source on GitHub w/Apache 2 license
• extensibility hooks for diagnostics, debugging and tracing
• central HTTP pipeline handler design using the BCL's portable HttpClient
• secure by default with HTTPS

I'm very happy about the design effort that went into the libraries as well as the general approach. The guidance created as part of this effort should help to continue to blur the lines between the dozens of service teams and approaches within the organization.

Some of the design features and principles that went into this product are noted in this section.

Portable Class Libraries (Profile158)

Each individual management library that makes up MAML is built for the portable-net45+sl50+wp80+win (Profile158) portable class library target, and for a limited time, also .NET 4.0.

The PCL target enables the most broad use across areas of interest to today's developers as we continue to see strong adoption of .NET Framework 4.5 and newer, as well as Windows Phone 8 and the recently-announced Windows Phone 8.1 universal platform.

Asynchronous-first: Await/Async/Task (AAT) design

.NET Framework 4.5 introduced the await/async asychronous programming model and today the async pattern and compiler support is even available in portal class libraries and earlier frameworks using the Microsoft.Bcl.Async NuGet package that supports .NET 4, Silverlight 4-5, Windows Phone 7.5 and newer, and PCL.

All of the management libraries are async first and by default.

Extension methods for common scenarios

We also bundle extension methods for all of the service methods to make the lives of our developers that much easier.

Simple async method: Although the core implementation of a method is its asynchronous method that takes a CancellationToken parameter, we know that many developers do not need to use the cancellation token system very often, and so the additional parameter is a nuisance.

An extension method is available that does not have the cancellation token parameter, passing CancellationToken.None automatically.

Synchronous method: Although modern client developers should always be using the async/await system in their .NET and Windows Store applications, we know that a lot of people are building simple command line tools, using PowerShell, or other scenarios where a synchronous call is actually desired for quick and dirty work.

We generate a synchronous extension method as well for these users. Please do not use them in modern client applications!

Long-running operation support

Some Azure service management APIs are actually long-running operations: calling the REST API begins the operation, returning a operation identifier that can be then polled for information about the operation.

MAML supporst these long-running operations and offers easy methods that will return after the entire LRO is complete, as well as the building blocks, such as BeginDeleting, that can then be used with your own smarter logic to do runtime polling and operation management.

A good example API of this it the Virtual Machine Delete operation.

Platform-specific light-up

Although individual libraries are PCL, the Common library and its NuGet package provide a base Common library which is PCL and then additional platform-specific light-up libraries.

Shipping in the NuGet today are light-up components for Windows Store apps as well as Windows Phone 8 apps.

Unfortunately the PCL surface area for Windows Phone 8 does not include the X.509 Certificate capabilities that .NET Framework 4.5 does; the X509Certificate2 type therefore cannot be used. This means that RDFE management certificate (the majority of today's management uses) won't be able to be used or called from phone apps.

Brady has a post on Windows Phone 8 and MAML that offers some suggestions to working around these challenges, such as using OAuth and Azure Active Directory.

Professionally-designed API surface area

We've taken the time to project many of the REST endpoint specifications into more useful .NET libraries. Our first priority is to offer simple REST wrappers as opposed to "fat" client libraries, but we can do some peanut-buttering over some pain points to make the library as friendly as possible.

Taking into account the .NET Framework Naming Guidelines, we've really studied all our options to build a great .NET experience and then map to the underlying wire protocol defined by the REST spec.

Known Values static string helpers

There are places that we have had to make trade-offs: instead of offering a rich enumeration for all the possible virtual machine sizes, and taking that enum as a parameter in the virtual machine creation process, we've instead had to expose just a string for the virtual machine instance size.

This is to enable the broad set of scenarios that this area has in the real world: as we introduce additional compute sizes, experiment with specialized hardware and other specific opt-in projects for customers, we need to use the string parameter for maximum flexibility instead.

We then ship a "known values" static class helper, though, for the most simple scenarios: if you are using the Compute library and have included its management library namespace, you can then type VirtualMachineSizes and be greeted with an enum-like object exposing known common sizes like ExtraLarge and A7, returning the string value.

You'll find that MAML libraries for other languages, like Node.js, will be generated to look native and properly named and patterned for those platforms, part of the excellent generation system.

IntelliSense

While working to build the right .NET surface area, we've also made sure to have top-notch documentation available through IntelliSense. Working with the MSDN documentation teams and our staff writers, we've been able to include a lot of useful service information that will pop right inside VS as you are using the library.


Operation Groups

Having IntelliSense is an important part of the developer experience, but so is reducing the cluster and noise that many REST libraries tend to have.

By placing types of service and management operations into "operation groups", client instances are much more useful and fun to use in an IDE.


A basic library like the infrastructure management library (Microsoft.WindowsAzure.Management) has these groups:

• AffinityGroups
• Locations
• ManagementCertificates
• RoleSizes
• Subscriptions

A client such as Compute will have these logical groups:

• Deployments
• HostedServices
• OperatingSystems
• ServiceCertificates
• VirtualMachineDisks
• VirtualMachineExtensions
• VirtualMachineOSImages
• VirtualMachines
• VirtualMachineVMImages

Expanding the OperatingSystems operation group will give you all of these methods:

Method Name	Parameters	Async/Await?	Implementation Detail
List	None	Synchronous	Extension method
ListAsync	None	Yes	Extension method
ListAsync	CancellationToken	Synchronous	Method
ListFamilies	None	Synchronous	Extension method
ListFamiliesAsync	None	Yes	Extension method
ListFamiliesAsync	CancellationToken	Yes	Method

Semantic Versioning

Each component in the set of libraries is free to independently version, following the SemVer pattern.

During the preview period for these assemblies, when the Azure compute and fabric teams added new high-memory virtual machine sizes (A6, A7), the addition of the new sizes to the libraries was an incremental minor feature, so the increment did not alter the major or minor versions of that assembly at release time.

If the Compute team were to introduce a major new feature without any breaking changes, they would increment the minor version, moving from 1.0 to 1.1, and effectively letting people know that it is relatively safe to update the NuGet packages in their projects, as the work is non-breaking and iterative.

When more libraries come online for the next generation of Azure APIs for management, you will likely see these independent components move into v2.x and beyond, given there will be breaking changes in many cases.

Rest easy knowing that the teams will be rapidly iterating and following SemVer to make sure that you can trust NuGet updates. The open source releases are tagged as well offering the ability to see what changes and differences are in releases.

Composable component libraries: independent service packages

Azure has many services and more are being added regularly. Instead of building all of these services into a single large assembly, we ship many packages, giving developers the power to opt-in to which components they need to work with.

This helps reduce the memory footprint, performance, and enables Azure service teams to iteratively innovate, hotfix and release their work independent of other teams and their schedules.

This is much more efficient than having to wait for an Azure-wide update like you might be used to seeing with the primary Azure SDK for .NET that tends to ship alongside major developer events.


We also expect that more services and components will begin to leverage the management library engineering system instead of rolling their own. Within Azure itself teams are now able to use these libraries, improving engineering efficiency, and making sure that our developer customers use the same essential bits that we do in running our cloud-scale products & services.

CloudContext for centralized client discoverability

As a result of shipping a large number of libraries and offering a "pay-for-play" model when it comes to selecting assemblies to use in a project, we've created a discovery mechanism powered by extension methods to identify the available clients within an application, again making IntelliSense development much more enjoyable.

The CloudContext type is located inside the Common library and offers centralized services, namely a static CloudContext.Clients method & type that loaded libraries expose extension methods for creation into.

If you've added, for example, only the NuGet package for Web Sites Management to your project, and no other packages, you will have only its 2 client types available to you with IntelliSense:


The Web Site management package includes a single portable assembly that has 2 clients: a general WebSiteManagementClient and then a WebSiteExtensionsClient: one for core management (creating, deleting and configuring Azure Web Sites) and another for interacting with the extensions service.

If we instead go ahead and add the uber-package Microsoft.WindowsAzure.Management.Libraries, it will pull in all the other released packages that make up MAML 1.0.

If you again take a look in the IDE at CloudContext.Clients, you will see many more clients available with handy creation methods:


CloudContext also is a good place for general configuration and extensions, so it is where we have exposed the ability to dynamically grab and parse connection strings using Service Runtime.

Modern and minimal package dependencies

Dependency management is a real nightmare in complicated applications. We've worked to offer a very minimal spanning set of dependencies that all the management libraries use.

Teams that decide to offer more full-featured packages can either take a dependency on the basic REST wrapper library from the MAML group, add additional dependencies, and then ship that component as a separate NuGet package. This will help to keep the most basic REST surface area completely available to most applications without having to place the dependency management game.

These core dependencies today are:

HttpClient from the BCL team

By using the BCL team's HttpClient implementation, libraries are able to span multiple platforms and build on the good work the team did to support the set of Microsoft frameworks. Their NuGet package supports PCL and many platforms with the modern async/await HttpClient methods.

This is dependency #1 of 2 for the MAML (although it pulls in the good BCL packages for async/await and enabling build support).

Json.NET

The Json.NET component is a favorite of developers across all the .NET frameworks today and is where we have chosen to depend for the MAML libraries when it comes to JSON parsing.

The component's developer, James Newton-King, is a great member of the community and we are proud to be able to use his important library in lighting up Azure management.

Smart HTTP retry handler

The Patterns and Practices team at Microsoft has done excellent work in supporting real customers & enterprises on Azure. We've been able to work together to deliver a built-in TransientFaultHandling system that has a smart retry policy turned on by default.

Of course you can build your own fault handling and retry policies, but the default experience is fantastic and should meet the needs of most developers. The entire HttpClient pipeline is available for your own handler improvements to be integrated. This is also super useful for unit testing.

Flexible Authentication Choices

Various credential providers ship in both the Common library and various platform-specific light-ups.

When you're using .NET Framework 4.5 or newer, after including the Common NuGet package you'll see references to both Microsoft.WindowsAzure.Common and Microsoft.WindowsAzure.Common.NetFramework. The .NET-specific assembly includes the CertificateCloudCredentials type that takes a X509Certificate2 management certificate and a subscription ID string.

Other credential types are in Common including TokenCloudCredentials for use with Azure Active Directory, AnonymousCloudCredentials and then BasicAuthenticationCloudCredentials for specialized uses in the right APIs.

Brady Gaster has a post about using Azure Active Directory to authenticate with and use the management libraries, and I'm sure the team will work to continue to drive improvements in this space over time.

Extension/light-up packages for logging and more

You'll find that components such as an extension for log4net, supporting Windows Store applications, an ETW tracing provider are all additive and unique NuGet packages, making great new functionality a click away inside the NuGet Package Manager, keeping simple apps light and robust.

By packaging extensions as separate packages, we again adhere to the opt-in, pay-for-play model, while allowing such extensions to then further depend on other systems that not every developer would want to pull in.

We look to you to help provide input on which additional extensions we should build, and would love your open source contributions as well.

Support for service wrapper libraries, too

Although MAML is focused mostly on management surface area, the technology can power general REST libraries for Microsoft Azure as well.

Microsoft Azure Scheduler allows you to invoke actions such as calling web endpoints or posting a message to a storage queue on any schedule. If you install the Scheduler management library, you'll find it contains 2 clients: a SchedulerClient and a SchedulerManagementClient. The management library allows you to create job collections and then the actual SchedulerClient lets you manage the jobs within that collection.

Over time we want to support more and more Azure services as proper MAML components, using the same consistent API patterns, Common library, and multi-language support.

A commitment to minimal Common library changes

The Common library is the core component that all of the service team's management libraries depend on. It is so important that it should not be updated or broken very often. It contains the core hooks to add tracing, diagnostics, HTTP pipeline code, smart retry policies, and the CloudContext type.

The team is committed to try and minimize breaking changes to the Common library, ideally taking incremental work (SemVer minor version changes) but not breaking changes more than once a year if possible. This isn't a SLA commitment but rather a goal that the team has. Clearly hotfixes, security issues and other scenarios might lead to a newer release at some point in the future.

Note that the v2.0 of the Common assembly will likely ship sooner, as the next generation of Resource Management APIs (announced at Build) will likely move the management libraries forward to v2.0. I'm sure the team will have further guidance in this space in the future.

Supported REST services

Many components have now shipped at the 1.0 GA quality bar, and others are in preview. Here's a look at the release packages as of early April.

NuGet Package	Clients	Status
Microsoft.WindowsAzure.Management.Libraries	All current GA+ clients	1.0
Microsoft.WindowsAzure.Management	ManagementClient, SubscriptionClient	1.0
Microsoft.WindowsAzure.Management.Network	NetworkManagementClient	1.0
Microsoft.WindowsAzure.Management.Compute	ComputeManagementClient	1.0
Microsoft.WindowsAzure.Management.ExpressRoute	ExpressRouteManagementClient	Preview
Microsoft.WindowsAzure.Management.MediaServices	MediaServicesManagementClient	1.0
Microsoft.WindowsAzure.Management.Monitoring	AlertsClient, AutoscaleClient, MetricClient	Preview
Microsoft.WindowsAzure.Management.Scheduler	Scheduler, SchedulerManagement	1.0
Microsoft.WindowsAzure.Management.ServiceBus	ServiceBusManagement	Preview
Microsoft.WindowsAzure.Management.Storage	StorageManagementClient	1.0
Microsoft.WindowsAzure.Management.Store	StoreManagementClient	Preview
Microsoft.WindowsAzure.Management.Sql	SqlManagementClient	Preview
Microsoft.WindowsAzure.Management.WebSites	WebSiteManagement, WebSiteExtensions	1.0

Supporting Azure Pack and other environments

Each client has a constructor that by default tends to just take a credential instance that will contain information about your subscription ID, the management certificate used to communicate over secured client HTTPS (or an Azure Active Directory bearer token that is in a properly-configured domain environment for service management).

The constructors also offer an optional additional parameter to provide the Base URI to the management service surface area.

This enables using the libraries with other environments, where supported, such as the Azure Pack or the Microsoft Azure China cloud (or, in the case of Microsoft internal service operations teams, the ability to target staging and test environments and clusters).

Support across key languages

Azure has to be the best platform for Microsoft developers, so of course we take great pride in this new set of .NET libraries; but it's a new world and the only way that we're going to be successful as a cloud platform is if we also take the time to fully support the other languages and frameworks that our developers are asking for.

Thankfully Azure is very serious about Java, Node.js, and many other languages like Python, Ruby and PHP.

On the SDK team there's been a real excitement over Node.js as it has enabled the great cross-platform tooling, and so today there are MAML-style libraries for .NET and Node.js, with more on the way in the future I hope!

Open Source

Like all of the Azure SDK components we have been introducing over the past few years, we are continuing to maintain the open source story here:

• Apache 2 license
• Ship on GitHub at https://github.com/azure/
• We accept contributions, especially to extension libraries and common components
• Part of the .NET Foundation

How to best contribute today

If you'd like to contribute to the Common libraries, build extensions such as the tracing components, we welcome your contributions per the standard Azure open source code contribution guidelines.

Until the code generation technology can be open sourced entirely, please consider opening Issues in GitHub if you think you find any typos, REST call bugs, etc. Submitting code changes to files marked with the header indicating code generation is not ideal.

For Microsoft Azure teams

If you're like to learn more about the MAML technologies and learn about onboarding, please visit the Azure SDK SharePoint.

If you read this far...

I hope you love this new set of libraries. Let us know what you build! Install the Microsoft.WindowsAzure.Management.Libraries NuGet uber package today.