JWT and JOSE specifications approved for publication as RFCs

It seems the JSON Web Token (JWT) specs are finally ready to become RFCs. I’ve wrote about security tokens before in the past: it was 2008, XML, SAML and WS-Security were still hot subjects and JWT didn’t existed yet. The more recent “Designing Evolvable Web APIs with ASP.NET” book already includes a discussion of JWT in its security chapter. However, I think this announcement deserves a few more words and a colorful diagram.

A security token is a data structure that holds security related information, during the communication between two parties. For instance, on a distributed authentication scenario a security token may be used to transport the identity claims, asserted by the identity provider, to the consuming relying party.

As a transport container, the security token structure must provide important security properties:

  • Integrity – the consuming party should be able to detect any modifications to the token while in transit between the two parties. This property is usually mandatory, because the token information would be of little use without it
  • Confidentiality – only the authorized receiver should be able to access the contained information. This property isn’t required in all scenarios.

Kerberos tickets, SAML assertions and JSON Web Tokens are all examples of security tokens. Given the available prior art, namely SAML assertions, one may ask what’s the motivation for yet another security token format. JWT tokens where specifically designed to be more compact than the alternatives and also to be URL-safe by default. These two properties are very important for the modern usage scenarios (e.g. OpenID Connect protocol), where tokens are transported in URIs query strings and HTTP headers. Also, JWT tokens use the JavasScript Object Notation (JSON) standard, which seems to be the data interchange format du jour for the Web.

The following diagram presents an example of an encoded token, the contained information and how it relates to the token issuer, the token recipient and and the token subject.

jwt

A JWT is composed by multiple base64url encoded parts, separated by the ‘.’ character. The first part is the header and is composed by a single JSON object. In the example, the object’s properties, also called claims, are:

  • "typ":"JWT" – the token type.
  • "alg":"HS256" – the token protection algorithm, which in this case is only symmetric signature (i.e. message authentication code) using HMAC-SHA-256.

The second part is the payload and is composed by the claim set asserted by the issuer. In the example they are:

  • "iss":"https://issuer.webapibook.net" (issuer) – the issuer identifier.
  • "aud":"https://example.net" (audience) – the intended recipient.
  • "nbf":1376571701 (not before).
  • "exp":1376572001 (expires).
  • "sub":"alice@webapibook.net" (subject) – the claims subject (e.g. the authenticated user).
  • "email":"alice@webapibook.net" (email) – the subject’s email.
  • "name":"Alice" (name) – the subject’s name.

The first five claims (iss to sub) have their syntax and semantics defined by the JWT spec. The remaining two (email and name) are defined by other specs such as OpenID Connect, which rely on the JWT spec.

Finally, the last part is the token signature produced using the HMAC-SHA-256 algorithm. In this example, the token protection only includes integrity verification. However, it is possible to also have confidentiality by using encryption techniques.

The signature and encryption procedures, as well as the available algorithms and the ways to represent key information (e.g. public keys and key metadata) are defined on a set of auxiliary specs produced by the Javascript Object Signing and Encryption (JOSE) IETF working group.

Finally, a reference to the excellent JWT debugger and library list, made available by Auth0.

Recollections on 2014 – the soul of a new book

Last March 11, while waiting for the subway to head home, I received an email from our O’Reilly editor telling us that “Designing Evolvable Web APIs with ASP.NET” had finally gone to print. More than 2 years had passed on a journey that started with an email from Pablo, asking me if I was interested in co-authoring a book on ASP.NET Web API.

designevolvecover

“Designing Evolvable Web APIs with ASP.NET” is the result of the combined knowledge, experience and passion of five authors (Darrel, Glenn, Howard, Pablo and me), with different backgrounds but a common interest for the Web, its architecture and possibilities.

Writing a book with five authors, living in three continents and four time zones is a challenging endeavor. However, it is also an example of what can be accomplished with the cooperation technologies that we currently have available. The book was mostly written using Asciidoc, a textual format similar to Markdown but with added features. A private Git repo associated with a build pipeline was used to share the book source among the authors and create the derived artifacts, such as the PDF and the HTML versions. A GitHub organization was also used to share all the book’s code, which is publicly available at https://github.com/webapibook. For the many conversations and meetings, we used mostly Skype and Google Hangout.

One of my recollections of reading the “C++ Programming Language” book, by B. Stroustrup, almost 20 years ago, is the following quote attributed to Kristen Nygaard: “Programming is understanding”. For me, writing is also understanding. Many afternoons and evenings were spent trying to better grasp sparse and incomplete ideas by turning them into meaningful sequences of sentences and paragraphs. The rewarding feeling of finally being able to write an understandable paragraph made all those struggling hours worthwhile. I really hope the readers will enjoy reading them as much as I did writing them. There were some defeats also. For them, I apologize.

“Designing Evolvable Web APIs with ASP.NET” aims to provide the reader with the knowledge and skills required to build Web APIs that can adapt to change over time. It is divided in three parts.
The first one is composed by four chapters and contains an introduction to the Web architecture, Web APIs and related specs, such as HTTP. It also contains an introduction to the ASP.NET Web API programming model and runtime architecture.

The second and core part of the book addresses the design, implementation and use of an evolvable Web API, based on a concrete example: issue tracking. It contains chapters on problem domain analysis, on media type selection and design, on building and evolving the server and on creating clients.

The third and last part is a detailed description of the ASP.NET Web API technology, addressing subjects such as the HTTP programming model, hosting and OWIN, controllers and routing, client-side programming, model binding and media type formatting, and also testing. It also includes two chapters about Web API security, with an emphasis to the authentication and authorization aspects, namely the OAuth 2.0 Authorization framework.

“Designing Evolvable Web APIs with ASP.NET” is available for purchase at the O’Reilly shop. A late draft is also freely available at O’Reilly Atlas. Also, feel free to drop by our discussion group.

(the title for this post was inspired by the “The Soul of a New Machine” book, authored by Tracy Kidder)

Using HttpClient with SSL/TLS-based client side authentication

The HttpClient is a the new class for making HTTP requests and receiving the associated HTTP responses, introduced in the 4.5 version of .NET. In a previous post  – The new .NET HttpClient class – we described its the core concepts and architecture. One of the main takeaways was the concept of client handler, which is just a message handler responsible for delivering the HTTP messages to the network stack and receiving the responses from it. Typically, the client handler will be at the end of the message handler pipeline used by the client.

The current’s post goal is to describe how to configure this new HTTP client class with client-side authentication based on the TLS (Transport Layer Security) protocol.

The HttpClient class does not contain any configuration properties or methods related to TLS. This is because the HttpClient class is independent of the used HTTP message transport mechanism. Namely, it is even possible to connect a HttpClient directly to an HttpServer instance, without any network intervention, as described in another post.

Instead, the TLS configuration requires dealing directly with one of the available client handlers presented in the previous post : HttpClientHandler and WebRequestHandler.

The first option is to explicitly configure the HttpClient with a HttpClientHandler instance, containing its ClientCertificateOptions property set to Automatic.

The resulting HttpClient can then be used normally: if during a connection handshake the server requires the client certificate, the HttpClientHandler instance will automatically select a compatible client certificate for the user’s personal certificate store.

var client = new HttpClient(

 new HttpClientHandler{
   ClientCertificateOptions = ClientCertificateOption.Automatic
 });
// ...

This option is the only one available for Windows Store applications.

For classical scenarios (e.g. console, WinForms or WPF applications) there is a second option using the WebRequestHandler, which provides more control over the configuration.

var clientHandler = new WebRequestHandler()
clientHandler.ClientCertificates.Add(cert);
var client = new HttpClient(clientHandler)

where cert is a X509Certificate2 instance representing the client certificate.
This instance can be constructed directly from a PFX file or obtained from a Windows certificate store

X509Store store = null;
try
{
  store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
  store.Open(OpenFlags.OpenExistingOnly | OpenFlags.ReadOnly);
  // select the certificate from store.Certificates ...
}
finally
{
  if(store != null) store.Close();
}

And that’s it. Hope it helps.

JSON Web Tokens and the new JWTSecurityTokenHandler class

Last week, Vittorio Bertocci announced the developer preview of the new JWT Security Token Handler, which provides support for a important piece of the modern identity and access control management puzzle.

What are security tokens?

In the context of the claims model, a security token is an interoperable  container of security-related information, typically identity claims, securely packaged for communication between two or more parties. This packaging ensures properties such as:

  • Confidentiality – only the authorized receiver should be able to access the contained information
  • Integrity – the authorized receiver should be able to detect any modifications to token, while in transit between the two parties.

On Web Single-Sign On protocols, security tokens are used to securely transport the identity information from the identity provider to the identity consumer. On a delegated authorization protocol, such as OAuth 2.0, security tokens can be used to convey the authorization information from the client to the resource server.

For instance, the SAML (Security Assertion Markup Language) assertion is an example of a very popular token format, used by  Single-Sign On protocols such as: Shibboleth, the SAML protocols and WS-Federation. SAML assertions are XML-based and use the XML Digital Signature and XML Encryption standards for providing integrity and confidentiality.

What is JWT?

JWT stands for  JSON Web Token, and is a new format for packing and protecting security information. It is based on the JSON (JavaScript Object Notation) syntax and aims to be usable in “space constrained environments such as HTTP Authorization headers and URI query parameters”.

The following example (taken from the spec), represents a unprotected token (line breaks added for display purposes)

eyJhbGciOiJub25lIn0
.
eyJpc3MiOiJqb2UiLA0KICJleHAiOjEzMDA4MTkzODAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ

 

The encoded token is composed by a sequence of parts, separated by the ‘.’ character. Each part is the base64url encoding of an octet stream. In this example, both octet stream result from the UTF-8 encoding of JSON objects. The first object (encoded in the first part) is the JWT header

{“alg”:”none”}

The header defines the token cryptographic protection, which is ‘none’ in this case.

The second object (encoded in the second part) is the JWT Claims Set

{“iss”:”joe”,

“exp”:1300819380,

http://example.com/is_root”:true}

The JWT Claims Set object is a container of claims, where the object’s property corresponds to the claim type and the property’s value contains the claims value. Some claims types are defined by the JWT spec (e.g. “iss” and “exp”), while others are context specific (e.g. “http://example.com/is_root”).

We will see more examples of JWT tokens after presenting the JWT Security Token Handler.

What are Security Token Handlers?

Security Token Handlers are a concept introduced by the  WIF (Windows Identity Foundation) framework, which is now an integral part of the .NET 4.5 framework. A token handler has multiple responsibilities, namely:

  • Serialize and deserialize tokens between a XML or string format and a SecurityToken-derived instance;
  • Validate security tokens and extract the contained claims into a claims identity or a claims principal;
  • Create a token from a token description.

This behavior is defined by the abstract SecurityTokenHandler class, with multiple concrete derived classes for each token type (e.g. the Saml2SecurityTokenHandler class).

The JTWSecurityTokenHandler class

The recently announced Microsoft.IdentityModel.Tokens.JWT NuGet package contains a new token handler for the JWT token format – the JWTSecurityTokenHandler class – depicted in the next diagram.

TokenHandlers

As a token handler,  the JWTSecurityTokenHandler can be used to create and validate JWT tokens, as shown by the next example

 

 

  • First, we create a token handler and a symmetric key, that will be used by both the sending and the receiving party to sign and validate the token, respectively.
  • Then, we create a token descriptor, defining the token contents, namely:
    • The contained claims, i.e., the subject of the token,
    • The token issuer name,
    • The intended recipient of the token (AppliesToAddress),
    • The token lifetime, defined by a not before and an expires date-time,
  • The token descriptor also contains the signing credentials, namely the symmetric key and the MAC (Message Authentication Code) algorithm identifier (“http://www.w3.org/2001/04/xmldsig-more#hmac-sha256”).
  • Then, we use the token handler to create the token from the token descriptor. We also use the token handler to serialize the token into a string.
  • On the receiving side, we begin by defining the validation parameters, namely:
    • The allowed audience, i.e., the value defined in the AppliestoAddress property of the token descriptor. This value should be an identifier of the receiving party.
    • The validation cryptographic, in the form of a BinarySecretSecurityToken containing the shared symmetric key.
    • The name of the accepted issuer.
  • Finally, we use the token handler to simultaneously deserialize the token, validate its signature and extract the contained claims into a claims principal.
  • We end the example by asserting that the claims principal does contains the Name and Role claims inserted in the token by the issuer.

 

The serialized token is just the concatenation of three base64url encoded parts (line breaks added for display purposes).

 

eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9

.

eyJhdWQiOiJodHRwOi8vd3d3LmV4YW1wbGUuY29tIiwiaXNzIjoic2VsZiIsIm5iZiI6MTM1Mzk3NDczNiwi

ZXhwIjoxMzUzOTc0ODU2LCJodHRwOi8vc2NoZW1hcy54bWxzb2FwLm9yZy93cy8yMDA1LzA1L2lkZW50aXR5

L2NsYWltcy9uYW1lIjoiUGVkcm8iLCJodHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL3dzLzIwMDgvMDYv

aWRlbnRpdHkvY2xhaW1zL3JvbGUiOiJBdXRob3IifQ

.

a-Tu5ojQSyiGSzTb9E5QbEYxyhomywzh2wqKs4El7lc

 

The first part contains the JWT Header

{“typ”:”JWT”,”alg”:”HS256″}

The second part contains the claims, including the audience, issuer and validity

{“aud”:”http://www.example.com”,

“iss”:”self”,”nbf”:1353973142,”exp”:1353973262,

http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name“:”Pedro”,

http://schemas.microsoft.com/ws/2008/06/identity/claims/role”:”Author”}

Finally, the third part is the signature value, computed by the MAC algorithm over the first two parts.

ASP.NET Web API: Creating an Host using Azure Service Bus

In the last posts, I’ve presented the new ASP.NET Web API processing architecture and described three different hosting capabilities, supported “out of the box”: web hosting, in-memory hosting and self-hosting.

In this post, I will describe the development of a custom host using the Azure Service Bus relaying capabilities. This new host enables the exposure of a Web API on the public cloud, while running on a private machine (e.g. my laptop), that is, a machine without inbound connectivity (e.g. private addresses, firewall, NAT).

ServiceBusRelay

This host design is inspired in the self-host architecture, namely the usage of WCF and its integration with the service bus and WCF. Is composed by the following main components, shown in the following diagram.

 

 

ServiceBusHosting

 

  • The HttpServiceBusConfiguration class derives from HttpConfiguration and adds a couple of properties specific to the this scenario, such as the bus authentication credentials (IssuerName and IssuerSecret).
  • The HttpServiceBusServer is initialized with a HttpServiceBusConfiguration and internally performs the following:
  • When a request message is received, the WCF runtime delivers it to the DispatcherService, containing generic asynchronous operations to handle GET requests (BeginGet and EndGet methods) and other HTTP methods (BeginInvoke and EndInvoke).
    [ServiceContract]
    [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)]
    internal class DispatcherService
    {
        private readonly HttpServer _server;
        private readonly HttpServiceBusConfiguration _config;

        public DispatcherService(HttpServer server, HttpServiceBusConfiguration config)
        {
            _server = server;
            _config = config;
        }

        [WebGet(UriTemplate = "*")]
        [OperationContract(AsyncPattern = true)]
        public IAsyncResult BeginGet(AsyncCallback callback, object state)
        {
            var context = WebOperationContext.Current;
            return DispatchToHttpServer(context.IncomingRequest, null, 
				context.OutgoingResponse, _config.BufferRequestContent, callback, state);
        }

        public Message EndGet(IAsyncResult ar)
        {
            var t = ar as Task;
            var stream = t.Result;
            return StreamMessageHelper.CreateMessage(MessageVersion.None, "GETRESPONSE", stream ?? new MemoryStream());
        }

        [WebInvoke(UriTemplate = "*", Method = "*")]
        [OperationContract(AsyncPattern = true)]
        public IAsyncResult BeginInvoke(Stream s, AsyncCallback callback, object state)
        {
            var context = WebOperationContext.Current;
            return DispatchToHttpServer(context.IncomingRequest, s, 
				context.OutgoingResponse, _config.BufferRequestContent, callback, state);
        }

        public Message EndInvoke(IAsyncResult ar)
        {
            var t = ar as Task;
            var stream = t.Result;
            return StreamMessageHelper.CreateMessage(MessageVersion.None, "GETRESPONSE", stream ?? new MemoryStream());
        }
		...
    }

 

  • These generic operations convert the WCF requests, represented by the older IncomingWebRequestContext class, into instances of the new HttpRequestMessage class. Then, they pushe these messages into the HttpServer pipeline. When the server finally returns the responses’ HttpResponseMessage instances, the generic operations convert them back into WCF messages.

The code, still in alpha/”works in my machine” status, is available from https://github.com/pmhsfelix/WebApi.Explorations.ServiceBusRelayHost.

Feedback is appreciated.

ASP.NET Web API: self-hosting

In the last posts, I’ve been exploring the new ASP.NET Web API Beta architecture. I started by the high-level processing architecture, then described web hosting and in-memory hosting, i.e., directly connecting a client to the server without going through the network.

This post describes a third “out of the box” hosting option: self-hosting. The following code excerpt exemplifies the usage of the HttpSelfHostServer class to host a server on a console application.

    var config = new HttpSelfHostConfiguration("http://localhost:8080");
    config.Routes.MapHttpRoute("default", "{controller}/{id}", 
                                    new { id = RouteParameter.Optional });
    var server = new HttpSelfHostServer(config);
    server.OpenAsync().Wait();
    Console.WriteLine("Server is opened");

 

The HttpSelfHostServer class derives from HttpServer and and is configured by a HttpSelfHostConfiguration instance, as shown in the following diagram.

SelfHosting

 

Internally, the HttpSelfHostServer uses a WCF channel stack layer to obtain messages from the transport medium and then pushes them into the upper message handler pipeline.

The following section briefly presents the WCF high-level architecture, setting the ground for the description of Web API self-hosting characteristics.

WCF architecture

The WCF architecture is divided into two layers: the channel stack layer and the service model layer, as depicted in the following diagram.

WcfArchitecture

The lower channel stack layer is composed by a stack of channels and behaves similarly to a classical network protocol stack.  The channels are divided into two types: transport channels and protocol channels. Transport channels are responsible by the interface with the transport medium (e.g. TCP, MSMQ, HTTP) (yes, I know, HTTP is not a transport protocol), namely by receiving and sending messages. Protocol channels process the messages that flow up and down through the stack. A typical use case for a protocol channel is the addition of digital signatures at the sending side and the verification of those signatures at the receiving side. The transport channels use encoders to convert between the transport medium byte streams and message instances.

The upper service model layer performs the interface between the messages and methods calls, dealing with tasks such as:

  • transforming a received message into a parameter sequence;
  • obtaining the service instance to use;
  • selecting the method to call;
  • obtaining the thread where to call the method.

However, the HttpSelfHostServer doesn’t use the service model layer. Instead, it directly consumes the messages retrieved from channel stack layer.

The concrete channel stack layer organization is described by bindings, as presented in the following diagram.

WcfBindingElementsChannels

A binding is a ordered collection of binding elements, where each element roughly describes one channel or encoder. The first binding element describes the upper channel and the last element describes the lower channel, which is always a transport channel.

 

The HttpSelfHostServer and HttpSelfHostConfiguration classes

Internally, the HttpSelfHostserver.OpenAsync method creates and configures a HttpBinding instance, based on the HttpSelfHostConfiguration instance properties. Then it uses this binding to asynchronously create a WCF channel stack. It also creates a pump that pulls messages from this channel stack, converts them into HttpRequestMessage instances and pushes these new requests into the HttpServer, that is, the message handler pipeline.

SelfHostingDiagram

When self-hosting, most of the WCF HTTP binding capabilities and settings are available. The configuration of the internally created HttpBinding instance can be accomplished in two ways. The first is to use HttpSelfHostConfiguration properties, such as MaxBufferSize and TransferMode, that will be used to configure the internal HttpBinding instance. The second is to create a HttpSelfHostConfiguration derived class and override the OnConfigureBinding method, which receives the internally created binding instance. This method has the opportunity to change the binding settings before it is used to create the channel stack.

SelfHostingDetail

In a future post, I will show how to create a custom host. Until then, comments are welcomed.