WARNING: This server provides a static reference view of the NetKernel documentation. Links to dynamic content do not work. For the best experience we recommend you install NetKernel and view the documentation in the live system .

Design Overview

Broadly speaking the RESTOverlay is an endpoint routing switch for HTTP REST services/applications.

For a given internal resource request from the HTTPBridge, it will select and invoke a target endpoint based upon a best match with the external HTTP client's expressed preferences and the set of internal target endpoints.

Target endpoints declare their capabilities by specifying user-supplied endpoint metadata. This is ROC metadata and can loosely be considered as a richer generalisation of Java annotations.

Features

The RESTOverlay supports the following capabilities...

  • flexible simple grammars
  • dynamic content negotiation
  • dynamic language negotiation,
  • dynamic compression negotiation
  • HTTP method routing
  • automatic Etag generation
  • automatic 404/406 handlers
  • automatic exception handling - global or fine-grained
  • targeted pre-processes
  • lazily evaluated functional wrapping of target endpoints

The RESTOverlay manages the ROC spacial scope so that the caching semantics of the responses of targeted endpoints are conserved.

Architecture

The RESTOverlay is a companion to the HTTPBridge and must be implemented downstream of the HTTPBridge since it interacts with the httpRequest:/ and httpResponse:/ resource sets.

A schematic diagram of where the RESTOverlay sits and how it routes requests to target endpoints is shown below...

Instantiation

The RESTOverlay is instantiated from the prototype found in the urn:org:netkernel:tpt:http space. The broad structure of an instance will look like this...

<rootspace>
  <overlay>
    <prototype>RESTOverlay</prototype>
    <config>
      <basepath>/somepath/</basepath>
    </config>
    <space>
      < !-- Space containing target endpoints with suitable REST metadata -->
    </space>
  </overlay>
  <import>
    <private />
    <uri>urn:org:netkernel:tpt:http</uri>
  </import>
</rootspace>

The space contained within the RESTOverlay is the location where the managed REST endpoints should be present.

Configuration

The RESTOverlay requires a parameter with the following values

  • mandatory the basepath from which managed rest services will be offset. All paths must start with "/". If the root is to be the basepath then use "/" alone.
  • optional automatically handle 404 responses for any request into the basepath that is not matched. If the tag has a value this is user-specified target id for a custom 404 handler (see below).
  • optional automatically handle 406 responses for found resources but with insufficient acceptability for the client. The tag must contain a user-specified target id for a custom 406 handler (see below).
  • optional the target id of a catch all exception handler endpoint (see below for interface requirements)
  • optional when specified it indicates that the RESTOverlay must use strict mode. In strict mode matching of target endpoints must be exact. The default is to be tolerant and find a best match target endpoint.
  • optional when specified it indicates the maximum size in bites to accept in POST, PUT and PATCH requests. Local endpoint value will override this global.

Endpoint REST Metadata Declaration

To make an endpoint a potential routing target for the RESTOverlay it must declare suitable metadata. An endpoint declares metadata by providing a tag on its declaration...

<endpoint>
  <meta> ...User specified metadata... </meta>
  <grammar>foo</grammar> ...etc...
</endpoint>

The RESTOverlay searches for a metadata tag within the declaration. For example the following endpoint declares that it wishes to handle GET requests for a REST sub-path with a simple grammar of "hello"...

<endpoint>
  <meta>
    <rest>
      <method>GET</method>
      <simple>hello</simple>
    </rest>
  </meta>
  <grammar>res:/helloImpl</grammar> ...
</endpoint>

Grammars

A target REST endpoint must always provide a grammar definition. This is a full simple grammar and so it may use the pattern matching and named arguments capabilities of the simple grammar.

The value of the tag is prefixed with the value of the RESTOverlay configuration parameter to construct a logical endpoint.

For example if the RESTOverlay has a basepath of /foo/ and a taret endpoint has a simple grammar of baa the RESTOverlay will automatically create a resolvable logical REST service with the simple grammar...

res:/foo/baa

If the declared simple grammar has pattern matching fields, these are passed through intact. For example if you have a basepath of /foo/ and a simple grammar of {country}/{state}/{city} the RESTOverlay will automatically create resolvable logical REST service with the simple grammar...

res:/foo/{country}/{state}/{city}

Multiple Targets

You may provide any number of target endpoints with the same grammar. At the level of the RESTOverlay these are logically combined into a single logical endpoint with the common constructed grammar.

When a request is resolved by the logical endpoint's grammar the RESTOverlay will then start its matching algorithm to determine which of the internal target endpoints that share the same grammar should receive the request.

Internal "True Grammar"

Whilst many target endpoints can share the same external grammar - declared in the /endpoint/meta/rest/simple declaration. Each target endpoint must have its own unique endpoint grammar - declared as /endpoint/grammar. To distinguish these, lets call this the "true grammar".

The "true grammar" must be unambiguous so that a potential external request can be uniquely routed to that endpoint.

The RESTOverlay will detect if any target endpoints have ambiguous "true grammars" and will log a warning. It will also show that it is incorrectly configured in the space explorer. If you see this warning, you must locate the endpoint with the ambiguous true grammar and make sure it is changed to be unique.

Argument Relaying

The external REST service logical endpoint may have a simple grammar containing argument fields. If you use arguments in your grammar you must declare the target endpoint's "true grammar" with the corresponding named arguments so that they can be relayed to the target endpoint.

For example this target endpoint implements the /country/state/city REST path grammar and its "true grammar" accepts the same named arguments.

<endpoint>
  <meta>
    <rest>
      <method>GET</method>
      < !--The simple external logical grammar-->
      <simple>{country}/{state}/{city}</simple>
    </rest>
  </meta>
  < !--The true grammar-->
  <grammar>
    <active>
      <identifier>active:CityService</identifier>
      <argument name="country" />
      <argument name="state" />
      <argument name="city" />
    </active>
  </grammar> ...
</endpoint>

The external simple grammar arguments (country, state, city) are relayed onto the request to the target endpoint. So for example a request

res:/foo/USA/California/LA

would be resolved by the RESTOverlay and then internally routed as a request for

	active:CityService
		+country@USA
		+state@California
		+city@LA

Note: In this case we declared the "true grammar" using an active grammar syntax. You don't have to make your internal endpoints use active grammars but we recommend it since they are normalized and ensure that you cannot get confused between external logical REST paths and normalized internal ROC resources.

Endpoint REST Declarations

We have seen that meta/rest/ tag must contain a tag. The following optional tags are also supported...

Routing Options

Purpose

Indicates which HTTP method or methods will be routed to this target endpoint.

Tag Value

A comma separated list of one or more GET, POST, PUT, PATCH, HEAD, DELETE etc.

Purpose

The RESTOverlay can perform dynamic content negotiation in order to locate a target endpoint that satisfies the expressed content requirements of the HTTP client (via the incoming HTTP Accept header).

If the Accept header preference is not matched by any tags and the RESTOverlay is not in strict mode then the request will be resolved to the first endpoint matching the REST path's simple grammar.

The value of the produces tag must be the mimetype which this endpoint produces.

Tag Value

A single valued mimetype.

Multiple tags and @withTransform

If multiple produces tags are specified then the Accept header preferences will be resolved against each of the mimetypes. The required mimetype resolved during this process is subsequently checked against the mimetype returned in the endpoint's response.

In the event that the endpoint returns a mimetype that differs from the required mimetype then the response will be passed as the operand argument to the service specified by the withTransform attribute on the resolved .

Example

Say an HTTP request has an Accept header requiring text/html...

HTTP Request:
Accept text/html

and your RESTOverlay metadata for the target endpoint has multiple tags...

<rest> ...
  <produces>text/xml</produces>
  <produces withTransform="active:styleMyXMLResponse">text/html</produces>
</rest>

Then the request will be issued to the endpoint but by default it will return a text/xml representation.

The RESTOverlay will detect that the required mimetype is text/html. It will also determine that the tag that specified text/html has an @withTransform attribute.

To convert the received response to the required mimetype the RESTOverlay will make a request to the service specified by the withTransform attribute and provide the endpoint's response as the operand argument.

In this example it will issue a request for

active:styleMyXMLResponse+operand@response from the endpoint

The withTransform service must be able to transform the mimetype of the endpoint response to the mimetype declared by the matched tag.

Any number of tags may be added. The single requirement is that the withTransform service is implemented and has an active grammar that accepts the operand argument with the original response.

Multiple tags and NO @withTransform

It is preferable that an endpoint should only produce one mimetype. As shown above the @withTransform feature allows mimetype based representation transformations to be automatically applied.

However, in some cases it can be useful to support multiple representations from one endpoint and have the endpoint decide which representation type to return. In such cases you may specify multiple tags such that any of the specified mimetypes are valid for content-negotiated resolution to this endpoint.

However, since the @withTransform (above) is not being used, it is essential that the representation type is distinguished for each of the possible types. To enable this the RESTOverlay will provide, on the request to the target endpoint, the special active URI argument...

+requiredMime@....

It follows that the grammar of your multi-produces endpoint must match this argument - something like this...

<grammar>
  <active>
    <identifier>active:myMultiProduceEndpoint</identifier>
    <argument name="requiredMime" min="0" max="1" />
  </active>
</grammar>

Inside your endpoint a request for

acceptmimetype=context.source("arg:requiredMime")

will provide the value of the external HTTP Accept header mimetype that was successfully resolved to this endpoint.

You must ensure that the endpoint returns a representation of this mimetype.

Purpose

The RESTOverlay will perform dynamic content delivery for entity bearing methods (POST, PUT, PATCH) and will use the mimetype of the body content to select a target endpoint. The tag indicates which mimetypes of the body content the target endpoint is prepared to accept.

The values of the consumes tag must be the set of mimetypes which this endpoint is able to consume.

Tag Value

A comma separated list of one or more mimetypes.

Purpose

The RESTOverlay can perform dynamic language negotiation in order to locate a target endpoint that satisfies the expressed language requirements of the HTTP client (via the incoming HTTP Accept-Language header).

Tag Value

The value of the language tag must be an HTTP standard language code. eg "en" or "fr" etc

Purpose

The RESTOverlay will ensure that the requested URL was on the https: SSL channel. If it was not it will automatically 302 redirect to the same URL with https: scheme.

Content Processing Options

Purpose

If specified the RESTOverlay will automatically perform content hashing and associate an Etag header with the resource.

Tag Value

The algorithm to use for the hash. Must be one of MD2, MD5, SHA1, SHA256, SHA384, SHA512

Purpose

If specified the RESTOverlay will automatically attempt to compress the representation using a negotiated compression algorithm that is acceptable to the HTTP client. Implemented formats include gzip and deflate compression.

The HTTP Header Content-Encoding is automatically set as appropriate.

Purpose

When commissioning the RESTOverlay will attempt to detect any potentially ambiguous grammars such as, for example,

users/{id:0-9+}
users/{id}

which are potentially ambiguous. However it may be the case that these grammars are intentional - the first matches numberic user ids where the second is a default that matches any other type of id.

If you are certain that your grammars are correct you can suppress the warning by declaring in the rest metadata, even so an info message will be logged for these endpoints just to be sure.

Processing Options

Purpose

Override the default auto generated endpoint id for this endpoint with the given id.

<rest>
  <id>myendpoint</id> ...
</rest>

Purpose

The RESTOverlay will catch any exception in the main target request and issue a request to the specified exception handling target.

The target endpoint must have a grammar that accepts request and exception arguments. For example...

<endpoint>
  <id>MyExceptionHandler</id>
  <grammar>
    <active>
      <identifier>active:MyExceptionHandler</identifier>
      <argument name="request" />
      <argument name="exception" />
    </active>
  </grammar>
</endpoint>

The request argument will be the INKFRequest which was issued to the failed internal target endpoint. The exception argument will be an org.netkernel.http.rest.RESTOverlayWrappedException which wraps the thrown exception.

A local declaration of will override any setting of the config parameter.

The RESTOverlayWrappedException has this interface...

package org.netkernel.http.rest;

public class RESTOverlayWrappedException
{
    public Exception mWrappedException;
    
    public RESTOverlayWrappedException(Exception e)
    {   mWrappedException=e;        
    }
    
    public Exception getException()
    {   return mWrappedException;
    }
    
    public void throwWrappedException() throws Exception
    {   throw mWrappedException;        
    }
}

Purpose

Before the main request to the resolved target, the RESTOverlay will issue a request to the endpoint specified by the preTarget identifier.

The target endpoint must have a grammar that accepts a request argument. The request argument will be the INKFRequest which will be issued to the the target endpoint.

The preTarget endpoint can perform authentication, validation, logon etc. If necessary it can make requests to the httpRequest:/ and httpResponse:/ to do any necessary process.

Response

The preTarget endpoint must return a boolean indicate if the RESTOverlay should proceed with issueing the request to the target endpoint.

Purpose

The RESTOverlay will automatically wrap the target request with a request to the wrapperTarget endpoint.

A wrapper endpoint must accept an operand argument. The operand is a pass-by-request containing the request to the primary target. Only when the wrapper actually SOURCE's its operand argument is the primary target request triggered.

This pattern is functional lazy evaluation and looks like this: w(f(x))

The beauty of the pattern is that before sourcing the operand, the wrapper is able to perform a pre-processing phase. After sourcing the operand, it can move to a post-processing phase.

The RESTOverlay carefully constructs a spacial scope so that requests to the wrapper are cacheable and have a dependency expiry. The net effect is that GET endpoints that generate a cacheable resource can be combined into a composite in a wrapper handler and the composite resource is also cacheable.

Response

The response from the wrapper is returned as the final response from the overlay.

Purpose

A per-endpoint setting for maximum Content-Length on POST, PUT and PATCH methods. This setting will override any global setting in RESTOverlay config.

If Content-Length exceeds this value, or the requestor does not state Content-Length, a 413 response is sent to the requestor.

Purpose

The RESTOverlay can perform dynamic REST to ROC verb translations. If this tag is specified the following default translations will occur

  • GET: SOURCE
  • POST: SINK
  • PUT: SINK
  • DELETE: DELETE

Otherwise default to SOURCE.

If the target verb is SINK, the httpRequest:/body resource will be SOURCEd and then used as the Primary argument in the SINK request.

Tag Value

If the tag has a value it may provide arbitrary overrides for method to verb mappings. The syntax is a comma separated list of method:verb mappings. For example:

POST:NEW,HEAD:EXISTS

These values will override any setting in the defaults and any unchanged defaults will be used.

Note - if you choose to use verb translation, you should be aware that the opportunity which ROC enables to cache POSTs (and other entity bearing HTTP methods) will no longer be available. There are deep and subtle differences between REST and ROC. ROC is a more general abstraction and can systemically normalize state - which means that it is possible that state transfer is not required at all. But we're also pragmatic - so if this verb translation is useful for you then its simple to set up.

Examples

The restoverlay-test package can be installed with apposite and provides a comprehensive set of examples of the use of the RESTOverlay.

The installed module is located as modules/urn.test.org.netkernel.tpt.http.RESTOverlay-x.x.x/

The unit tests can be executed here

The demo-addressbook package has a simple CRUD address-book that presents both a web-application interface and a REST service interface using the RESTOverlay for content negotiation, a decorated wrapping template pattern, compression and Etag generation.