WARNING: This server provides a static reference view of the NetKernel documentation. Links to dynamic content do not work. For the best experience we recommend you install NetKernel and view the documentation in the live system .

This document discusses caching from the perspective of developing applications. Although caching "just works" it may require tuning. Design considerations in your applications may determine how effective the caching is. With this in mind this page gives you a background in how the caching operates and discusses the tools and approaches to help you get the best from it.

For an overview of caching concepts see the ROC in a Nutshell book.

Caching Mindset

For the most part caching should be orthogonal to application development. Caching is just an optimisation - that said, knowing that you will get caching changes the way you develop for the better. The reason for this is that if you think in terms of resources and stateless processes that compose resources then you don't need to worry about keeping and managing state because you just request it when you need it. This will result in less code and stateless endpoints making your system quicker to develop, more maintainable, and more scalable. In order to feel comfortable with this approach it's important to remember that commonly requested resources will always be pulled from cache rather than recomputed each time.

When you are developing, initially you should not be concerned with caching; endpoints have default behaviour that is sensible for many situations. For example the default expiry mechanism on endpoints is that they accumulate dependencies of all sub-requests such that if any dependent resource expires then it's response will expire too. However if an endpoint doesn't explicitly set a response then the response it returns will be always expired - this is because the assumption is that the endpoint caused some desirable side-effect that should be repeated if we re-request. Overriding the expiration is probably the first thing that you will want to understand. One common situation where you'll want to override is when you want something to always be recomputed, for example if we are creating a random number generator.

Scoping is the most difficult concept to grasp in regard to caching. The concept of scope is critical to ROC because it becomes part of the identity of resources when all resolution is performed relative to a set of address spaces rather than the global namespace of the WWW. Scope becomes important to understand when your application structure injects dynamic spaces into the request scope and then during evaluation endpoints issue requests to resources in those injected spaces. (This happens when you receive a request through the HTTP Bridge, when you use the pluggable overlays and when you pass-by-value as a few examples. ) In essence the response is sensitive to the context that it is requested in. This doesn't stop a response from being cached but it may inhibit it from being retrieved. Later in this page we will discuss the tools that help you understand scoping.

Request Lifecycle

The NetKernel kernel performs cache operations during the lifecycle of event processing. These operations put and get responses into and out of the resolution and representation cache.

Retrieval

When a request is issued the NetKernel kernel first performs a representation cache lookup using the issued request as the key. If a response is found then this is returned otherwise the kernel performs a lookup on the resolution cache to see if there is a previous resolution for the given request. (resolution comprises both a resolved endpoint and an evaluation scope) If a resolution is found then it is passed to the request evaluation stage otherwise the resolution process is performed and the resolution put into the resolution cache.

Once the resolution is obtained the kernel then performs a second lookup on the representation cache using the evaluation scope. This second lookup is valuable when a resource is being requested with a different request scope each time and it's full identity is only known after resolution. If a response is found on this second lookup then an additional key is put into the representation cache against the found response using the request scope and the response it returned. Because of this, you may more than one request scope against a given response. If no response is found then the resolved endpoint must be executed. The endpoint is passed the issued request but with the evaluation scope substituted for the request scope.

Storage

When the endpoint has executed and generated it's response the NetKernel kernel offers the representation cache an opportunity to cache that response. The response is cached under both the request scope and the evaluation (resolved) scope.

The cache performs a number of tests on the response before actually caching it. These tests are:

Request verb must be SOURCE, EXISTS or META - Only responses from idempotent requests which have cause no change to the state of underlying resources are cacheable. This excludes SINK, DELETE, NEW. Explicit TRANSREPT requests are also uncacheable - this is because they don't have a well defined key to be cached with. However implicit TRANSREPTs that occur whilst servicing a SOURCE request will not inhibit caching.

Response must not be expired - Any response whose expiration function returns true will never be placed into the cache. Expiration functions are discussed in more detail below.

Response must not be marked with a no-cache header - If a response has been marked with a no-cache header by the implementing endpoint then the response will not be eligible for caching. See Response Configuration for more details.

Request must not be marked with a no-cache or forget-dependencies header - If a request was issued with a no-cache of forget-dependencies headers then the response will not be eligible for caching. See Kernel Request Headers for more details.

Response cost must be above the caches cost threshold - The standard NetKernel cache has a threshold configuration property which, as an optimisation, stops extremely cheap representations from being cached.

Cache Keying

Responses are stored in the representation cache using a cache key which comprises the resource identifier, the request verb and a scope, either a request scope or evaluation scope. Usually at least two keys are stored for each response unless the request and evaluation scope are identical. Understanding the semantics of the identifier and verb are relatively straight forward because when a lookup is performed they must match exactly.

Scope comparison is more complex. A looked-up scope is considered equivalent to a stored scope if it is equal (i.e. they have the same number of spaces and each corresponding space is equal) or it is a super set (i.e. each corresponding space is equal but the looked-up scope may contain additional items on it's tail.) Equality of spaces may mean that they are exactly the same space. However some space implementations such as value spaces (as used when adding pass-by-value arguments) implement equality functions that mean spaces can be considered equal when they contain the exact same set of resources. This becomes important to gain cacheability when dynamic spaces are injected at runtime.

Expiry

NetKernel builds a rich dependency model for all resource representations by default such that expiration semantics propagate up into dependent resources when sub-requests are issued. The default behaviour can be overridden where necessary with a number of preset modifiers which can be attached to the response in either NKF or DPML. These are:

  • EXPIRY_ALWAYS - Always expire a resource representation
  • EXPIRY_NEVER - Never expire a resource representation and ignore any dependencies that might cause it to expire
  • EXPIRY_DEPENDENT - (default) Expire a resource when any of it's dependent resources expire
  • EXPIRY_CONSTANT - Expire a resource at a specific time
  • EXPIRY_MIN_CONSTANT_DEPENDENT - Expire a resource when any of it's dependent resources expire OR at a specific time, which ever comes soonest.
  • EXPIRY_MAX_CONSTANT_DEPENDENT - Expire a resource when any of it's dependent resources expire AND after specific time is passed.
  • EXPIRY_FUNCTION - provide a customised expiration function to determine when the representation should expire.
  • EXPIRY_MIN_FUNCTION_DEPENDENT - provide a customised expiration function but also expire early if any dependencies have expired.
  • EXPIRY_POLLED_DEPENDENT - like dependent only we only look at the dependent resources every poll period so it is more performant for resources with a large number of dependencies

Custom expiration functions

Custom expiration functions can be implemented for any number of application or technology specific purposes.

Two examples of custom expiration functions are:

  • Filesystem backed resources - these are polled for changes according to the "Filesystem Poll" period determined in the "Configure NetKernel" tool. And will automatically expire when the file is changed.
  • Golden thread - these provide cross resource expiration coupling when utilising external technologies such as databases which don't have the ability to easily define expiry mechanisms.

Cache Management

The representation cache cannot keep every valid representation because it doesn't have infinite storage. Besides managing a large cache has costs of it's own which may outweigh the benefit it could deliver. Therefore the size of the cache is bounded based on a number of configurable factors. You can view and modify these on the Configure Netkernel tool. The standard cache in NetKernel watches the available heap space to determine how many items it can keep. When it is determined that the cache has grown too large, a sweep of the entire cache is made and all entries are ranked according to a function of merit based up how many times it has been used, when it was last used and how much work went into generating it (cost).

The kernel automatically computes and accumulates cost during evaluation. Complex computations have a high cost and simple computations a low cost. The cost is accumulated over all sub-requests that are evaluated to create a response and will also include costs retrieved from cache. The absolute value of the cost is not important - different implementations on different platforms may use different methods to compute this value using elapsed time or CPU core specific timings. The cache uses differences between costs to determine relative merit.

Once all cache items have been ranked, a configurable percentage of the lowest ranked items are permanently culled. During ranking and culling the cache will still accept gets and puts - it is fully asynchronous.

Tools

Status Tab

To get an macroscopic view of how the cache is operating with your system look at the status tab on the backend fulcrum. Here you can get a historical view of the cache size and heap usage over the last five minutes. (this duration is configurable) This view is useful for checking your cache is sensibly configured. Things to look out for include:

  • A large gap between committed heap use and peak may indicate that you are not making the most of available heap to cache values. Consider raising the maximum cache size.
  • A very small gap between heap baseline, peak and committed in combination with a small cache size may indicate that you don't have enough heap or there is a memory leak consuming heap. See Memory Leaks for more details.

Representation Cache Viewer

The representation cache viewer can give an insight into the contents of the cache. Items are listed and this list can be sorted and filtered. For each item we can see the cost, number times the item has been used, time since last use and the computed index that is used to rank and remove least valuable items when culling. Below each item we get a list of scopes that the item is keyed under. You should see at least one evaluation scope and zero or more request scopes. Clicking on an item will return it's representation (transrepting to a binary stream if necessary, or calling a Java .toString() if no transreption is available.

Useful functions of the cache viewer include:

  • Filtering on a specific identifier to see how many instances of a resource exists. Multiple instances where you only expect one, may indicate a scoping issue is effecting cache retrieval. You can confirm this by looking at the hit counts on individual entries.

Visualizer

The NetKernel Visualizer is a power tool which can capture the full details of request execution. It has many purposes but certainly understanding caching is one them. When the visualizer captures a request, each and every request that executes is shown in the recursive structure that it actually executes in. Each row in a request trace shows a request. If at any stage a resource representation is pulled from the cache, this is shown. Likewise if you see a request being executed rather than it's response being pulled from cache, you can see that too. On each row their is a cacheability "traffic light" that indicates cacheability. When the traffic light is red, caching of this request is not possible, when it is green caching will occur, and when amber caching will occur but the retrieval depends upon requests scope.

Clicking on a request line and selecting "View Request Details" you will get detailed information on the request. In the response section a breakdown of the reason for cacheability is shown along with details of the response scope and expiration dependencies.

Enterprise Edition Tools

NetKernel Enterprise Edition contains a number of additional plugin tools to the visualizer that make understanding and controlling caching easier. This include "Find Expiry Determinants", "Find Scope Determinants" and "Compare Caching". For more details see here (Enterprise Edition only).

Patterns and Anti-patterns

Anti patterns

  • SINK - any requests using the resource mutating verbs of SINK, DELETE, NEW will be expired and hence any requests that contain nested requests using these verbs will be expired and un-cacheable by default.
  • Pass-by-request - use of pass-by-request will make make cacheability harder because of the complex request scopes that can occur using this method.
  • Pluggable Overlay - the pluggable overlay places both the request and response in a pass by value space when calling the pre and post process services. Because of this these services themselves will not be cacheable. Though the response from the pluggable overlay and the main passed request may still be cacheable.
  • HTTP request headers - use of many aspects of the HTTP request will render the requestor un-cacheable.

Patterns

  • Golden thread - Golden Threads can be used to externalize response validity from the underlying resource endpoint. This is particularly effective with relational database queries and legacy systems with no ability to directly determine invalidation.
  • Timed Expiry - overriding the expiration of a response that would otherwise be always expired with a (suitable) timed expiry (EXPIRY_CONSTANT) is a good way to increase performance and caching.
  • Cacheable layer - thinking about and creating cacheable layers with well defined interfaces will ensure you can achieve caching. This may mean factoring out dependencies on transports using a bridge layer or mapper.