RiOS HTTP Optimization Techniques
The HTTP optimization module was first introduced in RiOS v4.x. At the time, the only feature available was URL learning. Subsequently, in RiOS v5.x, the parse-and-prefetch and metadata response features were added to the family. In RiOS v6.0, the metadata response feature was replaced with the object prefetch table. In RiOS v6.1, major enhancements were made specifically to optimize HTTP authentication.
RiOS v7.0 introduces HTTP automatic configuration. With HTTP automatic configuration you can:
enable HTTP automatic configuration to profile HTTP applications. In other words, HTTP automatic configuration collects statistics on applications and dynamically generates a configuration entry with the proper optimization settings and self-tunes its HTTP parameter settings. You can also store all objects permitted by the servers—you do not need to specify the extension type of specific objects to prefetch.
expand the HTTP authentication options by including end-to-end Kerberos authentication. All previous configurations are still available for configuration.You must run RiOS v7.0 or later on both the client-side SteelHead and server-side SteelHead to use automatic HTTP configuration and end-to-end Kerberos. For more details, see
HTTP Automatic Configuration and
HTTP Authentication Optimization.
Primary Content Optimization Methods
The following list explains the primary content optimization methods:
Strip compression - To conserve bandwidth, nearly all browsers support compression. The browser specifies what encoding schemes it supports in the Accept-Encoding line in the request. Before the server responds with a reply, it compresses the data with the encoding scheme that the client supports. To maximize the benefit of SDR, data coming from the server must decompress to allow SDR to de-duplicate this data. When Strip Compression is enabled, the HTTP optimization module removes the Accept-Encoding line from the request header before sending the request to the server. Because the modified request does not contain any supported compression scheme, the server responds to the request without any compression. This allows SDR to deduplicate the data. With this option enabled, the amount of LAN-side traffic increases as the server no longer sends the traffic in a compressed format. Insert cookie - The HTTP optimization module relies on cookies to distinguish between different users. If the server does not support cookies, the HTTP optimization module inserts its own cookie so that it can distinguish between different users. Cookies that are inserted by the HTTP optimization module start with rbt-http= and are followed by a random number. Enable this option only if the server does not issue its own cookies.The SteelHead removes the Riverbed cookie when it forwards the request to the server.
Insert keep-alive - Keep-alive, or persistent connection, is required for the HTTP optimization module to perform prefetches. If the client does not support keep-alive and the server does, the client-side SteelHead inserts a Connection: keep-alive to the HTTP/1.0 response, unless the server explicitly instructed to close the connection by adding Connection: close. This option does not apply to situations where the client supports keep-alive but the server does not. If the server does not support keep-alive, then prefetching is not possible and changes must be made on the server to support keep-alive.
URL learning - The SteelHead learns associations between a base request and a follow-on request. This feature is most effective for Web applications with large amounts of static-content images, style sheets, and so on. Instead of saving each object transaction, the SteelHead saves only the request URL of object transactions in a knowledge base, and then generates related transactions from the list. URL learning uses the Referrer header field to generate relationships between object requests and the base HTML page that referenced them, and to group embedded objects. This information is stored in an internal HTTP database. The benefit of URL Learning is faster page downloads for subsequent references to the same page, from the original browser of the requester or from other clients in the same location. When the SteelHead finds a URL in its database, it immediately sends requests for all of the objects referenced by that URL, saving round trips for the client browser. You can think of URL Learning as an aggressive form of prefetching that benefits all users in a location instead of just a single user, and that remembers what was fetched for subsequent accesses.Parse-and-prefetch - The SteelHead includes a specialized algorithm that determines which objects are going to be requested for a given Web page, and prefetches them so that they are available when the client makes a request. This feature complements URL Learning by handling dynamically generated pages and URLs that include state information. Parse-and-prefetch reads a page, finds HTML tags that it recognizes as containing a prefetchable object, and sends out prefetch requests for those objects. Typically, a client needs to request the base page, parse it, and then send out requests for each of these objects. This still occurs, but with parse-and-prefetch the SteelHead has quietly prefetched the page before the client receives it and has already sent out the requests. This allows the SteelHead to serve the objects as soon as the client requests them, rather than forcing the client to wait on a slow WAN link.
For example, when an HTML page contains the tag <img src="my_picture.gif">, the SteelHead prefetches the image my_picture.gif
Like URL Learning, parse-and-prefetch benefits many users in a location rather than just a single user. Unlike URL Learning, parse-and-prefetch does not remember which objects were referenced in a base request, so it does not waste space on dynamic content that changes each request for the same URL, or on dynamic URLs, which is not an efficient use of space.
Object prefetch table - The SteelHead stores objects per the metadata information contained in the HTTP response. The object prefetch table option helps the client-side SteelHead respond to If-Modified-Since (IMS) requests from the client, cutting back on round trips across the WAN. This feature is useful for applications that use a lot of cacheable content.Although URL learning and parse-and-prefetch can request information from the HTTP server much sooner than the client would have requested the same information, the Object Prefetch Table stores information to completely eliminate some requests on the WAN. The client browsers receive these objects much faster than they would if the object had to be fetched from the server.
In releases previous to RiOS v8.6, all HTTP responses above 1 Mb triggered the RiOS HTTP traffic optimization to bypass the remainder of established HTTP sessions. In RiOS v8.6 and later, environments running both client-side and server-side Steelhead appliances remove the 1-Mb response limit. However, single HTTP objects returned as part of the response, which are greater than 1 Mb, are not stored in the HTTP object prefetch table by default.
Although features such as URL learning, parse-and-prefetch, and the object prefetch table (the replacement to metadata response) can help speed up applications, other factors, such as authentication, can negatively impact the performance of applications.
HTTP Vary Headers
RFC 2616 provides HTTP 1.1 the ability for client and server to determine if they can retrieve HTTP responses from local cache or if they must retrieve the responses from the origin server. The vary header consists of HTTP field names that the client evaluates. In releases previous to RiOS v8.6, responses that contained a HTTP vary header were excluded from SteelHead HTTP optimization.
In RiOS v8.6 and later, if HTTP responses contain a vary HTTP header and are not compressed, RiOS HTTP optimization caches the response for client-side consumption. However, if RiOS receives a response with a vary HTTP header and the response is content-encoded, the local RiOS HTTP object cache does not store the HTTP object.
The following example shows an HTTP response with an HTTP vary header set without content encoding enabled:
HTTP/1.1 200 OK
Date: Sat, 01 Mar 2014 23:07:28 GMT
Server: Apache
Last-Modified: Fri, 28 Feb 2014 10:23:10 GMT
ETag: "183-4f374d3157f80-gzip"
Accept-Ranges: bytes
Vary: User-Agent
Content-Encoding: gzip
Content-Length: 265
Keep-Alive: timeout=15, max=98
Connection: Keep-Alive
Content-Type: text/css
As a best practice, enable the HTTP Strip Compression option on the HTTP page. You can apply this as part of the HTTP automatic configuration settings or manually per HTTP server. For more information about HTTP automatic configuration, see
HTTP Automatic Configuration.
Figure 4‑12. HTTP Optimization Configuration
Connection Pooling
Connection pooling preestablishes 20 inner channel connections between each pair of SteelHeads. HTTP traffic benefits the most from connection pooling, although connection pooling is not specific to HTTP. When the SteelHead requires an inner channel, it picks one from the pool and therefore eliminates the time for the TCP three-way handshake. The reason HTTP traffic benefits the most is because those connections are typically short-lived.
Connection pooling is available only when using the Correct Addressing as the WAN visibility mode. For details, see the SteelHead Deployment Guide.