Managing Web Proxies
You configure web proxy for Internet-bound traffic in the Web Proxy page. This section includes these topics:
A single-ended web proxy transparently intercepts all traffic bound to the Internet. The web proxy improves performance by providing optimization services such as web object caching and SSL decryption to enable content caching and logging services. The efficient caching algorithm provides a significant advantage for video traffic. The benefit comes in the form of multiple users viewing the same video content, thereby saving significant WAN bandwidth and providing efficient network use. YouTube caching is handled as a special case given its growing popularity in the enterprise.
Web proxy improves HTTP performance and reduces congestion on Internet traffic. It also provides performance benefits when you access HTTP(S) servers on the Internet directly from a branch office. It provides visibility to all Internet activity at any given branch as long as that destined traffic passes through the web proxy. Web proxy is only supported on the SteelHead CX and the xx70.
You enable the web proxy in a single-ended or asymmetric SteelHead deployment; a server-side SteelHead is not required. Only in-path deployments are supported. Web proxy is supported on SteelHead CX and the xx70.
HTTPS web proxy integrates with the certificate authority (CA) service on the SCC to generate server certificates and decrypt traffic for a predefined whitelist.
Web object caching includes all objects delivered through HTTP(S) that can be cached, including large video objects like static VoDs and YouTube video. The size of objects that can be cached is limited only by the total available cache space, determined by the SteelHead appliance model. The proxy cache is separate from the RiOS data store. When objects for a given website are already present in the cache, the system terminates the connection locally and serves the content from the cache. This saves the connection setup time and also the bytes to be fetched over WAN.
The maximum size of a single object is unlimited. An object remains in the cache for the amount of the time specified in the cache control header. When the time limit expires, the SteelHead evicts the object from the cache. The cache sizes range from 50GB to 500GB.
The proxy cache is separate from the RiOS data store. When objects for a given website are already present in the cache, the system terminates the connection locally and serves the content from the cache. This saves the connection setup time and also reduces the bytes to be fetched over the WAN.
You can view web proxy connections in the SteelHead in the Current Connections report as a new connection type: web proxy. SteelHead log messages display SEPIA_YES if web proxy is successful.
You can view the cache hits for all SteelHeads configured with web proxy in the SCC on the Web Proxy page.
Configuring HTTP Web Proxy
The HTTP web proxy has these characteristics:
• The cache sizes range from 50 GB to 500 GB on SteelHeads.
• The cache content is persistent after reboots and service restarts.
• There is not individual object size limitation.
• The cache storage space has been expanded on SteelHeads. The xx55 models have 50 GB of cache space for Web Proxy storage.
• The request logging format has been expanded to improve visibility, debugging, and diagnostics.
• You can use the Web proxy with virtual in-path deployments such as WCCP and PBR.
The in-path rule table includes a default web proxy rule set to Auto. By default, all traffic not specified in user-configured rules is web proxied for Internet-bound traffic. This includes all traffic destined to public IP addresses not included in Request for Comments (RFC) 1918 on port 80 and 443. Only IPv4 is supported for web proxy. SteelHead Cloud Acceleration (SCA) takes priority over web proxy, when web proxy is configured as Auto.
If you need a more fine-grained rule for public IP addresses, then you must add a new in-path rule with these options:
• Type: Auto Discover
• Web Proxy: Auto
• Source Subnet: IPv4 address or subnet
• Destination Subnet: IPv4 address or subnet
• Port: No matter what port is specified, only port 80 and 443 traffic is directed to the web proxy
If you need an in-path rule for private or intranet IP addresses, specify these options:
• Type: Pass Through
• Web Proxy: Force
• Port: Any port or port-label specified is proxied. This value results in plain TCP proxying without optimizations if the traffic is not detected to be HTTP or HTTPS.
To enable HTTP web proxy
2. Choose Manage > Optimization: Web Proxy to display the Web Proxy page.
Figure: Configuring Web Proxy
3. Select Enable Web Proxy to enable HTTP Web caching.
4. Click Save to save your settings.
Enabling HTTPS Web Proxy
HTTPS web proxy allows caching content that is SSL encrypted. HTTPS web proxy is required for YouTube caching.
Server Certificates are autogenerated and auto-renewed based on a domain whitelist of the SCC. The decrypting key and certificate are stored on the secure store on the client-side SteelHead.
Prerequisites
These prerequisites are required for HTTPS web proxy:
• Certificate Authority (CA) service must be configured on the SCC.
• CA certificate must be trusted by the clients and browsers.
• CA certificate has a default validity of 365 days.
• CA certificates are automatically renewed when within two days of expiration.
• CA certificate validity checks occur every 24 hours.
• If a CA certificate cannot be renewed, the default behavior is to no longer serve the expired certificate.
• If renewal fails, an error is logged, and traffic is not decrypted for that domain.
YouTube Caching
YouTube caching is enabled by default. Caching for YouTube uses a heuristic algorithm based on observed traffic flow that automatically learns the key to cache YouTube traffic. Because YouTube traffic is typically encrypted, HTTPS web proxy optimization must be enabled. You must add these domains to the HTTPS domain whitelist:
• *.googlevideo.com
• *.youtube.com
YouTube caching is not supported on Firefox and mobile browsers.
In-Path Pass-Through Rule
There is a default pass-through rule for all secure ports traffic above the default in-path rule that prevents all traffic to port 443 from being intercepted.
If HTTPS proxying is required, then the pass-through rule must be added above the secure ports rule, to direct SSL traffic to the web proxy with these options:
• Type: Pass Through
• Web Proxy: Force
• Port: Any port or port-label specified is proxied. This value results in plain TCP proxying without optimizations if the traffic is not detected to be HTTP or HTTPS.
To enable HTTPS decryption and caching
3. Make sure that all domains defined in the web proxy whitelist have the CA configured on the client browser.
4. Choose Manage > Optimization: Web Proxy to display the Web Proxy page.
5. Select Enable HTTPS Optimization to enable HTTPS caching.
6. Click Save to save your setting permanently.
7. If necessary, define a pass-through rule for port 443 traffic and push it to appliances.
Adding Domains to the Whitelist for HTTPS
You add a list of domains that you want to decrypt for HTTPS caching. The domain names can either be hostnames (for example, myhost.riverbed.com) or wildcard domain names (for example, *.riverbed.com).
To add domains to the global HTTPs whitelist
1. Choose Manage > Optimization: Web Proxy to display the Web Proxy page.
2. Under Global HTTP Whitelist, click the + Add Domain to display the pop-up window.
Figure: Adding Domains to the Global HTTPS Whitelist
3. Specify the domain name and click Add Domain. The domain appears in the global HTTPS whitelist table.
Important: For HTTPS web proxy, before adding domains, make sure that the SCC CA is trusted by all client browsers defined in the domain whitelist.
To configure exceptions to the whitelist
1. Choose Manage > Optimization: Web Proxy to display the Web Proxy page.
2. Under Global HTTP Whitelist, click the + Add Exception to display the pop-up window.
Figure: Adding Domain Exceptions to the Whitelist
3. Complete the configuration as described in this table.
Control | Description |
Exception Site | Click Sites or Site Types to specify the exception type. Specify the site or site type name in the text box. |
Settings | Click Enable Web Proxy to enable caching for this site or site type. Click Enable HTTPS Proxy to enable increased performance for HTTPS traffic on the specified site of site type. |
Add Exception | Adds the site exception to the exceptions table. |
Pushing Your Settings and Viewing Push Status
You can push your settings to sites or site types from the Policy Push Control on the right side of the page. You can also view push status from the Push Status panel on the right side of the page.
If the SCC and SteelHeads are both running 9.2.0 or later, for the initial configuration the SCC pushes the entire configuration. For SteelHeads and an SCC running 9.2, any changes made after the initial push, the SCC pushes only the modified settings to ensure improved response times and throughput performance. If the SCC and SteelHeads are both running 9.0 and 9.1, when you push configuration changes, whether the initial push or after, the SCC deletes the entire configuration and replaces it with the new configuration settings, which can slow response times and performance.
Note: When you perform a policy push, the SCC is the master configuration; any local changes made on SteelHeads are overwritten.
To push settings
1. Under Policy Push Control on the right side of the page, click Include in Push to expand the page and display the Push to Appliances panel.
Figure: Pushing Settings
Note: To exclude appliances from the push, under Push Control on the right side of the page, click Exclude from Push. (This option only appears if you have clicked Include in Push.)
2. Complete the configuration as described in this table.
Control | Description |
Push to Appliances | Select to push your path selection rules: • Site Types - Click the text box to display site types to choose from. Select the site types one at a time to add them to the text box. After you select the site type, it is displayed in the text box. To remove a site type, click the X. To view what sites make up the site type, click See More. Riverbed recommends that you choose site types rather than sites to organize your rules as site types make the management of rules easier. • Sites - Click the text box to display sites to choose from. Select the sites one at a time to add them to the text box. After you select the site, it is displayed in the text box. To remove a site, click the X. To view site details, click See Details. |
Push | Pushes configuration settings to the selected sites or site types. Click Clear to clear your settings. |
Viewing Push Status
You can view the current status of your pushes on the right side of the page in the Push Status panel.
To view current status of configuration pushes
• Under Push Status on the right side of the page, click More to be directed to the Operation History page.
Figure: Displaying Push Status
The current operations (that is, pushes) and status are displayed in the Operations table.
Viewing the Cache Hit Ratio
You can view the ratio of cache hits on the Web Proxy page.
The cache hits ratio is aggregated across all the SteelHeads. The report displays hourly data points for the last two weeks.
To view the cache hits ratio
1. Choose Manage Optimization > Web Proxy to display the Web Proxy page.
2. Scroll down to view the Cache Hits Ratio report.
Figure: Cache Hits Ratio
3. Hover over the date to display details.