CIFS and HTTP Prepopulation
CIFS and HTTP Prepopulation
You can avoid the penalties of a cold transfer by using Common Internet File System (CIFS) and HTTP prepopulation to warm the SteelHead with data not yet requested by end users. Prepopulation is fully integrated into RiOS 7.0 or later.
The prepopulation of data is extremely beneficial for end users who know of data crossing the WAN: for example, operating system patch updates, antivirus update files, a video training session posted on an internal file server for offline viewing, or a directory of a user share located at the data center.
You configure prepopulation only on the client-side SteelHead. You must configure and connect the client-side SteelHead primary interface properly for prepopulation to work. The primary interface must be able to establish connection with the server.
The primary interface of the SteelHead acts as a client and requests data from the share you want to use to warm the RiOS data store. To warm both SteelHeads, data flows from the server, passes the server-side SteelHead, and ends at the client SteelHead.
The traffic leaves the primary interface and it must traverse a client-side in-path connection (for example, LAN0_0/WAN0_0 for an inline deployment, or WAN0_0 for a logical in-path deployment) before reaching the server-side SteelHead. Prepopulation does not work if the primary interface bypasses client-side optimization first.
This chapter includes the following sections:
•  CIFS Prepopulation
•  HTTP Prepopulation
CIFS Prepopulation
CIFS prepopulation uses a flexible scheduler to prepopulate the RiOS data store. When a client on the remote LAN requests data again, only new or modified data is sent over the WAN, which dramatically increases the rate of data transfers. Unlike with Proxy File Service (PFS), the client-side SteelHead drops the actual data files it has received, but it maintains SDR data in the RiOS data store.
Because CIFS prepopulation uses the CIFS/SMB protocol, you can use a Windows machine, or any other CIFS/SMB capable NAS filer, for the file server.
In RiOS 7.0 or later, CIFS shares that require SMB signing are fully supported. Depending on the file server configuration, the authentication for SMB signing can be NTLM or Kerberos, but both methods are supported by CIFS prepopulation. To populate SMB signed CIFS shares, you must join the SteelHead to the domain in which the shares are published or to a trusted domain.
If the file server authentication is NTLM, then you must configure NTLM transparent mode on the server-side SteelHead. If the file server authentication is Kerberos, then you must configure the AD environment that the server-side SteelHead has joined (by creating a replication user account) and configure the server-side SteelHead to support Kerberos.
For more information, see Encrypted MAPI Optimization and Kerberos. For more information about SMB signing, see .
Figure: Traffic Flow for CIFS Prepopulation
When you create a share using CIFS prepopulation, the SteelHead attempts to connect to the destination server on port 139. You must enable NetBIOS over TCP/IP on a Microsoft Windows machine (server or client) to accept the connection on port 139. On a Windows server, enter the following command to see if it is listening on port 139:
C:\>netstat -an | find ":139"
If you do not see similar output with the correct host IP, NetBIOS over TCP/IP is not enabled on the interface. You can enable it on the relevant interface from the WINS tab in the Advanced TCP/IP settings.
In RiOS 8.5 or later, you can create policies and rules for a higher level of control over which files the system transfers to warm the RiOS data store. A policy is a group of rules that select particular files to prepopulate; for example, you can create a policy that selects all PDF files larger than 300 MB created since January 1st of 2013.
To configure CIFS prepopulation and add a CIFS share using the Management Console
1. From the Management Console, choose Optimization > Protocols: CIFS Prepopulation.
2. Select Enable CIFS Prepopulation and click Apply.
Figure: CIFS Prepopulation Page
3. Select Add a Prepopulation Share.
4. Specify the remote path where data is stored in UNC standard: for example, \\bw-sfowad1\shared.
5. Specify the account username. This username is the login name that has a minimum of read access to the remote path.
6. Specify your password and confirm your password.
7. Add a comment about the share. Comments cannot include an ampersand (&).
8. Specify a time limit that the synchronization job should not exceed.
9. Specify a limit on the amount of data in the synchronization job.
10. Select either current files for synchronization or use the latest share snapshot.
If no snapshots are available, the system uses the current files.
11. To schedule regular prepopulation events, select Enable Scheduled Synchronization (optional).
You can schedule full synchronization or incremental synchronization. Use Full Sync for data store wrap to keep potentially evicted data warm.
If a Full Sync and an Incremental Sync occur simultaneously, the Full Sync takes precedence.
12. Click Add Prepopulation Share.
13. Click Apply.
Figure: CIFS Prepopulation Share Information
Share synchronization begins, based on the parameters you entered.
Note: You cannot configure CIFS shares to be guest, anonymous, or public Samba shares without a password.
You can create synchronization policies in RiOS 8.5 or later. Synchronization polices enable you to define granular synchronization using filters for inclusion and exclusion specific data types.
To configure CIFS prepopulation policies
1. Open an existing prepopulation share.
2. Select Add a Policy.
3. Specify the policy name and a description.
4. Select data types from the drop-down list to create filers for the policy:
•  File extension
•  File size
•  Access time
•  Creation time
5. Modify time.
Figure: CIFS Synchronization Policy Filers
For more information about the CIFS prepopulation CLI commands and configuration, see the Riverbed Command-Line Interface Reference Manual.
Design Considerations
This section describes several design considerations.
Prewarming the RiOS data store during off-hours helps to reduce WAN bandwidth consumption during peak hours. When you configure CIFS prepopulation, it tries to use all available bandwidth. If you want to control bandwidth usage with CIFS prepopulation, you can configure QoS so that only a portion of the bandwidth is used for CIFS prepopulation.
To configure QoS to work with CIFS prepopulation
1. On the server-side SteelHead, enable QoS and set the proper link speed.
2. Set up a QoS class. Name it PREPOP and set an upper limit (for example, 10%).
3. Set up a QoS rule for the class PREPOP with a destination of the client-side SteelHead primary IP and give it a lower priority.
For more information about QoS, see the SteelHead Deployment Guide.
In RiOS 7.0 or later, clients can authenticate to servers requiring signing and synchronizing transparently from a Windows 2003 server DFS share.
Note: NTLM authentication mode is supported in RiOS 7.0 or later. In previous RiOS versions, CIFS prepopulation could only be performed against unsigned servers and required the use of transparent prepopulation support and the Riverbed Copy Utility (RCU) on the server.
In RiOS 7.0 or later, you can set the length of time a CIFS prepopulation command is to run. The time is reflected in seconds. You can run a test report on the client-side SteelHead to view the expected changes to be synchronized. A sample report follows:
(config) # prepop share modify remote-path '\\fileserver\share' max-duration 1800
#--- Set the maximum duration of a prepopulate run.
(config) # prepop share dry-run share-name '\\fileserver\share'
#--- Generate a dry run report where they can view a report of the expected changes.
(config) # show prepop log dry-run remote-path '\\fileserver\share'
RiOS 8.5 or later enhances CIFS prepopulation with a new policy-based system available in the CLI and the SteelHead Management Console. You can create specific policies and associate them with the CIFS prepopulation shares previously created.
The use of policies is a function of the client-side SteelHead. When you use policies, you can synchronize files:
•  created since a given set time.
•  modified since a given time.
•  matching an expression.
•  matching a given size range.
•   accessed since a given time.
An example policy creation follows:
(config) # prepop share policy share-name '\\fileserver\share' policy-name policy1
#--- To create a policy 'policy1' using share '\\fileserver\share'.
(config) # prepop share policy share-name '\\fileserver\share' policy-name policy1 create-time time '2011/10/01 00:00:00' compare-op after
#--- To prepopulate files created after that specific date.
(config) # prepop share policy share-name '\\fileserver\share' policy-name policy1 create-time time '2011/10/05 00:00:00' compare-op before
#--- To prepopulate files created before the earlier specific date.
(config) # show prepop share policy share-name '\\fileserver\share' policy-name policy1
#--- To display the specifics of the earlier mentioned policy
(config) # prepop share policy share-name '\\fileserver\share' policy-name policy1 file-name 'A_*.pdf;*.txt' compare-op matches
#--- To only include files with the.pdf and.txt extensions to be prepopulated
#--- Notice the use of the wild character "*". Additional matches need to be separated by a
#--- semicolon.
(config) # prepop share policy share-name '\\fileserver\share' policy-name policy1 file-name '*.dat' compare-op not-matches
#--- To exclude files with the.dat extension to be prepopulated.
config) # prepop share policy share-name '\\fileserver\share' policy-name policy1 file-size 10M compare-op less
#--- To prepop files less than 10meg in size.
(config) # prepop share policy share-name '\\fileserver\share' policy-name policy1 file-size 5M compare-op greater
#--- To prepopulate files greater than 5 MB in size.
You can delete a complete policy directly or delete a specific rule within a policy with the following command.
(config) # no prepop share policy share-name '\\fileserver\share policy-name policy1
HTTP Prepopulation
HTTP prepopulation, in RiOS 7.0 or later, is an enhanced HTTP-based data delivery method. HTTP prepopulation delivers data to the remote site by using the HTTP protocol to prewarm your RiOS data store. HTTP prepopulation uses the HTTP protocol to read and deliver data. The feature delivers any data content residing on a web server, including on-demand video streams at remote site. Remote users can have an enhanced viewing experience. For more information about video optimization, see the SteelHead Deployment Guide.
HTTP prepopulation requires that you configure and connect the primary interface for full functionality. HTTP prepopulation is only available through the command line. Configuration is only on the client-side SteelHead.
Note: If you are using HTTP prepopulation over IPv6, you must have a an IPv6 address on the primary interface. For more information about IPv6, see the SteelHead Deployment Guide.
To configure HTTP prepopulation you create a list composed of URLs that contain the data you want optimized. You can configure up to 100 lists, and you can include an unlimited number of URLs within each list.
For example, you can combine URL links to multiple Human Resource training videos in one list named HRvideos. The URL link must reference a filename with an extension in the URL link such as, and not reference a general URL link such as
Figure: Web Server Directory Listing shows a web server directory listing. The HTTP prepopulation functionality recursively downloads the entire folder content.
Figure: Web Server Directory Listing
The following example shows how to configure HTTP prepopulation with the list name HRvideos. List names are case sensitive.
To configure HTTP prepopulation
1. On the client-side SteelHead, connect to the CLI in configuration mode.
2. Build the list name, for example:
(config)# protocol http prepop list HRvideos
3. Add URLs to the list:
protocol http prepop list HRvideos url http://intranet/hr/HowToInterview.mp4
You can optimize content located on a secured web server (HTTPS). If HTTPS is specified as a protocol, you must configure SSL on the client-side SteelHead.
4. Start the process to transfer data across and prewarm your RiOS data store:
protocol http prepop list HRvideos start
The CLI session is unresponsive until the prepopulation runs to completion.
HTTP prepopulation time varies, depending on transfer time. If you want to maintain access to the CLI, Riverbed recommends that you use the job command to run the transfer at a specified time to run the transfer during off hours.
An example of the job command is as follows:
(config) # job 1 name prepop
(config) # job 1 command 1 "protocol http prepop list HRvideos start"
(config) # job 1 date-time 12:15:00
(config) # job 1 enable
To cancel an HTTP prepopulation job that has not begun
•  On the client-side SteelHead, connect to the CLI and enter the following command:
job <job-number> disable
To cancel HTTP prepopulation in progress
•  On the client-side SteelHead, connect to the CLI and enter the following command:
protocol http prepop list HRvideos cancel
Additional various supporting commands follow:
show protocol http prepop lists
show protocol http prepop status all
show protocol http prepop status list HRvideos
For more information about HTTP prepopulation commands, see Microsoft Silverlight, Apple HLS, and Adobe Flash and the Riverbed Command-Line Interface Reference Manual.
Microsoft Silverlight, Apple HLS, and Adobe Flash
In RiOS 7.0 and later, the SteelHead supports prepopulation for Microsoft Silverlight On-demand streaming content. In RiOS 8.5 and later, the SteelHead supports prepopulation for Microsoft Silverlight Video, Apple HLS, and Adobe Flash.
To prepopulate video, you must specify the manifest URL containing the manifest file. The manifest file includes information about the different bit rates available, the timeline markers, and so on. Manifest files are generally produced by video encoding software that supports their related format: for example, the Microsoft expressions encoder for Microsoft Silverlight is MP4.
In RiOS 8.5 and later, you can specify a URL (as opposed to specifically referencing the manifest file in the URL link as in previous RiOS versions). RiOS 8.5 and later automatically:
•  parses the content.
•  identifies the manifest file.
•  extracts the proper information.
•  extracts the constituent segment URLs.
•  represents the videos for all bit rates to download.
You need a browser plug-in on the client machine to play and watch the video. For example, Microsoft Silverlight is compatible with Internet Explorer, Firefox, and Safari.
The Microsoft Silverlight manifest file is an XML file, with a .ISM extension, that describes the live or on-demand video presentation. The Apple HLS manifest file is a text-based manifest file with a .M3U8 or .M3U extension that directs the player to additional manifest files for each of the encoded streams. The Adobe HTTP Dynamic Streaming manifest file has a .F4M extension.
Note: A manifest file is invisible to the end user; only end-user video players use it.
To configure HTTP prepopulation, use the following commands on the client-side SteelHead:
•  Create or delete a new prepopulation list with a user-specified name:
[no] protocol http prepop list *
•  Add or remove URL to the list that needs to be recursively prepopulated. All the objects in the page are fetched recursively (1 level deep):
[no] protocol http prepop list * url *
•  Start the prepopulation operation:
protocol http prepop list * start
•  Cancel the prepopulation operation:
protocol http prepop list * cancel
•  Display HTTP prepopulation status:
show protocol http prepop status all
•  Display specified list status info:
show protocol http prepop status list *
•  Display specified list configured URLs:
show protocol http prepop list *
•  Display all the configured list:
show protocol http prepop lists *