Deployment Best Practices
  
Deployment Best Practices
Every deployment of the SteelFusion product family differs due to variations in specific customer needs and types and sizes of IT infrastructure.
The following recommendations and best practices are intended to guide you to achieving optimal performance while reducing configuration and maintenance requirements. However, these guidelines are general; for detailed worksheets for proper sizing, contact your Riverbed account team.
Note: For additional advice and best practices, see the SteelFusion Best Practices Guide in the SteelFusion documents section of the Riverbed Splash site, https://splash.riverbed.com/community/product-lines/steelfusion.
This chapter includes the following sections:
•  “Core best practices” on page 109
•  “Edge best practices” on page 114
•  “Operating system patching” on page 120
•  “Related information” on page 121
Core best practices
This section describes best practices for deploying the Core. It includes the following topics:
•  “Deploy on gigabit Ethernet networks” on page 110
•  “Core hostname and IP address” on page 110
•  “Segregate storage traffic from management traffic” on page 110
•  “When to pin and prepopulate the export” on page 110
•  “Core configuration export” on page 111
•  “Core in HA configuration replacement” on page 111
•  “WAN usage consumption for a Core to Edge VMDK data migration” on page 111
•  “Reserve memory and CPU resources when deploying Core-v” on page 112
•  “Network path redundancy” on page 112
•  “Interface selection” on page 112
•  “Editing files on exported fileshares” on page 112
•  “Permissions on exported fileshares” on page 112
•  “Size of exported fileshares” on page 112
•  “Resizing exported fileshares” on page 113
•  “Replacing a Core” on page 113
•  “Replacing a Core in an HA deployment” on page 113
Deploy on gigabit Ethernet networks
The NFS protocol enables file-level traffic over IP networks. However, NFS is both latency and bandwidth sensitive. To optimize performance reliability, deploy Core and the storage array on Gigabit Ethernet networks.
Core hostname and IP address
If the branch DNS server runs on VSP and its DNS datastore is deployed on an export used with SteelFusion, use the Core IP address instead of the hostname when you specify the Core hostname and IP address.
If you must use the hostname, deploy the DNS server on the VSP internal storage, or configure host DNS entries for the Core hostname on the SteelHead.
Segregate storage traffic from management traffic
To increase overall security, minimize congestion, minimize latency, and simplify the overall configuration of your storage infrastructure, segregate storage traffic from regular LAN traffic using VLANs (Figure 10‑1).
Figure: Traffic segregation
When to pin and prepopulate the export
SteelFusion technology has built-in file system awareness for NTFS and VMFS file systems. You will likely need to pin and prepopulate the export if frequent, prolonged periods of WAN outages are expected.
Data availability at the branch during a WAN link outage
When the WAN link between the remote branch office and the data center is down, data no longer travels through the WAN link. Hence, SteelFusion technology and its intelligent prefetch mechanisms no longer functions. Pin and prepopulate the export if frequent, prolonged periods of WAN outages are expected.
By default, the Edge keeps a write reserve that is 10 percent of the blockstore size. If prolonged periods of WAN outages are expected, appropriately increase the write reserve space.
Core configuration export
Store and back up the configuration on an external server in case of system failure. Enter the following CLI commands to export the configuration:
enable
configure terminal
configuration bulk export scp://username:password@server/path/to/config
Complete this export each time a configuration operation is performed or you have some other changes on your configuration.
Core in HA configuration replacement
If the configuration has been saved on an external server, the failed Core can be seamlessly replaced. Enter the following CLI commands to retrieve what was previously saved:
enable
configure terminal
no service enable
configuration bulk import scp://username:password@server/path/to/config
service enable
WAN usage consumption for a Core to Edge VMDK data migration
When provisioning VMs as part of a data migration, it is possible to see high traffic usage across the WAN link. This can be due to the type of VMDK that is being migrated. This table gives an example of WAN usage consumption for a Core to Edge VMDK data migration containing a 100 GiB VMDK with 20 GiB used.
VMDK type
WAN traffic usage
Space used on array thick LUNs
Space used on array thin LUNs
VMDK fragmentation
Thin
20 GB
20 GiB
20 GiB
High
Thick eager zero
100 GB + 20 GB = 120 GB
100 GiB
100 GiB
None (flat)
Thick lazy zero (default)
20 GB + 20 GB = 40 GB
100 GiB
100 GiB
None (flat)
For more details, see this Knowledge Base article on the Riverbed Support site:
https://supportkb.riverbed.com/support/index?page=content&id=S23357
Reserve memory and CPU resources when deploying Core-v
When deploying Core-v, see the SteelFusion Core Installation and Configuration Guide to understand what hardware requirements (memory, CPU, and disk) are required to support your Core-v model. We strongly recommend that you allocate and reserve the correct amount of resources as specified in the SteelFusion Core Installation and Configuration Guide. Reserving these resources ensures that the memory and recommended CPU are dedicated for use by the Core-v instance, enabling it to perform as expected.
Network path redundancy
Core does not support Multipath input/output (MPIO). For redundancy between the Core and the backend NFS file server, consider using the Edge Virtual IP (VIP) capability on the NFS file server interfaces if it is supported by the vendor.
Interface selection
In a deployment where multiple network interfaces are configured on the Core to connect to backend NFS file servers, there is no interface selection for the NFS traffic. Any interface that is able to find a route to the NFS file server could be used. If you must use specific interfaces on the Core to connect to specific NFS file server ports, then consider using static routes on the Core to reach the specific NFS file server IP addresses.
Editing files on exported fileshares
Do not edit files on the exported fileshares by accessing them directly, unless the fileshare has already been unmounted from the Core. Where possible, use controls to restrict direct access to exported files on the file server.
Permissions on exported fileshares
Ensure that only the Core has write permission on the exported fileshares. This can be achieved by using IP or hostname based access control on the NFS file server configuration settings for the exports. If necessary, the permissions on exported fileshares can be configured to allow mounting read-only by other hosts.
Size of exported fileshares
Exported fileshares up to 16 TiB (per export) are supported on the Core. However, the number of mounted exports that are supported will depend on the model of Core.
Note: EMC Isilon only - if you are provisioning and exporting NFS storage on an Isilon storage system, we recommend that you set a storage quota for the directory so the export does not exceed 16 TiB. For details, see “Deploying SteelFusion with Dell EMC Isilon for NFS” on the Riverbed Splash site.
Resizing exported fileshares
Exported fileshares can be expanded on the backend NFS file server without first unmounting them from the Core. The resized export will be detected by the Core automatically within approximately one minute. Reduction of exported fileshares is not supported.
Replacing a Core
Replacing a Core as part of an RMA operation will require an extra configuration task if the Core is part of an NFS deployment. By default, a replacement Core will be shipped with a standard software image that would normally operate in block storage mode. As part of the procedure to replace a Core, if necessary, the Core software should first be updated to a version of supporting NFS (minimum version 5.0). If the Core needs to be upgraded to a version that supports NFS, make sure both image partitions are updated so the Core isn’t accidentally rebooted to a version that doesn’t support NFS/file mode. Once the required version of software is installed, the Core needs to be instructed to operate in NFS/file mode. This is achieved by entering the following two commands using the Core command line interface:
service reset set-mode-file
service reset set-mode-file confirm
The first command will automatically respond back with a request for confirmation, at which point, you can enter the second command.
Note: This operation will clear any storage configuration on the Core; therefore care must be taken to ensure the command is performed on the correct Core appliance.
Once the Core is configured for NFS operations, it can be configured with the relevant storage settings.
Replacing a Core in an HA deployment
If a Core that is part of an HA deployment needs to be replaced, use the following sequence of steps as a guideline to help with replacement of the Core:
•  Consider opening a ticket with Riverbed Support. If you are not familiar with the replacement process, Riverbed Support can guide you safely through the required tasks.
•  Stop the service on the Core to be replaced. This will cause the peer Core to transition to ActiveSolo state and take control of fileshares that had been serviced by the Core to be replaced. The ActiveSolo Core will change all the fileshares to read-only mode for the Edges.
•  If the replacement Core is expected to take an extended period of time (days) before it is installed, contact Riverbed Support for guidance on forcing the filesystems back to a read-write mode.
•  Once the replacement Core is installed and configured to operate in NFS mode, it can be updated with the relevant storage configuration settings by sending them from the ActiveSolo Core. This is achieved by performing the following command on the ActiveSolo Core:
device-failover peer set <new-peer-ip> local-if <interface> force
In this command, <new-peer-ip> is the IP address of the replacement Core appliance and <interface> is the local interface of the ActiveSolo Core used to connect to the replacement Core IP address. For more details on this command, see the SteelFusion Command-Line Interface Reference Manual.
Note: Ensure that you enter commands on the correct Core appliance.
Edge best practices
This section describes best practices for deploying the Edge. It includes the following topics:
•  “Segregate traffic” on page 114
•  “Pin the export and prepopulate the blockstore” on page 115
•  “Ports and type of traffic” on page 115
•  “Changing IP addresses on the Edge, ESXi host, and servers” on page 115
•  “Disk management” on page 116
•  “Rdisk traffic routing options” on page 116
•  “Deploying SteelFusion with third-party traffic optimization” on page 117
•  “Set up memory reservation for VMs running on VMware ESXi in the VSP” on page 117
•  “Running antivirus software” on page 118
•  “Running disk defragmentation software” on page 118
•  “Running backup software” on page 118
•  “Configure jumbo frames” on page 118
•  “Removing Core from Edge and re-adding Core” on page 119
•  “General purpose file server” on page 119
•  “Alternative ports for VIP address” on page 119
•  “Replacing an Edge” on page 120
•  “Unmounting ESXi datastores” on page 119
•  “Sparse files” on page 119
•  “Changing Edge VIP address” on page 120
•  “Replacing an Edge” on page 120
Segregate traffic
At the remote branch office, separate storage traffic and WAN/Rdisk traffic from LAN traffic. This practice helps to increase overall security, minimize congestion, minimize latency, and simplify the overall configuration of your storage infrastructure.
Pin the export and prepopulate the blockstore
In specific circumstances, you should pin the export and prepopulate the blockstore. Additionally, you can have the write-reserve space resized accordingly; by default, the Edge has a write-reserve space that is 10 percent of the blockstore.
To resize the write-reserve space, contact your Riverbed representative.
We recommend that you pin the export in the following circumstances:
•  Database applications - If the export contains database applications that use raw disk file formats or proprietary file systems, pin the export and prepopulate the blockstore. If the export contains a VM that runs a database application, pin the export and prepopulate the blockstore.
•  WAN outages are likely or common - Ordinary operation of SteelFusion depends on WAN connectivity between the branch office and the data center. If WAN outages are likely or common, pin the export and prepopulate the blockstore.
Ports and type of traffic
You should only allow NFS traffic on primary and auxiliary interfaces. Riverbed does not recommend that you configure your external NFS initiators to use the IP address configured on the in-path interface. Some appliance models can optionally support an additional NIC to provide extra network interfaces. You can also configure these interfaces to provide NFS connectivity.
Changing IP addresses on the Edge, ESXi host, and servers
When you have an Edge and ESXi running on the same converged platform, you must change IP addresses in a specific order to keep the task simple and fast. You can use this procedure when staging the Edges in the data center or moving them from one site to another.
This procedure assumes that the Edges are configured with IP addresses in a staged or production environment. You must test and verify all ESXi, servers, and interfaces before making these changes.
To change the IP addresses on the Edge, ESXi host, and servers
1. Starting with the Windows server, use your vSphere client to connect to the console, login and change it to DHCP or the new destination IP address, and shut down the Windows server from the console.
2. To change the IP address of the ESXi host:
SteelFusion Edge
Connect to the ESXi console serial port or run the following command at the RiOS command line to show the ESXi console on the screen and allow you to change the IP address.
hypervisor console
3. On the Edge Management Console, choose Networking > Networking: In-Path Interfaces, and then change the IP address for inpath0_0 to the new destination IP address.
4. Modify the VIP interface on the Edge configuration. Click edit next to Edge Virtual IP Address Configuration. Make the change and click Update to confirm.
5. Use the included console cable to connect to the console port on the back of the Edge appliance and log in as the administrator.
6. Enter the following command to change the IP address to your new destination IP address.
enable
config terminal
interface primary ip address 1.7.7.7 /24
ip default-gateway 1.7.7.1
write memory
7. Enter the following command to shut down the appliance:
reload halt
8. Move the Edge appliance to the new location and boot up the Edge.
Disk management
You can create Edge Local exports using the Core management console. Choose Configure > NFS: Exports and use the “Create an Edge Local Export” tool. Once you have created the Edge Local exports, they can be mapped to the Edge.
Rdisk traffic routing options
You can route Rdisk traffic out of the primary or the in-path interfaces. This section contains the following topics:
•  “In-path interface” on page 116
•  “Primary interface” on page 117
For more information about Rdisk, see “Network Quality of Service” on page 105. For information about WAN redundancy, see “Configuring WAN redundancy” on page 77.
In-path interface
Select the in-path interface when you deploy the SteelFusion Edge W0 appliance. When you configure Edge to use the in-path interface, the Rdisk traffic is intercepted, optimized, and sent directly out of the WAN interface toward the Core deployed at the data center.
Use this option during proof of concepts (POC) installations or if the primary interface is dedicated to management.
The drawback of this mode is the lack of redundancy in the event of WAN interface failure. In this configuration, only the WAN interface needs to be connected. Disable link state propagation.
Primary interface
We recommend that you select the primary interface when you deploy the SteelFusion Edge W1-W3 appliance. When you configure Edge to use the primary interface, the Rdisk traffic is sent unoptimized out of the primary interface to a switch or a router that in turn redirects the traffic back into the LAN interface of the Edge RiOS node to get optimized. The traffic is then sent out of the WAN interface toward the Core deployed at the data center.
This configuration offers more redundancy because you can have both in-path interfaces connected to different switches.
Deploying SteelFusion with third-party traffic optimization
The Edges and Cores communicate with each other and transfer data-blocks over the WAN using six different TCP port numbers: 7950, 7951, 7952, 7953, 7954, and 7970.
Figure 10‑2 shows a deployment in which the remote branch and data center third-party optimization appliances are configured through WCCP. You can optionally configure WCCP redirect lists on the router to redirect traffic belonging to the six different TCP ports of SteelFusion to the SteelHeads. Configure a fixed-target rule for the six different TCP ports of SteelFusion to the in-path interface of the data center SteelHead.
Figure: SteelFusion behind a third-party deployment scenario
Set up memory reservation for VMs running on VMware ESXi in the VSP
By default, VMware ESXi dynamically tries to reclaim unused memory from guest virtual machines, while the Windows operating system uses free memory to perform caching and avoid swapping to disk.
To significantly improve performance of Windows virtual machines, configure memory reservation to the highest possible value of the ESXi memory available to the VM. This configuration applies whether the VMs are hosted within the hypervisor node of the Edge or on an external ESXi server in the branch that is using exports from SteelFusion.
Setting the memory reservation to the configured size of the virtual machine results in a per virtual machine vmkernel swap file of zero bytes, which consumes less storage and helps to increase performance by eliminating ESXi host-level swapping. The guest operating system within the virtual machine maintains its own separate swap and page file.
Running antivirus software
There are two antivirus scanning modes:
•  On-demand - Scans the entire export data files for viruses at scheduled intervals.
•  On-access - Scans the data files dynamically as they are read or written to disk.
There are two common locations to perform the scanning:
•  On-host - Antivirus software is installed on the application server.
•  Off-host - Antivirus software is installed on dedicated servers that can access directly the application server data.
In typical SteelFusion deployments in which the exports at the data center contain the full amount of data and the remote branch cache contains the working set, run on-demand scan mode at the data center and on-access scan mode at the remote branch. Running on-demand full file system scan mode at the remote branch causes the blockstore to wrap and evict the working set of data leading to slow performance results.
However, if the exports are pinned, on-demand full file system scan mode can also be performed at the remote branch.
Whether scanning on-host or off-host, the SteelFusion solution does not dictate one way versus another, but in order to minimize the server load, we recommend off-host virus scans.
Running disk defragmentation software
Disk defragmentation software is another category of software that can possibly cause the SteelFusion blockstore cache to wrap and evict the working set of data. Do not run disk defragmentation software. Disable default-enabled disk defragmentation on Windows 7 or later.
Running backup software
Backup software is another category of software that will possibly cause the Edge blockstore cache to wrap and evict the working set of data, especially during the execution of full backups. In a SteelFusion deployment, run differential, incremental, synthetic full, and a full backup at the data center.
Configure jumbo frames
If jumbo frames are supported by your network infrastructure, use jumbo frames between Core and storage arrays. We make the same recommendation for the Edge and any external application servers (not hosted within VSP) that are using exports from the Edge. The application server interfaces must support jumbo frames. For details, see “Configuring Edge for jumbo frames” on page 47.
Removing Core from Edge and re-adding Core
When a Core is removed from the Edge with the “preserve config” setting enabled, the Edge local exports are saved and the offline remote exports are removed from the Edge configuration. On the Core, there is no change, but the exports show as “Not connected.” In most scenarios, the reason for this procedure is replacement of the Core.
If a new Core is added to the Edge, the Edge local storage is merged from the Edge to the Core and normal operations are resumed.
However, if for some reason the same Core is added back, the Edge local storage information on the Core must be cleared from the Core by removing any entries for the specific Edge local exports, before the add operation is performed.
Note: As long as the Edge and Core are truly disconnected when the Edge local storage entries are removed, no Edge local storage is physically deleted. Only the entries themselves are cleared.
Once the same Core is added back again, the Edge local storage information, on the Edge, is merged to the Core configuration. At the same time, the remote offline storage information on the Core that is mapped to the Edge is merged across to the Edge.
Failure to clear the information prior to re-adding the Core will result in the Core rejecting the Edge connection and both the Core and Edge logfiles reporting a “Config mismatch.”
Further details are available on the Riverbed Support site, in Knowledge Base article S30272 - https://supportkb.riverbed.com/support/index?page=content&amp;id=S30272.
General purpose file server
SteelFusion Edge with NFS is not designed to be a general purpose NFS file server and is not supported in this role.
Alternative ports for VIP address
If additional nonbypass ports are available in the RiOS node due to the installation of an optional network interface card (NIC), the VIP address can be assigned on those ports.
Unmounting ESXi datastores
Before unmapping or taking offline any NFS exports on the Edge, ensure that the ESXi datastore that corresponds to the exported fileshare has been unmounted.
Sparse files
Exported fileshares that contain sparse files can be mounted by the Core and projected to the Edge. However, SteelFusion does not maintain the sparseness and, therefore, pinning the exported fileshare may not be possible. If a sparse file is created through the Edge, it will be correctly synchronized back to the backend NFS file server. But if the export that contains the sparse file is off-lined or unmounted, the sparseness is not maintained following a subsequent on-line or mount operation.
Changing Edge VIP address
If you need to change the VIP address on the Edge appliance, you may need to remount the datastores. In this scenario, a VIP address change will also affect the UUID generated by the ESXi server. For more details on additional administration steps required, see the Knowledge Base article on the Riverbed Support site: https://supportkb.riverbed.com/support/index?page=content&id=S30235.
Replacing an Edge
Replacing an Edge as part of an RMA operation will require an extra configuration task if the Edge is part of an NFS deployment. By default, a replacement Edge will be shipped with a standard software image that would normally operate in block storage mode. As part of the procedure to replace an Edge, if necessary, the Edge software should first be updated to a version supporting NFS (minimum version 5.0). If the Edge needs to be upgraded to a version that supports NFS, make sure both image partitions are updated so the Edge isn’t accidentally rebooted to a version that doesn’t support NFS mode. Once the required version of software is installed, the Edge needs to be configured to operate in NFS mode. To complete this task, you will need to enter the following commands using the CLI:
Edge> enable
Edge# config t
Edge(config)#service reset set-mode-file
Edge(config)#service reset set-mode-file confirm
After the command is executed, the SteelFusion appliance reboots. Use the show service CLI command to verify the mode of the appliance.
If you have any concerns about performing this task, seek guidance by opening a case with Riverbed Support.
For a more detailed step-by-step guide, see this Knowledge Base article on the Riverbed support site:
https://supportkb.riverbed.com/support/index?page=content&id=S30298
Once the Edge is configured for NFS operations, it can be configured with the relevant storage settings.
Operating system patching
This section contains the following topics:
•  “Patching at the branch office for virtual servers installed on NFS exports” on page 120
•  “Patching at the data center for virtual servers installed on NFS exports” on page 120
Patching at the branch office for virtual servers installed on NFS exports
You can continue to use the same or existing methodologies and tools to perform patch management on physical or virtual branch servers booted over the WAN using SteelFusion appliances.
Patching at the data center for virtual servers installed on NFS exports
If you want to perform virtual server patching at the data center and save a round-trip of patch software from the data center to the branch office, use the following procedure.
To perform virtual server patching at the data center
1. At the branch office:
•  Power down the virtual machine.
•  Take the NFS datastore offline.
2. At the data center:
•  Take the export on the Core offline.
•  Mount the export to a temporary ESX server.
•  Power up the virtual machine, and apply patches and file system updates.
•  Power down the virtual machine.
•  Take the NFS datastore offline.
•  Bring the export on the Core online.
3. At the branch office:
•  Bring the NFS datastore online.
•  Boot up the virtual machine.
If the export was previously pinned at the edge, patching at the data center can potentially invalidate the cache. If this is the case, you might need to prepopulate the export.
Related information
•  SteelFusion Core User Guide
•  SteelFusion Edge User Guide
•  SteelFusion Core Installation and Configuration Guide
•  SteelFusion Command-Line Interface Reference Manual
•  Fibre Channel on SteelFusion Core Virtual Edition Solution Guide
•  Riverbed Splash at https://splash.riverbed.com/community/product-lines/steelfusion