Viewing Diagnostic Reports and Logs
This chapter describes how to view diagnostic reports and generate user and system logs to evaluate performance or troubleshoot the system. It includes these sections:
SCC System
Viewing SCC alarm status reports
Viewing SCC CPU utilization reports
Viewing SCC memory paging reports
SCC Logs
Viewing SCC user and system logs
Downloading SCC user and system logs
SCC Dumps
Generating SCC system dumps
Viewing SCC process dump files
Creating SCC TCP dump files
Appliance System
Viewing appliance details reports
Viewing health check details reports
Viewing appliance CPU utilization reports
Viewing appliance memory paging reports
Viewing appliance TCP memory reports
Appliance Logs
Downloading appliance logs
Generating appliance system dumps
Creating Appliance TCP dumps
Viewing appliance expiring certificates
Viewing SCC alarm status reports
The appliance tracks key hardware and software metrics and alerts you of any potential problems so you can quickly discover and diagnose issues.
RiOS 7.0 and later feature better alarm reporting using hierarchical alarms. The system groups certain alarms into top-level categories, such as the SSL Settings alarm. When an alarm triggers, its parent expands to provide more information. For example, the System Disk Full top-level alarm aggregates over multiple partitions. If a specific partition is full, the System Disk Full alarm triggers and the Alarm Status report displays more information regarding that partition caused the alarm to trigger.
The health of the SCC falls into one of these states:
Healthy - The SCC is in a healthy state.
Needs Attention - The SCC is in a healthy state indicating management-related issues aren’t affected but something may need to be looked it. For example, the license may need to be reviewed.
Degraded - The SCC has detected an issue.
Critical - The SCC has encountered a critical issue that needs to be addressed immediately.
The health of a managed appliance on the SCC falls into one of these states:
Needs Attention - Accompanies a healthy state to indicate management-related issues not affecting the ability of the SteelHead to optimize traffic.
Degraded - The SteelHead is optimizing traffic but the system has detected an issue.
Admission Control - The SteelHead is optimizing traffic but has reached its connection limit.
Critical - The SteelHead might or might not be optimizing traffic; you must address a critical issue.
Unsupported - The SteelHead is unsupported.
The Alarm Status report provides the status for the SCC alarms and includes this alarm information.
Alarm
Reason
CPU Utilization
Displays an alarm when the system has reached the CPU threshold for any of the CPUs in the appliance. If the system has reached the CPU threshold, check your settings.
If your alarm thresholds are correct, reboot the appliance.
If more than 100 MBs of data is moved through an appliance while performing PFS synchronization, the CPU utilization can become high and result in a CPU alarm. This CPU alarm isn’t cause for concern.
Disk Full
Displays an alarm when the system partitions (not the RiOS data store) are full or almost full. For example, RiOS monitors the available space on /var that’s used to hold logs, statistics, system dumps, TCP dumps, and so on.
This alarm monitors these system partitions:
Partition “/boot Full” Free Space
Partition “/bootmgr Full” Free Space
Partition “/config Full” Free Space
Partition “/data Full” Free Space
Partition “/proxy” Free Space
Partition “/var” Free Space
Hardware
Flash Error - Indicates an error with the flash drive hardware.
IPMI - Indicates an Intelligent Platform Management Interface (IPMI) event. (Not supported on all appliance models.)
This alarm triggers when there has been a physical security intrusion. These events trigger this alarm:
Chassis intrusion (physical opening and closing of the appliance case)
Memory errors (correctable or uncorrectable ECC memory errors)
Hard drive faults or predictive failures
Power supply status or predictive failure
By default, this alarm is enabled.
Licensing
Displays an alarm when your licenses are current.
Autolicense critical event - This alarm triggers on a SteelHead (virtual edition) appliance when the Riverbed Licensing Portal can’t response to a license request with valid licenses. The Licensing Portal can’t issue a valid license for one of these reasons:
A newer SteelHead (virtual edition) appliance is already using the token, so you can’t use it on the SteelHead (virtual edition) appliance displaying the critical alarm. Every time the SteelHead (virtual edition) appliance attempts to refetch a license token, the alarm retriggers.
The token has been redeemed too many times. Every time the SteelHead (virtual edition) appliance attempts to refetch a license token, the alarm retriggers.
Autolicense informational event - This alarm triggers if the Riverbed Licensing Portal has information regarding the licenses for a SteelHead (virtual edition) appliance. For example, the SteelHead (virtual edition) appliance displays this alarm when the portal returns licenses that are associated with a token that has been used on a different SteelHead (virtual edition) appliance.
Insufficient Appliance Management License(s) - This alarm triggers if there aren’t enough licenses to manage all connected appliances.
Invalid License(s) - This alarm triggers if there is any invalid license.
Licenses Expired - This alarm triggers if one or more features has at least one license installed, but all of them are expired.
Licenses Expiring - This alarm triggers if the license for one or more features is going to expire within two weeks.
License(s) Missing - This alarm triggers if any licenses are missing.
Note: The licenses expiring and licenses expired alarms are triggered per feature. For example: if you install two license keys for a feature, LK1-FOO-xxx (expired) and LK1-FOO-yyy (not expired), the alarms don’t trigger, because the feature has one valid license.
Link Duplex
Displays an alarm and sends an email notification when an interface wasn’t configured for half-duplex negotiation but has negotiated half-duplex mode. Half-duplex significantly limits the optimization service results.
The alarm displays which interface is triggering the duplex alarm.
By default, this alarm is enabled.
Link I/O Errors
Displays an alarm when the error rate on an aux or primary interface has exceeded 0.1 percent while either sending or receiving packets. This threshold is based on the observation that even a small link error rate reduces TCP throughput significantly. A properly configured LAN connection experiences very few errors. The alarm clears when the error rate drops below 0.05 percent.
The alarm clears when the rate drops below 0.05 percent.
Link State
Displays an alarm and sends an email notification if an Ethernet link is lost due to an unplugged cable or dead switch port. Depending on that link is down, the system can no longer be optimizing and a network outage could occur.
This condition is often caused by surrounding devices, like routers or switches, interface transitioning. This alarm also accompanies service or system restarts on the appliance.
For aux and primary interfaces.
By default, this alarm is disabled.
Memory Paging
Displays an alarm when the system has reached the memory paging threshold. If 100 pages are swapped approximately every two hours the SteelHead is functioning properly. If thousands of pages are swapped every few minutes, then reboot the SteelHead. If rebooting doesn’t solve the problem, contact Riverbed Support at https://support.riverbed.com.
Process Dump Creation
Displays an alarm when the system has detected an error while trying to create a process dump. This alarm indicates an abnormal condition where RiOS can’t collect the core file after three retries. It can be caused when the /var directory that’s used to hold system dumps is reaching capacity or other conditions. When this alarm is raised, the directory is blacklisted.
SCC Appliance Configuration Backup
Displays an alarm when the daily back up has failed.
SCC External Configuration Backup/Restore
Displays an alarm when the external configuration backup has failed. It updates every 30 seconds.
SCC External Statistics Backup/Restore
Displays an alarm when the external statistics backup has failed. It updates every 30 seconds.
SCC Underprovisioned Virtual Machine
Displays an alarm when the an under provisioned virtual SteelHead is detected.
Secure Vault
Enables an alarm and sends an email notification if the system encounters a problem with the secure vault:
Secure Vault Locked - Needs Attention - Indicates that the secure vault is locked. To optimize SSL connections or to use RiOS data store encryption, the secure vault must be unlocked. Choose Appliance > Secure Vault and unlock the secure vault.
SSL
Enables an alarm if an error is detected in your SSL configuration:
Non-443 SSL Servers - Indicates that during a RiOS upgrade (for example, from 5.5 to 6.0), the system has detected a preexisting SSL server certificate configuration on a port other than the default SSL port 443. SSL traffic can’t be optimized. To restore SSL optimization, you can add an in-path rule to the client-side SteelHead to intercept the connection and optimize the SSL traffic on the nondefault SSL server port.
After adding an in-path rule, you must clear this alarm manually by entering this CLI command:
stats alarm non_443_ssl_servers_detected_on_upgrade clear
SSL Certificates Error - Indicates that an SSL peering certificate has failed to reenroll automatically within the Simple Certificate Enrollment Protocol (SCEP) polling interval.
SSL Certificates Expiring - Indicates that an SSL certificate is about to expire.
SSL Certificates SCEP - Indicates that an SSL certificate has failed to reenroll automatically within the SCEP polling interval.
Temperature
Critical Temperature - Enables an alarm and send an email notification of the CPU temperature exceeds the rising threshold. When the CPU returns to the reset threshold, the critical alarm is cleared. The default value for the rising threshold temperature is 70ºC; the default threshold temperature is 67ºC.
Warning Temperature - Enables an alarm and sends an email notification if the CPU temperature approaches the rising threshold. When the CPU returns to the reset threshold, the waning alarm is cleared.
What this report tells you
The Alarm Status report answers this question:
What is the current status of the SCC?
About report data
The SCC is designed to retain statistics for up to a maximum of three years, based on daily statistics for 2,000 appliances monitoring 50 to100 TCP ports per SteelHead. Factors that can influence this number include the number of monitored TCP ports, the number of active interfaces on managed appliances, and changes in types amounts of data collected in RiOS releases.
The SCC polls data every five minutes. In general, the SCC retains five-minute granularity data points for a maximum of 30 days. One-hour granularity data points are stored for a maximum of 90 days. Beyond 90 days, SCC retains one-day granularity data points for up to three years. In case of stats in excess of capacity, the SCC deletes the oldest data from each of the three granularities, while attempting to preserve as much recent data as it can.
Be aware that if the SCC and remote appliances lose connectivity with each other, the bandwidth and connection data during the period of lost connectivity can be skewed. For example, if a remote appliance loses connectivity with the SCC for six hours, data for the missing six hours appears to be 0 in reports for periods of Last Day or Custom intervals smaller than one day. However, when the remote appliance reestablishes connectivity, it sends an aggregate data point for the last day. Thus, report for periods longer than Last Day do reflect bandwidth and connection data accurately. If you need to analyze data on the remote SteelHead for the missing period, you can view this in the SCC for the individual remote appliance.
To view the SCC Status report
Choose Diagnostics > SCC System: Alarm Status to display the Alarms Status page.
Alarm Status report
Related topics
Configuring alarm parameters
Configuring SNMP basic settings
Viewing SCC CPU utilization reports
The CPU Utilization report summarizes the percentage of the CPU used within the time period specified.
Typically, a SteelHead operates on approximately 30 to 40 percent CPU capacity during nonpeak hours and approximately 60 to 70 percent capacity during peak hours. No single SteelHead CPU usage should exceed 90 percent.
What this report tells you
The CPU Utilization report answers these questions:
How much of the CPU is being used?
What is the average and peak percentage of the CPU being used?
About report graphs
Mouse over a specific data point to see the y values and exact time stamp were in relation to peaks.
About report data
The Riverbed system reports for periods up to one month. Due to performance and disk space considerations, the display granularity decreases with time passed since the data was sampled.
For detailed information about data grouping, see “Data grouping” on page 333.
To view the CPU Utilization report
1. Choose Diagnostics > SCC System: CPU Utilization to display the CPU Utilization page.
CPU Utilization report
2. Use the controls to customize the reports as described in this table.
Control
Description
Time Interval
Select a report time interval of 1 hour (1h), 1 day (1d), 1 week (1w), 30 days (30d), yesterday, last week, or last month.
Time intervals that don’t apply to a particular report are dimmed.
For a custom time interval, enter the start time and end time using the format yyyy/mm/dd hh:mm:ss
You can quickly see the newest data and see data points as they’re added to the chart dynamically. To display the newest data, click Show newest data.
Display Mode
Select one of these displays from the drop-down list:
Brief - Displays the CPU percentages for each RiOS core individually. The individual cores appear with a number and a color in the data series. To hide or display a core in the plot area, select or clear the check box next to the core name.
Detailed - Displays the CPU utilization percentage of all cores combined as a system-wide average.
Cores
Select the cores from the drop-down list.
3. Under Export, complete the configuration as described in this table.
Control
Description
Export To
Select either the Email or URL option from the drop-down list.
Email Addresses
(Only displays when Email is selected.) Specify the email address of the recipient.
Email Subject
(Only displays when Email is selected.) Specify the subject.
Destination URL
(Only displays when URL is selected.) Specify the URL.
Format
Select HTML, CSV, or PDF from the drop-down list.
Per Appliance Report
Enables appliance report settings.
Note: Generates graphs per appliance for HTML/PDF reports.
Note: Generates one CSV per appliance for CSV reports.
Export Now
Select Export Now and click Export to start the export immediately.
Schedule Export
Select Schedule Export and specify the start date, time, and frequency of the export. Use this format: yyyy/mm/dd hh:mm:ss
Export
Click Export to export the configuration.
Printable view
Displays the print menu.
Viewing SCC memory paging reports
The Memory Paging report provides the total number of memory pages, per second, utilized in the time period specified. It includes a table of statistics that describes memory paging activity for the time period you specify.
Field
Description
Pages Swapped Out
Specifies the total number of pages swapped. If 100 pages are swapped approximately every two hours the SteelHead is functioning properly. If thousands of pages are swapped every few minutes, contact Riverbed Support at https://support.riverbed.com.
What this report tells you
The Memory Paging report answers these questions:
How much memory is being used?
What is the average and peak amount of memory pages swapped?
About report graphs
Mouse over a specific data point to see the y values and exact time stamp.
For detailed information about data grouping, see “Data grouping” on page 333.
To view the memory paging report
1. Choose Diagnostics > SCC System: Memory Paging to display the Memory Paging page.
Memory Paging report
2. Use the controls to customize the reports as described in this table.
Control
Description
Time Interval
Select a report time interval of 1 hour (1h), 1 day (1d), 1 week (1w), 30 days (30d), yesterday, last week, or last month.
Time intervals that don’t apply to a particular report are dimmed.
For a custom time interval, enter the start time and end time using the format yyyy/mm/dd hh:mm:ss
You can quickly see the newest data and see data points as they’re added to the chart dynamically. To display the newest data, click Show newest data.
Page Swap Out Rate
Specifies the total number of pages swapped per second. If 100 pages are swapped approximately every two hours the SteelHead is functioning properly. If thousands of pages are swapped every few minutes, contact Riverbed Support at https://support.riverbed.com.
3. Under Export, complete the configuration as described in this table.
Control
Description
Export To
Select either the Email or URL option from the drop-down list.
Email Addresses
(Only displays when Email is selected.) Specify the email address of the recipient.
Email Subject
(Only displays when Email is selected.) Specify the subject.
Destination URL
(Only displays when URL is selected.) Specify the URL.
Format
Select HTML, CSV, or PDF from the drop-down list.
Per Appliance Report
Enables appliance report settings.
Note: Generates graphs per appliance for HTML/PDF reports.
Note: Generates one CSV per appliance for CSV reports.
Export Now
Select Export Now and click Export to start the export immediately.
Schedule Export
Select Schedule Export and specify the start date, time, and frequency of the export. Use this format: yyyy/mm/dd hh:mm:ss
Export
Click Export to export the configuration.
Printable view
Displays the print menu.
Viewing SCC user and system logs
SCC logs provide a high-level view of network activity.
Viewing user logs reports
Viewing system logs reports
Viewing user logs reports
You can view user logs in the View User Logs page. View users logs to monitor user activity. The user log filters messages from the system log to display messages that are of immediate use to the system administrator.
View user logs to monitor system activity and to troubleshoot problems. For example, you can monitor who logged in, who logged out, and who entered particular CLI commands, alarms and errors. The most recent log events are listed first.
To view user logs
1. Choose Diagnostics > SCC Logs: User Logs to display the User Logs page.
User Logs report
2. Use the controls to customize the report as described in this table.
Control
Description
Show
Select Current Log or one of the archived logs from the drop-down list.
Lines per page
Specify the number of lines you want to display on the page.
Jump to
Select one of these options from the drop-down list:
Page - Specify the number of pages you want to display.
Time - Specify the time (yyyy/mm/dd hh:mm:ss) of the pages you want to display.
Filter
Select one of these options from the drop-down list:
Regular Expression - Specifies only those connections that match the expression used to filter the display.
Use this format in the text field:
x.x.x.x[/mask][:port]
Error or higher - Displays Error level logs or higher.
Warning or higher - Displays Warning level logs or higher.
Notice or higher - Displays Notice level logs or higher.
Info or higher - Displays Info level logs or higher.
3. Click Go to apply the changes to the report display.
Viewing system logs reports
You can view system logs reports in the System Logs page. View System logs to monitor system activity and to troubleshoot problems. The most recent log events are listed first.
To view system logs
1. Choose Diagnostics > SCC Logs: System Logs to display the System Logs page.
System Logs report
2. Use the controls to customize the report as described in this table.
Control
Description
Show
Select Current Log or one of the archived logs from the drop-down list.
Lines per page
Specify the number of lines you want to display on the page.
Jump to
Select one of these options from the drop-down list:
Page - Specify the number of pages you want to display.
Time - Specify the time for the log you want to display.
Filter
Select one of these options from the drop-down list:
Regular Expression - Specify only those connections that match the expression used to filter the display.
Use this format in the text field:
x.x.x.x[/mask][:port]
Error or higher - Displays the Error level logs or higher.
Warning or higher - Displays the Warning level logs or higher.
Notice or higher - Displays the Notice level logs or higher.
Info or higher - Displays the Info level logs or higher.
3. Click Go to apply the changes to the report display.
Downloading SCC user and system logs
This section describes how to download user and system log files. It includes these sections:
Downloading user logs reports
Downloading system log files reports
Downloading user logs reports
You can download user logs in the User Logs Download page.
To download user logs
1. Choose Diagnostics > SCC Logs: User Logs Download to display the User Logs Download page.
User Logs Download report
2. Click the name of the log to save the log to disk.
You can download both compressed and uncompressed logs.
3. Click Rotate Logs to archive the current log to a numbered archived log file and then clear the log so that it is empty again.
Downloading system log files reports
You can download system logs reports in the System Logs Download page. Download system logs to monitor system activity and to troubleshoot problems.
To download system logs
1. Choose Diagnostics > SCC Logs: System Logs Download to display the System Logs Download page.
2. Click the name of the log to save the log to disk.
You can download both compressed and uncompressed logs.
3. Click Rotate Logs to archive the current log to a numbered archived log file and then clear the log so that it is empty again.
Generating SCC system dumps
You can generate, display, and download system dumps in the System Dump page. A system dump contains a copy of the kernel data on the system. System dump files can help you diagnose problems in the system.
To view system dump files
1. Choose Diagnostics > SCC Dumps: System Dumps to display the System Dumps page.
System Dumps report
2. Use the controls to customize the report as described in this table.
Control
Description
Appliance System Dump
Select the appliance from the drop-down list.
System Dump
Select the check box next to the system dump.
Generate System Dump
Select the type of information to include in the report:
Include Statistics- Select the check box to collect and include CPU, memory, and other statistics in the system dump (this option is enabled by default). The statistics are useful while analyzing traffic patterns to correlate to an issue. The system adds the statistics to a file in the sysdump called stats.tgz.
Include All Logs - Select the check box to remove the 50 MB limit for compressed log files, to include all logs in the system dump.
Generate System Dump
Click Generate System Dump to generate the new system dump.
System Dumps generates on the selected appliance and displays in the Operations section with a pending status.
Remove Selected
Select the check box next to the name and click Remove Selected.
Because generating a system dump can take a while, a spinner appears during the system dump creation. When the system dump is complete, it appears in the list of links to download.
To upload to Riverbed Support
1. Choose Diagnostics > SCC Dumps: System Dumps to display the System Dumps page.
2. Select a system dump file to view the information.
3. Use the controls to customize the report as described in this table.
Control
Description
Download
Click Download to resize a copy of the system dump file.
Upload to Riverbed Support
Enter the case number or the URL and click Upload to start the upload process.
Viewing SCC process dump files
You can display and download process dumps in the Process Dumps page. A process dump is a saved copy of memory including the contents of all memory, bytes, hardware registers, and status indicators. It is periodically taken to restore the system in the event of failure. Process dump files can help you diagnose problems in the system.
To view system dump files
1. Choose Diagnostics > SCC Dumps: Process Dumps to display the Process Dumps page.
Process Dumps report
2. Click the filename to open a file or save the file to disk.
3. Click Download to receive a copy of the snapshot file.
4. To remove an entry, select the check box next to the name and click Remove Selected.
To upload to Riverbed Support
1. Choose Diagnostics > SCC Dumps: Process Dumps to display the Process Dumps page.
2. Select a process dump file to view the information.
Uploading to Riverbed Support
3. Use the controls to customize the report as described in this table.
Control
Description
Download
Click Download to receive a copy of the snapshot file.
Upload to Riverbed Support
Enter the case number or the URL and click Upload to start the upload process.
Creating SCC TCP dump files
You can capture, download, and upload TCP dumps in the TCP Dumps page.
Capture files contain summary information for every Internet packet received or transmitted on the interface to help diagnose problems in the system.
RiOS provides an easy way to create and retrieve multiple capture files from the SCC. You can create capture files from multiple interfaces at the same time, limit the size of the capture file, and schedule a specific date and time to create a capture file. Scheduling and limiting a capture file by time or size allows unattended captures.
The top of the TCP Dumps page displays a list of existing capture files and the bottom of the page displays controls to create a capture file. The bottom of the page also includes the capture files that are currently running, and controls to create a trigger that stops a capture when a specific event occurs. The Running Capture Name list includes captures running at a particular time. It includes captures started manually and also any captures that were scheduled previously and are now running.
To view TCP dump files
1. Choose Diagnostics > SCC Dumps: TCP Dumps to display the TCP Dumps page.
2. Complete the configuration as described in this table.
Control
Description
Add a New TCP Dump
Displays the controls for creating a TCP trace dump.
Capture Name
Specify the name of the capture file. Use a unique filename to prevent overwriting an existing TCP dump. The default filename uses this format:
<hostname>_<interface>_<time-stamp>.cap
Where <hostname> is the hostname of the SCC, <interface> is the name of the interface selected for the trace (for example, lan0_0, wan0_0), and <time-stamp> is in the yyyy/mm/dd hh:mm:ss format.
If this trace dump relates to an open Riverbed Support case, specify the capture filename case_<number> where <number> is your Riverbed Support case number: for example, case_12345.
Note: The .cap file extension isn’t included with the filename when it appears in the capture queue.
Capture Traffic Between
IPs - Specify the source IP addresses. Separate multiple IP addresses with a comma to include all addresses bidirectionally. The default setting is all IP addresses.
Ports - Specify the source ports. Separate multiple ports with a comma. The default setting is all ports.
and:
IPs - Specify the destination IP addresses. Separate multiple IP addresses with a comma to include all addresses bidirectionally. The default setting is all IP addresses.
Ports - Specify the destination ports. Separate multiple ports with a comma. The default setting is all ports.
Capture Interfaces
Captures the TCP trace dump on the selected interface(s). You can select all interfaces or a base, in-path, or RSP interface. The default setting is none. You must specify a capture interface.
If you select several interfaces at a time, the data is automatically placed into separate capture files.
Capture Parameters
Specify the parameters:
Capture Untagged Traffic Only - Captures only traffic without a VLAN tag. Enabling this setting filters the trace dump by capturing all untagged packets.
Capture VLAN-Tagged Traffic Only - Captures only VLAN-tagged packets within a trace dump for a trunk port (802.1Q). Enabling this setting filters the trace dump by capturing only VLAN-tagged packets. This setting applies to physical interfaces only because logical interfaces (inpath0_0, mgmt0_0) don’t recognize VLAN headers.
Capture both VLAN and Untagged Traffic - Captures VLAN-tagged and untagged packets within a trace dump.
Capture Duration - Specify how long the capture runs, in seconds. The default value is 30. Specify 0 or continuous to initiate a continuous trace. When a continuous trace reaches the maximum space allocation of 100 MB, the oldest file is overwritten.
Maximum Capture Size (MB) - Specify the maximum capture file size, in megabytes. The default value is 100. We recommend a maximum capture file size of 1024 MB (1 GB).
Buffer Size - Optionally, specify the maximum amount of data, in kilobytes, allowed to queue up while awaiting processing by the TCP trace dump. The default value is
154 KB.
Snap Length - Optionally, select the snap length value for the capture file or specify a custom value. The snap length equals the number of bytes the report captures for each packet. Having a snap length smaller than the maximum packet size on the network enables you to store more packets, but you might not be able to inspect the full packet content. The default value is 1518 bytes.
Select 65535 for a full packet capture (recommended for CIFS, MAPI, and SSL captures). When using jumbo frames, we recommend selecting 9018. The default custom value is 16383 bytes.
Number of Files to Rotate - Specify how many TCP trace dump files to rotate. The default value is 5.
Custom Flags - Specify custom flags to capture unidirectional traces. Examples:
To capture all traffic to or from a single host
host x.x.x.x
To capture all traffic between a pair of hosts
host x.x.x.x and host y.y.y.y
To capture traffic between two hosts and two SteelHead inner channels:
(host x.x.x.x and host y.y.y.y) or (host a.a.a.a and host b.b.b.b)
Schedule Dump
Schedules the trace dump to run at a later date and time.
Start Date - Specify a date to initiate the trace dump in this format: yyyy/mm/dd
Start Time - Specify a time to initiate the trace dump in this format: hh:mm:ss
Add
Adds the TCP trace dump to the capture queue.
Tip: To remove an entry, select the check box next to the name and click Remove Selected.
To view TCP trace dump files
1. Choose Diagnostics > SCC Dumps: TCP Dumps to display the TCP Dumps page.
2. Under Stored TCP Dumps, click the trace dump name to open the file.
Tip: To print the TCP dump, select the trace dump filename under Download Link. When the file opens, choose File > Print in your web browser to open the Print dialog box.
Tip: To remove an entry, check the box next to the name in the TCP dump list and click Remove Selected.
To stop a running TCP trace dump
1. Choose Diagnostics > SCC Dumps: TCP Dumps to display the TCP Dumps page.
2. Click the trace dump filename in the Running Capture Name list.
3. Click Stop Selected Captures.
To upload the trace to Riverbed Support
In continuous mode, after you complete the capture, perform these steps:
(For timed TCP dumps, start with step 2.)
1. On the TCP Dumps page, select the running TCP Dump and click Stop Selected Captures.
The trace appears as a download link in the list of TCP Dumps stored on the SteelHead.
2. Click the top file in the TCP Dumps list and save it locally.
This file should contain the current date.
3. Compress (zip) the file and follow the upload instructions to share it with Riverbed Support:
Attach the files to your case at
https://support.riverbed.com/cases/viewcases.htm
-or-
Upload the files to FTP://ftp.riverbed.com/incoming
(for FTP, be sure the file is prefixed with case_<number>).
ftp ftp.riverbed.com
User: anonymous
Password: your_email@address
ftp> cd /incoming
ftp> bi
ftp> put case_12345-tcpdump.zip
Viewing appliance details reports
The Appliance Details report displays details about the connected appliances such as status, performance, connection counts, and peers.
What this report tells you
The Appliance Details report answers these questions:
What is the model number?
What is hardware revision type?
What is the serial number and the software version number currently installed on the appliance?
Is the synchronization connection active?
Is the SteelHead in the Catch-up or Keep-up phase of RiOS data store synchronization?
What percentage of the RiOS data store is unused?
About report data
The SCC is designed to retain statistics for up to a maximum of three years, based on daily statistics for 2,000 appliances monitoring 50 to100 TCP ports per SteelHead. Factors that can influence this number include the number of monitored TCP ports, the number of active interfaces on managed appliances, and changes in types amounts of data collected in RiOS releases.
The SCC polls data every five minutes. In general, the SCC retains five minute granularity data points for a maximum of 30 days. One hour granularity data points are stored for a maximum of 90 days. Beyond 90 days, SCC retains one day granularity data points for up to three years. In case of stats in excess of capacity, the SCC deletes the oldest data from each of the three granularities, while attempting to preserve as much recent data as it can.
For details about alarms for SteelHead CX, see the SteelHead User Guide for SteelHead CX.
For details about alarms for SteelHead EX, see the SteelHead User Guide for SteelHead EX.
For details about alarms for the Interceptor, see the SteelHead Interceptor User Guide.
For details about alarms for Mobile Controller, see the SteelCentral Controller for SteelHead Mobile User Guide.
To view the appliance details report
1. Choose Diagnostics > Appliance System: Appliance Details to display the Appliance Details page.
Appliance Details report
2. Select the appliance from the drop-down list.
3. Under Peers, you can view this information for each appliance that is peered with the selected appliance.
Field
Description
IP Address
Displays the IP address of the peer appliance.
Name
Displays the name of the peer appliance.
Model
Displays the model of the peer appliance.
Version
Displays the software version of the appliance’s operating system.
Licenses
Displays the licenses applied to the appliance.
4. Under Latency Detected Peers, you can view this information about appliances peered with the selected appliance and that have low-latency connections.
Field
Description
Peer IP / Hostname
Displays the hostname or IP address of the latency-detected peer appliance.
Latency
Displays the amount of latency between the selected appliance and the peer in milliseconds.
Cumulative Optimized Connections
Displays the number of optimized connections between the selected appliance and the peer.
Cumulative Passthrough Connections
Displays the number of passthrough connections between the selected appliance and the peer.
Current Peer State
Displays the state of the peer ‑ Optimized or Passthrough ‑ based on the current latency between the selected appliance and the peer.
5. Under Status, you can view this information as described in this table.
Field
Description
Status
Displays the status of the appliance:
Appliance Reported Health - Provides the health status as reported by the managed appliance: Healthy, Healthy: Needs Attention, Degraded, and Critical. Also provides hardware model number, software version details, and links to the appliance logs.
SCC Evaluated Health - Displays the appliance health status as evaluated by the SCC. The SCC can detect additional problems that the managed appliance can’t.
Model - Displays the model number.
Appliance Version - Displays the appliance version.
Detailed Appliance Version - Displays detailed information about the appliance.
RiOS Version - Displays the RiOS version.
SteelFusion Version - Displays the SteelFusion version.
Current ESXi Version - Displays the current ESXi version.
Original ESXi Version - Displays the original ESXi version.
ESXi Support Status - Displays the ESXi support status.
SCC Managed Appliance Alarms
Displays the different SCC alarms. It displays these SCC alarms:
Appliance too slow to respond - It updates every five minutes.
Configuration Change - It updates every five minutes.
Duplex Interface - It updates every five minutes.
High Appliance Usage Warning - It updates every five minutes.
PFS and RSP enabled together - It updates every five minutes.
Time drift - It updates every five minutes.
Too Many Half Open/Closed Connections - It updates every 30 seconds.
Unmanaged Appliances - It updates every 3 hours.
For details about alarms, see “Configuring alarm parameters” on page 76.
6. Under Appliance Reported Alarms, you can view the information as described in this table.
Control
Description
Admission Control
Enables an alarm and sends an email notification if the appliance enters admission control. When this occurs, the appliance optimizes traffic beyond its rated capability and is unable to handle the amount of traffic passing through the WAN link. During this event, the appliance continues to optimize existing connections, but new connections are passed through without optimization.
Connection Limit - Indicates the system connection limit has been reached. Additional connections are passed through unoptimized. The alarm clears when the appliance moves out of this condition.
CPU - The appliance has entered admission control due to high CPU use. During this event, the appliance continues to optimize existing connections, but new connections are passed through without optimization. The alarm clears automatically when the CPU usage has decreased.
MAPI - The total number of MAPI optimized connections has exceeded the maximum admission control threshold. By default, the maximum admission control threshold is 85 percent of the total maximum optimized connection count for the client-side appliance. The appliance reserves the remaining 15 percent so that the MAPI admission control doesn’t affect the other protocols. The 85 percent threshold is applied only to MAPI connections. RiOS is now passing through MAPI connections from new clients but continues to intercept and optimize MAPI connections from existing clients (including new MAPI connections from these clients). RiOS continues optimizing non-MAPI connections from all clients. The alarm clears automatically when the MAPI traffic has decreased; however, it can take one minute for the alarm to clear.
In RiOS 7.0, RiOS preemptively closes MAPI sessions to reduce the connection count in an attempt to bring the appliance out of admission control by bringing the connection count below the 85 percent threshold. RiOS closes the MAPI sessions in this order:
MAPI prepopulation connections
MAPI sessions with the largest number of connections
MAPI sessions with most idle connections
Most recently optimized MAPI sessions or oldest MAPI session
MAPI sessions exceeding the memory threshold
Memory - The appliance has entered admission control due to memory consumption. The appliance is optimizing traffic beyond its rated capability and is unable to handle the amount of traffic passing through the WAN link. During this event, the appliance continues to optimize existing connections, but new connections are passed through without optimization. No other action is necessary; the alarm clears automatically when the traffic has decreased.
TCP - The appliance has entered admission control due to high TCP memory use. During this event, the appliance continues to optimize existing connections, but new connections are passed through without optimization. The alarm clears automatically when the TCP memory pressure has decreased.
By default, this alarm is enabled.
Asymmetric Routing
Enables an alarm if asymmetric routing is detected on the network. This is usually due to a failover event of an inner router or VPN.
By default, this alarm is enabled.
Connection Forwarding
Enables an alarm if the system detects a problem with a connection-forwarding neighbor. The connection-forwarding alarms are inclusive of all connection-forwarding neighbors. For example, if an appliance has three neighbors, the alarm triggers if any one of the neighbors are in error. In the same way, the alarm clears only when all three neighbors are no longer in error.
Cluster IPv6 Incompatible - Enables an alarm and sends an email notification if a connection-forwarding neighbor is running a RiOS version that’s incompatible with IPv6, or if the IP address configuration between neighbors doesn’t match. Neighbors must be running RiOS 8.5.
Multiple Interface - Enables an alarm and sends an email notification if the connection to an appliance in a connection forwarding cluster is lost.
Single Interface - Enables an alarm and sends an email notification if the connection to a SteelHead connection forwarding neighbor is lost.
By default, this alarm is enabled.
CPU Utilization
Enables an alarm and sends an email notification if the average and peak threshold for the CPU utilization is exceeded. When an alarm reaches the rising threshold, it is activated; when it reaches the lowest or reset threshold, it is reset. After an alarm is triggered, it isn’t triggered again until it has fallen below the reset threshold.
By default, this alarm is enabled, with a rising threshold of 90 percent and a reset threshold of 70 percent.
Rising Threshold - Specify the rising threshold. When an alarm reaches the rising threshold, it is activated. The default value is 90 percent.
Reset Threshold - Specify the reset threshold. When an alarm reaches the lowest or reset threshold, it is reset. After an alarm is triggered, it isn’t triggered again until it has fallen below the reset threshold. The default value is 70 percent.
Data Store
Data Store Clean Required - Enables an alarm and sends an email notification if you need to clear the RiOS data store.
Corruption - Enables an alarm and sends an email notification if the RiOS data store is corrupt or has become incompatible with the current configuration. To clear the RiOS data store of data, restart the optimization service and click Clear the Data Store.
If the alarm was caused by an unintended change to the configuration, the configuration can be changed to match the old RiOS data store settings again and then a service restart (without clearing) will clear the alarm.
Encryption Level Mismatch - Enables an alarm and sends an email notification if a data store error such as an encryption, header, or format error occurs.
Synchronization Error - Enables an alarm if RiOS data store synchronization has failed. The RiOS data store synchronization between two SteelHeads has been disrupted and the RiOS data stores are no longer synchronized.
By default, this alarm is enabled.
Disk Full
Enables an alarm if the system partitions (not the RiOS data store) are full or almost full. For example, RiOS monitors the available space on /var that’s used to hold logs, statistics, system dumps, TCP dumps, and so on.
By default, this alarm is enabled.
This alarm monitors these system partitions:
/Full
/boot Full
/bootmgr
/config Full
/esxi Full
/proxy Full
/scratch Full
/tmp/mnt/config Full
/var Full
Domain Authentication Alert
Indicates that the system is either unable to communicate with the domain controller, or has detected an SMB signing error, or that delegation has failed. CIFS-signed and Encrypted-MAPI traffic is passed through without optimization.
By default, this alarm is enabled.
Domain Join Error
Enables an alarm if an attempt to join a Windows domain has failed. The number one cause of failing to join a domain is a significant difference in the system time on the Windows domain controller and the appliance. A domain join can also fail when the DNS server returns an invalid IP address for the domain controller.
By default, this alarm is enabled.
Link Duplex
Enables an alarm and sends an email notification when an interface wasn’t configured for half-duplex negotiation but has negotiated half-duplex mode. Half-duplex significantly limits the optimization service results.
The alarm displays which interface is triggering the duplex alarm.
By default, this alarm is enabled.
Hardware
Disk Error - Enables an alarm when one or more disks is offline. To see that disk is offline, enter this CLI command from the system prompt:
show raid diagram
By default, this alarm is enabled.
This alarm applies only to the appliance RAID Series 3000, 5000, and 6000.
Fan Error - Enables an alarm and sends an email notification if a fan is failing or has failed and needs to be replaced. By default, this alarm is enabled.
Flash Error - Enables an alarm when the system detects an error with the flash drive hardware. By default, this alarm is enabled.
IPMI - Enables an alarm and sends an email notification if an Intelligent Platform Management Interface (IPMI) event is detected. (Not supported on all appliance models.)
This alarm triggers when there has been a physical security intrusion. These events trigger this alarm:
Chassis intrusion (physical opening and closing of the appliance case)
Memory errors (correctable or uncorrectable ECC memory errors)
Hard drive faults or predictive failures
Power supply status or predictive failure
By default, this alarm is enabled.
Memory Error - Enables an alarm and sends an email notification if a memory error is detected. For example, when a system memory stick fails.
Other Hardware Error - Enables an alarm if a hardware error is detected. These issues trigger the hardware error alarm:
The appliance doesn’t have enough disk, memory, CPU cores, or NIC cards to support the current configuration
The appliance is using a memory Dual In-line Memory Module (DIMM), a hard disk, or a NIC that’s not qualified by Riverbed
Other hardware issues
By default, this alarm is enabled.
Power Supply - Enables an alarm and sends an email notification if an inserted power supply cord doesn’t have power, as opposed to a power supply slot with no power supply cord inserted. By default, this alarm is enabled.
RAID - Enables an alarm and sends an email notification if the system encounters an error with the RAID array (for example, missing drives, pulled drives, drive failures, and drive rebuilds). An audible alarm can also sound. To see if a disk has failed, enter this CLI command at the system prompt:
show raid diagram
For drive rebuilds, if a drive is removed and then reinserted, the alarm continues to be triggered until the rebuild is complete.
Rebuilding a disk drive can take four to six hours.
This alarm applies only to the SteelHead RAID Series 3000, 5000, and 6000.
By default, this alarm is enabled.
 
SSD Write Cycle Level Exceeded - Enables an alarm if the accumulated SSD write cycles exceed a predefined write cycle 95 percent level on appliance models 7050L and 7050M. If the alarm is triggered, the administrator can swap out the disk before any problems arise. For details, see the Riverbed Command-Line Interface Reference Manual.
By default, this alarm is enabled.
Licensing
Enables an alarm and sends an email notification if a license on the appliance is removed, is about to expire, has expired, or is invalid. This alarm triggers if the appliance has no MSPEC license installed for its currently configured model.
Appliance Unlicensed - This alarm triggers if the appliance has no BASE or MSPEC license installed for its currently configured model.
Autolicense Critical Event - This alarm triggers on a SteelHead (virtual edition) when the Riverbed Licensing Portal can’t response to a license request with valid licenses. The Licensing Portal can’t issue a valid license for one of these reasons:
A newer SteelHead (virtual edition) is already using the token, so you can’t use it on the Virtual SteelHead displaying the critical alarm. Every time the SteelHead (virtual edition) attempts to refetch a license token, the alarm retriggers.
The token has been redeemed too many times. Every time the SteelHead (virtual edition) attempts to refetch a license token, the alarm retriggers.
Autolicense Informational Event - This alarm triggers if the Riverbed Licensing Portal has information regarding the licenses for a SteelHead (virtual edition). For example, the SteelHead (virtual edition) displays this alarm when the portal returns licenses that are associated with a token that has been used on a different SteelHead (virtual edition).
Licenses Expired - This alarm triggers if one or more features has at least one license installed, but all of them are expired.
Licenses Expiring - This alarm triggers if the license for one or more features is going to expire within two weeks.
Note: The licenses expiring and licenses expired alarms are triggered per feature. For example, if you install two license keys for a feature, LK1-FOO-xxx (expired) and LK1-FOO-yyy (not expired), the alarms don’t trigger, because the feature has one valid license.
By default, this alarm is enabled.
Link I/O Errors
Enables an alarm and sends an email notification when the link error rate exceeds 0.1 percent while either sending or receiving packets. This threshold is based on the observation that even a small link error rate reduces TCP throughput significantly. A properly configured LAN connection experiences very few errors.
The alarm clears when the rate drops below 0.05 percent.
You can change the default alarm thresholds by entering the alarm link_errors threshold xxxxx CLI command at the system prompt. For details, see the Riverbed Command-Line Interface Reference Manual.
By default, this alarm is enabled.
Link State
Enables an alarm and sends an email notification if an Ethernet link is lost due to a network event. Depending on that link is down, the system can no longer be optimizing and a network outage could occur.
This is often caused by surrounding devices, like routers or switches interface transitioning. This alarm also accompanies service or system restarts on the appliance.
For WAN/LAN interfaces, the alarm triggers if in-path support is enabled for that WAN/LAN pair.
By default, this alarm is disabled.
Memory Paging
Enables an alarm and sends an email notification if memory paging is detected. If 100 pages are swapped every couple of hours, the system is functioning properly. If thousands of pages are swapped every few minutes, contact Riverbed Support at https://support.riverbed.com.
By default, this alarm is enabled.
Neighbor Incompatibility
Enables an alarm if the system has encountered an error in reaching an appliance configured for connection forwarding.
By default, this alarm is enabled.
Network Bypass
Enables an alarm and sends an email notification if the system is in bypass failover mode.
By default, this alarm is enabled.
NFS V2/V4 Alarm
Enables an alarm and sends an email notification if the SteelHead detects that either NFSv2 or NFSv4 is in use. The appliance only supports NFSv3 and passes through all other versions.
By default, this alarm is enabled.
Optimization Service
Internal Error - Enables an alarm and sends an email notification if the RiOS optimization service encounters a condition that can degrade optimization performance. By default, this alarm is enabled.
Service Status - Enables an alarm and sends an email notification if the RiOS optimization service encounters a service condition. By default, this alarm is enabled. The message indicates the reason for the condition. These conditions trigger this alarm:
Configuration errors.
An appliance reboot.
A system crash.
An optimization service restart.
A user enters the CLI command no service enable or shuts down the optimization service from the Management Console.
A user restarts the optimization service from either the SteelCentral Controller for SteelHead appliance or CLI.
Unexpected Halt - Enables an alarm and sends an email notification if the RiOS optimization service halts due to a serious software error. By default, this alarm is enabled.
Process Dump Creation Error
Enables an alarm and sends an email notification if the system detects an error while trying to create a process dump. This alarm indicates an abnormal condition where RiOS can’t collect the core file after three retries. It can be caused when the /var directory is reaching capacity or other conditions. When the alarm is raised, the directory is blacklisted.
By default, this alarm is enabled.
Proxy File Service
Indicates that there has been a Proxy File Service (PFS) operation or configuration error:
Proxy File Service Configuration - Indicates that a configuration attempt has failed. If the system detects a configuration failure, attempt the configuration again.
Proxy File Service Operation - Indicates that a synchronization operation has failed. If the system detects an operation failure, attempt the operation again.
By default, this alarm is enabled.
Path Down
Indicates that one of the predefined paths for a connection is unavailable because it has exceeded either the timeout value for path latency or the threshold for observed packet loss.
When a path fails, the SteelHead directs traffic through another available path. When the original path comes back up, the appliance redirects the traffic back to it.
Proxy File Service
Indicates that there has been a PFS operation or configuration error:
Proxy File Service Configuration - Indicates that a configuration attempt has failed. If the system detects a configuration failure, attempt the configuration again.
Proxy File Service Operation - Indicates that a synchronization operation has failed. If the system detects an operation failure, attempt the operation again.
By default, this alarm is enabled.
Riverbed Service Platform
Enables an alarm for RSP.
By default, this alarm is enabled.
Secure Vault
Enables an alarm and sends an email notification if the system encounters a problem with the secure vault:
Secure Vault Locked - Indicates that the secure vault is locked. To optimize SSL connections or to use RiOS data store encryption, the secure vault must be unlocked. Choose Appliance > Secure Vault and unlock the secure vault.
Secure Vault New Password Recommended - Indicates that the secure vault requires a new, nondefault password. Reenter the password.
Secure Vault Not Initialized - Indicates that an error has occurred while initializing the secure vault. When the vault is locked, SSL traffic isn’t optimized and you can’t encrypt the RiOS data store.
Software Compatibility
Enables an alarm and sends an email notification if the system encounters a problem with software compatibility:
Peer Mismatch - Needs Attention - Indicates that the appliance has encountered another appliance that’s running an incompatible version of system software. Refer to the CLI, SteelCentral Controller for SteelHead, or the SNMP peer table to determine that appliance is causing the conflict. Connections with that peer will not be optimized, connections with other peers running compatible RiOS versions are unaffected. To resolve the problem, upgrade your system software. No other action is required as the alarm clears automatically.
Software Version Mismatch - Degraded - Indicates that the appliance is running an incompatible version of system software. To resolve the problem, upgrade your system software. No other action is required as the alarm clears automatically.
By default, this alarm is enabled.
SSL
Enables an alarm if an error is detected in your SSL configuration.
Non-443 SSL Servers - Indicates that during a RiOS upgrade (for example, from 5.5 to 6.0), the system has detected a preexisting SSL server certificate configuration on a port other than the default SSL port 443. SSL traffic can’t be optimized. To restore SSL optimization, you can add an in-path rule to the client-side appliance to intercept the connection and optimize the SSL traffic on the nondefault SSL server port.
After adding an in-path rule, you must clear this alarm manually by entering this CLI command:
stats alarm non_443_ssl_servers_detected_on_upgrade clear
SSL Certificates Error (SSL CAs) - Indicates that an SSL peering certificate has failed to reenroll automatically within the Simple Certificate Enrollment Protocol (SCEP) polling interval.
SSL Certificates Error (SSL Peering CAs) - Indicates that an SSL peering certificate has failed to reenroll automatically within the Simple Certificate Enrollment Protocol (SCEP) polling interval.
SSL Certificates Expiring - Indicates that an SSL certificate is about to expire.
SSL Certificates SCEP - Indicates that an SSL certificate has failed to reenroll automatically within the SCEP polling interval.
By default, this alarm is enabled.
Storage Profile Switch Failed
Enables and alarm if the storage profile switch encountered a problem.
By default, this alarm is enabled.
System Detail Report
Enables an alarm if a system component has encountered a problem.
By default, this alarm is enabled.
Temperature
Critical Temperature - Enables an alarm and sends an email notification if the CPU temperature exceeds the rising threshold. When the CPU returns to the reset threshold, the critical alarm is cleared. The default value for the rising threshold temperature is 70ºC; the default reset threshold temperature is 67ºC.
Warning Temperature - Enables an alarm and sends an email notification if the CPU temperature approaches the rising threshold. When the CPU returns to the reset threshold, the warning alarm is cleared.
Rising Threshold - Specify the rising threshold (ºC). When an alarm reaches the rising threshold, it is activated. The default value is 70ºC.
Reset Threshold - Specify the reset threshold (ºC). When an alarm reaches the lowest or reset threshold, it is reset. After an alarm is triggered, it isn’t triggered again until it has fallen below the reset threshold. The default value is 67ºC.
 
7. Under Branch Storage, you can view the information as described in this table.
Field
Description
Main Hostname/IP
Displays the main hostname.
IP Address
Displays the IP address of the branch.
Port
Displays the port number.
SteelFusion Edge Identifier
Displays the identifier for SteelFusion Edge. The value must match the value configured on the SteelFusion Edge.
SteelFusion Edge identifiers are case sensitive.
Redundant Remote Interfaces
Displays whether there is a redundant remote interface.
Local Interfaces
Displays information for Target Details, Initiators, Initiator Groups, LUNs, and MPIO.
8. Under Disk Management, you can view the information as described in this table.
Field
Description
Disk Layout Mode
Displays the disk layout mode that allows space for the SteelFusion block store in the Disk Management page on the SteelFusion Edge.
Free disk space is divided between the VSP and the Core block store.
VSP Volume
Displays the free disk space available on the VSP.
SteelFusion Volume
Displays the free disk space available on the SteelFusion block store.
9. Under Performance, you can view the information as described in this table.
Field
Description
Reduction
Displays the total decrease of data transmitted over the WAN.
Peak Throughput
Displays the peak data transmitted.
Data store Usage
Displays the percent of RiOS data store usage.
10. Under Connection Counts, you can view the information as described in this table.
Field
Description
Established (Optimized)
Displays the total established active connections.
Half Opened (Optimized)
Displays the total half-opened active connections. A half-opened connection is a TCP connection in that the connection hasn’t been fully established. Half-opened connections count toward the connection count limit on the appliance because, at any time, they can become a fully opened connection.
If you’re experiencing a large number of half-opened connections, you can consider a more appropriately sized appliance.
Half Closed (Optimized)
Displays the total half-closed active connections. Half-closed connections are connections that the appliance has intercepted and optimized but are in the process of becoming inactive. These connections are counted toward the connection count limit on the appliance. (Half closed connections can remain if the client or server doesn’t close their connections cleanly.)
If you’re experiencing a large number of half-closed connections, you can consider a more appropriately sized appliance.
Pass Through
Displays the total connections passed through, unoptimized when the connection limit has been reached.
Total
Displays the sum of the counts described above.
11. Under Data Store Status, you can view the information as described in this table.
Field
Description
Synchronization Connection
Indicates the status of the connection between the synchronized appliances.
Synchronization Catch-Up
Indicates the status of transferring data between the synchronized appliances. Catch-Up is used for synchronizing data that wasn’t synchronized during the Keep-Up phase.
Synchronization Keep-Up
Indicates the status of transferring new incoming data between the synchronized SteelHeads.
Data Store Percentage Used (Since Last Clear)
Specifies the percentage of the RiOS data store that’s used.
12. Under Cloud Accelerator Status, you can view the information as described in this table.
Field
Description
Portal
Displays the portal name.
Status
Displays the status of the portal (registered/unregistered).
Service Ready
Displays whether the service is ready.
Redirection
Displays the redirection of the cloud accelerator.
Reason
Displays the reason for the redirection.
Last Contact
Displays the name of the last contact.
13. Under Windows Domain/Local Status, you can view in domain mode status and the primary DNS IP address.
14. Under Peers, you can view the IP address, name, model, version, and license information for peer appliances. Select Show Latency Detected Peers to filter the list for peers that pass through traffic because they are close enough that simply passing through traffic is faster than optimizing the traffic.
15. Click View Appliance Config to view the appliance configuration.
16. Under System Details, you can view the information as described in this table.
Field
Description
Module
Specifies the SteelHead module. Select a module name to view details. A right arrow to the left of a module indicates that the report includes detailed information about a submodule. Click the > to view submodule details.
This report examines these modules:
CPU - Displays information on idle time, system time, and user time per CPU.
Memory - Displays information on total, used, and free memory by percentage and in kilobytes.
CIFS - Click the right arrow and the submodule name to view details for unexpected shut downs and round trip statistics.
HTTP - Click the right arrow and the submodule name to view details for the URL Learning, Parse and Prefetch, and Object Prefetch Table optimization schemes.
Intercept - Click the right arrow to view statistics for message queue, GRE, and WCCP. Also includes table length and watchdog status.
 
MAPI - Click the right arrow and the submodule name to view details for:
Accelerators - Displays how many accelerator objects have been created for readahead, writebehind, and cached-mode folder synchronization. One accelerator object corresponds to the optimization of one particular Outlook action:
Readahead - Downloads an email attachment (in noncached Outlook mode or for public folders).
Writebehind - Uploads an email attachment.
Cache-sync - Downloads the new contents of a folder (in cached mode).
 
Requests and responses - Displays the number of MAPI round trips used and saved. Includes the number of responses and faults along with the fault reason. For example, access denied.
MAPI decryption and encryption (RPCCR) - Displays whether MAPI decryption and encryption is enabled. Includes the number of client-side and server-side SteelHead encrypted MAPI sessions, along with details on how many sessions were not encrypted, how many sessions were successfully decrypted and encrypted, how many sessions were passed-through, and how many experienced an authentication failure.
Connection sessions - Displays the number of client-side and server-side SteelHead MAPI sessions, counting the number of MAPI 2000, 2003, 2007, and pass-through sessions.
Oracle Forms - Click the right arrow and submodule name to view details for native and HTTP mode key
Secure Peering - Click the right arrow and submodule name to view details for secure inner channels, including information on certificate and private key validity, peer SteelHead trust, and blacklisted servers.
Splice-policy - Displays future connections matching the entries in the table.
SSL - Displays whether SSL optimization is enabled and details about the SSL configuration such as that advanced settings are in use. Click the right arrow and the submodule name to view details for the SSL outer and inner channels.
Status
Displays one of these results:
OK (Green)
Warning (Yellow)
Error (Red)
Disabled (Gray). Appears when you manually disable the module.
Related topics
Configuring alarm parameters
Viewing health check details reports
The Health Check report displays details about the health of the SteelHead and SteelHead EX.
The SteelHead must be running 6.0 or later to view this page.
The Health Check details report provides these health checks for an appliance.
Field
Description
Gateway Test
Pings each configured gateway.
Cable Swap Test
Tests if LAN and WAN ports are correctly facing their respective networks.
For accurate results, ensure that traffic is running through the appliance and that the topology is supported for this test. For details about topologies, see the SteelHead Deployment Guide.
Duplex Test
Tests a given interface for correct duplex settings.
Peer Reachability Test
Sends a test probe to a specified peer.
IP-Port Reachability Test
Tests if a specified IP address and optional port are connected.
What this report tells you
The Health Check report answers this question:
If LAN or WAN are correctly facing the networks?
About report data
The SCC is designed to retain statistics for up to a maximum of three years, based on daily statistics for 2,000 appliances monitoring 50 to 100 TCP ports per SteelHead. Factors that can influence this number include the number of monitored TCP ports, the number of active interfaces on managed appliances, and changes in types amounts of data collected in RiOS releases.
The SCC polls data every five minutes. In general, the SCC retains five minute granularity data points for a maximum of 30 days. One hour granularity data points are stored for a maximum of 90 days. Beyond 90 days, SCC retains one day granularity data points for up to three years. In case of stats in excess of capacity, the SCC deletes the oldest data from each of the three granularities, while attempting to preserve as much recent data as it can.
Be aware that if the SCC and remote appliances lose connectivity with each other, the bandwidth and connection data during the period of lost connectivity can be skewed. For example, if a remote appliance loses connectivity with the SCC for six hours, data for the missing six hours appears to be 0 in reports for periods of Last Day or Custom intervals smaller than one day. However, when the remote appliance reestablishes connectivity, it sends an aggregate data point for the last day. Thus, report for periods longer than Last Day do reflect bandwidth and connection data accurately. If you need to analyze data on the remote SteelHead for the missing period, you can view this in the SCC for the individual remote appliance.
To view health check details report
1. Choose Diagnostics > Appliance System: Health Check to display the Health Check page.
2. Select the appliance you want to view from the drop-down list to display the Health Check Details page.
Health Check report
3. Select the check box next to the test, and click Run Selected to run the test.
4. Click View Test Output to display the test results.
Viewing appliance CPU utilization reports
The CPU Utilization report summarizes the percentage of the CPU used on the SCC machine within the time period specified. You can display individual cores or an overall average, or both.
For details about operating report controls, see “Overview” on page 330.
Typically, a SteelHead appliance operates on approximately 30 to 40 percent CPU capacity during nonpeak hours and approximately 60 to 70 percent capacity during peak hours. No single SteelHead CPU usage should exceed 90 percent.
What this report tells you
The CPU Utilization report answers these questions:
How much of the CPU is being used?
What is the average and peak percentage of the CPU being used?
About report graphs
Mouse over a specific data point to see the y values and exact time stamp were in relation to the peaks.
The CPU Utilization and Memory Paging report pages expect the SCC to receive data from any SteelHead every five minutes. If the report displays some discontinuities in the graph, it is because the two data points (five minutes after and ten minutes after the previous data sample) are missing from the SCC's perspective.
For detailed information about data grouping, see “Data grouping” on page 333.
To view the CPU Utilization report
1. Choose Diagnostics > Appliance System: CPU Utilization to display the CPU Utilization page.
CPU Utilization report
2. Use the controls to customize the report as described in this table.
Control
Description
Time Interval
Select a report time interval of 1 hour (1h), 1 day (1d), 1 week (1w), 30 days (30d), yesterday, last week, or last month.
Time intervals that don’t apply to a particular report are dimmed.
For a custom time interval, enter the start time and end time using the format yyyy/mm/dd hh:mm:ss
You can quickly see the newest data and see data points as they’re added to the chart dynamically. To display the newest data, click Show newest data.
Appliance
Select an appliance from the drop-down list.
Force best granularity
Select the check box to force the data.
3. Under Export, complete the configuration as described in this table.
Control
Description
Export To
Select either the Email or URL option from the drop-down list.
Email Addresses
(Only displays when Email is selected.) Specify the email address of the recipient.
Email Subject
(Only displays when Email is selected.) Specify the subject.
Destination URL
(Only displays when URL is selected.) Specify the URL.
Format
Select HTML, CSV, or PDF from the drop-down list.
Per Appliance Report
Enables appliance report settings.
Note: Generates graphs per appliance for HTML/PDF reports.
Note: Generates one CSV per appliance for CSV reports.
Export Now
Select Export Now and click Export to start the export immediately.
Schedule Export
Select Schedule Export and specify the start date, time, and frequency of the export. Use this format: yyyy/mm/dd hh:mm:ss
Export
Click Export to export the configuration.
Printable view
Displays the print menu.
Viewing appliance memory paging reports
The Memory Paging report provides the total number of memory pages, per second, utilized by the SCC in the time period specified.
For details about operating report controls, see “Overview” on page 330.
The Memory Page report includes these statistics that describe memory paging activity for the time period you specify.
Data Series
Description
Page Swap Out Rate
Specifies the total number of pages swapped per second. If 100 pages are swapped approximately every two hours the SteelHead is functioning properly. If thousands of pages are swapped every few minutes, contact Riverbed Support at https://support.riverbed.com
What this report tells you
The Memory Paging report answers this question:
How many memory pages are swapping out?
About report graphs
Mouse over a specific data point to see the y values and exact time stamp.
The CPU Utilization and Memory Paging report pages expect the SCC to receive data from any SteelHead every five minutes. If the report displays some discontinuities in the graph, it is because the two data points (five minutes after and ten minutes after the previous data sample) are missing from the SCC's perspective.
To view the Memory Paging report
1. Choose Diagnostics > Appliance System: Memory Paging to display the Memory Paging page.
Memory Paging report
2. Use the controls to customize the report as described in this table.
Control
Description
Time Interval
Select a report time interval of 1 hour (1h), 1 day (1d), 1 week (1w), All, or type a custom date. All includes statistics for the last 30 days.
Time intervals that don’t apply to a particular report are dimmed.
For a custom time interval, enter the start time and end time using this format:
yyyy/mm/dd hh:mm:ss
You can quickly see the newest data and see data points as they’re added to the chart dynamically. To display the newest data, click Show newest data.
Appliance
Select an appliance from the drop-down list.
Force best granularity
Select the check box to force the data.
3. Under Export, complete the configuration as described in this table.
Control
Description
Export To
Select either the Email or URL option from the drop-down list.
Email Addresses
(Only displays when Email is selected.) Specify the email address of the recipient.
Email Subject
(Only displays when Email is selected.) Specify the subject.
Destination URL
(Only displays when URL is selected.) Specify the URL.
Format
Select HTML, CSV, or PDF from the drop-down list.
Per Appliance Report
Enables appliance report settings.
Note: Generates graphs per appliance for HTML/PDF reports.
Note: Generates one CSV per appliance for CSV reports.
Export Now
Select Export Now and click Export to start the export immediately.
Schedule Export
Select Schedule Export and specify the start date, time, and frequency of the export. Use this format: yyyy/mm/dd hh:mm:ss
Export
Click Export to export the configuration.
Printable view
Displays the print menu.
Viewing appliance TCP memory reports
The TCP Memory report simplifies the analysis of unexplainable throughput degradations, stalled and timed-out connections, and other network-related problems by providing the history of the TCP memory consumption and any TCP memory pressure events detected during network traffic processing. Use this report to gather preliminary information before calling Riverbed Support to troubleshoot an issue.
For details about operating report controls, see “Overview” on page 330.
The TCP Memory report includes two graphs. The TCP usage graph provides the absolute number of memory bytes allocated by the TCP subsystem. This graph includes these statistics that describe TCP memory activity for the time period you specify.
Data Series
Description
Max Threshold
Displays the absolute maximum amount of memory bytes that the TCP stack can allocate for its needs.
Cutoff Threshold
Displays the number of memory bytes allocated until the TCP memory allocation subsystem doesn’t apply memory saving mechanisms and rules. As soon as the TCP memory consumption reaches the cutoff limit, the TCP stack enters a “memory pressure” state. This state applies several important limitations that restrict memory use by incoming and transmitted packets. In practice, this means that part of the incoming packets can be discarded, and user space code is limited in its abilities to send data.
Enable Threshold
Displays the lower boundary of TCP memory consumption, when the memory pressure state is cleared and the TCP stack can use the unlimited memory allocation approach again.
Memory Usage
Displays the average memory consumption by the TCP/IP stack.
Memory Pressure
Displays the maximum percentage of time that the kernel has spent under TCP memory pressure.
The navigator shadows the memory usage series.
In many cases, even an insignificant increase in network traffic can cause TCP memory pressure, leading to negative consequences. There are many conditions that can cause TCP memory pressure events. However, all of the events can be sorted into these two categories to identify the bottleneck in the data transfer chain:
Slow client cases - Occur when the receiver (client) isn’t able to accept data at the rate the client-side SteelHead or the server-side SteelHead transfers data. This condition usually causes two TCP memory pressure points—one on the sender's side and another one on the receiver's (client's) side. The slow client on the sender's side (usually the client-side SteelHead) is characterized by a large amount of unsent data collected in the send socket buffers. Incorrect SteelHead settings, such as overly large send buffers, can trigger TCP memory pressure, even with relatively normal network traffic.
Fast server cases - Occur when the sender is able to transfer data faster than the receiver can accept it. This condition can be triggered not only because of insufficient CPU resources, but also because of an insufficient disk transfer rate (especially with a cold and warm data pattern). The most common cause of this problem is a lack of processing power on the SteelHead.
What this report tells you
The TCP Memory report answers these questions:
How much time is the kernel spending under TCP memory pressure?
What is the average memory consumption for the SteelHead?
About report graphs
Mouse over a specific data point to see the y values and exact time stamp.
For detailed information about data grouping, see “Overview” on page 330.
To view the TCP Memory report
1. Choose Diagnostics > Appliance System: TCP Memory to display the TCP Memory page.
TCP Memory report
2. Use the controls to customize the report as described in this table.
Control
Description
Time Interval
Select a report time interval of 1 hour (1h), 1 day (1d), 1 week (1w), 30 days (30d), yesterday, last week, or last month.
Time intervals that don’t apply to a particular report are dimmed.
For a custom time interval, enter the start time and end time using the format yyyy/mm/dd hh:mm:ss
You can quickly see the newest data and see data points as they’re added to the chart dynamically. To display the newest data, click Show newest data.
Appliance
Select an appliance from the drop-down list.
3. Under Export, complete the configuration as described in this table.
Control
Description
Export To
Select either the Email or URL option from the drop-down list.
Email Addresses
(Only displays when Email is selected.) Specify the email address of the recipient.
Email Subject
(Only displays when Email is selected.) Specify the subject.
Destination URL
(Only displays when URL is selected.) Specify the URL.
Format
Select HTML, CSV, or PDF from the drop-down list.
Per Appliance Report
Enables appliance report settings.
Note: Generates graphs per appliance for HTML/PDF reports.
Note: Generates one CSV per appliance for CSV reports.
Export Now
Select Export Now and click Export to start the export immediately.
Schedule Export
Select Schedule Export and specify the start date, time, and frequency of the export. Use this format: yyyy/mm/dd hh:mm:ss
Export
Click Export to export the configuration.
Printable view
Displays the print menu.
Downloading appliance logs
You can download all the messages, user messages, web access logs, and web error log in the tar.gz format from an appliance or group appliances in the Download Logs page. The name of the tar files includes a serial number to distinguish between the different appliances.
The Transfer Appliance Logs option enables you to collect individual appliance logs.
The Transfer Appliance Group Logs option enables you to collect logs for several appliances at once as a group.
About report data
The SCC is designed to retain statistics for up to a maximum of three years, based on daily statistics for 2,000 appliances monitoring 50 to 100 TCP ports per SteelHead. Factors that can influence this number include the number of monitored TCP ports, the number of active interfaces on managed appliances, and changes in types amounts of data collected in RiOS releases.
The SCC polls data every five minutes. In general, the SCC retains 5 minute granularity data points for a maximum of 30 days. One hour granularity data points are stored for a maximum of 90 days. Beyond 90 days, SCC retains one day granularity data points for up to three years. In case of stats in excess of capacity, the SCC deletes the oldest data from each of the three granularities, while attempting to preserve as much recent data as it can.
Be aware that if the SCC and remote appliances lose connectivity with each other, the bandwidth and connection data during the period of lost connectivity can be skewed. For example, if a remote appliance loses connectivity with the SCC for six hours, data for the missing six hours appears to be 0 in reports for periods of Last Day or Custom intervals smaller than one day. However, when the remote appliance reestablishes connectivity, it sends an aggregate data point for the last day. Thus, report for periods longer than Last Day do reflect bandwidth and connection data accurately. If you need to analyze data on the remote SteelHead for the missing period, you can view this in the SCC for the individual remote appliance.
To transfer appliance log files report
1. Choose Diagnostics > Appliance Logs: Download Logs to display the Download Logs page.
2. Under Transfer Appliance Logs, select the appliance from the drop-down list.
Transfer appliance logs section
3. Select Transfer to a remote path to upload logs to an SCP or FTP server, and enter the URL.
- or-
Select Download in Browser to download files locally, and click Transfer Logs.
The logs are now available as a tar file.
To transfer group log files report
1. Choose Diagnostics > Appliance Logs: Download Logs to display the Download Logs page.
2. Under Transfer Group Logs, select the group you want to transfer from the drop-down list.
Transfer group logs section
3. Specify the URL in the Select Destination for logs to upload logs to a HTTP or FTP server.
4. Click Transfer Logs.
The logs are now available as a tar file.
Generating appliance system dumps
You can generate, display, and download system dumps in the System Dumps page. A system dump contains a copy of the kernel data on the system. System dump files can help you diagnose problems in the system.
To generate a system dump
1. Choose Diagnostics > Appliance Logs: System Dumps to display the System Dumps page.
System Dumps report
2. Under Appliance System Dump, select the appliance from the drop-down list to include in the report.
3. Under Generate System Dump, select the type of information to include in the report:
Include Statistics - Select to collect and include CPU, memory, and other statistics in the system dump (this option is enabled by default). The statistics are useful while analyzing traffic patterns to correlate to an issue. The system adds the statistics to a file in the sysdump called stats.tgz.
In RiOS 8.5, you can collect and include application visibility statistics in a compressed archive file called app_vis.db.
Include All Logs - Removes the 50 MB limit for compressed log files, to include all logs in the system dump.
Include RSP - (xx50 only) Collects and includes RSP information in the system dump.
Include VSP - Collects and includes VSP ESXi information in the system dump.
4. Click Generate System Dump.
Because generating a system dump can take a while (especially when including ESXi information on a SteelHead EX), a spinner appears during the system dump creation. When the system dump is complete, it appears in the list of links to download.
Managing TCP Dumps
You create, view, upload, and stop TCP dumps in the TCP Dumps page. This section contains these topics:
Creating Appliance TCP dumps
Capturing TCP dumps for Interceptor clusters
Troubleshooting
Custom flag use examples
Stopping a TCP dump after an event occurs
Viewing a TCP dump
Uploading a TCP dump
Creating Appliance TCP dumps
You can create, download, and upload TCP dump capture files for appliances in the TCP Dumps page.
Capture files contain summary information for every internet packet received or transmitted on the interface to help diagnose problems in the system.
RiOS provides an easy way to create and retrieve multiple capture files. You can create capture files from multiple interfaces at the same time, limit the size of the capture file, and schedule a specific date and time to create a capture file. Scheduling and limiting a capture file by time or size allows unattended captures.
The top of the TCP Dumps page displays a list of existing capture files and the bottom of the page displays controls to create a capture file. The bottom of the page also includes the capture files that are currently running, and controls to create a trigger that stops a capture when a specific event occurs. The Running Capture Name list includes captures running at a particular time. It includes captures started manually and also any captures that were scheduled previously and are now running.
The SCC automatically uploads the tcpdumps to itself and zips them once the capture is finished. You may want to ensure that the TCP dumps don’t saturate their WAN links by either minimizing the size of the captures or employing QoS to rate limit the transfers.
Capturing TCP dumps for Interceptor clusters
With RiOS 9.6, the SCC customizes the display in the TCP Dumps page to guide you to specify the correct endpoints (that is, IP addresses) so that you can capture all relevant packets to debug Interceptor cluster configurations. This feature reduces the number of TCP dumps taken for debugging and cluster set up.
Previously you would have to create multiple TCP dumps, such as Interceptor—>SteelHead, Server—>Client, and so forth to obtain all the data you need to debug cluster configuration. With RiOS 9.6, you can capture all relevant packets in a single TCP dump. TCP Dump capture supports IPv4 traffic with correct addressing and full-transparency mode. For IPv6 traffic, all relevant packets are captured as long as there are no IPv6 extended headers in the data packet originating on the client or server.
In the SCC you can select an Interceptor appliance or a SteelHead local to the Interceptor to capture flows for Client or Server endpoints. You can also specify whether you want to capture packets from the inner channel.
This table summarizes possible inner channel capture use cases.
Capture use cases
Packets captured
Interceptor without inner channel capture (lan0_0)
All responses from the server
All GRE messages between the Interceptor and the SteelHead
Interceptor without inner channel capture (wan0_0)
All probe messages
Interceptor with inner channel capture (lan0_0)
All requests from client
All responses from the server
All GRE messages between the Interceptor and the SteelHead
All heartbeat messages between the Interceptor and the SteelHead
Interceptor with inner channel capture (wan0_0)
All packets from client
All responses from the server
All probe messages
SteelHead (local to the Interceptor) without inner channel capture (wan0_0)
All requests from client
All responses from the server
All GRE messages between the Interceptor and the SteelHead
SteelHead (local to the Interceptor) with inner channel capture (wan0_0)
All requests from client
All responses from the server
All GRE messages between the Interceptor and the SteelHead
All heartbeat messages between the Interceptor and the SteelHead
SteelHead not local to the Interceptor
No option to specify the location of the Interceptor (client/server). You can only capture traffic between SteelHead IP addresses in a comma separated list.
To add or remove stored TCP trace dumps
1. Choose Diagnostics > Appliance Logs: TCP Dumps to display the TCP Dumps page.
TCP Dumps displaying the Interceptor option
2. Complete the configuration as described in this table.
Control
Description
Add a New TCP Dump
Displays the controls for creating a TCP trace dump.
Capture Name
Specify the name of the capture file. Use a unique filename to prevent overwriting an existing TCP dump. The default filename uses this format:
<hostname>_<interface>_<time-stamp>.cap
Where <hostname> is the hostname of the SCC, <interface> is the name of the interface selected for the trace (for example, lan0_0, wan0_0), and <time-stamp> is in the yyyy/mm/dd hh:mm:ss format.
If this trace dump relates to an open Riverbed Support case, specify the capture filename case_<number> where <number> is your Riverbed Support case number: for example, case_12345.
Note: The .cap file extension isn’t included with the filename when it appears in the capture queue.
Appliances
Select an appliance from the list. The SCC displays Interceptors, SteelHeads, and SteelHeads that are local to the Interceptor.
If you select an Interceptor appliance or a SteelHead that is local to the Interceptor, the Endpoints section displays the Select Interceptor Location: Client/Server option so that you can capture data based on the location of the Interceptor in your network.
If you select an appliance that isn’t local to the Interceptor the page displays only the Endpoints option for SteelHead deployments for capturing packets between endpoints.
Endpoints (SteelHead deployments)
Specify source and destination IP addresses and corresponding ports to capture packets between endpoints. For example, specify the client-side IP addresses and server-side addresses to capture packets between these endpoints.
Capture traffic between:
IPs - Specify All to capture all IP addresses on one side of the network or specify particular IP addresses separated by commas. You can specify IPv4 or IPv6 addresses. The default setting is All.
Ports - Specify All to capture all corresponding ports or specify particular ports separated by commas. The default setting is All.
and
IPs - Specify All to capture all IP addresses on the other side of the network or specify particular IP addresses separated by commas. You can specify IPv4 or IPv6 addresses. The default setting is All.
Ports - Specify All to capture all corresponding ports or specify particular ports separated by commas. The default setting is All.
Note: To capture traffic flowing in only one direction or to enter a custom command, use the CLI tcpdump command. For details, see the Riverbed Command-Line Interface Reference Manual.
Endpoints (Interceptor clusters)
Select Interceptor Location - Select either Client or Server from the drop-down list. Your choice determines the endpoints (that is, IP addresses) that you can specify.
If you select Client:
IPs - Specify All to capture all the client-side endpoints or specify one or more IP addresses separated by commas. Specify client-side addresses only. You can specify IPv4 or IPv6 addresses. The default setting is All.
Ports - Specify All to capture all corresponding ports or specify one or more ports separated by commas. The default setting is All.
If you select Server:
IPs - Either specify All to capture all server-side endpoints or specify one or more IP addresses separated by commas. Specify client-side addresses only. You can specify IPv4 or IPv6 addresses. The default setting is All.
Ports - Specify All to capture all corresponding ports or specify one or more ports separated by commas. The default setting is All.
Capture Inner Channel Data - Captures all inner channel requests between the endpoints. The default setting is off.
Appliance IP address - Specify the in-path IP address of the local SteelHead.
Service Port - Specify the service port of the in-path local SteelHead. The default service port number is 7800.
Capture Interfaces
Captures packet traces on the selected interfaces. You can select all interfaces or a base or in-path interface. The default setting is none. You must specify a capture interface.
If you select several interfaces at a time, the data is automatically placed into separate capture files.
When path selection is enabled, we recommend that you collect packet traces on all LAN and WAN interfaces.
Capture Parameters
These parameters let you capture information about dot1q VLAN traffic. You can match traffic based on VLAN-tagged or untagged packets, or both. You can also filter by port number or host IP address and include or exclude ARP packets. Select one of these parameters for capturing VLAN packets:
Capture Untagged Traffic Only - Select this option for these captures:
All untagged VLAN traffic.
Untagged 7850 traffic and ARP packets. You must also specify or arp in the custom flags field in this page.
Only untagged ARP packets. You must also specify and arp in the custom flags field in this page.
Capture VLAN-Tagged Traffic Only - Select this option for these captures:
Only VLAN-tagged traffic.
VLAN-tagged packets with host 10.11.0.6 traffic and ARP packets. You must also specify 10.11.0.6 in the IPs field, and specify or arp in the custom flags field in this page.
VLAN-tagged ARP packets only. You must also specify and arp in the custom flags field in this page.
Capture both VLAN and Untagged Traffic - Select this option for these captures:
All VLAN traffic.
Both tagged and untagged 7850 traffic and ARP packets. You must also specify these values in the custom flags field in this page:
(port 7850 or arp) or (vlan and (port 7850 or arp))
Both tagged and untagged 7850 traffic only. You must also specify 7850 in one of the port fields in this page. No custom flags are required.
Both tagged and untagged ARP packets. You must also specify these values in the custom flags field in this page:
(arp) or (vlan and arp)
Capture Duration (Seconds)
Specify a positive integer to set how long the capture runs, in seconds. The default value is 30. Specify 0 or continuous to initiate a continuous trace.
For continuous capture, we recommend specifying a maximum capture size and a nonzero rotate file number to limit the size of the TCP dump.
Maximum Capture Size
Specify the maximum capture file size in megabytes. The default value is 100. After the file reaches the maximum capture size, TCP dump starts writing capture data into the next file, limited by the Number of Files to Rotate field.
We recommend a maximum capture file size of 1024 MB (1 GB).
Buffer Size
Optionally, specify the maximum amount of data, in kilobytes, allowed to queue while awaiting processing by the capture file. The default value is 154 kilobytes.
Snap Length (bytes)
Optionally, select the snap length value for the capture file or specify a custom value. The snap length equals the number of bytes the report captures for each packet. Having a snap length smaller than the maximum packet size on the network enables you to store more packets, but you might not be able to inspect the full packet content.
Select 65535 for a full packet capture (recommended for CIFS, MAPI, and SSL captures). The default value is 1518 bytes.
When using jumbo frames, we recommend selecting 9018.
The default custom value is 16383 bytes.
Number of Files to Rotate
Specify how many capture files to keep for each interface before overwriting the oldest file. To stop file rotation, you can specify 0; however, we recommend rotating files, because stopping the rotation can fill the disk partition.
This control limits the number of files created to the specified number and begins overwriting files from the beginning, thus creating a rotating buffer.
The default value is 5. The maximum value is 2147483647.
Custom Flags
Specify custom flags as additional statements within the filter expression. Custom flags are added to the end of the expression created from the Endpoints fields and the Capture Parameters radio buttons (pertaining to VLANs).
If you require an “and” statement between the expression created from other fields and the expression that you are entering in the custom flags field, you must include the “and” statement at the start of the custom flags field.
Do not use host, src, or dst statements in the custom flags field. Although it is possible in trivial cases to get these statements to start without a syntax error, they don’t capture GRE-encapsulated packets that some modes of SteelHead communications use, such as WCCP deployments or Interceptor connection-setup traffic. We recommend using bidirectional filters by specifying endpoints.
For complete control of your filter expression, use the CLI tcpdump command. For details, see the Riverbed Command-Line Interface Reference Manual.
Schedule Dump
Schedules the trace dump to run at a later date.
Start Date - Specify a date to initiate the trace dump in this format: yyyy/mm/dd
Start Time - Specify a time to initiate the trace dump in this format: hh:mm:ss
Add
Adds the TCP trace dump to the capture queue.
Remove Selected
Under Stored TCP Dumps, select the TCP Dump check box and click Remove Selected.
Troubleshooting
If the tcpdump command results in a syntax error with an immediate or scheduled TCP dump, this message appears:
Error in tcpdump command. See System Log for details.
Review the system log to see the full tcpdump command attempt. Check the expression for issues such as a missing “and” statement as well as contradictory instructions such as looking for VLAN-tagged traffic and nontagged traffic.
Custom flag use examples
The examples in this table focus on the custom flag entry but rely on other fields to create a complete filter.
Filter purpose
Custom flag
To capture all traffic on VLAN 10 between two specified endpoints: 1.1.1.1 and 2.2.2.2
and vlan 10
To capture any packet with a SYN or an ACK
tcp[tcpflags] & (tcp-syn|tcp-ack) != 0
To capture any packet with a SYN
tcp[tcpflags] & (tcp-syn) != 0
-or-
tcp[13] & 2 == 2
To capture any SYN to or from host 1.1.1.1
and (tcp[tcpflags] & (tcp-syn) != 0)
-or-
and (tcp[13] & 2 == 2)
IPv6 custom flag use examples
The examples in this table focus on the custom flag entry but rely on other fields to create a complete filter.
To build expressions for TCP dump, IPv6 filtering doesn’t currently support the TCP, UDP, and other upper-layer protocol types that IPv4 does. Also, these IPv6 examples are based on the assumption that only a single IPv6 header is present.
Filter purpose
Custom flag
To capture all FIN packets to or from host 2001::2002
and (ip6[53] & 1!=0)
To capture all IPv6 SYN packets
ip6 or proto ipv6 and (ip6[53] & 2 == 2)
Stopping a TCP dump after an event occurs
Capture files offer visibility into intermittent network issues, but the amount of traffic they capture can be overwhelming. Also, because rotating logs is common, after a capture logs an event, the SteelHead log rotation can overwrite debugging information specific to the event.
RiOS 8.5.x and later make troubleshooting easier because they provide a trigger that can stop a continuous capture after a specific log event occurs. The result is a smaller file to help pinpoint what makes the event happen.
The stop trigger continuously scans the system logs for a search pattern. When it finds a match, it stops all running captures.
To stop a capture after a specific log event
1. Choose Diagnostics > Appliance Logs: TCP Dumps to display the TCP Dumps page.
2. Schedule a capture.
TCP Dump Stop Trigger pane
3. Under TCP Dump Stop Trigger: Pattern, enter a Perl regular expression (regex) to find in a log. RiOS compares the Perl regex against each new line in the system logs and the trigger stops if it finds a match.
The simplest regex is a word or a string of characters. For example, if you set the pattern to “Limit,” the trigger matches the line “Connection Limit Reached.”
Notes:
Perl regular expressions are case sensitive.
Perl treats the space character like any other character in a regex.
Perl reserves some characters, called metacharacters, for use in regex notation. The metacharacters are:
{ } [ ] ( ) ^ $ . | * + ? \
You can match a metacharacter by putting a backslash before it. For example, to search for a backslash in the logs, you must enter two backslashes (\\) as the pattern.
The pattern follows Perl regular expression syntax. For details, go to:
http://perldoc.perl.org/perlre.html
You can’t change the pattern while a scan is running. You must stop the scan before changing a pattern.
You don’t need to wrap the pattern with the metacharacters to match the beginning or end of a line (^ $) or with the wildcard character (*).
4. Specify the amount of time to pause before stopping all running captures when RiOS finds a match. The time delay gives the system some time to log more data without abruptly cutting off the capture. The default is 30 seconds. Specify 0 for no delay; the capture stops immediately.
After a trigger has fired, the capture can stop by itself before the delay expires. For example, the capture duration can expire.
5. Click Start Scan.
When the scan stops, RiOS sends an email to all email addresses on the Administration: System Settings > Email page appearing under Report Events via Email. The email notifies users that the trigger has fired.
The page indicates “Last Triggered: Never” if a TCP Dump stop trigger has never triggered on the SteelHead. After the delay duration of the stop trigger, RiOS displays the last triggered time.
Before changing the Perl regular expression or amount of delay, you must first stop the process.
To stop a running scan
Click Stop Scan to halt the background process that monitors the system logs. RiOS dims this button when the stop trigger is idling.
Stop trigger limitations
These limitations apply to the trigger:
You can’t create a trigger to stop a specific capture; the trigger affects all running captures.
If the search pattern contains a typo, the trigger might never find a match.
Only one instance of a trigger can run at one time.
Viewing a TCP dump
The top of the TCP Dumps page displays a list of existing captures.
To view a capture file
1. Choose Diagnostics > Appliance Logs: TCP Dumps to display the TCP Dumps page.
2. Under Stored TCP Dumps, select the capture name to open the file.
3. Click Download to view a previously saved capture file.
4. To remove a capture file, select the check box next to the name and click Remove Selected.
To print a capture file
1. Choose Diagnostics > Appliance Logs: TCP Dumps to display the TCP Dumps page.
2. Under Download Link, select the capture filename to open the file.
3. When the file opens, choose File > Print in your web browser to open the Print dialog box.
To stop a running capture
1. Choose Diagnostics > Appliance Logs: TCP Dumps to display the TCP Dumps page.
2. Select the capture filename in the Running Capture Name list.
3. Click Stop Selected Captures.
Uploading a TCP dump
Riverbed offers a way to upload capture files to the support server for sharing with the support team while diagnosing issues.
To upload the capture file to Riverbed Support
1. In continuous mode, on the TCP Dumps page, select the running capture and click Stop Selected Captures.
For timed captures that are complete, skip to Step 2.
The capture appears as a download link in the list of Stored TCP Dumps.
2. Select the capture filename.
3. Optionally, specify a case number that corresponds to the capture. We recommend using a case number: for example, 194170.
To specify a URL instead of a case number, you must use the CLI. You can enter the CLI command file tcpdump upload url. When you specify a URL, the capture file goes directly to the URL.
If the URL points to a directory on the upload server, it must have a trailing backslash (/).
For example:
ftp://ftp.riverbed.com/incoming/
(not ftp://ftp.riverbed.com/incoming)
The filename as it exists on the appliance will then match the filename on the upload server.
For details, see the Riverbed Command-Line Interface Reference Manual.
4. Click Upload.
Because uploading a capture file can take a while, a progress bar displays the percentage of the total upload completed, the case number (if applicable), and the date and time the upload began. When the capture file finishes uploading, the date, time, and a status of either uploaded (appears in green) or failed (appears in red) are indicated.
Successful uploads show the status, the case number (if applicable), and the date and time the upload finished.
For uploads that fail, an explanation, the case number (if applicable), and the upload starting date and time appear.
Viewing appliance expiring certificates
The Expiring Certificates report displays the SSL certificates that have expired or will expire within sixty days.
It may take up to three hours for this page to update after the certificates have been removed.
The report displays this information.
Control
Description
Certificate Location
Displays the certificate location.
CA/Policy/Appliance
Displays the policy and appliance.
Certificate
Displays the certificates.
Expiration Date
Displays the expiration date of the certificate.
What this report tells you
The Expiring Certificates report answers these questions:
What certificates are expired or within sixty days of expiring?
Where are the certificates applied?
What is the certificate location?
About report data
The Riverbed reporting functionality polls bandwidth and connection metrics every 15 seconds and reports on performance for periods up to one month. Every 15-second sample is used for calculating its average and peak value. However, due to performance and disk space considerations, data representation in reports for periods longer than the last five minutes are interpolated between data points obtained by aggregating more than one 15-second sample. The display granularity decreases with time passed since data was sampled.
To view the Expiring Certificates report
Choose Diagnostics > Appliance Logs: Expiring Certificates to display the Expiring Certificates page.
Expiring Certificates report
It may take up to three hours for the page to update after the certificates have been removed.
Related topics
Managing user permissions
Changing the account password