NetProfiler and NetExpress Flow Storage

This section describes estimating and sizing your flow rate and flow storage requirements. It includes the following sections:

Types of Flow Storage

Flow Rate Estimation

Flow Storage Size Estimation

Types of Flow Storage

The Standard and Enterprise NetProfiler support the following types of flow storage:

Local storage - Disk internal to the SteelCentral appliance.

Storage area network (SAN) - Near-line storage.

These storage mechanisms operate differently and provide their own benefits and disadvantages.

Local Storage

Every NetExpress, NetProfiler, and Enterprise cluster includes some amount of local storage. The following table lists the system storage type.

System Type	Physical Disks	Storage (Raw)
NetExpress (all physical models)	2	3 TB flow/4 TB packet
NetProfiler (all physical models)	12	10 TB
Enterprise NetProfiler analyzer, or expansion module	12	20 TB

The primary advantages to local storage are performance and cost. Because the storage is located within the system, it is extremely fast when performing reads and writes, limited only by the physical disks and bus connecting the disks to the core system. The NetProfiler includes storage, and no external storage is required.

The biggest disadvantage of local storage is that it limits expansion. There is no option for internal disk expansion.

Storage Area Network

Storage area network (SAN) provides the most robust external storage solution for Standard and Enterprise NetProfiler. SAN is a reliable solution that enables the NetProfiler to very quickly access an extremely large amount of storage—up to 50 TB.

When you connect a SAN to the NetProfiler, the SAN functions as a new disk partition. The NetProfiler treats the SAN as another disk that is part of the appliance, enabling it to easily access the data. You can also offload much of the processing to the add-in card you use to connect the SAN.

You must use a separate logical unit number (LUN) and fiber connection between each analyzer or expansion module and the SAN when you connect a SAN to an Enterprise NetProfiler cluster. This configuration increases the performance drastically, because each analyzer is given its own dedicated channel to the SAN and potentially has its own set of drives on the SAN (depending on the SAN configuration). Also, when you deploy SAN with an Enterprise Cluster you must either deploy SAN with all analyzer and expansion modules or no analyzer and expansion modules. You cannot partially deploy SAN with an Enterprise Cluster.

Flow Rate Estimation

The most important aspect of system sizing is the number of flows your system receives every minute. To accurately estimate the number of flows, you must know if a host accesses internal resources or internal and external resources.

You must take into account the following when estimating flows:

Estimate your flows per minute.

In general, you can expect a minimum of 40 flows per host per minute across all internal systems. Because different systems send different numbers of flows, it is probable that some hosts send much more than 40 fpm, while some systems send fewer.

Riverbed recommends that consider another system when the estimate approaches the limits of a system: for example, if the estimate indicates 49,000 fpm you might want to consider if a system capable of supporting 50,000 flows is too restrictive.

Consider how much of the traffic the NetProfiler detects.

A network with 10,000 hosts, only 25 percent of which forwards traffic to the NetProfiler, has a very different requirement than a network with 5,000 hosts, that all forward traffic. The network with 10,000 hosts has an estimated 100,000 fpm, and the networkw tih 5,000 hosts has an estimated 200,000 fpm. You must know the total number of hosts on a network and the number of hosts that have their flows forwarded.

Estimate the current traffic level and what is expected to occur in the near future.

Sizing a system for 2,000 hosts when an additional 500 seats will be added over the next year (or conversely 500 seats will be removed) might result in a significant missizing of the system. Additionally, if you are performing a proof of concept (PoC) on a small portion of a network, ensure that the final solution is sized for the actual implementation and not the PoC implementation.

Flow Storage Size Estimation

Another primary sizing decision factor is how long you want to store flows. Because the amount of space occupied by each flow is dynamic depending on the number of hops being stored. Theoretically, you can store up to tens of billions of individual unique flows.

Note: This estimation assumes that you have 50 percent of available flow storage dedicated to storing flow records. On normal implementations, half of the storage is dedicated to storing the minute-by-minute flows and half is used to store presummarized data, called rollups.

If a NetExpress receives one flow per minute, it can store flows covering a time span of more than 4,500 years. On a more realistic network sending 50,000 fpm, the same NetExpress can store more than 60 days worth of flow data.

If you must store flows (at the realistic flow rate) for at least 180 days, then the storage included on the NetExpress is inadequate. Because NetExpress does not support a SAN alternative, you must upgrade to a larger system (such as a Standard NetProfiler) with additional storage capacity. When you determine the upgrade path to take, ensure that the path you choose is the most logical approach. You can move up from one system to another but not backwards. For example, you can move from a NetExpress to Standard NetProfiler or from Standard NetProfiler to Enterprise NetProfiler cluster. You cannot move from Enterprise NetProfiler cluster to a Standard NetProfiler.

When you choose the appropriate upgrade, take into account the desired flow storage capacity and the cost of the system. While the Enterprise NetProfiler cluster might meet the storage needs of the network in the example above, unless you expect significant growth in the near future, it is unlikely to be cost effective.