This chapter provides several best practices to leverage Generic Metric Collection (GMC) by using NetIM Metric Import REST API. Best practices include:

The NetIM Metric Import REST API enables you to extend SNMP-based metric polling.

Examples of use cases include:

 

Once these metrics have been imported, they are stored in the NetIM metrics database as any other metrics polled from conventional SNMP polling. That also means that these NetIM Import REST API metrics can be:

This list compares the previous Generic Metric Collection (GMC) feature and the newer NetIM Metric Import REST API:

The legacy Generic Metric Collection feature (GMC), based on the import of .mtr files, is still available. However, for most use cases we recommend the newer NetIM Metric Import REST API because of the way metrics are imported. The NetIM Metric Import REST API feature leverages a set of APIs within the NetIM swarm exposed by the network-metric import service. These APIs provide a better, more efficient, and more programmatic interface for importing externally produced metric data into the NetIM system.

To import metrics with the NetIM Metric Import REST API, you typically use a script in Perl, Python, PowerShell, Bash, or any scripting language.

The script can run within the NetIM system itself (for example, on the NetIM Core or on the NetIM Worker). The NetIM VMs have Python3 and PowerShell already installed. You may, however, have to install additional modules depending on the modules you import and use in your script. To configure the script to run on one of the NetIM VMs, you need access to the Bash shell. Contact Riverbed Technical Support or your Riverbed Account management team for access.

You can also run the script on an external platform (for example, external Linux or Windows Server).

In general, in order not to overload the NetIM VMs, this is the preferred option if you plan to deploy many scripts feeding several NetIM Metric Import REST API metrics.

You can then choose the script run interval by choosing between two options:

You can also run the script using a NetIM Synthetic Test by selecting an External Test Type and then selecting the script and the frequency. The advantage of this approach is that you are able to monitor the success/failure of the script run itself and the time taken to run the script, allowing you to evaluate the minimal frequency you can use for your script.

 

Using the NetIM Import REST API does not consume any license units.

Also, consider where to run the script that performs the REST API calls. In the case of a few metrics, you can likely run this on the NetIM Core or NetIM Worker (running script on NetIM Manager is not recommended). In case of a larger deployment, we recommend running the script(s) on an external Linux or Windows server.

In general, you have three options for displaying the NetIM Import REST API imported metrics:

The NetIM custom views have been extensively enhanced in recent releases and offer new panel types like Metrics Tables, Gauges, Metric/Attribute values, and so on. This allows new options for displaying the custom metrics directly in NetIM. This can be useful when Portal is not available or when you really want the custom metrics to be well integrated into the already existing out-of-the-box (OOTB) NetIM workflows and views.

Portal offers some more flexibility in terms of display (for example, grouping) and allows metrics at any level (group/device/interface) to be displayed in the same dashboard. Portal also allows for enhanced role-based access control (RBAC) allowing you better control of who and what can see. See the section “Viewing your imported metrics in NetIM and in Portal.”

Extracting the metrics for display in third-party tools is always an option, but it typically requires some scripting efforts.

NetIM supports numeric (integer, float) and enumerated values.

NetIM does not support other metric types such as Timeticks, IP/MAC addresses, or text strings. Although these types cannot be used as metrics, they can still be used as component identifiers (like indexes) and can be used for grouping, for example in Portal.

In some cases, you may be able to use the already existing OOTB metric classes. This could be the case if, for example, you want to import CPU utilization for a device that you can obtain via a REST API call to the device or a device manager but not via SNMP. In this case, you can use the existing metric class “CPU_UTIL” and insert the metrics into this metric class.

In most cases, your customization requires NetIM to learn about new metrics in new metric classes, which can be accomplished by manually inserting custom resource files into the NetIM system.

However, the recommended way to import new metric classes is to:

Using the above process ensures consistency, and any syntax errors will be caught during the import of the custom MIB. See Chapter 5, “Common Data Collection Challenges in a Multivendor Environment.”

Defining a metric class in a MIB is rather straightforward. It follows the classic ASN.1 (Abstract Syntax Notification) like any other public and proprietary MIB.

You want to collect Microsoft Exchange metrics from an Exchange server using PowerShell and the Microsoft Exchange API (see the section “Examples of usage of NetIM Metric Import REST API.”)

For each mailbox, you want to collect Total Size and Total Available (remaining) Size. These metrics can be obtained in a table from by the PowerShell command as depicted in this screenshot.

To import these metrics into NetIM, you need to teach NetIM about a new metric class with this table format. DisplayName is the Index for this table.

 

To teach NetIM about this new metric class, you create a MIB file with this contents defining the metric class:

Save the contents to a file with the suffix .my or .mib. You import this MIB via the User-Defined Metrics wizard.

If you want to be able to use a given non-numeric value as an index for grouping in Portal, then make sure to select the option “Component Identifier” for this value.

The new metric class is now ready for being used by the NetIM Metric Import REST API.

To use the NetIM Metric Import REST API, you need to create a local user account with ADMIN privileges because the NetIM Metric Import REST API writes to the NetIM system.

NetIM does not support SAML/TACACS accounts. Only Basic Authentication is supported (no support for token-based authentication).

As preparation for using the NetIM Metric Import REST API and before starting the actual scripting, Postman is a useful tool for running tests.

Postman is available in a web version as well as a local application. See an example below.

A “200 OK” message indicates that the message was received and accepted. You can now check NetIM if the metric was imported.

Any scripting language can be used for injecting metrics into NetIM using the NetIM Metric Import REST API. This includes scripting languages like Python, Perl, PowerShell, and Bash.

The NetIM Core VM folder /home/admin/rest_api_examples contains some examples of Python3 utilities that can be used by the script to obtain various useful information such as deviceID for a given device name, metric class ID for a given metric class name, and so on.

Most scripts function like this:

For any information on specific script implementation, contact your Riverbed Account management team.

For the NetIM Metric Import REST API, the primary API endpoint provided for publishing the metric samples is:

This endpoint requires HTTP Basic Authentication (like all the APIs exposed by NetIM Core).

The body of the request must be provided as type "application/json".

This example shows the various structural components that are required for a valid request to the network-metric-import API endpoint:

The "source" property is used to identify the collecting source of the data; this could be something like "poller", "wmi", or "external". In most cases use "poller".

The "metricClass" property is used to identify the metric class to which the data belongs. Each request to the API endpoint may represent samples from only a single metric class at a time. For information on the metric classes available, see information about the /metric-classes API endpoint available in the NetIM API documentation (under Help in the NetIM web UI). The value provided must be the "identifiers" property for a valid metric class currently provisioned within the system (such as CPU_UTIL or IFC_UTIL).

The "identifiers" property is used to provide unique identifying information about the base level object with which the data should be associated. The exact content of this property required depends on the metricClass provided.

For example, if a device "level" metric class, such as DEV_AVL or CPU_UTIL is provided, then only VNES_OE identifying attributes are required. If an interface "level" metric class, such as IFC_UTIL or COS_UTIL, is provided instead, then VNES_OE and VNES_IFC identifying attributes are required.

Although you can publish metric data associated with a group, AppNetwork Path, or synthetic test, this document only discusses the common customer use cases of device and interface-level data association.

At the device (VNES_OE) level, to uniquely identify the device, the "deviceID" (also known as the "deviceAccessInfoId") must be provided (not the “ID”). The “deviceAccessInfoId” is typically a 13-digit number.

Example:

At the interface (VNES_INTERFACE) level, both interface identifying information and its parent device (VNES_OE) must be provided.

The previously described device identifying information applies here.

To identify the interface, various property combinations must be included. These combinations of properties can be provided:

These rules are evaluated in priority order. When the first rule that matches identifies the interface.

Example:

The "minTimestamp" and "maxTimestamp" properties are provided to set the time boundaries for all samples being published.

Both values should be provided as millisecond UTC or "Unix epoch" values.

The "sampleList" property is where the actual sample data is to be provided. This property should be provided as an array of sample objects. A maximum of 500 individual sample objects should be provided in any single request. If more than 500 samples are required to be published, they should be separated into separate requests, each containing a maximum of 500 samples.

Each sample object represents a value for one or more fields from the specified metric class corresponding to a single time stamp.

The exact structure of the sample object varies depending on the metric class. However, each sample has some common elements.

Here is a basic example representing a sample of CPU_UTIL data:

A sample representing a different CPU metric at the same time might look like this:

The "sampleInfo" property is used to provide meta information about the sample. This property is primarily used when the sample being published represents a nonraw (or "rollup") sample.

If the data represented is "raw" sampling, then the "sampleInfo" property can simply be set to "null" as shown in the above example.

Because the NetIM system provides roll-up capability on all incoming raw metric data, including data provided via these APIs, only raw data should be published. (Configuring the "sampleInfo" property is not discussed further in this document.)

The "fieldValues" property is used to provide values for the fields of the metric class being published.

The structure required is a key-value mapping where the keys should be the identifier of the metric class field that is populated, and the value should be whichever value to be populated, provided always as a string.

The values should be provided as strings even if they represent numeric values. These values are cast properly downstream depending on the definition of the field within the header file.

While not all fields are required to be present to populate a sample for the metric class, the time stamp is always required. The time stamp value should be provided as a millisecond UTC or "Unix epoch" time value. Depending on the definition of the metric class, other fields are also required.

Any fields within the metric class that are marked as "index" == true or "isComponentPart" == true are required for the sample to be properly mapped for persistence.

For example, within the CPU_UTIL metric class you’ll find these field definitions:

Where appropriate, the <fs:sp name="description"> is the name of the requested CPU utilization type.

When evaluating a metric class to see what fields are required, it’s important to find fields that contain either of these two property definitions:

or

If either of these properties are present for the field, then that field is required to be present.

For the CPU_UTIL example, these fields are required in each sample object:

Besides a purely failed request for some unknown reason, the API responds with HTTP code 200 and a body that provides additional detail about the success of the request. Because the request can be partially successful, you should inspect the contents of the response.

The response body looks something like this:

The "status" property provides a high-level indication of whether the request was successful or not.

The "totalSamples" and "validSamples" properties provide more detailed information about whether the request was successful or not. These values indicate the number of samples provided via the "sampleList" request property that were valid and accepted for ingestion into the system.

If any of the samples provided via "sampleList" were invalid and not accepted, this field provides additional information about why.

These examples show the request body published to the API endpoint for various metric classes at various levels of model association.

This example is for “DEV_AVL”:

This example is for “CPU_AVL”:

This example is for "COS_UTIL":

Always start by verifying that the metrics are imported in the Metrics tab for the object (device or interface) by selecting the Metrics tab for that object. In the drop-down list, select the metric class and then click Load.

NetIM Custom View templates and NetIM Custom view template rules allow you to design new NetIM views for the At-a-Glance Home page, Site/Group Page, Device page, and Interface page. You can add custom panels including the new NetIM Import Metric REST API metrics, and you can specify to which devices these views should be used.

NetIM allows you to create new Alert profiles and select your new NetIM Import Metric REST API metrics with proper thresholds. A device can have several alert profiles applied.

One of these alert profiles can also be enabled as a Health profile. This alert profile the controls the state (red, orange, yellow, green) of the object (device, interface). See the example below.

You can opt to display the new NetIM Import Metric REST API metrics in custom Portal dashboards.

Before you use this option, make sure to restart Portal so it learns about these new metrics. You then start creating the panels in the new dashboard with the Object Selection wizard, like in this example.

Then format the panel into rows and columns by Metric Display and the index or component identifier like in this table.

Most script languages allow you to insert print statements in the script. Print statements are useful when debugging your script.

Sometimes it can take several minutes before the first metrics are available in the table.

If a metric does not seem to be imported, inspect the NetIM swarm log for the service “Network Metric Import Service.”

At the device (VNES_OE) level, provide the "deviceID" (also known as the "deviceAccessInfoId") to uniquely identify the device (not the "ID"). The "deviceAccessInfoId" is typically a 13-digit number.

Verify the variable values inserted in the script. Python (like many other languages) are case sensitive.

Riverbed Technical Support supports the NetIM Metric Import REST API, which is the API interface with its endpoints. But Riverbed Technical Support does not support the script that is using this API interface.

For any of the examples in this section, contact your Riverbed account management team for more information about how these examples were implemented.

This customization displays and alerts on the VMware ESX server at the VM (Guest level), including details like CPU, Memory, Disk I/O, and Network usage at a per-VM level. For each metric, Max/Min and Average values are provided. All metrics are alertable.

This is how it is made:

This customization displays and alerts on specific Microsoft Exchange Metrics like DB and Index Status, Queue Length, Replication Health, and Mailbox usage. All metrics are alertable.

This is how it is made:

The customization and the script were developed before the availability of the newer NetIM Metric Import REST API, so it is injecting the legacy GMC file-based format.

The metrics are displayed in a custom Dashboard on the Portal Server. The metrics could also be displayed directly in NetIM on the device view for the Exchange server using the NetIM Custom Views feature.

This customization displays BGP Route Flapping and BGP Peer Transitions for BGP Routers. All metrics are alertable.

This customization uses the NetIM REST API in two ways: first to extract polled metrics (export), and then to insert new metrics (import).

This is how it is made:

This customization displays VoIP Trunk Utilization at a per-Dial Peer level. It displays average and maximum values for Trunk Utilization and total number of calls. All metrics are alertable.

This customization uses the NetIM REST API in two ways: first to extract polled metrics (export), and then to insert new metrics (import).

This is how it is made: