Overview¶
The documentation in this section describes the RESTful APIs included with AppResponse 11. It is assumed that the reader has practical knowledge of RESTful APIs, so the documentation does not go into detail about what REST is, and how to use it. Instead, the documentation focuses on what data can be accessed, and how to access it.
The APIs are versioned using the Semantic Versioning rules.
When it comes to programmatically accessing REST APIs, Riverbed strongly recommends using the SteelScript for Python SDK. With SteelScript for Python, you can:
- Get started quickly based on example scripts.
- Develop applications faster by leveraging the vast number of modules available for Python.
- Build upon shared code samples from the Riverbed and Python communities.
APIs to access data¶
Glossary¶
Report Instance - An object that contains one or multiple data definition requests, and the actual data pulled up from the backend data sources.
Data definition - An object that describes a data request, usually (but not necessarily) connected to one widget on the client. Once processed, the object contains “data” that can be retrieved.
Data source - An internal object responsible to handle a data definition request.
Report Instance object¶
The process of retrieving data basically comes down to creating a report instance object, then monitoring the state of each data request, and, finally, querying each data object for the rows of data.
The Report Instance object is made of these properties:
id - Integer (read-only). A unique identifier created on the server.
live - Boolean. This tells the backend that the report is “live”. Each data request will be queried periodically for data, and the end time will move forward. Live API is used in the “packet” data request.
data_defs - An array of data definition objects. Each of these is a question (set of data) that the client is interested in obtaining. If the client is a Web interface, then a data definition is created for each of the visual components (table/pie/bar/line/etc.).
The workflows for retrieving data are:
- Asynchronous API for getting peformance/historical data
- Synchronous API for getting performance/historical data
- Live API for retrieving current/recent packet data
Data Definition (request)¶
Data definition is an object that defines a set of data that the clients of the API need to get. The set of data looks like a database table. It has columns and rows. The data returned by the API is always in the same format.
The data definition (request) has the following properties: source, time, group_by, sort, limit, and filters.
Source property¶
This allows clients to explicitly pick the data source to handle the data request. The source property is an object. It has the following required sub-properties:
name - String (required).
path - String (optional). Source path, used for packet data. Can be: capture file or virtual file (fs/path/to/file.pcap); job (jobs/00000001); virtual interface group (vifgs/1000); clip (clips/000000010004).
type - enum (INDEX, PACKETS). Optional. Used for packed data to specify the source type.
Please check out the API discoverability section for a list of all possible data source names.
Example:
{
"source": {
"name": "aggregates"
}
}
Limit property¶
The limit is used when requesting top or bottom talkers.
Example of getting top 20 talkers:
{
"limit": 20
}
Columns property¶
Request columns. The client can specify the requested key/metric columns, as well as their order.
Example of getting throughput for host groups:
{
"columns": [
"host_group.id",
"host_group.name",
"avg_traffic.total_bytes_ps"
]
}
There are two special (time series) columns used in all data requests:
start_time - Use this to request time series data. Commonly used by Left-aligned clients. The data result timestamps will be aligned to the left of the time bucket. For example, when 5min granular data is requested, each timestamp represents the start of the 5min bucket.
end_time - Use this to request time series data. Commonly used by Right-aligned clients. The data result timestamps will be aligned to the right of the time bucket. For example, when 5min granular data is requested, each timestamp represents the end of the 5min bucket.
Group by property¶
The group by property specifies the keys in the request. It is usually used to determine what kind of data is requested. If the start_time (or end_time) column is in the group_by, then the request is considered “time series”. In time series requests, each data row represents a time bin. The time bin length is determined by the time.resolution and time.granularity properties (described later in this document).
An example of requesting top 20 host groups by Throughput:
{
"group_by": [
"host_group.id"
],
"columns": [
"host_group.id",
"host_group.name",
"avg_traffic.total_bytes_ps"
],
"top_by": [
{
"id": "avg_traffic.total_bytes_ps",
"direction": "desc"
}
],
"limit": 20
}
An example of requesting time series data for two host groups with a granularity of one minute for the last one hour:
{
"time": {
"granularity": "60",
"duration": "last 1 hour"
},
"group_by": [
"start_time",
"host_group.id"
],
"columns": [
"start_time",
"host_group.id",
"host_group.name",
"avg_traffic.total_bytes_ps"
],
"limit": 20
}
The columns that can be included in the group_by array are flagged as “grouped_by”: true in the npm.reports.sources API. For more information, please see API discoverability.
Top by property¶
The top_by property is an array that includes column id as well as direction - ‘acs’ for ascending and ‘desc’ for descending.
An example of requesting the top Talkers by Throughput:
{
"top_by": [
{
"id": "avg_traffic.total_bytes_ps",
"direction": "desc"
}
]
}
An example of requesting the bottom Talkers by Throughput:
{
"top_by": [
{
"id": "avg_traffic.total_bytes_ps",
"direction": "asc"
}
]
}
Retrieving bottom talkers is not a supported feature throughout all data sources. Due to the large volume of data, limited resources on the system, and the desire to respond in timely fashion, some data sources drop information for bottom talkers. You can discover if top (direction.desc) or bottom (direction.asc) talker requests are supported by looking at the column object in the npm.reports.sources API. For example, the aggregates data source does not support getting bottom talkers for Throughput:
{
"source_name": "aggregates",
"label": "Throughput",
"id": "avg_traffic.total_bytes_ps",
"topped_by": {
"asc": false,
"desc": true
}
}
Reference property¶
Clients can pass a String reference property. The server ignores the value, and simply returns it back. This is commonly used to connect presentation elements with data requests. For example, “reference_id”: “pie_chart_1234”.
Time property¶
Use this sub-object to specify the time duration of the data requests. The time property also includes a few properties that help refine time-series requests.
The time property is an object containing the following sub-properties:
start - String. Epoch start time of the request. The start time is inclusive. The unit is seconds. The precision can be up to nanoseconds. Notable examples: seconds precision: “1234567890”, millisecond precision: “1234567890.123”, nanosecond precision: “1234567890.123456789”.
end - String. Epoch end time of the request. The end time is exclusive. The unit is seconds. The precision can be up to nanoseconds. Notable examples: seconds precision: “1234567890”, millisecond precision: “1234567890.123”, nanosecond precision: “1234567890.123456789”.
granularity - String. This refers to the amount of time for which the data source computes a summary of the metrics it received. The data source examines all data and creates summaries for 1 second, 1 minute, 5 minute, 15 minute, 1 hour, 6 hours, and 1 day. You can choose which of these to use, or let the data source choose the most appropriate one automatically. Greater granularity (shorter time periods) results in greater accuracy. Lesser granularity (1 hour, 6 hours, 1 day) requires less processing and, therefore, the data is returned faster. Granularity must be specified as number of seconds. If you do not specify it, then the granularity is selected automatically. If you set it to 0, then it is selected automatically. Packet data sources have the ability to serve granularity in milliseconds or microseconds; therefore, it can be between 0 and 1. Some data sources support only pre-defined list of granularities. The supported values can retrieved by querying the npm.reports.sources API. An example:
{
"granularities": [
"60",
"300",
"3600",
"21600",
"86400"
]
}
Notable examples: “60” - 1 minute, “300” - 5 minutes, “0.001” - one millisecond, “0.000001” - one microsecond.
resolution - String. It is an additional granularity setting that tells the data source to aggregate the data further. For example, if you are getting one day of data and you specify a resolution of “3600” (one hour), then the result data will have 24 summary samples. But if you specify a resolution of “21600” (six hours), then the result data will have only four summary samples. Resolution must be multiple of the requested granularity. For example, if you specify a granularity of “300” (five minutes), then the resolution can be set to 5 mins, 10 mins, 15 mins, etc. If the resolution is set to be equal to the granularity, then it has no effect on the number of returned samples. The resolution is optional.
duration - String. Calendar unit of duration. Notable examples: “previous 2 months”, “this week”, “last 5 days”, “5 seconds”, “12 min”, “6 hours”, “1 day”, “4 weeks”, “2 quarters”, “2 years”.
There are the following different ways that clients will be able to specify time:
- Getting data for well-known absolute time frames. Example: retrieve data between two absolute time points, from one minute granular source, aggregate the results into 3600 (one hour) buckets. Please note that if granularity and resolution are not specified, the server will pick the best values automatically.
{
"time": {
"start", "1461703405.123",
"end": "1461709465.123",
"granularity": "60",
"resolution": "3600"
}
}
- Requesting relative data for calendar units.
Retrieve data starting from a time point and ending one calendar month later, where month is defined based on the current system timezone.
{
"time": {
"start", "1461703405",
"duration": "1 month"
}
}
In this case, the end time will be calculated based on the calendar for the timezone. For example: If the start time is Feb 3rd, and the duration is “1 month”, then the end time will be March 3rd. In case the end time is not a valid calendar time, we will default to the previous valid time. Example: Start time is Jan 30th, duration is “1 month”; the end time will be Feb 28th (end of day).
Retrieve data ending at time point and starting two months earlier, where month is defined based on the current system timezone.
{
"time": {
"end", "1461703405.123456789",
"duration": "2 months"
}
}
Similar to the previous example, in this case we’ll compute start time going back two months.
The required properties are [start, duration] or [end, duration]. Optional properties are granularity and resolution. The API will return an error if these three properties are provided: star, end, duration.
- Requesting “this” calendar units
Retrieve data ending at a time point and starting at the beginning of this month, where month is defined based on the current system timezone.
{
"time": {
"end", "1461703405.123456",
"duration": "this month"
}
}
If it is March 5th at the moment, “this month” will set the start time to March 1st (beginning of day).
Other duration examples: “this hour”, “this day”, “this week”, “this month”, “this quarter”, “this year”.
When using “this week”, the week starts on Sunday.
The required properties are [end, duration]. Optional properties are: granularity and resolution. The API will return an error if these three properties are provided: star, end, duration.
- Requesting “previous” calendar units
Retrieve data starting at the beginning of 1 month ago, ending at the end of last month, where month is defined based on the current system timezone.
{
"time": {
"duration": "previous 1 month"
}
}
If it is March 5th at the moment, previous 1 month will set the start time to Feb 1st and the end time to Feb 28th (end of day). If it is March 5th at the moment, previous 2 months will set the start time to Jan 1st and the end time to Feb 28th (end of day).
- Requesting “last” calendar units
Retrieve data ending at a time point and starting from 1 month ago, where month is always defined as 30 days.
{
"time": {
"duration": "last 1 month"
}
}
If it is March 5th 18:30:30 at the moment, last 1 month will set the start time to Feb 3rd 18:30:30 (30 days ago), and the end time to March 5th 18:30:30. If it is March 5th 18:30:30 at the moment, last 2 months will set the start time to Jan 4th 18:30:30(2 * 30 = 60 days ago), and the end time to March 5th 18:30:30.
Other duration examples: “last second”, “last minute”, “last hour”, “last day” (1 day ago), “last week” (7 days ago), “last month” (30 days ago), “last quarter” (90 days ago), “last year” (365 days ago).
Filters property¶
The filters property is an array with filter objects. Each filter object can be of type STEELFILTER, WIRESHARK, or BPF. STEELFILTER is supported syntax for all data sources. If type is not specified, then it is defaulted to STEELFILTER. WIRESHARK and BPF types are supported for packet data source requests.
Example:
{
"filters": [
{
"type": "STEELFILTER",
"value": "(column_id == 'value1' or column_id >= 200) and (coumn_id == 'value2')"
},
{
"type": "WIRESHARK",
"value": "ip.src==192.168.0.0/16 and ip.dst==192.168.0.0/16"
},
]
]
If more than one filter is specified, then the data source will put them together using an AND operation. These two examples are equivalent:
{
"filters": [
{
"value": "(column_id == 'value1' or column_id >= 200) and (coumn_id == 'value2')"
}
]
]
{
"filters": [
{
"value": "column_id == 'value1' and column_id >= 200"
},
{
"value": "coumn_id == 'value2'"
}
]
]
The filters refer to column IDs. The list of all possible columns for each data source can be retrieved by querying the npm.reports.sources API. Please see the API discoverability section for more information. Generally, columns flagged as “grouped_by”: true (key columns) can be included in filters. Some data sources support threshold filters on metrical columns, such as avg_traffic.total_bytes_ps > 1000. To find out if a given data source supports filters on metrical columns, look inside the capabilities property:
{
"capabilities": {
"filters_on_metrics": true
}
}
STEELFILTER syntax¶
STEELFILTER provides a simple but powerful filter language that allows you to build quite complex filter expressions.
Below are the basic STEELFILTER expression formats:
- “{column_id} {operator} {value}”
- “{unary_operator} {expression}”
column_id - The string identifier of a data source column. The list of all possible columns for each data source can be retrieved by querying the npm.reports.sources API. For more information, please see API discoverability.
operator - The filter operation applied to the column using the value. The supported operators are:
| Operator | Meaning | Example |
|---|---|---|
| =, == , is, eq | Equal | ip.addr == ‘10.0.0.1’ |
| !=, <>, ne | Not Equal | ip.addr != ‘10.0.0.1’ |
| <=, le | Less than or equal | avg_traff ic.total_ bytes_ps <= 10000 |
| >=, ge | Greather than or equal | avg_traff ic.total_ bytes_ps => 10000 |
| >, gt | Greather than | avg_traff ic.total_ bytes_ps > 10000 |
| <, lt | Less than | avg_traff ic.total_ bytes_ps < 10000 |
| or | Or | ip.addr == ‘10.0.0.1’ or ip.addr == ‘10.0.0.2’ |
| in, in any | In | ip.addr IN (‘10.0.0.1 ‘, ‘10.0.0.2’ ) |
| (, ) | Parenthesis | (ip.addr == ‘10.0.0.1’ ) or (cli_ip.a ddr == ‘10.0.0.2’ and srv_ip.ad dr == ‘10.0.0.3’ ) |
value - comparison value to compare with column data based on the filter operation.
Special filter values
- Fully qualified IPv4 CIDR. Example: ip.addr == ‘10.0.0.0/8’
- Fully qualified IPv6 CIDR. Example: ip.addr == ‘2001:db8::/32’
- IPv4 wildcard. Example: 10.91., 10.91.., 10.91.126. Wildcards in the middle of the address are not allowed.
- IPv6 wildcard. Example: 2A03:2A03:2A03:2880:F000:F000::; 2A03:2A03:2A03:2880:F000:F000:F000:* Wildcards in the middle of the address are not allowed.
- DNS names are accepted. Example: ip.addr == ‘host.com’. DNS needs to be configured. The actual filter applied is only the first IP address that comes from forward DNS resolution. If host.com resolves to two addresses, only the first one will be applied. The response data object contains the “actual_filters” property; it contains the actual filter applied, so please inspect this property to ensure that the desired filter was applied.
Data Definition (response)¶
In addition to the requested properties, there are a number of read-only properties assigned to the data object when it is proccesed.
actual_time - Contains the actual time frame of the requested data. In some cases, this can be different from the requested time. The most common case is data not being available up to the start or end time. Clients should look into this information to make sure their request was fulfilled.
actual_filters - This tells what was actually used as a filter for the data request. It can be different from what was requested when DNS names are used to filter on IP columns.
status - This tells the current state of the data request. It is used to determine if the data is ready for retrieval.
For more information, please see the resource API documentation.
Gaps in data¶
Gaps in time series data
When there is a gap in the data, the corresponding rows will be omitted. For example, requesting 9:00am to 9:10am with a gap between 9:03 and 9:07:
[
[9:00am, "10.0.0.1", "10.0.0.2", "2000"],
[9:01am, "10.0.0.2", "10.0.0.1", "1000"],
[9:02am, "10.0.0.3", "10.0.0.1", "1000"]
[9:07am, "10.0.0.3", "10.0.0.1", "1000"],
[9:08am, "10.0.0.3", "10.0.0.1", "1000"],
[9:09am, "10.0.0.3", "10.0.0.1", "1000"]
]
Note: The values in the first column will be UNIX times, such as “1522684800”.
Gaps within one resolution bucket
In cases in which there is a gap within one resolution bucket, the start/end times are not adjusted to report the gap. For example, imagine a client has requested 9:00am to 9:10am. There are gaps between 9:00:20 to 9:00:45 and 9:05 - 9:08. The result will be the same.
Not applicable and No data (not available)¶
There are two special values:
N/A - not applicable¶
N/D - not available (no data)¶
In this example, a metric can be computed for protocol TCP, but cannot be computed for UDP, and the API returns #N/A (not applicable):
[
["tcp", "324234.23"],
["udp", "#N/A"]
]
In this example, a DNS cannot be retrieved for 10.0.0.2, the API returns #N/D (no data):
[
["10.0.0.1", "sitename.com"],
["10.0.0.2", "#N/D"]
]
Errors¶
When working with data APIs, clients must pay attention to two kinds of error conditions: HTTP errors and data objects in an error state.
HTTP 400 and 500 error responses¶
When the HTTP response code is not in the 200 range, the operation has failed. The data APIs follow the HTTP standards for response codes. If you get an error in 400 to 499 space, then it’s a client error - there is something in the HTTP request that must be corrected. Once corrected, you can re-try the request. If you get an error in 500 to 599 space, then it is a server error - there is something not quite right with the server (most likely the client request is well formatted). You can re-try the request sometime later. If the error persists, please contact Riverbed Technical Support to troubleshoot the issue further. HTTP redirects (300 to 399 range) are not used at the moment.
Common errors in the 400 to 499 space:
400 Bad request - There is something syntactically incorrect with the request. Usually this is a malformed JSON document, passing an invalid property or passing an invalid value for property. The response body will include an error object, and the error_text property should give enough details to point out what’s wrong with the request.
An example of sending a malformed JSON request:
--> POST /api/npm.reports/1.0/instances
{ xyz malformed json
"data_defs" : []
}
<-- 400
{
"error_id": "REQUEST_INVALID_INPUT",
"error_text": "Request parsing error: parsing failed: Invalid characters at line 1, column 2, processed 2"
}
401 Missing, expired or invalid access token - The data API is protected. Each data request requires a valid access token. If you are getting this error, you must authenticate, or re-authenticate. Please check the Authentication overview document for more details.
403 Forbidden. The request is valid, but the access token is not granting you with enough permissions to access this resource. Contact the system administrator to grant more permissions to the user account used in your script. Please check the Authentication overview document for more details.
404 - Not found. The request URL is invalid. Make sure the API URI (namespace and version) you are requesting is valid. This error can also happen if the system is in the process of rebooting. If the error persists, and your request is properly valid according to the API documentation, please contact Riverbed Technical Support.
405 - Method not supported. The request HTTP method is not supported for this URL.
Common errors in the 500 to 599 space:
500 Internal server error. If this error is persistent, please contact Riverbed Technical Support.
502 Bad Gateway. This error can happen if the system is in the process of rebooting. If this error is persistent, please contact Riverbed Technical Support.
504 Gateway timeout. If this error is persistent, please contact Riverbed Technical Support.
When data requests are successful, then the response code is in the range of 200 to 299. Typical success codes:
- 201 - Data object created successfully (POST methods)
- 204 - The operation is successful, and the response body is empty (typical for DELETE requests)
- 200 - The operation is successful, usual for GET and PUT (update) methods
Data objects in error state¶
There are two aspects of the data API that make it hard to always return HTTP errors when the data requests are invalid:
- The API is asynchronous in nature. The requests are briefly validated and the response is returned to the client. Then data requests are validated more deeply and processed, and, if an error condition occurs after the request is already accepted, it’s set to error state.
- The API is bulk in nature. One data request can include an array of data questions, and, while one can succeed, others can fail.
Example of sending a request with two data objects. The second one has a malformed time duration (xyz seconds).
--> POST /api/npm.reports/1.0/instances/sync
{
"data_defs" : [
{
"reference": "Top 10 IPs",
"source" : {
"name" : "aggregates"
},
"group_by" : [ "tcp.ip" ],
"time" : {
"duration" : "1 hour"
},
"limit": 10,
"top_by": [
{"id": "sum_traffic.total_bytes", "direction": "desc"}
],
"columns" : [
"tcp.ip",
"tcp.dns",
"sum_traffic.total_bytes"
]
},
{
"reference": "Top host groups",
"source" : {
"name" : "aggregates"
},
"group_by" : [ "host_group.id" ],
"time" : {
"duration" : "xyz seconds"
},
"top_by": [
{"id": "sum_traffic.total_bytes", "direction": "desc"}
],
"columns" : [
"host_group.id",
"host_group.name",
"sum_traffic.total_bytes"
]
}
]
}
<-- 201
{
"data_defs": [
{
"status": {
"state": "completed"
...
},
{
"status": {
"state": "error",
"messages": [
{
"module_id": "report_manager",
"details": {
"duration_tense": "xyz"
},
"id": "invalid_time_duration_tense",
"text": "Invalid duration tense: xyz"
}
]
...
}
]
}
You can see that while the first data objects gets into ‘completed’ state, and data can be retrieved out of it, the second one gets into ‘error’ state.
The error details are included in the messages sub-object. The messages sub-object is an array of (error) objects. In the majority of cases, it’s an array of one error. Presently, there are only a few cases in which it’s possible to return multiple errors. The error sub-object has the following properties:
module_id - The system (internal) module that triggered the error.
id - The unique identifier of the error within this module. The id property together with the module_id uniquely identify an error condition.
text - The error message (formatted in EN_US locale). While the data API supports returning column labels, descriptions and enums labels in ZH_CN locate, the errors are always returned in EN_US. Clients can re-format the errors using the ‘details’ property.
details - This provides more details in certain error conditions. It allows clients to re-format the error text.
API discoverability¶
There is an API provided to allow developers, who have the task to write custom scripts to retrieve data, to discover what data sources are available, and also find out more about the type/kind of data types (columns). Only a specific subset of data sources has been tested and certified for API-based use in end users’ scripts and other programs. These components are packets, aggregates, flow_ip and audit. In addition, the JSON data returned by each REST API resource includes an attribute “external_api_access”. The value of this attribute indicates whether that resource should be used in end users’ scripts. A value of “supported” indicates that resource is available for API-based use; a value “not_supported” indicates the opposite. API-based access to unsupported resources could cause the following problems: (a) Resource-specific APIs are not guaranteed to be present in future releases; (b) Using the API to interact with unsupported resources may also cause data processing gaps in the appliance.
For historical/performance data sources, the API is:
GET /api/npm.reports.sources/1.0/
For performance reasons, the packet data source (live API) is in a separate namespace:
GET /api/npm.probe.reports.sources/1.0/
Both APIs have identical structure.
The discovery API answers the following questions:
- What data sources are available
- What columns each data source supports
- What is the column set version (it changes when new columns are added or removed)
- For each column, what is the column type, unit, label, etc.
- Which columns can be used as keys (put in group_by property)
- Which columns can be used to retrieve Top talkers or Bottom talkers.
- Which metric columns can be used as threshold filters in STEELFILTER
- What are the suported granularities
Getting information about all data sources:
--> GET /api/npm.reports.soutrces/1.0/sources
{
"items": [
{
"info": {
"external_api_access": "supported",
"generated_at": "20/03/2018 14:20:30",
"version": "1.5",
"vendor": "Riverbed"
},
"columns": [{}, {}, ...],
"name": "aggregates",
"capabilities": {
"filters_on_metrics": true
},
"granularities": [
"60",
"300",
"3600",
"21600",
"86400"
]
},
{...}
]
Getting information about one data source, in this example “aggregates”:
--> GET /api/npm.reports.soutrces/1.0/sources/items/aggregates
{
"info": {
"external_api_access": "supported",
"generated_at": "20/03/2018 14:20:30",
"version": "1.5",
"vendor": "Riverbed"
},
"columns": [{}, {}, ...],
"name": "aggregates",
"capabilities": {
"filters_on_metrics": true
},
"granularities": [
"60",
"300",
"3600",
"21600",
"86400"
]
}
The response has the following properties:
info.version - Semantic versioning, telling the version of the available column set. Whenever a new column is addded, or a minor change is made (changed column label), there is a minor version bump (e.g., 1.3 -> 1.4). Whenever a column is removed, or the returned values are changed, there is major version bump (1.4 -> 2.0).
info.vendor - Reserved for future use. At the moment it returns “Riverbed” for all sources.
info.generated_at - For informational purpose only, the date/time when the columns were generated. This usually matches the date/time of the last product build.
name - The data source name. When making data requests, this matches the source.name property.
capabilities.filter_on_metrics - Whether this data source supports threshold filters on metrical columns, such as avg_traffic.total_bytes_ps > 1000.
granularities - An optional array with supported granularities.
columns - An array of supported columns. Each column can have the following properties:
- id - Unique identifier. This identifier is used in all properties in the data API - “columns”, “group_by”, “top_by”, “filters”, etc.
- field - The primary (most important) property that tells what the column is about. This is essentially the phenomenon being measured or the object that information is returned for. Notable examples: host_group.name; dns.response_time. The fields are shared across all data sources. One field can only have one “type” and one only one “unit”. Type and unit are described below.
- label - The property that tells the client how to label the column when present in grid views, chart views, column choosers, etc. All labels are properly internationalized. Currently only two locales are supported: en_US (American English) and zh_CN (Simplified Chinese). All labels are returned in en_US locale. If you want the Simplified Chinese labels, please send HTTP request header “Accept-Language: zh_CN”.
- description - This gives the client a more detailed description of the field. This property is not widely localized at this time.
- type - This is designed to tell the form in which the values will be returned
from the API. A given column field can be assigned only one type.
Possible types:
- string
- boolean
- number - float number
- integer - integer number
- ipaddr - IPv4 or IPv6 address
- ether - MAC address
- port - TCP/UDP port number
- timestamp - high precision timestamp
- duration - high precision duration
- array - the data values are a JSON array with items from one of the other types.
- enum - Boolean. This is set to true if the column values are limited to enumerated list. Clients can send separate GET request to retrieve all posible values, e.g: GET /api/npm.reports.sources/1.0/fields/items/ip.protocol.type_name
- minimum - This specifies the minumum possible value for number or integer columns.
- maximum - This specifies the maximum possible value for number or integer columns.
- unit - This tells what quantity (measure) the values returned from the
API will be in. Notable examples: bytes; seconds. A given column
field can be assigned only one unit. The assignment is done inside
the dimensions/fields.yaml source file. When unit is not applicable,
it is set to none. Possible values:
- bytes - For simplicity and consistency, all values returned from the datra APIs (from all data sources) are measured in bytes and never in KB, MB, GB, etc. Conversions in other units must be performed in the clients of the API.
- seconds - For simplicity and consistency, all values returned from the data APIs (from all data sources) are measured in seconds and never in minutes, milliseconds, microseconds etc. Columns of types timestamp and duration use the unit seconds. timestamp Values are in epoch form - “.”, where the number of partial seconds is optional. Notable examples: “1522772201”; “1522772201.123”; “1522772201.123456789”. The lowest precision can be nanoseconds - used in low-level packet data source. duration Values are always measured in seconds form - “.”, where the number of partial seconds is optional. Notable examples: “60” - sixty seconds; “1.001” - one second and one millisecond; “2.000000001” - two seconds and one nanosecond. The lowest precision can be nanoseconds.
- rate - This tells what rate (frequency of sampling) the values returned from the API will have. Notable examples: ps - per second; pgs - per granularity sample, used in packet data source, indicates that the sampling varies based on the requested granularity.
- operation - This tells what mathematical operation will be applied to values when the results need to be grouped by or aggregated to higher granularity or higher resolution.
- direction - Directionality is a common (across many networking
data sources) event. When measuring traffic data flow, for example,
we can say that from client to server there are X number of bytes
transferred; from server to client there are Y number of bytes; or
when measuring different objects we can say that X number of bytes
were send from this object to its peers and Y number of bytes were
sent from peers to self. Because direction is a very common event of
most data sources, allow clients to find related columns;
some columns are flagged with directions. Possible values:
- in - inbound
- out - outbound
- c2s - client to server
- s2c - server to client
- m2p - member to peer
- p2m - peer to member
- role - Similar to direction, this tells what kind of role the
object performs in a network event. In order to allow clients to find
related columns, some columns are flagged with role. Possible values:
- cli - Client
- srv - Server
- conn - Connected
- parent - Parent
- src - Source
- dst - Destination
- requested - Requested
- granted - Granted
- request - Request
- reply - Reply
- lower - Lower
- upper - Upper
- ctrl - Control
- caller - Caller
- receiver - Receiver
- user - User
- grouped_by - Boolean flag that tells whether the data source can group the data based on a given column field. Columns with grouped_by: true can be used in the group_by array property.
- top_by - {asc: true|false, desc: true|false} This tells whether the data source can top the data in ascending or descending order (provide top talkers or bottom talkers). Retrieving bottom talkers is not a supported feature throughout all data sources. Due to the large volume of data, limited resources on the system, and the desire to respond in a timely fashion, some data sources drop information for bottom talkers.
- sortable: Boolean; this tells whether the back-end can sort on this column. The property is optional. It is set only for columns that are not sortable (sortable: false). When this property is missing, assume it’s true. Presently, the only columns that are not sortable are DNS names due to forward DNS server limitations/performance reasons. If this is false, then using this column id in GET /data?sort_by=id will return an error.
- metric: Boolean, reserved for use by Riverbed. It tells whether the column is metrical. The property is optional. It is set only for columns that are metrics (metric: true). When this property is missing, assume it’s false. Riverbed clients use it to build a chart widget column chooser based on it. This property allows for separation between key columns and metrical columns and any other (informational) columns.
- custom: Boolean, reserved for use by Riverbed. This flag indicates that this field is configurable by the client (via separate/custom API). Clients can declare that they want a special field or measurement to be tracked or calculated by a data source. This is currently used only in the web transaction analysis data source.
- custom_id - Reserved for use by Riverbed. Unique identifier of client configured columns. This property is optional. It is set to true only for fields that are configurable by the client (custom: true). This is currently used only in the the web transaction analysis data source.
- default_if_absent - Reserved for used by Riverbed. Tells the clients what values should be defaulted to if they are absent. There are two possible values at this time - “#N/A” and “0”. The use case for “0” is to change Line charts to draw the line to the 0 axis rather than draw missing value line (e.g. dashed-line).
- synthesized - Reserved for used by Riverbed.
- categories - An array of categories. Reserved for use by Riverbed, used to categorize columns in diffrent groups for presentation purpoeses.
- groups - An array of groups. Similar to categories. Reserved for used by Riverbed, used to further categorize columns in diffrent groups for presentation purposes.
Workflows for accesing data¶
Asynchronous API (performance/historical data)¶
The asynchronous API goes through the following steps:
- Step 1: POST (Create data request)
- Step 2: GET status (multiple calls until completed)
- Step 3: GET data
- Step 4: Delete the data request
Step 1: Create an instance object with one or multiple data request objects (in this example, two):
The client performs HTTP POST request to the following end-point:
--> POST /api/npm.reports/1.0/instances
{
"data_defs": [
{...},
{...}
]
}
<-- 201 Created
Response Headers:
Location: /api/npm.reports/1.0/instances/items/1234
...
Response body:
{
"id": 1234,
"data_defs": [
{"id": 1, "status": {"state": "pending"}, ...},
{"id": 2, "status": {"state": "pending"}, ...}
]
}
When the request is received on the server, an instance ID:1234 is generated to serve it. In this example, the instance contains two data objects, and their IDs are 1 and 2, respectively. The data objects can be addressed separately:
/api/npm.reports/1.0/instances/items/1234/data_defs/items/1
/api/npm.reports/1.0/instances/items/1234/data_defs/items/2
The Location response header contains the newly created instance URI. It’s recommended that you use it as a base for constructing additional requests.
Step 2: Wait for data request to complete
At this point, the initial POST is completed. Some basic validation has been performed, and the data request(s) have been accepted and are being processed. Each data object goes through a state machine with the following states:
Successful data retrieval:
pending --> initializing --> executing --> completed
Error cases:
pending --> initializing --> error
pending --> initializing --> executing --> error
The client should perform ‘status’ polls and wait for each data request to complete. The data request objects are complete when they get into “completed” or “error” state. In case the instance object contains multiple data objects, the data from one object can be requested as soon as it’s done, while other data request are still being processed. There are two methods for getting the data objects state:
Getting status for one data object within one instance:
GET /api/npm.reports/1.0/instances/items/1234/data_defs/items/1/data_defs/items/1/status
<-- 200 OK
{
"id": 1,
"status": {
"state": "completed",
"progress": {
"percent": 100,
"remaining_sec": 0
}
}
}
Getting status for all data objects within one instance:
GET /api/npm.reports/1.0/instances/items/1234/data_defs/items/1/data_defs/status
<-- 200 OK
[
{
"id": 1,
"status": {
"state": "completed",
"progress": {
"percent": 100,
"remaining_sec": 0
}
},
{
"id": 1,
"status": {
"state": "error",
"progress": {
"percent": 100,
"remaining_sec": 0
},
"messages": [
{
"module_id": "module_id",
"id": "error_id",
"text": "error text",
"details": {"key1": "value1", ...}
}
]
}
}
}
Step 3: request the data
When a data object gets into the “completed” state, data can be requested. To request all the data for one data object:
GET /api/npm.reports/1.0/instances/items/1234/data_defs/items/1/data_defs/data
<-- 200 OK
{
"meta": {
"count": 5,
"total": 5
},
"data": [
[
"1",
"Default-10.x.x.x",
"2528527700"
],
[
"4",
"Default-192.168.x.x",
"2058158942"
],
[
"2",
"Default-Internet",
"1726708312"
],
[
"0",
"Other Group",
"1357016354"
],
[
"3",
"Default-172.x.x.x",
"87192"
]
],
"columns": [
"host_group.id",
"host_group.name",
"sum_traffic.total_bytes"
]
}
The “data” results is always represented as an array of arrays. The data property can be omitted if the data response is empty.
The data response includes the “columns” array, which tells the order of the columns returned. The order of columns should match the columns in the request. Different column order can be requested by providing columns in the comma-separated URL parameter.
Clients can request a subset of the data, by providing URL parameters.
URL parameters: /data?limit=&offset=&sort_by=&columns=
limit - number, limit the returned rows to this value.
offset - number, return rows offset by this value, commonly used in paging views.
sort_by - string, comma-separated list of metrics to re-order the data. An example:
&sort_by=column_id1%20desc,column_id2%20asc
‘%20’ is the URL-encoded space character. You can pass multiple columns to sort. Please note that sort_by will re-order the rows that were retreieved already by the data object; it will not produce new top talkers, and it should not be confused with the ‘top_by’ property passed in the data (request JSON) object. If the request has top_by of ‘Throughput’ desc with limit of 1000, then the data object will contain the top 1000 talkers by Throughput, requesting a sort_by ‘Response Time’ would return the worst ‘Response Time’ objects among the 1000 top talkers by Throughput.
columns - A string, comma-separated list of columns. By default, the data response will include the columns in the order specified in the JSON request; however, providing a columns property allows changing the order, or getting a subset.
An example of getting name and id for the first two host groups, sorted by id descending:
GET /api/npm.reports/1.0/instances/items/1234/data_defs/items/1/data?limit=2&offset=0&sort_by=host_group.id%20desc&columns=host_group.name,host_group.id
<-- 200 OK
{
"meta": {
"count": 2,
"total": 5,
"offset": 0,
"limit": 2,
"sort_by": "host_group.id desc"
},
"data": [
[
"Default-192.168.x.x",
"4"
],
[
"Default-172.x.x.x",
"3"
]
],
"columns": [
"host_group.name",
"host_group.id"
]
}
The data response includes a “meta” object that includes the following properties:
total - The total number of records ready to be retrieved. The total number of rows can be equal to or less than the “limit” in the data request object.
count - The number of records in the current response. It matches the length of the “data” array.
sort_by - Sorted by column. It is included when sort_by URL parameter is specified.
limit - The current requested limit. It is included when the limit URL parameter is specified.
offset - The current requested offset. It is included when the offset URL parameter is specified.
columns - The current requested list of columns. It is included when columns URL parameter is specified.
Clients can also (bulk) request data for all data objects within one instance:
GET /api/npm.reports/1.0/instances/items/1234/data
<-- 200 OK
{
"data_defs": [
{
"datasource_id": "152455",
"meta": {
"count": 5,
"total": 5
},
"reference_id": "",
"data": [
[
"1",
"Default-10.x.x.x",
"2528527700"
],
[
"4",
"Default-192.168.x.x",
"2058158942"
],
[
"2",
"Default-Internet",
"1726708312"
],
[
"0",
"Other Group",
"1357016354"
],
[
"3",
"Default-172.x.x.x",
"87192"
]
],
"id": 1,
"columns": [
"host_group.id",
"host_group.name",
"sum_traffic.total_bytes"
]
},
{...}
]
}
When getting (bulk) data results for all data objects, the URL parameters to get a subset of the data are not supported.
** Step 5:** Delete the data request
Deleting the data request is an optional step. It is recommended to send a DELETE request after the data is retrieved, and won’t be referenced anymore. If a DELETE request is not sent, then the system will automatically purge the data objects within one day.
DELETE /api/npm.reports/1.0/instances/items/1234
<-- 204 Accepted
Synchronous API (performance/historical data)¶
The synchronous API exists for performance and debugging reasons. Its goal is to reduce the number of HTTP messages between client and server. It allows sending a data request and receiving the data in a single HTTP call. It’s convenient for quick tests. While in the general case, this API call should take only one HTTP call, please note that the server automatically closes HTTP connections if they take more than 60 seconds. In order to work around this HTTP limitation, if the data retrieval takes more than 50 seconds, then the server will respond with unfinished status, and the client is supposed to re-attach by sending another request that would block for another 50 seconds. This method of data retrieval also exposes a risk of occupying too many connections; please do not issue more than a few sync requests at the same time.
The asynchronous API goes through the following steps:
Case 1: Data request takes less than 50 seconds to complete
- Step 1: POST (Create data request)
- Step 2: Delete the data request
Step 1: Send Synchronous data request
--> POST /api/npm.reports/1.0/instances/sync
{
"data_defs": [
{...},
{...}
]
}
<-- 201 Created
Response Headers:
Location: /api/npm.reports/1.0/instances/items/1234
...
Response body:
{
"id": 1234,
"data_defs": [
{"id": 1, "status": {"state": "completed"}, "data": [{}, {}...], ...},
{"id": 2, "status": {"state": "completed"}, "data": [{}, {}...], ...}
]
}
** Step 2:** Delete the data request
Deleting the data request is an optional step. It is recommended to send a DELETE request after the data is retrieved and won’t be referenced anymore. If a DELETE request is not sent, then the system will automatically purge the data objects within one day.
DELETE /api/npm.reports/1.0/instances/items/1234
<-- 204 Accepted
Case 2: Data request takes more than 50 seconds to complete
- Step 1: POST (Create data request)
- Step 2: Re-attach (it takes more than 50 seconds to process the data request, this step can be repeated multiple times)
- Step 3: Delete the data request
Step 1: POST (Create sync data request)
--> POST /api/npm.reports/1.0/instances/sync
{
"data_defs": [
{...},
{...}
]
}
<-- 201 Created
Response Headers:
Location: /api/npm.reports/1.0/instances/items/1234
...
Response body:
{
"id": 1234,
"data_defs": [
{"id": 1, "status": {"state": "executing"}, ...},
{"id": 2, "status": {"state": "executing"}, ...}
]
}
Step 1: Re-attach to sync request. Note that at this point, the client can choose to continue using the asynchronous workflow.
--> GET /api/npm.reports/1.0/instances/items/1234/sync
<-- 200 OK
{
"id": 1234,
"data_defs": [
{"id": 1, "status": {"state": "completed"}, "data": [{}, {}...], ...},
{"id": 2, "status": {"state": "completed"}, "data": [{}, {}...], ...}
]
}
Live API for retrieving current/recent packet data¶
When it comes to getting packet data, the first action is to choose a packet source. There are four different types of packet sources:
Virtual interface group:
{
"source": {
"name": "packets",
"path": "vifgs/<vifg id>"
}
}
Note: a list of all virtual interface groups can be retrieved by doing:
GET /api/npm.packet_capture/2.0/vifgs
Job source:
{
"source": {
"name": "packets",
"path": "jobs/<job id>"
}
}
Note: a list of all capture jobs can be retrieved by doing:
GET /api/npm.packet.capture/2.0/jobs
Clip source:
{
"source": {
"name": "packets",
"path": "clips/<clip id>"
}
}
Note: a list of all clips can be retrieved by doing:
GET /api/npm.clips/1.0/clips
File system source:
{
"source": {
"name": "packets",
"path": "fs/<username>/mytrace.pcap"
}
}
Note: a list of all files can be retrieved by doing:
GET /api/npm.filesystem/1.0/fs
The Live API goes through the following steps:
- Step 1: POST (Create live data request)
- Step 2: GET status (multiple calls until the state becomes ‘collecting’)
- Step 3: GET info (multiple calls until desired)
- Step 4: GET data (multiple calls until desired)
- Step 5: Delete the live data request
Step 1: Create a live instance object with one or multiple data request objects (in this example two):
The client performs HTTP POST request to the following end-point:
--> POST /api/npm.probe.reports/1.0/instances
{
"live": true,
"data_defs": [
{
"source": {
"name": "packets",
"path": "jobs/c69a9a14-fb82-46a7-b7c1-70751b70f69b"
},
"time": {"retention_time": "86400", "granularity": "1"},
...
},
{
"source": {
"name": "packets",
"path": "jobs/c69a9a14-fb82-46a7-b7c1-70751b70f69b"
},
"time": {"retention_time": "3600", "granularity": "0.001"},
...
},
]
}
<-- 201 Created
Response Headers:
Location: /api/npm.probe.reports/1.0/instances/items/1234
...
Response body:
{
"id": 1234,
"live": true,
"data_defs": [
{"id": 1, "status": {"state": "pending"}, ...},
{"id": 2, "status": {"state": "pending"}, ...}
]
}
When the request is received on the server, an instance ID:1234 is generated to serve it. In this example, the instance contains two data objects, and their IDs are 1 and 2, respectively. The first data object will have a retention time of one day. The second data object will have a retention time of one hour. Since the request is “live”, new data will automatically become available, the end time will keep going forward, while the available start time will remain constant until the retention time duration is achieved, and then the oldest data will be purged. Live requests require granularity to be specified. In this example, data object one will be filled up with data bins, each containing summarized packet metrics for a one-second interval. The second data object will be filled up with data bins, each containing summarized packet metrics for a one-millisecond interval.
The data objects can be addressed separately:
/api/npm.reports/1.0/instances/items/1234/data_defs/items/1
/api/npm.reports/1.0/instances/items/1234/data_defs/items/2
The Location response header contains the newly created instance URI. It’s recommended that you use it as a base for constructing additional requests.
Step 2: Wait for data request to be initialized and get into the ‘collecting’ state
At this point, the initial POST is completed. Some basic validation have been performed and the data request(s) have been accepted and are being processed. Each data object goes through a state machine with the following states:
Successful data retrieval:
pending --> initializing --> collecting
Error case:
pending --> initializing --> error
The client should perform ‘status’ polls and wait for each data request to become “collecting”. In case the instance object contains multiple data objects, the data from one object can be requested as soon as it’s available, while other data requests are still being processed. The packet data source provides additional progress information while it’s being initialized. The percent property goes from 0 to 100.
There are two methods for getting the data objects state:
Getting status for one data object within one instance. In this example, it’s 45% done initializing:
GET /api/npm.reports/1.0/instances/items/1234/data_defs/items/1/data_defs/items/1/status
<-- 200 OK
{
"id": 1,
"status": {
"state": "initializing",
"progress": {
"percent": 45
}
}
}
Getting status for all data objects within one instance:
GET /api/npm.reports/1.0/instances/items/1234/data_defs/items/1/data_defs/status
<-- 200 OK
[
{
"id": 1,
"status": {
"state": "collecting",
"progress": {
"percent": 100,
}
},
{
"id": 1,
"status": {
"state": "error",
"progress": {
"percent": 100
},
"messages": [
{
"module_id": "module_id",
"id": "error_id",
"text": "error text",
"details": {"key1": "value1", ...}
}
]
}
}
}
Step 3: Request information about one live data object
When a data object gets into the “collecting” state, data is constantly retrieved and made available. Clients may want to find out what’s the current available start/end time that can be retrieved.
Perform this HTTP call to find out if the data objects remain in a healthy “collecting” state, and if there is a new end_time available (never than previously retrieved):
GET /api/npm.reports/1.0/instances/items/1234/data_defs/items/1
<-- 200 OK
{
"status": {
"state": "collecting",
"progress": {
"percent": 100
}
},
"actual_time": {
"time_ranges": [
{"start": "1522346400", "end": "1522432800"}
]
}
},
...
}
Clients can also request (bulk) information for all data objects:
GET /api/npm.reports/1.0/instances/items/1234/
<-- 200 OK
[
{
"status": {
"state": "collecting",
"progress": {
"percent": 100
}
},
"actual_time": {
"time_ranges": [
{"start": "1522346400", "end": "1522432800"}
]
}
},
...
},
{...}
]
Step 4: GET data (multiple calls until desired)
When the object is in “collecting” state, clients can continuously get the latest live bin(s) for an indefinite time. Clients normally memorize the last end_time retrieved, and request new data when a newer end_time is observed at step 3.
The “data” result is always represented as array of arrays. The data property can be omitted if the data response is empty.
The data response includes a “columns” array, which tells the order of the columns returned. The order of columns should match the columns in the request. A different column order can be requested by providing columns in a comma-separated URL parameter.
Clients can request subset of the data, by providing URL parameters.
URL parameters: /data?resolution=&limit=&offset=&sort_by=&columns=&nsamples=true
resolution - request the data to be aggregated using resolution. It must be equal to or higher than the granularity. The resolution must be a multiple of the requested granularity.
Note: Granularity and resolution are explained in details in the API details sections below.
limit - Number, limit the returned rows to this value.
offset - Number, return rows after this value, commonly used in paging views.
sort_by - String, comma-separated list of metrics to re-order the data. An example:
&sort_by=column_id1%20desc,column_id2%20asc
‘%20’ is the URL-encoded space character. You can pass multiple columns to sort. Please note that sort_by will re-order the rows that were retrieved already by the data object; it will not produce new top talkers, and it should not be confused with the ‘top_by’ property passed in the data (request JSON) object. If the request has top_by of ‘Throughput’ desc with limit of 1000, then the data object will contain the top 1000 talkers by Throughput; requesting a sort_by ‘Response Time’ would return the worst ‘Response Time’ objects among the 1000 top talkers by Throughput.
columns - String, comma-separated list of columns. By default, the data response will include the columns in the order specified in the JSON request; however, providing a columns property allows changing the order, or getting a subset.
nsamples - Boolean. When nsamples=true is specified in the URL, all average columns would be returned in the format “total|samples”, e.g., 41222|55 (avgerage value: 41222 / 55 = 749.4909090909091). This allows clients to properly re-aggregate weighted average metrics.
The data response includes a “meta” object that includes the following properties:
total - The total number of records ready to be retrieved. The total number of rows can be equal to or less than the “limit” in the data request object.
count - The number of records in the current response. It matches the length of the “data” array.
sort_by - Sorted by column(s). It is included when sort_by URL parameter is specified.
limit - The current requested limit. It is included when the limit URL parameter is specified.
offset - The current requested offset. It is included when the offset URL parameter is specified.
columns - The current requested list of columns. It is included when the columns URL parameter is specified.
resolution - The current resolution being used.
nsamples - The current requested nsamples value. It is included when the columns URL parameter is specified.
An example of getting live data - retrieve five milliseconds from 1522432799.999 to 1522432800.004:
GET /api/npm.reports/1.0/instances/items/1234/data_defs/items/1/data?resolution=0.001
&start_time=1522432799.999&end_time=1522432800.004
<-- 200 OK
{
"meta": {
"count": 5,
"total": 5,
"start_time": "1522432799.999",
"end_time": "1522432800.004",
"resolution": "0.001"
},
"data": [
[
"1522432799.999",
"4324234.23444"
],
[
"1522432800.000",
"12223.0011"
],
[
"1522432800.001",
"98352.2344"
],
[
"1522432800.002",
"36333455.7245"
],
[
"1522432800.003",
"22123.66678"
]
],
"columns": [
"start_time",
"sum_tcp.total_bytes"
]
}
Getting live data, in this example, retrieve five seconds from 1522432800 to 1522432805 bucketed into one data bin (using resolution of 5 seconds):
GET /api/npm.reports/1.0/instances/items/1234/data_defs/items/1/data?resolution=5start_time=1522432800&end_time=1522432805
<-- 200 OK
{
"meta": {
"count": 1,
"total": 1,
"start_time": "1522432799.999",
"end_time": "1522432800.004",
"resolution": "5"
},
"data": [
[
"15224327800",
"98774324234.23444"
]
],
"columns": [
"start_time",
"sum_tcp.total_bytes"
]
}
Note: Granularity and resolution are explained in details in the API details sections below.
There is a Boolean URL parameter, nsamples=true; when passed, all average columns would be returned in the format “total|samples”, e.g. 41222|55 (avgerage value: 41222 / 55 = 749.4909090909091). This allows clients to properly re-aggregate weighted average metrics.
** Step 5:** Delete the live data request
Deleting the live data request is a highly recommended step. If the DELETE request is not sent, then the system will automatically delete the data object one hour after the last seen request to it.
DELETE /api/npm.probe.reports/1.0/instances/items/1234
<-- 204 Accepted
To increase the auto-delete timeout, please set the idle_timeout property in the initial request to create live object (Step 1). In this example, the idle timeout is set to 1 day:
--> POST /api/npm.probe.reports/1.0/instances
{
"live": true,
"idle_timeout": "86400",
"data_defs": [
...
APIs to access packets¶
Export Handle object¶
The process of retrieving packets basically comes down to creating a export handle object, then monitoring the state of the handle, and, finally, downloading the packets using the streaming resource.
The Export Handle object has the following properties: id, config, status.
id - Hex Integer (read-only). A unique identifier created on the server.
Config property¶
This allows clients to explicitly pick the packet source to handle the export request and customize the query.
The config property is an object with the following sub-properties:
path - string (required). It can be: capture file or virtual file (fs/path/to/file.pcap); job (jobs/ef3b2a1b-0bd8-47e5-8d2c-ac81bb8766ae); virtual interface group (vifgs/1000); clip (clips/ef3b2a1b-0bd8-47e5-8d2c-ac81bb8766ae0004).
mode - enum (LEGACY, LIVE_STREAM, BUFFER_STREAM, STREAM_TO_FILE, PREPARE) (optional). If mode is not specified, then it is defaulted to LEGACY.
Used to specify the workflow for retrieving packets:
- Synchronous API (LEGACY, LIVE_STREAM) to filter and get stream of packets
- Asynchronous API (BUFFER_STREAM, STREAM_TO_FILE) to store the filtered packets on the appliance and retrieve them once the filtering is completed.
- Prepare API (PREPARE) to check packet availability and estimate the export size before deciding which workflow to use
start_time - string (optional). Used to specify the start time of the export.
end_time - string (optional). Used to specify the end time of the export.
time_intervals - array of time_interval objects (optional). Used to specify multiple distinct time intervals.
filters - array of filter objects (optional). Each filter object can be of type STEELFILTER, WIRESHARK, or BPF.
Note: only STEELFILTER type on IP addresses allows packet access optimizations when using capture jobs in “Optimize for read” mode.
stop_rule - object (optional). Used to specify export limits. It has the following sub-properties:
- time_limit - integer (optional). Used to specify max number of seconds.
- size_limit - integer (optional). Used to specify max number of bytes (This property is required in BUFFER_STREAM workflow).
- packet_limit - integer (optional). Used to specify max number of packets.
snap_length - integer (optional). Used to specify packet snap length. If snap_length is not specified, then it is defaulted to 65535.
output_format - enum (PCAP_US, PCAP_NS, PCAPNG_US, PCAPNG_NS). Optional. If output_format is not specified, then it is defaulted to PCAP_US.
output_filename - string (optional). Used to specify the filename path within the npm file system when using the STREAM_TO_FILE mode.
connection_timeout_msec - integer (optional). Used to specify the amount of time the Export Handle will be available once the export stream it’s ready to be downloaded.
If connection_timeout_msec is not specified, then it is defaulted to 180000 (3min). This property does not apply to BUFFER_STREAM and STREAM_TO_FILE workflows.
handle_timeout_msec - integer (optional). Used to specify the amount of time the Export Handle will be available once the export stream is terminated.
If handle_timeout_msec is not specified, then it is defaulted to 0 and the handle will be destroyed automatically once the download is terminated.
This property does not apply to BUFFER_STREAM workflows.
Example:
{
"config": {
"path": "jobs/ef3b2a1b-0bd8-47e5-8d2c-ac81bb8766ae",
"mode": "BUFFER_STREAM",
"filters": [
{
"type": "STEELFILTER",
"value": "(column_id == 'value1' and column_id == 'value2')"
}
],
"start_time": "123456789",
"end_time": "2345678890",
"stop_rule": {
"size_limit": 1000000000
}
}
}
Status property¶
This allows clients to retrieve the Export handle state and stats. The status property is an object with the following sub-properties:
state - enum (UNINITIALIZED, INITIALIZING, READY, RUNNING, STREAMING, ERRORS, NODATA, DONE). Used to understand the Export Handle state and if it’s possible to retrieve the packets.
- UNINITIALIZED: The handle has not been initialized yet
- INITIALIZING: The handle has been create and the pre-filtering process is running
- READY: The handle is configured and ready to start the processing/streaming. This state is used only for LIVE_STREAM, BUFFER_STREAM and PREPARE workflows
- RUNNING: The handle is processing/streaming the data. This state is used only for LEGACY and STREAM_TO_FILE workflows
- STREAMING: The handle is processing/streaming the data. This state is used only for LIVE_STREAM and BUFFER_STREAM workflows
- ERRORS: The handle is in an error state.
- NODATA: The handle has no data available for the download.
- DONE: The processing for the handle has completed.
progress - float. Used to check the progress of the packet processing, that includes filtering, buffering and streaming.
creation_time - string. Time at which the export handle was created.
owner - string. User who initiated the export.
stats - object (optional). Statistics of the export handle It has the following sub-properties:
- exported_packets - integer. Number of exported packets.
- exported_bytes - integer. Number of exported bytes.
- total_bytes - integer. Approximate number of bytes in the input.
- estimated_bytes - integer (optional). Estimated number of bytes that will be exported. This property is set only if microflow index pre processing is performed
- estimated_packets - integer (optional). Estimated number of packets that will be exported. This property is set only if microflow index pre processing is performed
- processed_packets - integer. Number of processed packets, does not take in consideration packets pre filtered by npm_capture.
- dropped_packets - integer (optional).
- truncated - boolena (optional). Set to True if the output has been truncated because we reached the limit or the disk was full before we processed all the requested packets.
error - object (optional). Details in case of error. It has the following sub-properties:
- error_id - string.
- error_text - string. Number of exported bytes.
- error_metrics - object (optional). Array of key/value error metrics, used for detailed error responses.
Example:
{
"status": {
"state": "STREAMING",
"progress": 50.4,
"creation_time": "123456789",
"owner": "username",
"stats": {
"exported_packets": 12345,
"exported_bytes": 10002984,
"total_bytes": 19902452,
"processed_packets": 12345,
}
}
}
Workflows for accesing packets¶
Asynchronous API (BUFFER_STREAM, STREAM_TO_FILE)¶
Using the asynchronous modes the packets are filtered and stored in the appliance, the clients can retrieve the pre-filtered packet stream or use the pcap to access packet data using the npm.probe.reports service.
The asynchronous API goes through the following steps:
- Step 1: POST (Create export request)
- Step 2: GET status (multiple calls until ready)
- Step 3: GET stream of packets
- Step 4: DELETE export handle
BUFFER_STREAM Mode¶
Step 1: Create an export handle:
The client performs HTTP POST request to the following end-point:
--> POST /api/npm.packet_export/1.3/exports
{
"config": {
"mode": "BUFFER_STREAM"
...
}
}
<-- 201 Created
Response Headers:
Location: /api/npm.packet_export/1.3/exports/items/00000012
...
Response body:
{
"id": 00000012,
"config": { ... },
"status": { ... }
}
When the request is received on the server, an instance ID:00000012 is generated to serve it.
The Location response header contains the newly created instance URI. It’s recommended that you use it as a base for constructing additional requests.
Step 2: Wait for packet stream request to be ready
At this point, the initial POST is completed. Some basic validation has been performed, and the export handle starts to storing the packets to disk.
The Export handle goes through a state machine with the following states:
Successful data retrieval:
INITIALIZING --> READY <-> STREAMING
Error cases:
INITIALIZING --> NODATA
INITIALIZING --> ERRORS
INITIALIZING --> READY -> ERRORS
INITIALIZING --> READY <-> STREAMING -> ERRORS
The client should perform ‘status’ polls and wait for the ‘state’ to be “READY”, “NODATA” or “ERRORS”.
Getting status for export handle:
GET /api/npm.packet_export/1.3/exports/items/00000012
<-- 200 OK
{
"id": 00000012,
"status": {
"state": "READY",
...
}
}
Step 3: Request packets stream
When an export handle gets into the “READY” state packets can be requested.
To request the packets:
GET /api/npm.packet_export/1.3/packets/items/00000012
<-- 200 OK
<-- binary stream containing packets in the format set in the configuration (output_format)
The export state changes to “STREAMING” and, using a different connection, the client can query the status and the stats:
GET /api/npm.packet_export/1.3/exports/items/00000012
<-- 200 OK
{
"id": 00000012,
"status": {
"state": "STREAMING",
...
}
}
Once the stream is completed the Export handle state goes back to “READY” and the client can perform a new download of the packets.
Step 4: Delete the export handle
A BUFFER_STREAM export handle will be available until the client deletes it.
When the packets have been retrieved and there’s no plan to use them again it is recommended to perform a DELETE request to cleanup disk space in the appliance.
DELETE /api/npm.packet_export/1.3/exports/items/00000012
<-- 204 Accepted
STREAM_TO_FILE Mode¶
STREAM_TO_FILE export handle stores the packets in the npm file system making the pcap available to be accessed by the npm.filesystem and npm.probe.reports services.
In this mode the ‘output_filename’ property is required and it defines the path to use to access the pcap with npm.filesystem and npm.probe.report api.
Note: using “LEGACY” mode and specifying ‘output_filename’ is equivalent to use “STREAM_TO_FILE” mode.
Step 1: Create an export handle:
The client performs HTTP POST request to the following end-point:
--> POST /api/npm.packet_export/1.3/exports
{
"config": {
"mode": "STREAM_TO_FILE"
...
}
}
<-- 201 Created
Response Headers:
Location: /api/npm.packet_export/1.3/exports/items/00000012
...
Response body:
{
"id": 00000012,
"config": { ... },
"status": { ... }
}
When the request is received on the server, an instance ID:00000012 is generated to serve it.
The Location response header contains the newly created instance URI. It’s recommended that you use it as a base for constructing additional requests.
Step 2: Wait for packet stream request to be ready
At this point, the initial POST is completed. Some basic validation has been performed, and the export handle starts to store the packets to disk.
The Export handle goes through a state machine with the following states:
Successful data retrieval
UNINITIALIZED --> RUNNING -> DONE
Error cases:
UNINITIALIZED --> ERRORS
UNINITIALIZED --> RUNNING --> ERRORS
The client should perform ‘status’ polls and wait for the ‘state’ to be “DONE” or “ERRORS”.
Getting status for export handle:
GET /api/npm.packet_export/1.3/exports/items/00000012
<-- 200 OK
{
"id": 00000012,
"status": {
"state": "DONE",
...
}
}
Step 3: Request packet stream
When an export handle gets into the “DONE” state, packets are stored in the npm file system and they can be accessed with the npm.filesystem api.
Getting packets with the npm.filesystem api:
GET /api/npm.filesystem/1.0/fileop/download/PATH_TO_FILE (output_filename)
<-- 200 OK
<-- stream of packets in the format specified in the configuration
Step 4: Delete the export handle
Deleting the data request is an optional step. By default STREAM_TO_FILE export handles are deleted when the stream is in DONE state.
In case the client wants to track errors or stats at the end of the export it is recommended to set ‘handle_timeout_msec’ to a couple of minutes, that will allow the handle to stay active after the stream is closed.
When the status has been retrieve it is recommended to perform a DELETE request. If a DELETE request is not sent, then the system will automatically purge the export handle objects within ‘handle_timeout_msec’.
DELETE /api/npm.packet_export/1.3/exports/items/00000012
<-- 204 Accepted
Synchronous API (LEGACY, LIVE_STREAM)¶
Using the synchronous modes the packets are filtered and immediately provided to the clients using a binary stream.
Once the stream is completed the packets cannot be retrieved with the same Export Handle.
The synchronous API goes through the following steps:
- Step 1: POST (Create export request)
- Step 2: GET status (multiple calls until ready)
- Step 3: GET stream of packets
- Step 4: DELETE export handle (optional)
LIVE_STREAM Mode¶
Step 1: Create an export handle:
The client performs HTTP POST request to the following end-point:
--> POST /api/npm.packet_export/1.3/exports
{
"config": {
"mode": "LIVE_STREAM"
...
}
}
<-- 201 Created
Response Headers:
Location: /api/npm.packet_export/1.3/exports/items/00000012
...
Response body:
{
"id": 00000012,
"config": { ... },
"status": { ... }
}
When the request is received on the server, an instance ID:00000012 is generated to serve it.
The Location response header contains the newly created instance URI. It’s recommended that you use it as a base for constructing additional requests.
Step 2: Wait for packet stream request to be ready
At this point, the initial POST is completed. Some basic validation has been performed and the packet stream request starts the pre processing phase.
The Export handle goes through a state machine with the following states:
Successful data retrieval:
INITIALIZING --> READY -> STREAMING -> DONE
Error cases:
INITIALIZING --> ERRORS
INITIALIZING --> READY -> ERRORS
INITIALIZING --> READY -> STREAMING -> ERRORS
The client should perform ‘status’ polls and wait for the ‘state’ to be “READY” or “ERRORS”.
Getting status for export handle:
GET /api/npm.packet_export/1.3/exports/items/00000012
<-- 200 OK
{
"id": 00000012,
"status": {
"state": "READY",
...
}
}
Step 3: Request packet stream
When an export handle gets into the “READY” state, packets can be requested.
To request the packets:
GET /api/npm.packet_export/1.3/packets/items/00000012
<-- 200 OK
<-- binary stream containing livestream protocol messages
The export state changes to “STREAMING” and, using a different connection, the client can query the status and the stats:
GET /api/npm.packet_export/1.3/exports/items/00000012
<-- 200 OK
{
"id": 00000012,
"status": {
"state": "STREAMING",
...
}
}
LIVE_STREAM protocol
The LIVE_STREAM protocol has been implemented to avoid timeout when the synchronous request stream does not receive data for a long time, usually that can happen when the packet source contains a lot of data and the filter cannnot be optimized (e.g. filter on tcp ports).
To retrieve packets with the LIVE_STREAM mode the client needs to parse the LIVE_STREAM message protocol used to stream the data.
A live stream message has the following format:
+--------+--------+----------------+-----------------------+
| Magic | Type | Data Length | Data
+--------+--------+----------------+-----------------------+
magic - 2 bytes. Always set to 0xF0CA
type - 2 bytes. It can be set to one of the following values:
- 0 (DATA) - data message
- 1 (EOF) - stream completed
- 2 (NODATA) - no data message, keepalive message
- 3 (ERROR) - error, connection will be closed
data length - 4 bytes. Length of the data following the header
data - N bytes (optional). Binary data containing the packets in the format specified in the configuration (output_format)
Step 4: Delete the export handle
Deleting the data request is an optional step. By default LIVE_STREAM export handles are deleted when the stream is closed.
In case the client wants to track errors or stats at the end of the export it is recommended to set ‘handle_timeout_msec’ to a couple of minutes, that will allow the handle to stay active after the stream is closed.
When the status has been retrieve it is recommended to perform a DELETE request. If a DELETE request is not sent, then the system will automatically purge the export handle objects within ‘handle_timeout_msec’.
DELETE /api/npm.packet_export/1.3/exports/items/00000012
<-- 204 Accepted
LEGACY Mode¶
Step 1: Create an export handle:
The client performs HTTP POST request to the following end-point:
--> POST /api/npm.packet_export/1.3/exports
{
"config": {
"mode": "LEGACY"
...
}
}
<-- 201 Created
Response Headers:
Location: /api/npm.packet_export/1.3/exports/items/00000012
...
Response body:
{
"id": 00000012,
"config": { ... },
"status": { ... }
}
When the request is received on the server, an instance ID:00000012 is generated to serve it.
The Location response header contains the newly created instance URI. It’s recommended that you use it as a base for constructing additional requests.
Step 2: Wait for packet stream request to be ready
At this point, the initial POST is completed. Some basic validation has been performed and the packet stream request starts the pre processing phase.
The Export handle goes through a state machine with the following states:
Successful data retrieval
UNINITIALIZED --> RUNNING -> DONE
Error cases:
UNINITIALIZED --> ERRORS
UNINITIALIZED --> RUNNING --> ERRORS
The client should perform ‘status’ polls and wait for the ‘state’ to be “RUNNING” or “ERRORS”.
Getting status for export handle:
GET /api/npm.packet_export/1.3/exports/items/00000012
<-- 200 OK
{
"id": 00000012,
"status": {
"state": "RUNNING",
...
}
}
Step 3: request packet stream
When an export handle gets into the “RUNNING” state, packets can be requested.
Getting packets for export handle:
GET /api/npm.packet_export/1.3/packets/items/00000012
<-- 200 OK
<-- binary stream containing packets in the format set in the configuration (output_format)
The export state stays “RUNNING” and, using a different connection, the client can query the status and the stats:
GET /api/npm.packet_export/1.3/exports/items/00000012
<-- 200 OK
{
"id": 00000012,
"status": {
"state": "RUNNING",
...
}
}
Step 4: Delete the export handle
Deleting the data request is an optional step. By default LEGACY export handles are deleted when the stream is closed.
In case the client wants to track errors or stats at the end of the export it is recommended to set ‘handle_timeout_msec’ to a couple of minutes, that will allow the handle to stay active after the stream is closed.
When the status has been retrieve it is recommended to perform a DELETE request. If a DELETE request is not sent, then the system will automatically purge the export handle objects within ‘handle_timeout_msec’.
DELETE /api/npm.packet_export/1.3/exports/items/00000012
<-- 204 Accepted
Prepare API (PREPARE)¶
Using the PREPARE modes the packets are pre processed by the microflow index and an estimate of the export is provided when the input is a indexed clip or a indexed capture job set in optimize for write.
Once the estimate is completed, it is provided in the stats and it’s possible to pick the desired synchronous or the asynchronous mode to retrieve the packets.
The Prepare API goes through the following steps:
- Step 1: POST (Create export request)
- Step 2: GET status and estimate (multiple calls until ready)
- Step 3: PUT export handle (pick sync or async mode)
- Step 4: DELETE export handle (optional)
Step 1: Create an export handle:
The client performs HTTP POST request to the following end-point:
--> POST /api/npm.packet_export/1.3/exports
{
"config": {
"mode": "PREPARE"
...
}
}
<-- 201 Created
Response Headers:
Location: /api/npm.packet_export/1.3/exports/items/00000012
...
Response body:
{
"id": 00000012,
"config": { ... },
"status": { ... }
}
When the request is received on the server, an instance ID:00000012 is generated to serve it.
The Location response header contains the newly created instance URI. It’s recommended that you use it as a base for constructing additional requests.
Step 2: Wait for packet stream request to ready
At this point, the initial POST is completed. Some basic validation has been performed and the packet stream request starts the pre processing/estimate phase.
The Export handle goes through a state machine with the following states:
Successful data retrieval:
INITIALIZING --> READY
Error cases:
INITIALIZING --> ERRORS
INITIALIZING --> READY -> ERRORS
The client should perform ‘status’ polls and wait for the ‘state’ to be “READY” or “ERRORS”.
Getting status for export handle:
GET /api/npm.packet_export/1.3/exports/items/00000012
<-- 200 OK
{
"id": 00000012,
"status": {
"state": "READY",
...
}
}
Step 3: pick mode
When an export handle gets into the “READY” state, the estimate is completed and the client can update the mode to retrieve the packets.
Besides the ‘mode’ the following properties can be updated in this step:
- snap_length
- output_format
- output_filename
- stop_rule
To update the mode:
PUT /api/npm.packet_export/1.3/packets/items/00000012
<-- 200 OK
<-- binary stream containing livestream protocol messages
The export state changes to the one of the following values based on the mode that has been picked:
- “INITIALIZING” (BUFFER_STREAM)
- “RUNNING” (STREAM_TO_FILE and LEGACY)
- “READY” (LIVE_STREAM)
and the client can follow the corresponding step based on the workflow associated to that mode.
Workflow to setup NetProfiler Export¶
This section describes how to setup NetProfiler or NetflowV9 Collector Export. The configuration involves calls to multiple api services going through the following steps:
- Step 1: Update Flow Export Settings
- Step 2: Set Traffic Selection
- Step 3 (optional): Set Traffic Selection for autodiscovered Virtual Interface Groups
- Step 4 (optional): Get Flow Export Status and Stats
Flow Export Settings¶
The first step consists of updating the global flow export settings using the ‘npm.profiler_export’ settings to specify the destination devices and some global parameters.
The client performs HTTP PUT request to the following end-point:
--> PUT /api/npm.profiler_export/1.0/settings
{
"enabled": true,
"voip_enabled": false,
"profilers": [
{
"address": "10.1527.12.94",
"sync": false
},
{
"address": "my_netprofiler",
"sync": true
}
],
"collectors": [
{
"address": "my_v9collector.domain",
"port": 1324
}
]
}
<-- 200 OK
...
Response body:
{
"enabled": true,
"voip_enabled": false,
"profilers": [
{
"address": "10.1527.12.94",
"sync": false,
"status":
{
"state": "OK"
}
},
{
"address": "my_netprofiler",
"sync": true,
"status":
{
"state": "OK"
}
}
],
"collectors": [
{
"address": "my_v9collector.domain",
"port": 1324,
"status":
{
"state": "OK"
}
}
]
}
The Flow Export Configuration object has the following properties: enabled, voip_enabled profilers, collectors.
enabled - boolean (required). Enable/disable the NetProfiler Export module.
voip_enabled - boolean. Enable/disable the export of voip metrics.
Profilers property (required)¶
This allows clients to define which NetProfiler(s) are used to export the flows.
The profilers property is an array of profiler object with the following sub-properties:
address - string (required). NetProfiler address, either IP or full qualified name.
sync - boolean (optional). Enable/disable the NetProfiler sync of applications, urls and ports definitions. If sync is not specified, then it is defaulted to false.
Note: only one NetProfiler in the list can have the sync property enabled
status - exporter_status object (read only). Information on a NetProfiler connection.
Collectors property (required)¶
This allows clients to define which NetFlowV9 Collector(s) are used to exports the flows.
The collectors property is an array of collector object with the following sub-properties:
address - string (required). NetFlowV9 Collector address, either IP or full qualified name..
port - integer (required). NetFlowV9 Collector UDP port.
status - exporter_status object (read only). Information on a NetFlowV9 Collector connection.
Example:
{
"enabled": true,
"voip_enabled": false,
"profilers": [
{
"address": "10.1527.12.94",
"sync": false
},
{
"address": "my_netprofiler",
"sync": true
}
],
"collectors": [
{
"address": "my_v9collector.domain",
"port": 1324
}
]
}
exporter_status object¶
This allows clients to retrieve the connection export state. The exporter_status is an object with the following sub-properties:
state - enum - values: OK, ERROR, DISABLED.
- OK: The connection with the NetProfiler/Collector has been successfully established.
- ERROR: The connection with the NetProfiler/Collector failed.
- DISABLED: The NetProfiler/Collector Export is disabled or unavailable.
error - object. Connection error object with the following sub-properties:
- error_id - string - Error internal identifier.
- error_text - string - Error text.
- error_metrics - object (optional). Array of key/value error metrics, used for detailed error responses.
Example:
{
"state": "OK"
}
{
"state": "ERROR",
"error": {
"error_id": "profiler-export-01",
"error_text": "Disconnected, connection closed by the peer"
}
}
Flow Traffic Selection¶
Once the global settings have been updated, the user can pick which traffic is exported enabling the flow export in the corresponding VIfG. In case he’s interested in only a subset of the traffic an optional filter can be specified (empty value for all traffic).
The client performs HTTP GET request to the following end-point to get the current list of VIfGs:
--> GET /api/npm.packet_capture/X.Y/vifgs
<-- 200 OK
...
Response body:
{
"count": 1,
"items": [
{
"state": {
...
},
"config": {
...
"enabled": true,
"flow_export": {
"filter": {
"type": "BPF",
"value": ""
},
"enabled": false
},
"name": "my_vifg"
},
"id": 1004
}
]
}
For each VIfG the user would like to use, the client updates the flow_export properties as needed, and he performs a HTTP PUT request to the following end-point:
--> PUT /api/npm.packet_capture/X.Y/vifgs/items/1004
{
"config": {
...
"enabled": true,
"flow_export": {
"filter": {
"type": "BPF",
"value": "tcp port 80"
},
"enabled": true
},
"name": "my_vifg"
}
}
<-- 200 OK
...
Response body:
{
"state": {
...
},
"config": {
...
"enabled": true,
"flow_export": {
"filter": {
"type": "BPF",
"value": "tcp port 80"
},
"enabled": true
},
"name": "my_vifg"
},
"id": 1004
}
Traffic Selection for autodiscovered Virtual Interface Groups¶
In case the user is using VIfGs autodiscovering, it’s possible to specify the default flow export properties for the new discovered VIfG.
The client performs HTTP PUT request to the following end-point:
--> PUT /api/npm.packet_capture/X.Y/vifgs/settings
{
"autodiscovery": {
"default": {
...
"flow_export": {
"filter": {
"type": "BPF",
"value": "tcp or icmp"
},
"enabled": true
},
"dedup": true
},
"enabled": true
},
...
}
<-- 200 OK
...
Response body:
{
"supported_grouping_types": [
...
],
"autodiscovery": {
"default": {
...
"flow_export": {
"filter": {
"type": "BPF",
"value": "tcp or icmp"
},
"enabled": true
},
"dedup": true
},
"enabled": true
},
...
}
Flow Export Status and Stats¶
Now that the setup is completed, it is possible to start monitoring the state of the connections and the flow export statistics. The statistics includes exported flows, bytes and packets for the last minute and last week. If the limit of licensed flows is reached the statistics provide the number of rejected flows.
Note: the connection state is available for NetProfiler and NetflowV9 Collectors, however the V9 collectors uses a UDP connection and a successful bind in the AR11 side does not necessary mean the collector is receiving the data.
To get the connection state the client performs HTTP GET request to the following end-point:
--> GET /api/npm.profiler_export/1.0/settings
<-- 200 OK
...
Response body:
{
"voip_enabled": false,
"enabled": true,
"collectors": [],
"profilers": [
{
"status": {
"state": "OK"
},
"sync": false,
"address": "my_profiler.com"
}
]
}
To get the export statistics the client performs HTTP GET request to the following end-point:
--> GET /api/npm.profiler_export/1.0/stats
<-- 200 OK
...
Response body:
{
"collector_export": {
...
},
"profiler_export": {
"last_minute": {
"exported": {
"ip_bytes": 15475194346,
"packets": 59849423,
"flows": 150054
},
"rejected": {
"ip_bytes": 0,
"packets": 0,
"flows": 0
}
},
"last_week": {
"exported": {
"ip_bytes": 155989959007680,
"packets": 603282183840,
"flows": 4936821120,
"average_flows": 489764,
"peak_flows": 975673
},
"rejected": {
"ip_bytes": 0,
"packets": 0,
"flows": 0,
"average_flows": 0,
"peak_flows": 0
}
}
}
}
