This chapter provides a summary of the application layer gateway (ALG) security rules and the procedures for ensuring security compliance. It includes these sections:

As a supplement to this guide, consult Securing SteelHeads in the SteelHead Deployment Guide 9.1 or later. This guide provides additional guidance regarding security best practices for SteelHead deployments.

This section includes this rule:

Rule Title: RiOS must be configured to ensure inbound and outbound traffic is forwarded to be inspected by the firewall and IDPS in compliance with remote access security policies.

STIG ID: RICX-AG-000037

Rule ID: SV-77303r1_rule Severity: CAT II

Vuln ID: V-62813 Class: Unclass

Automated monitoring of remote access traffic allows organizations to detect cyber attacks and also ensure ongoing compliance with remote access policies by inspecting connection activities of remote access capabilities.

Remote access methods include both unencrypted and encrypted traffic. Inbound traffic must be inspected prior to being allowed on the enclave's trusted networks. Outbound traffic inspection must occur prior to being forwarded to destinations outside of the enclave.

Optimally, the SteelHead must be architecturally placed at the perimeter of the network in front of the perimeter router. Thus, traffic is directed for firewall and IDPS inspection for inbound and outbound traffic in compliance with DoD policy. Additionally, from an operational perspective, this architecture avoids the need to open many ports and services in the firewall to accommodate TCP options 76 and 78 and ports 7800, 7810, and 7870. Some other configurations might involve even more ports and services.

Inspect the architectural placement of the device. Verify the traffic from the device is directed to the firewall and intrusion detection system (IDS) or intrusion prevention system (IPS) for inspection.

If RiOS is not configured to ensure inbound and outbound traffic is forwarded to be inspected by the firewall and IDPS in compliance with remote access security policies, this is a security vulnerability finding.

Architecturally place the SteelHead device to avoid the need to open TCP ports in the firewall. The best practice for the SteelHead appliance is to install it at the perimeter of the network in front of the perimeter router configure it to direct traffic to the router. Thus, inbound and outbound traffic is forwarded to be inspected by the firewall and IDPS in compliance with remote access security policies.

This section includes this rule:

Rule Title: If TLS optimization is used, signed SMB and encrypted MAPI must ensure the integrity and confidentiality of data transmitted over the WAN.

STIG ID: RICX-AG-000032

Rule ID: SV-77277r1_rule Severity: CAT II

Vuln ID: V-62787 Class: Unclass

Protecting the end-to-end security of transport layer security (TLS) is required to ensure the integrity and confidentiality of the data in transit.

Signed SMB and encrypted MAPI traffic use techniques to protect against unauthorized man-in-the-middle devices from making modifications to their exchanged data. Additionally, encrypted MAPI traffic and encrypted SMB3 traffic ensure data confidentiality by transmitting data with protection across the network.

To securely optimize this traffic, a properly configured client and server-side SteelHead appliance with WAN optimization must:

To query the Windows domain controller for the necessary cryptographic information to optimize this traffic, the server-side SteelHead appliance must join a Windows domain. The SteelHead appliance can require other configuration settings, both on the SteelHead appliance, and in the Windows domain. This cryptographic information is only useful for the lifetime of an individual connection or session. The information is obtained at the beginning of a connection, and transferred to the client-side SteelHead appliance as needed, using the secure-inner channel feature. You must configure the secure-inner channel to ensure maximum security.

Only the server-side SteelHead appliance is required to join the domain, and it does so using a machine account in the same way that a Windows device joins the domain using a machine account. The SteelHead appliance joins the domain to obtain a client-user session key (CUSK) or server-user session key (SUSK), which allows the SteelHead appliance to sign and/or decrypt MAPI on behalf of the Windows user that is establishing the relevant session.

The server-side SteelHead appliance must join a domain that is either:

Verify that signed SMB and encrypted MAPI is configured to ensure the integrity and confidentiality of data transmitted over the WAN.

Configure signed SMB and encrypted MAPI optimization services to ensure the integrity and confidentiality of data transmitted over the WAN.

This section includes these rules:

Rule Title: If TLS WAN optimization is used, SSL optimization must protect private keys ensuring that they stay in the data center by ensuring end-to-end security.

STIG ID: RICX-AG-000038

Rule ID: SV-77305r1_rule Severity: CAT II

Vuln ID: V-62815 Class: Unclass

Protecting the end-to-end security of TLS is required to ensure integrity and confidentiality of the data in transit.

RiOS TLS optimization accelerates data transfers that are encrypted using TLS for SteelHead appliances that are deployed locally to both the client-side and server-side of the network. All of the same optimized connections that are applied to normal nonencrypted TCP traffic can also apply to encrypted TLS traffic. SteelHead appliances accomplish this without compromising end-to-end security and the established trust model. Private keys remain in the data center and are not exposed in remote locations where they might be compromised.

The RiOS TLS optimization solution starts with SteelHead appliances that have a configured trust relationship, enabling them to exchange information securely over their own dedicated TLS connection. Each client uses unchanged server addresses and each server uses unchanged client addresses; no application changes or explicit proxy configuration is required. RiOS uses a unique technique to split the TLS handshake. The handshake is the sequence of message exchanges at the start of a TLS connection. In an ordinary TLS handshake, the client and server first establish identity using public-key cryptography, and then negotiate a symmetric session key to use for data transfer. When using RiOS TLS acceleration, the initial TLS message exchanges take place between the client application (for example, a Web browser) and the server-side SteelHead appliance.

SteelHead WAN optimization platform works to ensure that TLS acceleration delivers the following:

Verify that TLS optimization services are configured to ensure end-to-end security and protect private keys from unauthorized access.

Configure TLS optimization services to provide end-to-end security and protection for private keys.

Rule Title: RiOS must protect the authenticity of communication sessions by configuring securing pairing trust relationships for SSL and secure protocols.

STIG ID: RICX-AG-000123

Rule ID: SV-77323r1_rule Severity: CAT II

Vuln ID: V-62833 Class: Unclass

Authenticity protection provides protection against man-in-the-middle attacks, session hijacking, and the insertion of false information into sessions.

This authenticity protection control focuses on communications protection for the application session rather than for the network packet and establishes grounds for confidence at both ends of communication sessions in ongoing identities of other parties and in the validity of information transmitted. Depending on the required degree of confidentiality and integrity, web services and service-oriented architecture (SOA) will require the use of mutual authentication (that is, two-way or bidirectional).

Verify RiOS is configured to support TLS v1.1 as a minimum and preferably TLS v1.2.

Configure RiOS to support TLS v1.1 as a minimum and preferably TLS v1.2.

Rule Title: RiOS must validate certificates used for TLS functions by performing RFC 5280-compliant certification path validation.

STIG ID: RICX-AG-000098

Rule ID: SV-77321r1_rule Severity: CAT II

Vuln ID: V-62831 Class: Unclass

A certificate's certification path is the path from the end-entity certificate to a trusted-root certificate authority (CA). Certification path validation is necessary for a relying party to make an informed decision regarding acceptance of an end entity certificate.

Certification path validation includes checks such as certificate issuer trust, time validity, and revocation status for each certificate in the certification path. Revocation status information for CA and subject certificates in a certification path is commonly provided through certificate revocation lists (CRLs) or online certificate status protocol (OCSP) responses.

Verify that RiOS is configured to validate certificates used for TLS functions by performing certificate path validation.

Configure RiOS to validate certificates used for TLS functions by performing certificate path validation.

This section includes these rules:

Rule Title: If TLS optimization is enabled RiOS must use encryption services that implement NIST FIPS-validated cryptography to protect the confidentiality of TLS.

STIG ID: RICX-AG-000039

Rule ID: SV-77307r1_rule Severity: CAT II

Vuln ID: V-62817 Class: Unclass

Without confidentiality protection mechanisms, unauthorized individuals might gain access to sensitive information through a remote access session.

Remote access is access to DoD-nonpublic information systems by an authorized user (or an information system) communicating through an external, nonorganization-controlled network. Remote access methods include broadband and wireless connections. Remote access methods include, for example, proxied remote encrypted traffic (for example, transport layer security (TLS) gateways, web content filters, and web email proxies).

Encryption provides a means to secure the remote connection to prevent unauthorized access to the data traversing the remote access connection, thereby providing a degree of confidentiality. The encryption strength of the mechanism is selected based on the security categorization of the information.

This requirement applies to ALGs providing remote-access proxy services as part of its intermediary services (for example, OWA or TLS gateway).

Verify RiOS is configured to support TLS v1.1 as a minimum and preferably TLS v1.2.

For detailed information, see Verifying the TLS Version.

Configure RiOS to support TLS v1.1 as a minimum and preferably TLS v1.2.

For detailed information, see Configuring TLS Version for Peer, Client, and Server Ciphers.

Rule Title: If TLS optimization is used, RiOS, which stores secret or private keys, must use FIPS-approved key management technology and processes in the production and control of private/secret cryptographic keys.

STIG ID: RICX-AG-000040

Rule ID: SV-77309r1_rule Severity: CAT II

Vuln ID: V-62819 Class: Unclass

Private key data is used to prove that the entity presenting a public key certificate is the certificate's rightful owner. Compromise of private key data allows an adversary to impersonate the key holder.

Private key data associated with software certificates, including those issued to an ALG, is required to be generated and protected in at least a FIPS 140-2 Level 1 validated cryptographic module.

The RiOS secure vault contains sensitive information from your SteelHead appliance configuration, including SSL private keys and the data store encryption key. These configuration settings are encrypted on the disk using AES 256-bit encryption.

The secure vault always runs in FIPS mode.

Rule Title: RiOS must be configured to comply with the required TLS settings in NIST SP 800-52.

STIG ID: RICX-AG-000041

Rule ID: SV-77311r1_rule Severity: CAT II

Vuln ID: V-62821 Class: Unclass

Class: Unclass

SP 800-52 provides guidance on using the most secure version and configuration of the TLS/SSL protocol. Using older unauthorized versions or incorrectly configuring protocol negotiation makes the gateway vulnerable to known and unknown attacks that exploit vulnerabilities in this protocol.

This requirement applies to TLS gateways (also known as the SSL gateways) and is not applicable to virtual private network (VPN) devices. Application protocols such as HTTPS and DNSSEC use TLS as the underlying security protocol and are in scope for this requirement. The NIS SP 800-52 provides additional guidance.

NIST SP 800-52 sets TLS v1.1 as a minimum version, thus all versions of SSL are not allowed (including for client negotiation) either on DoD-only or on public facing servers.

Verify that RiOS is configured to support TLS v1.1 as a minimum and preferably TLS v1.2.

For detailed information, see Verifying the TLS Version Support.

Configure RiOS to support TLS v1.1 as a minimum and preferably TLS v1.2.

For detailed information, see Configuring TLS Version for Peer, Client, and Server Ciphers.

Rule Title: RiOS must use NIST FIPS-validated cryptography to protect the integrity of remote access sessions.

STIG ID: RICX-AG-000042

Rule ID: SV-77313r1_rule Severity: CAT II

Vuln ID: V-62823 Class: Unclass

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Remote access is access to DoD-nonpublic information systems by an authorized user (or an information system) communicating through an external, nonorganization-controlled network. Remote access methods include broadband and wireless connections. Remote access methods include, for example, proxied remote encrypted traffic (for example, TLS gateways, web content filters, and webmail proxies).

Cryptographic mechanisms used for protecting the integrity of information include, for example, signed hash functions using asymmetric cryptography enabling distribution of the public key to verify the hash information while maintaining the confidentiality of the secret key used to generate the hash.

This requirement applies to ALGs providing remote access proxy services as part of its intermediary services (for example, OWA or TLS gateway).

Verify that RiOS is configured to support FIPS-approved key management technology and processes in the production and control of private and secret cryptographic keys.

Configure RiOS to support FIPS-approved key management technology and processes in the production and control of private and secret cryptographic keys.

This section includes these rules:

Rule Title: RiOS must not have unrelated or unnecessary services enabled on the host.

STIG ID: RICX-AG-000086

Rule ID: SV-77315r1_rule Severity: CAT II

Vuln ID: V-62825 Class: Unclass

Typically, the SteelHead is installed in the architecture at the perimeter of the network. Installation of unnecessary functions and services on the same host increases the security risk by implementing these functions before the network inspection occurs and opens excessive ports on the firewall for these functions and services to operate. Loading functions that are outside the scope and unrelated to the WAN optimization function is unauthorized and might create an attack vector. Related services include content filtering, traffic analysis, decryption, caching, and traffic inspection tools (for example, firewall, IDS), unrelated services include email, DNS, and web server.

When the solution is deployed using a SteelHead appliance consisting of the RiOS software installed on the SteelHead, administrators are not allowed to install any software that is not part of a Riverbed software upgrade. RiOS enforces this restriction by performing a validity check when an upgrade is attempted.

However, the RiOS software is available in a virtual appliance that can be installed on an organization-provided host. This type of implementation adds a security risk because more ports might be opened in the firewall if placed in the recommended logical position in the architecture (that is, after the router and before the firewall and IDS). The traffic would then be routed for inspection after traversing the WAN optimizer.

If RiOS, as a virtual appliance, is installed on the SteelHead appliance, this is a security vulnerability finding.

Inspect the services and applications that are installed on the host with the RiOS application suite.

Ask the site representative if a security review using the applicable STIG has been performed on the operating system and applications that are cohosted.

If unrelated or unnecessary services are installed on the same host as the RiOS, this is a security vulnerability finding.

If a security review using the applicable STIG has not been performed on the operating system and applications cohosted on RiOS, this is a security vulnerability finding.

Disable or uninstall unrelated or unnecessary services from the host.

Rule Title: RiOS must not have unnecessary services and functions enabled.

STIG ID: RICX-AG-000087

Rule ID: SV-77317r1_rule Severity: CAT II

Vuln ID: V-62827 Class: Unclass

Unrelated or unneeded proxy services increase the attack vector and add excessive complexity to the securing of RiOS 8.x.x. Multiple application proxies can be installed on many ALGs. However, proxy types must be limited to related functions. At a minimum, the web and email gateway represent different security domains/trust levels. Organizations should also consider separation of gateways that service the DMZ and the trusted network.

Verify that RiOS is configured to disable unrelated or unneeded application proxy services.

Obtain documentation for which applications are approved and disapproved for optimization by the organization.

Check to see if services other than the authorized services are enabled for optimization.

Obtain documentation for which applications are approved and disapproved for optimization by the organization.

Rule Title: RiOS must be configured to prohibit or restrict the use of functions, ports, protocols, and/or services, as defined in the PPSM CAL and vulnerability assessments.

STIG ID: RICX-AG-000088

Rule ID: SV-77319r1_rule Severity: CAT II

Vuln ID: V-62829 Class: Unclass

In order to prevent unauthorized connection of devices, unauthorized transfer of information, or unauthorized tunneling (that is, embedding of data types within data types); organizations must disable or restrict unused or unnecessary physical and logical ports and protocols on information systems.

ALGs are capable of providing a wide variety of functions and services. Some of the functions and services provided by default might not be necessary to support essential organizational operations. DoD continually assesses the ports, protocols, and services that can be used for network communications. Some ports, protocols or services have known exploits or security weaknesses. Network traffic using these ports, protocols, and services must be prohibited or restricted in accordance with DoD policy. RiOS is a key network element for preventing these noncompliant ports, protocols, and services from causing harm to DoD information systems.

The network ALG must be configured to prevent or restrict the use of prohibited ports, protocols, and services throughout the network by filtering the network traffic and disallowing or redirecting traffic as necessary. Default and updated policy filters from the vendors will disallow older version of protocols and applications and will address most known nonsecure ports, protocols, and/or services. However, sources for further policy filters are the Information Assurance Vulnerability Management (IAVM) and the PPSM requirements.

Verify that RiOS is configured to disable unrelated or unneeded application proxy services.

Obtain documentation for which applications are approved and disapproved for optimization by the organization.

Check to see if services other than the authorized services are enabled for optimization.

Obtain documentation for which applications are approved and disapproved for optimization by the organization.