Signed SMB and Encrypted MAPI Optimization
  
Signed SMB and Encrypted MAPI Optimization
This chapter discusses high-level techniques and guidance for configuring signed Service Message Block (SMB) and encrypted MAPI traffic to ensure data integrity. This chapter includes the following sections:
•  Overview of SteelHeads Optimizing with Secure Windows Traffic
•  Windows Security Concepts
•  Domain Relationships
•  Choosing an Authentication Mode for the Server-Side SteelHead
•  Overview of Configuring SMB Signing and Encrypted MAPI
•  SMB3 Optimization with Windows 8, Windows 2012 Server, and Later Operating Systems
•  Joining a SteelHead to a Domain
•  Kerberos
•  Configuring the Server-Side SteelHead for Active Directory Integrated (Windows 2003/2008)
•  Best Practices for the SteelHead in a Secure Windows Deployment
•  Domain Authentication Scaling
•  Domain Health Check and Domain Authentication Automatic Configuration
•  Single Domain Example Configuration
Overview of SteelHeads Optimizing with Secure Windows Traffic
Signed SMB and encrypted MAPI traffic use techniques to protect against unauthorized man-in-the-middle devices from making modifications to their exchanged data. Additionally, encrypted MAPI traffic and encrypted SMB3 traffic ensure data confidentiality by transmitting data with protection across the network. To securely optimize this traffic, a properly configured client-side and server-side SteelHead:
•  decrypts and removes signatures on received LAN side data from the client or server.
•  performs bandwidth and application layer optimization.
•  uses the secure inner channel feature to maintain data integrity and confidentiality of data transmitted over the WAN.
•  converts the received optimized data back to its native form.
•  encrypts and applies signatures for LAN side transmission of data to the client or server.
To query the Windows domain controller for the necessary cryptographic information to optimize this traffic, the server-side SteelHead must join a Windows domain. The SteelHead can require other configurations, both on the SteelHead and in the Windows domain. This cryptographic information is only useful for the lifetime of an individual connection or session. The information is obtained at the beginning of a connection and transferred to the client-side SteelHead as needed, using the secure inner channel feature. You must configure the secure inner channel to ensure maximum security.
Only the server-side SteelHead is required to join the domain, and it does so using a machine account in the same way that a Windows device joins the domain using a machine account. The SteelHead joins the domain this way to obtain a client user session key (CUSK) or server user session key (SUSK), which allows the SteelHead to sign and/or decrypt MAPI on behalf of the Windows user that is establishing the relevant session.
The server-side SteelHead must join a domain that is either:
•  the user domain. The domain must have a trust with the domains that include the application servers (file server, Exchange server, and so on) you want to optimize.
•  a domain having a bidirectional trust with the user domain, or a one-way trust such that the user domain is the trusted domain.
The domain that the server-side SteelHead is joined to might include some or all of the Windows application servers (file server, Exchange server) for SteelHead optimization. The server-side SteelHead can only be a member of one domain, but where suitable domain trusts exist, it can also optimize connections for Windows application servers that reside in other domains. For more information, see Domain Relationships.
Production deployments can have multiple combinations of client and server Windows operating system versions, and they can include different configuration settings for signed SMB and encrypted MAPI. Therefore it is possible that the security authentication between clients and servers can use NT LAN Manager (NTLM) or Kerberos, or a combination of the two. This chapter includes more information about authentication types and SteelHead configuration requirements.
Windows Security Concepts
The Windows security framework is based on a formal structure known as a domain. Inside the domain is a logical group of host resources (primarily clients and servers, but also printers and other peripherals) that share a central directory database. The database resides on one or more servers known as domain controllers and contains user accounts and security information for all the resources in the domain. Other domains can coexist alongside and are joined together through a Trust Relationship to allow resources to securely communicate between each other even though they are in different domains with their own domain controllers.
When users and clients are accessing server resources (like file servers and Exchange servers), their credentials are validated against the database on the domain controller. This validation ensures that the client and the user have the correct security privileges to be able to access resources that provide signed SMB traffic or encrypted MAPI traffic.
Several techniques and protocols are used to validate the credentials of the user and client. These techniques vary according to the Windows operating system version and application configuration (for example, Microsoft Outlook and Microsoft Exchange) on both the client and server. Example protocols include Kerberos, NTLMv1, and NTLMv2. The earliest suitable RiOS version required to perform optimization in a Windows security environment is based on the authentication protocol used. Because Riverbed has added more capabilities in newer versions of RiOS, Riverbed recommends that you use the most recent version that meets your needs.
Domain Relationships
Some organizations might have more than one Windows domain in use in their environment. A SteelHead, like a Windows server, can only join a single domain. Therefore, the choice of which domain the SteelHead should join depends on the domain where the file or mail servers are located, the type of trust relationship between the SteelHeads potential domain, the file or mail servers domain, and the domain containing a user's credentials.
Figure: Single Domain Structure shows an example of a simple, single domain structure. All resources (such as clients and servers) that have joined the same domain are subject to the domain permissions and authentications, and they can access the other available resources. Only a single domain controller is required in this case, although you can have more than one domain controller for resilience. The SteelHead must join the one available domain as a precursor to secure Windows protocol optimization.
Figure: Single Domain Structure
Figure: Two Domains and One-Way Trust Structure shows an example where clients are in one domain and servers are in a second domain. There is a trust relationship between the two domains that allow the clients and servers to access each other. The trust relationship is a one-way trust. The client domain is described as the Trusted Domain and the server domain is described as the Trusting Domain. The arrow that indicates the direction of trust is from the trusting domain to the trusted domain. Because of the one-way trust, only the resources in the client domain are allowed to access the resources in the server domain and not the other way around. Each of the two domains has its own domain controller, each with their own database for the resources in its domain.
Figure: Two Domains and One-Way Trust Structure
Figure: Multiple Domains with Two-way Trust Structure shows an example of a configuration where there are multiple domains. In a two-way trust, each of the resources in the child domains can access the other domains through the top-level (parent) domain as long as the correct permissions have been set up within the child domains.
Figure: Multiple Domains with Two-way Trust Structure
There can be a mixture of domains with domain controllers with different Windows operating system versions and a mixture of trust types. Native mode is when all domain controllers run the same version of operating system, and mixed mode is when there are different versions of the operating system.
Choosing an Authentication Mode for the Server-Side SteelHead
There are many combinations and settings for the Windows client operating system, Windows domain level, Windows authentication method, and RiOS version. You can choose from several different configuration options on the server-side SteelHead. In some cases, you can select multiple options to provide the best possible flexibility for your deployment. The options are as follows:
•  Transparent mode - Authenticate using NTLM pass-through authentication to the Windows domain controller. This option is considered the easiest to deploy with the minimum of administrative overhead. To use this mode with RiOS 7.0 and later, you can join as Active Directory integrated (Windows 2008 and later) or Active Directory integrated (Windows 2003). Riverbed recommends that you use Active Directory integrated (Windows 2008 and later) because this mode enables the server-side SteelHead to securely communicate with domain controllers in other trusted domain in which there may be additional servers requiring optimized connections. For more information about join types, see Joining a SteelHead to a Domain.
•  End-to-end Kerberos - When you need the authentication between Windows clients and servers to be Kerberos from end to end, the server-side SteelHead can make use of a replication user account in the Windows domain. While this is the only choice available when the authentication is required to be end-to-end Kerberos, you can combine it with transparent mode to provide the most flexibility in the event that some clients still negotiate NTLM authentication.
•  Delegation mode - This is a legacy configuration option within RiOS on the SteelHead. The feature is retained for reasons of backward compatibility. Riverbed does not recommend that you use the configuration for deployments using versions later than RiOS 7.0. For information, see “Legacy Delegate User Configuration” in Appendix A of the SteelHead Deployment Guide - Protocols, July 2015.
Overview of Configuring SMB Signing and Encrypted MAPI
A deployment that successfully optimizes SMB signing and encrypted MAPI traffic requires gathering information about the Windows environment, configuring the SteelHeads with the appropriate RiOS version and configuration; when using delegation mode or end-to-end Kerberos, this same deployment potentially requires more configuration in the Windows domain.
The following is a general overview on how to configure a typical SMB signing and MAPI encrypted deployment (for complete details, see the SteelHead Management Console User’s Guide):
1. Determine if the domain containing users to optimize is different from the server domain.
If the user domain is different from the server domain, determine the trust relationship between the domains.
2. Gather version information about the Windows environment:
•  Client operating system versions.
•  Domain functional level of all server domains involved in the configuration (for example, Native 2003, mixed mode, Native 2008-R2, and Native 2012).
Typical environments allow the Windows client and server to negotiate either NTLM authentication or Kerberos authentication for SMB or MAPI traffic. In these environments, the SteelHead can force negotiation to NTLM authentication to provide latency and bandwidth optimization. Certain environments do not negotiate NTLM or Kerberos—instead they require Kerberos authentication. The Kerberos authentication is set either through settings on the client or through domain policy settings. RiOS 7.0 or later supports this type of end-to-end Kerberos authentication. If your environment is Kerberos only, see Kerberos.
3. For the best compatibility with the current versions of Microsoft Windows Active Directory environments, Riverbed recommends using a minimum of RiOS 7.0 on both the client-side and server-side SteelHeads. Additionally, Riverbed has incorporated new features designed to simplify the configuration and management of SteelHeads that are integrated with Active Directory. These features are available in RiOS 8.6.x and 9.x, therefore consider upgrading to a suitable recent version.
If you must use a version prior to RiOS 7.0, see Appendix A of the SteelHead Deployment Guide - Protocols, July 2015.
4. Upgrade the RiOS version on the client-side and server-side SteelHeads, if necessary.
5. If not already implemented, configure the secure inner channel feature on both the client-side and server-side SteelHead.
6. Join the server-side SteelHead to the domain. The domain can be the user domain or a domain that has a trust with the user domain: for example, the server domain. For details, see Joining a SteelHead to a Domain.
In environments where the server domain only has a one-way trust to the user domain, you need a special configuration when joining the server-side SteelHead to a domain. For details, see One-Way Trust Configuration.
7. If delegation mode is required, set up the delegation accounts in the server domains and configure Delegation mode on the server-side SteelHead. For information, see “Configuring Constrained Delegation for Delegation Mode” in Appendix A of the SteelHead Deployment Guide - Protocols, July 2015.
8. Verify successful optimization of the SMB signed or encrypted MAPI traffic.
SMB3 Optimization with Windows 8, Windows 2012 Server, and Later Operating Systems
Microsoft created an update to the SMB protocol with the release of Microsoft Windows 8 and Server 2012 operating systems. This new version is officially known as SMB3, but in some early documentation it might be referred to as SMB 2.2.
The following new features included in SMB3 are relevant to your SteelHead deployment:
•  Encryption between client and server
•  New SMB signing algorithm
•  Secure dialect negotiation
You can use all three features only if the client and server are Windows 8 and Windows 2012 servers respectively. If the client is Windows 7 or earlier, or the server is Server 2008-R2 or earlier, the client and server do not use the new features for reasons of backward compatibility—with the exception of secure dialect negotiation.
Secure dialect negotiation is enabled and used by default in Windows 8 and Server 2012 connections. You can optionally use secure dialect negotiation with SMB2 when you are setting up a connection to a server running Server 2008-R2, but this is not normally enabled for compatibility reasons.
With the release of Windows 8.1 and Windows Server 2012 R2, Microsoft has added support for SMB 3.02.
For SMB3 and SMB 3.02 specifications, go to http://msdn.microsoft.com/en-us/library/cc246482.aspx.
RiOS versions prior to 8.5 provide some limited optimization of SMB3, but in some cases there is no optimization at all. For more information about SMB3 and releases prior to RiOS 8.5, go to https://supportkb.riverbed.com/support/index?page=content&id=S16547.
In RiOS 8.5 or later, you can provide full optimization for signed SMB3 traffic between Windows 8 clients and Server 2012 servers, as well as other client-server combinations that result in the use of signed SMB3 traffic.
Additionally, in RiOS 8.5 or later, you can optimize SMB3 traffic that is encrypted at the file share or file server level.
To optimize SMB3 traffic, you must run RiOS 8.5 or later on both the client-side and server-side SteelHeads and join the server-side SteelHead to the domain. Additionally, you must enable SMB3 optimization on both the client-side and server-side SteelHeads.
Note: Even if you do not configure Windows 8 clients and Server 2012 servers to require signing and encryption, the SMB3 protocol continues to use secure dialect negotiation during the connection setup. Therefore, the configuration settings outlined in this section are mandatory to ensure SMB3 traffic is optimized.
If you correctly configure the SteelHeads to optimize signed SMB3 traffic, there is no additional configuration required to support the optimization of signed SMB3 traffic that is encrypted at the share or server level.
For more information about:
•  the configuration settings required for SMB3, see the SteelHead Management Console User’s Guide.
•  optimizing SMB traffic between combinations Windows 8.1 and Server 2012R2, go to https://supportkb.riverbed.com/support/index?page=content&id=S23689.
SMB 3.1.1 Optimization
Further updates to the SMB protocol were included with Microsoft's release of Windows 10 and Windows Server 2016 operating systems. The default SMB protocol used between combinations of client and server running these operating system versions is SMB 3.1.1. Similar to SMB3 and SMB 3.02, encryption for SMB 3.1.1 can be enabled at the share or fileserver level. The default cipher is AES-128-GCM, but SMB 3.1.1 can negotiate to AES-128-CCM to support older configurations. The ability to negotiate the cipher (unlike SMB 3.02) means that Microsoft can chose to include additional crypto algorithms in the future.
Along with signing, encryption, and secure dialect negotiation included with previous SMB3 dialects, an additional mechanism was introduced with SMB 3.1.1 called pre-authentication integrity. This is a form of integrity check that occurs between two hosts when establishing an SMB 3.1.1 session. This check is a mandatory feature with SMB 3.1.1 and is designed to protect against any tampering of the client-server session setup by using SHA-512 cryptographic hashing. The hash is used as one of the inputs to the key derivation function from which the session's secret keys are obtained.
For more information on pre-authentication integrity, go to http://blogs.msdn.com/b/openspecification/archive/2015/08/11/smb-3-1-1-pre-authentication-integrity-in-windows-10.aspx. This page also discusses the use of WAN accelerators in the Proxy and traffic accelerator section.
To provide full Layer-7 latency optimization for SMB 3.1.1 connections, both the client-side and server-side SteelHeads must be running RiOS 9.2 or later and must have SMB2 and SMB3 latency optimization settings enabled. The server-side SteelHead must also be configured with the following items:
•  SMB2 and SMB3 signing enabled.
•  Joined to a suitable Windows domain, preferably using Active Directory integrated (Windows 2008 and later).
•  NTLM Transparent Mode or Kerberos Authentication Support.
–  You can use NTLM Delegation Mode, but it is discouraged due to higher administration overheads compared with Transparent Mode or Kerberos.
•  Suitable replication or delegation service accounts to match the authentication modes in use.
With the exception of the service accounts, you can complete all of the above settings on the server-side SteelHead by using the Configure Domain Auth Easy Config.
When you have the client-side and server-side SteelHeads running RiOS 9.2 or later, and have problems with the preauthentication integrity hash or negotiated encryption algorithm, then the connection between the client and server is blacklisted and subsequent sessions between the two hosts are passed through with no optimization.
Note: You must use RiOS 9.2 or later for complete Layer-7 latency optimization of SMB 3.1.1. Previous versions of RiOS might provide some degree of bandwidth optimization. For more guidance on the use of earlier RiOS versions and optimizing SMB 3.1.1 traffic between combinations of Windows 10 and Server 2016, go to https://supportkb.riverbed.com/support/index?page=content&id=S25594.
Joining a SteelHead to a Domain
SteelHeads support joining domains that are operating in native or mixed modes for the following domain functional levels:
•  Windows 2000
•  Windows 2003 R2
•  Windows 2008
•  Windows 2008 R2
•  Windows 2012
For more information about maximum latency between a SteelHead and a domain controller, go to http://supportkb.riverbed.com/support/index?page=content&id=S22110.
Before you join a SteelHead to a domain, perform the following prerequisite verifications:
•  The primary interface on a SteelHead must have IP reachability to any domain controllers, DNS servers, and other resources that a Windows server or workstation typically has in the domain that the SteelHead is joining. By default, the SteelHead uses the primary interface as the source interface for the join operation. Other interfaces, such as the auxiliary and in-path management interfaces, might work if enabled, but they are not supported for authentication traffic in general.
•  The source interface for the SteelHead must not have its communication to the Windows domain controller blocked by a firewall or other security mechanism—for example, the RiOS Management ACL mechanism—either during the join or while optimizing connections. For information about the ports and protocols required for operation, go to http://support.microsoft.com/?id=179442.
•  The SteelHead must be configured to use the same DNS server that other members of the domain use. Active Directory uses the DNS protocol to discover domain resources. When a SteelHead is joining a domain, it needs to make DNS based requests to a DNS server that understands these Active Directory-specific operations.
•  The source interface for the join must have an A record in the DNS domain associating the SteelHead hostname to its IP address.
•  The SteelHead must have its clock synchronized to within a few seconds of the domain controller it communicates with. The best practice is to have the SteelHead use the NTP protocol to synchronize with the NTP server that is used for clock synchronization within the Windows domain.
Note: A clock that is not synchronized on the SteelHead is the most common cause of domain join errors.
•  Credentials for a domain account with sufficient privileges to join a machine to the domain must be temporarily available. Many organizations have specific accounts that are used only for the purpose of joining machines to the domain and have no other privileges. Enter the credentials on the SteelHead to perform the join operation. The credentials are not stored on the SteelHead. If you have multiple SteelHeads in the data center (for example, in a high-availability configuration), then you can add the same replication user account to the configuration of all the server-side SteelHeads. You can also create an account solely for the purpose of joining the domain, and can be used once. For more details, see https://supportkb.riverbed.com/support/index?page=content&id=S18097.
If the server-side SteelHead is running a version of RiOS between 6.1 and 6.5, it can join the domain only to appear as a Workstation. In RiOS 7.0 or later, the SteelHead can join the domain and appear as one of three different roles: Workstation, Active Directory integrated (Windows 2003), or Active Directory integrated (Windows 2008 and later).
Regardless of the domain functional level, as long as there is at least one domain controller running Windows 2008 or later, Riverbed recommends joining with the role of Active Directory integrated (Windows 2008 and later). As well as enabling the server-side SteelHead to communicate with domain controllers running Windows 2008 and later, this join type provides the broadest coverage of NTLM dialects used by clients and the ability for the SteelHead to operate more easily across multi-domain environments.
For more details, see Configuring the Server-Side SteelHead for Active Directory Integrated (Windows 2003/2008).
During the join operation, the SteelHead communicates with domain controllers and other entities related to the Windows domain, including the DNS server. The exact operations performed depend on the domain functional level, but are similar to the operations made when a new Windows server joins the domain.
The following process occurs during the domain join:
1. The SteelHead performs a lookup for Domain Controllers. It uses the DNS server that is specified in the Host Settings page of the SteelHead Management Console.
2. The SteelHead picks a domain controller from the list—usually the one that responds quickest.
3. The SteelHead establishes a short-lived SMBv1 and well-known named pipe session to the chosen domain controller.
4. Using these two short-lived sessions, and with the authority of the temporary user account entered into the Domain Join page of the SteelHead Management Console (or CLI), the SteelHead requests the flags for the UserAccountControl attribute for its machine account.
5. After the machine account is successfully created, the SteelHead establishes a long-lived secure channel to the NetLogon service of the chosen domain controller.
By default, the SteelHead appears in the Computers Organizational Unit (OU) in the domain. This is the default for Windows member servers joining a domain. You can use the CLI for a domain join and specify a different OU.
The following example shows a SteelHead joining the domain RVBD.COM and placing the SteelHead in the WAN-opt OU:
domain join domain-name RVBD.COM login join-account password join-password org-unit WAN-opt
One-Way Trust Configuration
You need RiOS 6.1 or later when the user is in a different domain from the servers, and when the server domain has a one-way trust relationship with the user domain. In this scenario, the user domain is the trusted domain, and the server domain is the trusting domain. This setup is occasionally deployed in organizations that have a number of subsidiary companies or between two companies that have recently merged but wish to keep their IT separate for a period of time.
More typically this configuration is found when the user organization has file or mail services hosted by an external service provider. For example, with Microsoft Office 365 Dedicated service, this setup enables the SteelHead to use authentication of the Exchange servers that provide Microsoft Exchange online services.
In environments in which there is a one-way trust between server domains and user domains, the server-side SteelHead is joined to the user domain, or any domain that trusts the domain of the client user. Because of the one-way trust, the server-side SteelHead cannot automatically build a list of domains on the other side of the trust. Therefore, you must explicitly configure the server domains that trust the user domain on the server-side SteelHead, using the protocol domain-auth oneway-trust command.
For example, if the users in the RVBD.COM domain use file or mail servers in the SERVERS.PROVIDER.COM domain (whose shortened NetBIOS domain name is SERVERS), you first ensure that the server-side SteelHead is joined to the RVBD.COM domain and then use the following commands on the server-side SteelHead:
protocol domain-auth oneway-trust dns-name SERVERS.PROVIDER.COM netbios-name SERVERS
You can view the list of configured one-way trust domains with the show protocol domain-auth oneway-trust command and, if needed, you can remove domains with the no protocol domain-auth oneway-trust dns-name command.
In RiOS 7.0 or later, the support for one-way trusts is further enhanced to include Windows 7 clients without requiring a registry change on the client. You must join the server-side SteelHead to the domain using the Active Directory integrated (Windows 2003/2008) mode and then execute the CLI command as indicated previously.
Support for Kerberos authentication through a one-way trust is included in RiOS 8.5 or later. Versions of RiOS prior to 8.5 support only NTLM authentication in one-way trust configurations.
Enabling Kerberos in a Restricted Trust Environment
When you use Kerberos authentication, there are some additional one-way trust restrictions if you use external managed services such as Microsoft Office 365 Dedicated. These restricted trust models are deliberately designed with split resource and management Active Directory domains.
Within a hosted services deployment, or any other deployment in which there is a restricted one-way trust, the server-side SteelHead cannot use the replication user account to contact the resource domain controller for the session key.
The server-side SteelHead joins the user account domain and uses the replication user account to communicate with the domain controller in the user domain.
In addition, the client-side and server-side SteelHeads use a Kerberos feature in RiOS 8.5 to intercept traffic over TCP port 88, the port most used by Kerberos.
By intercepting the Kerberos exchanges (which are initiated by the client when setting up an authenticated connection to a server), the server-side SteelHead can obtain a copy of the session key used between the client and server. This action enables the SteelHeads to optimize the signed SMB or encrypted MAPI session in a restricted trust deployment.
Note: The Kerberos feature must use TCP.
Windows XP clients, by default, use UDP for Kerberos authentication. You must reconfigure Windows XP clients to use TCP, if necessary.
For more information about configuring SteelHeads for restricted one-way trusts, see Configuring the Server-Side SteelHead for Active Directory Integrated (Windows 2003/2008) and the SteelHead Management Console User’s Guide.
Kerberos
Kerberos is a network authentication protocol. In a Microsoft Windows environment, the Active Directory domain controller maintains user account and login information to support the Kerberos service. From a corporate perspective, you can think of Kerberos as guarding against unauthorized access to your IT assets.
This section includes the following topics:
•  Overview of Kerberos
•  Optimization in a Native Kerberos Environment
•  Domain User with Replication Privileges
•  Configuring Traffic Optimization for HTTP (SharePoint), Encrypted MAPI, and Signed SMB/SMB2/SMB3
Note: Kerberos authentication is not unique to a Microsoft Windows environment. This guide contains explanations and details limited to a Windows configuration.
Overview of Kerberos
The three components of Kerberos are the key distribution center (KDC), the client user, and the server resource with the service that the client user wants to access. The KDC is a part of the Windows domain controller and performs two service functions: the authentication service (AS) and the ticket-granting service (TGS).
To enable the client user to access a server resource (for example, a Windows file server), three exchanges are involved:
•  The AS exchange
•  The TGS exchange
•  The AP (application protocol) exchange
When users initially log in to a Microsoft Windows network, they must provide a login name and password for access. These credentials are verified in the AS exchange of a KDC within the domain of the user. The KDC has access to Active Directory user account information. When a user is successfully authenticated, the authentication server grants the user a ticket to get tickets (TGT) that is valid for the local domain—you are first approved to get tickets, and next you are approved for each ticket as you ask for them. The TGT (sometimes referred to as the AS reply, although the AS reply actually contains more parts) has a default lifetime of 10 hours and can be renewed during the user's session without requiring the user to reenter a password.
The AS reply includes two components: the TGT itself, which is encrypted with a key that only the KDC (TGS) can decrypt, and the first of two session keys encrypted with the user's password hash. Any future communication with the KDC uses this initial session key (SK1). The client uses the TGT, which is cached in volatile memory space on the client machine, to request sessions with services throughout the network.
The client sends a request to the TGS and asks for credentials of the desired server. The client first creates an authenticator using the SK1. The authenticator contains the username, IP address, and a time stamp—all encrypted using SK1. The client next sends a copy of the TGT and authenticator to the TGS. The TGS responds with the server credentials in the form of a service ticket that also has two components: one encrypted in the server's key, which the client cannot read, and the other encrypted with the client's key, which is temporary. The encryption key is the second of the two session keys (SK2) issued during the sequence of exchanges. The SK2 is valid only for communicating with the desired server and lasts for the lifetime of the session with the desired server. A second or subsequent session with the same server needs a completely new SK2 session key requested by the client from the TGS.
Next, the client creates another authenticator, this time using SK2 for encryption. The client transmits the authenticator with the service ticket, which contains the client identity and a copy of the session key—SK2—all encrypted with the server's key, to the server. As a result of this exchange, the SK2 is now held by the client and server. SK2 authenticates the client and can optionally authenticate the server. The SK2 can also be used to encrypt further communication between the two hosts or to exchange a separate subsession key used for encrypting further communication.
The AP exchange is also known as the client-server exchange because the exchange is often completed with whatever application protocol is required between the client and server: for example, SMB and SMB2 for file sharing, MAPI for email, or HTTP for SharePoint.
The Kerberos exchange process is shown in Figure: Kerberos Exchanges.
Figure: Kerberos Exchanges
Multiple Domain Environments and Referral Tickets
The Kerberos exchange occurs when the client user, KDS, and resource server are in the same Windows domain. Where there are multiple domains, there are steps that include referral tickets.
The AS and TGS functions are separate within the key distribution center. This enables the user to use the TGT obtained from the AS in the domain to obtain service tickets from a TGS in other domains. This is accomplished through referral tickets. A referral ticket is similar to the service ticket that is granted during the TGS exchange described earlier. The difference between the two is that the referral ticket contains a ticket for the KDC in the adjoining domain rather than the server.
If there is a trust established between two domains, referral tickets are granted to clients that request authorization for services in other domains. Because of the trust relationship between the two domains, an interdomain key, based on the trust password, is available for authenticating KDC functions.
For example, consider a client user in the local domain who seeks services in a foreign domain. The two domains are directly linked through a two-way trust. The basic steps are as follows:
1. The client performs a TGS exchange with its local domain KDC. The KDC recognizes a request for a session with a foreign domain server and responds by returning a referral ticket for the KDC in the foreign domain.
2. The client contacts the KDC of the foreign domain with the referral ticket. This ticket is encrypted with the interdomain key. Given that the decryption works, the TGS service for the foreign domain returns a service ticket for the server service in the foreign domain.
3. The client performs the client-server exchange with the server in the foreign domain and begins the user session with the service.
In a multiple domain configuration where the client is in one domain and the server is in a foreign domain, communication between the two can only be through an intermediate domain. Again, assume that there are transitive two-way trusts between the local domain and the intermediate domain, and the intermediate domain and the foreign domain.
The process is similar to the two-domain example. The basic steps are as follows:
1. The client performs a TGS exchange with its local domain KDC. The KDC recognizes a request for a session with a foreign domain server and responds by returning a referral ticket for the KDC in the intermediate domain.
2. The client contacts the KDC of the intermediate domain with the referral ticket. This ticket is encrypted with the interdomain key. The TGS service for the intermediate domain returns a referral ticket for the foreign domain.
3. The client contacts the KDC of the foreign domain with the intermediate domain referral ticket. The TGS service for the foreign domain returns a service ticket for the server service in the foreign domain.
4. The client performs the client-server exchange with the server in the foreign domain and begins the user session with the service.
For more details and reference material on Kerberos that are beyond the scope of this guide, go to:
•  http://technet.microsoft.com/en-us/library/bb742516.aspx
•  http://web.mit.edu/kerberos/
•  http://www.ietf.org/rfc/rfc4120.txt
Optimization in a Native Kerberos Environment
From a Riverbed perspective, the optimization of secure protocols in a Microsoft Windows environment includes the following protocols:
•  Signed SMB
•  Signed SMB2
•  Signed SMB3
•  Encrypted MAPI
•  Encrypted Outlook-Anywhere
•  SharePoint
You must understand how RiOS performs optimization on these protocols when the authentication is end-to-end Kerberos before you proceed to more specific details. Remember the following points:
•  Secure protocols use a session key to encrypt or digitally sign protocol packets. This session key is exchanged during the authentication phase between the client and server.
•  In the case of Kerberos, the session key is extracted from the service ticket by the target server.
•  Decrypting the service ticket requires the server machine account password, which is set up during the domain join process and might be periodically refreshed.
•  The servers machine account password is only known to the target server and domain controller.
•  Active Directory supports account replication between domain controllers for high availability.
Therefore, the only way for RiOS to legitimately optimize these encrypted, or digitally signed, secure protocols is to obtain the machine credentials for any servers that require optimization. RiOS uses Active Directory replication by using the same protocols and tools as the Windows domain controllers do and by employing a domain user that has replication privileges.
When the server-side SteelHead uses this replication capability, it can transparently participate in the key exchange between the client and server, maintain security, and optimize relevant traffic.
Domain User with Replication Privileges
Although a user with generic domain administrator privileges meets the requirements for replication, these privileges could have some security implications.
Instead, you can assign a restricted set of privileges to a user, known as a replication user. You can configure the replication user on a per forest basis so that the user assigned to it can retrieve machine credentials from any domain controller in any trusted domain within the forest. Remember that a forest can comprise of multiple domains with trusts between them.
The replication user begins with a generic user account created by a Windows domain administrator, which might or might not be you. After this is done, you can apply the privileges and restrictions for the account to become a replication user on the SteelHead. If you have multiple SteelHeads in the data center (for example, in a high-availability configuration), then you can add the same replication user account to the configuration for all the server-side SteelHeads. For information about how to configure the replication user, see the Riverbed Command-Line Interface Reference Manual and the SteelHead Management Console User’s Guide.
The replication user has enough rights to calculate the S1 and S2 Kerberos session keys for the SteelHead to properly terminate the session and perform optimization. These rights are given by granting Replicate Changes and Replicate Changes All.
After the configuration is complete, the SteelHead retains the replication user account details by encrypting them in its secure vault.
Note: In RiOS 7.0 or later, the secure vault is locked on a new SteelHead. The secure vault is also locked on a SteelHead that is upgraded to RiOS 7.0 or later. You must unlock the secure vault to view, add, remove, or edit any replication or delegate user configuration details that are stored on the SteelHeads.
In Windows 2008 domains, you can apply further restrictions to the replication user by using a password replication policy (PRP). A PRP is a set of rules that describe a list of accounts that you can replicate between domain controllers. When you enable PRP in a Windows domain, the SteelHead is restricted, specifically to using the replication user to replicate accounts as determined by the PRP settings. Therefore, you can use the PRP to limit the accounts known to the replication user. This configuration can create additional administrative overhead in managing the PRP; however, it might be worth the additional security.
PRP is not available in Windows domains with a functional level lower than Windows 2008. For information about how to configure PRP, see the SteelHead Management Console User’s Guide.
Configuring Traffic Optimization for HTTP (SharePoint), Encrypted MAPI, and Signed SMB/SMB2/SMB3
In RiOS 7.0 or later, you can enable Kerberos authentication support for HTTP for SharePoint, encrypted MAPI, signed SMB, and signed SMB2. To fully optimize signed SMB3 traffic, you must use RiOS 8.5 or later. When you enable the Kerberos setting on the server-side SteelHead, you enable the SteelHead to use the relevant domain user privileges and to access to the session key used by the client and server.
You must have RiOS 7.0 or later on both the server-side and client-side SteelHeads. You must use RiOS 8.5 or later for SMB3.
For information about how to configure each application, see the SteelHead Management Console User’s Guide and HTTP Optimization for SharePoint.
Configuring the Server-Side SteelHead for Active Directory Integrated (Windows 2003/2008)
Note: Depending on the authentication mechanism used by the Windows clients and servers, an alternative to creating and using a specific replication user can be used. For more information, see Domain User with Replication Privileges.
Regardless of the client-server authentication type, the server-side SteelHead must join the Windows domain. But depending on the authentication type, you may not need to use the replication user to optimize secure Windows traffic. If you are using NTLM authentication, so long as the server-side SteelHead join type is Active Directory integrated Windows 2003 or, preferably, Active Directory integrated Windows 2008, the SteelHead can use NTLM Transparent Mode to ensure the secure Windows traffic is fully optimized.
For the server-side SteelHead to integrate into Active Directory, you must configure the role when the appliance joins the Windows domain. Irrespective of the domain functional level (2003, mixed, 2008, and so on), select the SteelHead join type based on the operating system version running on the domain controllers. For example, if you only have domain controllers running 2003 or 2003-R2, then select the join type of Active Directory integrated Windows 2003. If you have domain controllers with a mixture of 2003, 2003-R2, 2008, or later, then select Active Directory integrated Windows 2008 as the join type.
For information about how to configure Active Directory integration, see the Riverbed Command-Line Interface Reference Manual and the SteelHead Management Console User’s Guide.
Be aware, that when you integrate the server-side SteelHead in this way, it does not provide any Windows domain controller functionality to any other machines in the domain and does not advertise itself as a domain controller or register any SRV records (service records). In addition, the SteelHead does not participate in an active replication nor does it write any Active Directory objects to persistent storage. The server-side SteelHead has just enough privileges so that it can read necessary attributes of servers you want to optimize and then use transparent interception.
This scenario is successful only for servers and clients that can make use of NTLM authentication. If you configure any clients and servers exclusively to use Kerberos authentication, they cannot use NTLM authentication. Therefore, the only way to optimize secure Windows traffic between such hosts is to configure the server-side SteelHead for end-to-end Kerberos support with a replication user account. However, in the event of a Kerberos authentication failure, unless explicitly configured, the session falls back to NTLM. Also, any service requested by IP address or a name that cannot be validated by Active Directory falls back to NTLM authentication.
For information about replication users, see Domain User with Replication Privileges.
The following table shows the different combinations of Windows clients and authentication methods with the earliest required version of RiOS and Windows configuration (delegation, Kerberos, Active Directory integrated) for the server-side SteelHead.
Client OS
Authentication Method
RiOS 7.0
Active Directory Integrated Mode
RiOS 7.0
Kerberos
RiOS 9.x
Active Directory Integrated Mode
Windows 7
Negotiate authentication/SPNEGO
Optimized using NTLM
Optimized using Kerberos
Optimized transparent
Any client up to Windows 7
Kerberos
Optimized
Optimized
Optimized
Windows 8 and 8.1
NTLM
Optimized
Optimized
Optimized
Windows 8 and 8.1
Kerberos
Optimized (fallback)
Optimized
Optimized
For Windows 8 clients behavior, use Windows 7 information in the above table, and RiOS 8.5 or later. For Windows 8.1 and Windows 10 clients, use RiOS 9.0 or later.
Remember, if you configure the server-side SteelHead to support end-to-end Kerberos authentication, you can also join it to the domain in Active Directory integrated mode to support other clients that might be using NTLM authentication. This configuration can provide flexible and broad support for multiple combinations of Windows authentication types in use within the Active Directory environment.
For advice and suggestions on configuration best practices, see Best Practices for the SteelHead in a Secure Windows Deployment.
Best Practices for the SteelHead in a Secure Windows Deployment
There are many possible ways to configure the server-side SteelHead to support the different secure Windows application options. The following is a list of best practices to ensure that the majority, if not all, of the secure Windows traffic in your environment is fully optimized:
•  Make sure that the server-side SteelHead has a DNS entry. You need only an A record.
•  Make sure that the server-side SteelHead time-of-day is synchronized through NTP. Riverbed recommends that you synchronize to the same NTP service as the domain controllers.
•  Join the server-side SteelHead to the Windows domain of choice using the role of Active Directory integrated Windows 2008. If possible, use the domain of the user domain or preferably, the same domain as the majority of the Windows application servers on which you want optimized traffic.
•  Optionally, when joining the domain, consider specifying one or more domain controllers within the same domain that are geographically closest to the server-side SteelHead. If a list is not specified, the SteelHead automatically discovers domain controllers and builds its own list.
•  Enable native Kerberos authentication support for the relevant Layer-7 RiOS features (for example, signed CIFS, signed SMB2, signed SMB3, encrypted MAPI, and HTTP) on the server-side SteelHead.
Note: For encrypted MAPI and/or signed SMB3 optimization, the client-side SteelHead must also have encrypted MAPI and/or SMB3 optimization enabled. You do not need to enable SMB3 signing on the client-side SteelHead.
•  Configure a replication user within the Active Directory forest and enter the account details into the server-side SteelHead.
•  Configure a PRP within Active Directory to further restrict the replication user account (optional).
•  If the server-side SteelHead interacts with other domains through a one-way trust, use the CLI or the Management Console to enable this setting.
If there are no one-way trusts, then this step is not required.
•  Depending on the RiOS version you have on the server-side SteelHead, you can perform the Windows Active Directory configuration steps (such as domain join and user account creation) by using the domain authentication automatic configuration feature in the Management Console or through the CLI.
For information about the domain health and domain authentication automatic configuration features, see Domain Health Check and Domain Authentication Automatic Configuration.
Domain Authentication Scaling
This section describes the ability of the SteelHeads to scale the authentication communications that occur between the server-side SteelHead and the Windows domain controller. This section includes the following topics:
•  When to Use Domain Authentication Scaling
•  General Improvements in RiOS 8.6
•  Domain Controller Load Balancing
When to Use Domain Authentication Scaling
In releases previous to RiOS 8.6, the server-side SteelHead communicates only with a single domain controller at a time. Even if the domain that the SteelHead is part of contains additional domain controllers, the SteelHead can choose to communicate only with one. If the domain controller fails or is unavailable for some reason, then the SteelHead automatically discovers an alternative domain controller.
While this process works well in most deployments, the following are examples of environments in which an individual SteelHead can be limited by this single-instance approach and can benefit from domain authentication scaling. Domain authentication scaling is available in RiOS 8.6 and later.
A large-scale deployment (with tens of thousands of users) typically involves a larger number of authentication requests between clients and servers. Because the server-side SteelHead is involved in so many requests when optimizing secure Windows traffic, lightening its communication to an individual domain controller is important.
In another scenario, there might not be a large number of requests but there might be a high degree of latency between the server-side SteelHead and the single domain controller it is communicating with. For example, with a deployment to Microsoft Office 365, there might be non-LAN latency between the domain controller and the server-side SteelHead.
In general, Riverbed recommends deployment in which the latency between domain controllers and the server-side SteelHead is no more than a few milliseconds.
For more information about maximum latency between a SteelHead and a domain controller, go to http://supportkb.riverbed.com/support/index?page=content&id=S22110.
General Improvements in RiOS 8.6
With RiOS 8.6 or later, the server-side SteelHead can manage higher quantities of communication with the Windows Active Directory (AD) environment it is requesting authentication details from. This has no direct benefit to the optimization of the secure Windows traffic (such as signed SMB and encrypted MAPI), but it can improve the authentication workload that the server-side SteelHead is part of. These general improvements include multithreading the RiOS code associated with AD interaction, which enables the server-side SteelHead to respond to more requests in a shorter time. Because these improvements are embedded within the RiOS code, you do not need to configure any additional settings other than to have the server-side SteelHead running RiOS 8.6 or later.
Domain Controller Load Balancing
Windows domains often contain more than one domain controller for redundancy and scalability. With RiOS 8.6 or later you can configure the server-side SteelHead to communicate simultaneously with multiple domain controllers in the same domain. This enables the server-side SteelHead to:
•  sustain higher quantities of authentication requests.
•  tolerate the loss of a domain controller much more efficiently.
•  reduce the load it places on an individual domain controller.
To enable the server-side SteelHead to load balance domain controllers, enter this command on the server-side SteelHead:
protocol domain-auth configure load-balancing
You can configure this setting only on the CLI. For more information about this command, see the Riverbed Command-Line Interface Reference Manual.
When you enable domain controller load balancing, the server-side SteelHead balances the traffic load across up to six domain controllers within the same domain. You can configure load balancing across more domain controllers by specifying the desired number as part of the command. The following example shows how to configure the SteelHead to load balance traffic across six domain controllers:
protocol domain-auth configure load-balancing 6
You can balance traffic across a maximum of eight domain controllers. Use the show protocol domain-auth load-balancing command to display the current status.
You do not need to specify the individual domain controllers to which the SteelHead load balances traffic. After the SteelHead is joined to a domain it automatically discovers any domain controllers that exist by performing a DNS lookup. The results from the DNS lookup enable the server-side SteelHead to automatically build its own list of domain controllers to balance traffic across. However, if you have already configured the SteelHead with a static list of domain controllers, then the static list takes precedence over an automatically discovered list. You can create a static list either by specifying one or more domain controllers on the server-side SteelHead during the join domain procedure or by using the domain settings dc-list * command.
Domain Health Check and Domain Authentication Automatic Configuration
This section describes domain health check and domain authentication automatic configuration and includes the following topics:
•  Domain Health Check
•  Domain Authentication Automatic Configuration
To optimize secure Windows traffic using SteelHeads, you:
•  need to configure NTP-related and DNS-related settings.
•  must join the server-side SteelHead to the Windows domain.
•  must configure the necessary RiOS features based on the type of protocols you want optimize: for example, encrypted MAPI, signed SMB, signed SMB2, and signed SMB3.
•  must deploy one or more service accounts configured for delegation (if using constrained delegation) or replication (if using end-to-end Kerberos).
The domain health check comprises of a series of tests related to Active Directory configuration settings. These tests and checks provide troubleshooting help for domain-related problems that might occur between the server-side SteelHead and the Active Directory environment.
The domain authentication automatic configuration includes a series of graphical widgets to assist you in performing the relevant configuration tasks associated with both the SteelHead and the Active Directory setup.
Domain Health Check
In RiOS 8.5 or later, the domain health check feature is available in the SteelHead Management Console and the CLI. Prior to RiOS 8.5, domain health check was only available in the CLI. You can use the domain health check feature to execute a variety of tests that provide diagnostic reports about the status of domain membership, end-to-end Kerberos replication, both manual and automatic constrained delegation, and DNS resolution. This information enables you to resolve issues quickly.
Riverbed recommends that you use domain health check from the SteelHead Management Console rather than the CLI.
For a full description of how to use domain health check, see the SteelHead Management Console User’s Guide and the Riverbed Command-Line Interface Reference Manual.
Using the SteelHead Management Console to Test Domain Health Check
Use the Domain Health Check page to run tests on domain health. You can create test parameters by entering specific information into certain fields. Click Test to run the relevant test. You receive feedback on whether the test succeeds or fails, along with the option to display a detailed log file of the test as it progresses. The output of the log file can aid in troubleshooting issues that might be found during testing.
Figure: Domain Health Check Page
You can access the same tests by choosing Optimization > Active Directory: Domain Join and Optimization > Active Directory: Service Accounts pages of the SteelHead Management Console.
The following examples describe several ways to confirm that the domain health check feature is functioning correctly.
To test the DNS setting using the SteelHead Management Console
•  From the Management Console, choose either Reports > Diagnostics: Domain Health Check or Optimization > Active Directory: Domain Join.
Figure: Failed DNS Test in the Management Console shows the check for Test DNS. In this particular example, the test has failed. You can choose to display or hide the logs for the test.
Figure: Failed DNS Test in the Management Console
To test Domain Join using the SteelHead Management Console
•  From the Management Console, choose either Reports > Diagnostics: Domain Health Check or Optimization > Active Directory: Domain Join.
Figure: Successful Test Join in the Management Console shows the check for Test Join. In this particular example, the test has passed. You can choose to display or hide the logs for the test.
Figure: Successful Test Join in the Management Console
Using the RiOS CLI Commands to Test Domain Health Check
To use the RiOS domain health check CLI commands, you must understand that each command performs a test or configuration task, but the result of the command is displayed only by executing a follow-on command. This second command is displayed as part of the output of the previous command and is usually a show command.
For example, the protocol domain-auth test dns command is followed by the show protocol domain-auth test dns command to display the results of the test.
The main reason for this two-stage process is that the tests themselves perform a request or look-up that is outside of the SteelHead: for example, a DNS query to a DNS server can take a few moments to complete. The two-stage process means the SteelHead CLI does not hang while waiting for the test to execute. As each test is executed, the results are saved to a temporary log file on the SteelHead. After a test is complete, the contents of the log file are displayed in a more user-friendly format when you use the relevant show command.
The following table lists the test or configuration tasks and the associated commands.
Task
CLI Commands
Check DNS settings.
protocol domain-auth test dns
show protocol domain-auth test dns
Confirm that the SteelHead is correctly joined to the Windows domain.
protocol domain-auth test join
show protocol domain-auth test join
Ensure that the SteelHead can authenticate client connections.
protocol domain-auth test authentication username * password *
show protocol domain-auth test authentication
Auto-configure a previously created account in Active Directory with replication privileges over the entire domain.
protocol domain-auth auto-conf replication adminuser * adminpass * domain * dc *
show protocol domain-auth auto-conf replication
Determine if end-to-end Kerberos replication is correctly configured.
protocol domain-auth test replication try-repl domain * shortdom * rserver *
show protocol domain-auth test replication try-repl
The following example describes how to confirm that the domain health check feature is functioning correctly.
To check the DNS settings using the CLI
•  Connect to the SteelHead CLI in and enter the following commands:
protocol domain-auth test dns
show protocol domain-auth test dns
Figure: Successful DNS Test in the CLI shows a successful DNS test and Figure: Failed DNS Test in the CLI shows a failed DNS test.
Figure: Successful DNS Test in the CLI
Figure: Failed DNS Test in the CLI
Domain Authentication Automatic Configuration
Domain authentication automatic configuration is available in the SteelHead Management Console in RiOS 8.5 or later. Domain authentication automatic configuration is a powerful set of widgets designed to help you easily configure the server-side SteelHead and Active Directory. In RiOS versions prior to 8.5, you can configure the server-side SteelHead and Active Directory using only the CLI.
For example, Figure: Domain Authentication Automatic Configuration shows how, in RiOS 8.5 or later, the domain authentication automatic configuration guides you through the steps to join the SteelHead to the domain and enable the relevant Windows features (encrypted MAPI, signed SMB, signed SMB2, and signed SMB3).
Figure: Domain Authentication Automatic Configuration
You can use the domain authentication automatic configuration to configure a Windows user account that you can use for delegation or replication purposes. The domain authentication automatic configuration on the SteelHead does not create the delegate or replication user; the Windows domain administrator must create the account in advance, using the preferred standard Active Directory procedures.
After you create the basic user account in Active Directory, you can complete the remaining configuration steps using domain authentication automatic configuration on the SteelHead.
Along with configuring the delegation and replication user accounts, domain authentication automatic configuration enables you to add and remove entries in the lists of delegation servers.
For information about how to use domain authentication automatic configuration, see the SteelHead Management Console User’s Guide.
Single Domain Example Configuration
Figure: Single Domain Example shows a data center and a branch office that are both in the RVBD.COM domain hosted by a domain controller running Windows 2008-R2. The Windows 7 clients in the remote office want optimized CIFS access to the file servers in the Data Center. The file servers are configured with Signing Required. For ease of management, transparent mode is preferred by the customer.
Figure: Single Domain Example
The following is true for this configuration:
•  Riverbed recommends that all the SteelHeads are time synchronized for SteelHead deployments that involve some form of Windows authentication. This is especially true where Kerberos is involved in the authentication process. Consider using NTP to make the time synchronization task easier to maintain.
•  SteelHead A on the client side must have RiOS 7.0 or later running to support signed SMB, but no further client-side configuration is required.
•  SteelHead B on the server side needs to join the RVBD.COM domain.
•  SteelHead B needs to have RiOS 7.0 or later and transparent mode configured.
•  Secure inner channel is optional for this configuration.
To configure a data center and branch office in the same domain
1. SteelHead A does not need configuration.
2. On SteelHead B, connect to the CLI in configuration mode and enter the following commands:
domain join domain-name RVBD.COM login Administrator password ******** dc-list 192.168.0.70 short-name RVBD
protocol cifs smbv1-mode enable
protocol cifs smb signing enable
protocol cifs smb signing mode-type transparent
write memory
restart
Confirm the correct configuration settings display in the Current Connections report, shown in Figure: MAPI-ENCRYPTS and Figure: CIFS-SIGNED. For information about the Current Connection Report, see the SteelHead Management Console User’s Guide.
Figure: MAPI-ENCRYPTS
Figure: CIFS-SIGNED