ipsec tech - faq

8/3/2019 Ipsec Tech - Faq

1/52

Understanding IPSec

Monitoring IPSec

IPSec Security Considerations

IPSec Policies

Configuring and Managing IPSec

Start Here

Understanding IPSec

IPSec Overview

IPSec is a suite of protocols which was designed by Internet Engineering Task Force ( IETF) to

protect data by signing and encrypting data before it is transmitted over public networks. The IETF

Request for Comments (RFCs) 2401-2409 defines the IPSec protocols with regard to security

protocols, security associations and key management, and authentication and encryption

algorithms. IPSec is a framework of open standards for encrypting TCP/ IP traffic within

networking environments. IPSec works by encrypting the information contained in IP datagrams

through encapsulating. This in turn provides network level data integrity, data confidentiality, data

origin authentication, and replay protection.

The primary features of IPSec are:

Authentication; protects the private network and the private data it contains. IPSec secures

private data from man-in-the-middle attacks, from attackers attempting to access the network,

and from an attacker changing the contents of data packets.

Encryption; conceals the actual content of data packets so that it cannot be interpreted by

unauthorized parties.

IPSec can be used to provide packet filtering capabilities. It can also authenticate traffic between

two hosts and encrypt traffic passed between the hosts. IPSec can be used to create a virtual private

network (VPN). IPSec can also be used to enable communication between remote offices and remote

access clients over the Internet.

IPSec operates at the network layerto provide end-to-end encryption. This basically means that

data is encrypted at the source computer sending the data. All intermediate systems handle the

encrypted portion of the packets as payload. Intermediate systems such as routers merely forward

the packet to its end destination. Intermediate systems do not decrypt the encrypted data. The

encrypted data is only decrypted when it reaches the destination.

IPSec interfaces with the TCP/UDP transport layer and the Internet layer, and is applied transparently

to applications. IPSec is transparent to users as well. This basically means that IPSec can provide

security for most of the protocols within the TCP/IP protocol suite. When it comes to applications,

1
http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ietf-458http://www.tech-faq.com/microsoft-ipsec/glossary-1/e/encryption-48http://www.tech-faq.com/microsoft-ipsec/glossary-1/f/framework-422http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/networking-554http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/packet-257http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/network-layer-551http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/network-layer-551http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ietf-458http://www.tech-faq.com/microsoft-ipsec/glossary-1/e/encryption-48http://www.tech-faq.com/microsoft-ipsec/glossary-1/f/framework-422http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/networking-554http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/packet-257http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/network-layer-551http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248


2/52

all applications that use TCP/IP can enjoy the security features of IPSec. You do not have to

configure security for each specific TCP/IP based application. By using rules and filters, IPSec can

receive network traffic and select the required security protocols, determine which algorithms to use,

and can apply cryptographic keys required by any of the services.

The security features and capabilities of IPSec can be used to secure the private network and private

confidential data from the following

Denial-of-service ( Dos) attacks

Data pilfering.

Data corruption.

Theft of user credentials

In Windows Server 2003, IPSec uses the Authentication Header (AH) protocol and Encapsulating

Security Payload (ESP) protocol to provide data security on:

Client computers

Domain servers

Corporate workgroups

Local area networks (LANs)

Wide area networks (WANs)

Remote offices

The security functions and features provided by IPSec are summarized below:

Authentication; a digital signature is used to verify the identity of the sender of the information.

IPSec can use Kerberos, a preshared key, or digital certificates for authentication.

Data integrity; a hash algorithm is used to ensure that data is not tampered with. A

checksum called a hash message authentication code (HMAC) is calculated for the data of the

packet. When a packet is modified while in transit, the calculated HMAC changes. This change

will be detected by the receiving computer.

Data privacy; encryption algorithms are utilized to ensure that data being transmitted is

undecipherable.

Anti-replay; prevents an attacker from resending packets in an attempt to gain access to the

privatenetwork.

Nonrepudiation; public key digital signatures are used to prove message origin.

2
http://www.tech-faq.com/microsoft-ipsec/glossary-1/d/dos-386http://www.tech-faq.com/microsoft-ipsec/glossary-1/k/kerberos-493http://www.tech-faq.com/microsoft-ipsec/glossary-1/a/algorithm-138http://www.tech-faq.com/microsoft-ipsec/glossary-1/d/dos-386http://www.tech-faq.com/microsoft-ipsec/glossary-1/k/kerberos-493http://www.tech-faq.com/microsoft-ipsec/glossary-1/a/algorithm-138


3/52

Dynamic rekeying; keys can be created during data sending to protect segments of the

communication with different keys.

Key generation; the Diffie-Hellman key agreement algorithm is used to enable two computers

to exchange a shared encryption key.

IP Packet filtering; the packet filtering capability of IPSec can be used to filter and block

specific types of traffic, based on either of the following elements or on a combination of them:

o IP addresses

o Protocols

o Ports

What New in Windows Server 2003 IPSec

A few new IPSec features have been included in Windows Server 2003, together with enhancements

to some IPSec features which existed in previous Windows operating systems:

Windows Server 2003 includes the new IP Security Monitortool which is implemented as an

MMC snap-in. The IP Security Monitor tool provides enhanced IPSec security monitoring. With

the IP Security Monitor tool, you can perform the following administrative activities:

o Customize the IP Security Monitor display

o Monitor IPSec information on the local computer.

o Monitor IPSec information on remote computers.

o View IPSec statistics.

o View information on IPSec policies

o View security associations information.

o View generic filters

o View specific filters

o Search for specific filters based on IP address

You can configure IPSec using the Netsh command-line utility. The netsh command-line utility

replaces the previously used Ipsecpol.exe command-line utility.

IPSec supports the new Resultant Set of Policy (RSoP) feature of Windows Server 2003. The

Resultant Set of Policies (RSoP) calculator can be used to determine the policies which have

been applied to a particular user or computer. Resultant Set of Policy (RSoP) sums all group

policies which are applied to a user and computer in a domain. This includes all filters and

3
http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/monitor-533http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-address-68http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/monitor-533http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-address-68


4/52

exceptions. You can use the feature through the Resultant Set Of Policy (RSoP) Wizard or

from the command-line to view the IPSec policy that is applied.

IPSec integration with Active Directory enables you to centrally manage security policies.

Kerberos 5 authentication is the default authentication method used by IPSec policies to verify

the identity of computers.

IPSec is backward compatible with the Windows 2000 Security Framework.

If a local policy or Active Directory based policy cannot be applied to a computer, you now

have the option of creating a persistent policy for the specific computer. The characteristics of

persistent policies are:

o Persistent policies can only be configured through the Netsh command-line utility.

o Persistent policies are always positive.

o Persistent policies cannot be overridden.

In Windows Server 2003 IPSec deployments, only Internet Key Exchange ( IKE) traffic is

exempt from IPSec. Previously, Resource Reservation Protocol (RSVP) traffic, Kerberos

traffic, and IKE traffic was exempt from IPSec.

IPSec in Windows Server 2003 includes support for the Group 3 2048-bit Diffie-Hellman key

exchange. The Group 3 key is much stronger and more complex than the previous Group 2

1024-bit Diffie-Hellman key exchange. If however you need backward compatibility with

Windows 2000 and Windows XP, then you have to use the Group 2 1024-bit Diffie-Hellman

key exchange.

IPSec ESP packets can pass over Network Address Translation (NAT) through User Datagram

Protocol-Encapsulating Security Payload (UDP-ESP) encapsulation in Windows Server 2003

IPSec deployments.

Understanding IPSec Terminology

This section of the Article lists the commonly used IPSec terminology and concepts:

Authentication Header (AH): This is one of the main security protocols used by IPSec. AH

provides data authentication and integrity, and can therefore be used on its own when data

integrity and authentication are relevant factors and confidentiality is not. This is because AHdoes not provide for encryption, and therefore cannot provide data confidentiality.

Authentication Header (AH) and Encapsulating Security Payload (ESP) are the main security

protocols used in IPSec. These security protocols and can be used separately, or together.

Encapsulating Security Payload (ESP): This is one of the main security protocols used by

IPSec. ESP ensures data confidentiality through encryption, data integrity, data authentication,

and other features that support optional anti-replay services. To ensure data confidentiality, a

number of symmetric encryption algorithms are used.

4
http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ike-460http://www.topbits.com/nat-network-address-translation.htmlhttp://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ike-460http://www.topbits.com/nat-network-address-translation.html


5/52

Certificate Authorities (CAs): This is an entity that generates and validates digital certificates.

The CA adds its own signature to the public key of the client. CAs issue and revoke digital

certificates.

Diffie-Hellman groups: Diffie-Hellman Key Agreement enables two computers to create a

shared private key that authenticates data and encrypts an IP datagram. The different Diffie-

Hellman groups are listed here:

o Group 1; provides 768-bit key strength



Internet Key Exchange (IKE): The IKE protocol is used by computers to create a security

association (SA) and to exchange information to generate Diffie-Hellman keys. IKE manages

and exchanges cryptographic keys so that computers can have a common set of security

settings. Negotiation occurs on which authentication method, and encryption algorithm and

hashing algorithm the computers will use.

IPSec Driver: The IPSec driver performs a number of operations to enable secure network

communication, including the following:

o Creates IPSec packets

o Generates checksums.

o Initiates the IKE communication

o Adds the AH and ESP headers

o Encrypts data before it is transmitted.

o Calculates hashes and checksums for incoming packets.

IPSec Policies: IPSec policies define when and how data should be secured, and defines

which security methods to use for securing data. IPSec policies contain a number of elements:

o Actions.

o Rules

o Filter lists

o Filter actions.

IPSec Policy Agent: This is a service running on a computer running Windows Server 2003

that accesses IPSec policy information. The IPSec Policy Agent accesses the IPSec policy

information in either the Windows registry or in Active Directory.

5


6/52

Oakley key determination protocol: The Diffie-Hellman algorithm is used for two authenticated

entities to negotiate and be in agreement on a secret key.

Security Association (SA): A SA is a relationship between devices that define how they use

security services and settings.

Triple Data Encryption (3DES): This is a strong encryption algorithm used on client machines

running Windows, and on Windows Server 2003 computers. 3DES uses 56-bit keys for

encryption.

Understanding How IPSec Works

A security association (SA) has to first be established between two computers before data can be

securely passed between the computers. A Security Association (SA) is a relationship between

devices that define how they use security services and settings. The SA provides the information

necessary for two computers to communicate securely. Internet Security Association and Key

Management Protocol ( ISAKMP) and the IKE protocol are the mechanism that enables two

computers to establish security associations. When an SA is established between two computers, the

computers negotiate on which security settings to utilize to secure data. A security key is exchanged

and used to enable the computers to communicate securely.

The security association (SA) contains the following:

The policy agreement which dictates which algorithms and key lengths the two computers will

use to secure data.

The security keys used to secure data communication.

The security parameters index (SPI).

With IPSec, two separate SAs are established for each direction of data communication:

One SA secures inbound traffic.

One SA secures outbound traffic.

In addition to the above, there is a unique SA for each IPSec security protocol. There are therefore

basically two types of SAs:

ISAKMP SA: When traffic flow is two directional and IPSec needs to establish a connection

between computers, an ISAKMP SA is established. The ISAKMP SA defines and handlessecurity parameters between the two computers. The two computers agree on a number of

elements to establish the ISAKMP SA:

o Determine which connections should be authenticated.

o Determine the encryption algorithm to use.

o Determine the algorithm to verify message integrity.

6
http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/isakmp-476http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/isakmp-476


7/52

After the above elements have been negotiated between the two computers, the computers

use the Oakley protocol to agree on the ISAKMP master key. This is the shared master key

which will be used with the above elements to enable secure data communication.

After a secured communication channel is established between the two computers, the

computers start to negotiate the following elements:

o Determine whether the Authentication Header (AH) IPSec protocol should be used for

the connection.

o Determine the authentication protocol which should be used with the AH protocol for the

connection.

o Determine whether the Encapsulating Security Payload (ESP) IPSec protocol should be

used for the connection.

o Determine the encryption algorithm which should be used with the ESP protocol for the

connection.

IPSec SA: IPSec SAs pertain to the IPSec tunnel and IP packet, and define securityparameters to use during a connection. The IPSec SA is derived from the above four elements

just negotiated between the two computers.

To secure and protect data, IPSec uses cryptography to provide the following capabilities:

Authentication: Authentication deals with verifying the identity of the computer sending the

data, or the identity of the computer receiving the data. The methods which IPSec can use to

authenticate the sender or receiver of data are:

o Digital certificates: Provides the most secure means of authenticating identities.

Certificate authorities (CAs) such as Netscape, Entrust, VeriSign, and Microsoft provide

certificates which can be used for authentication purposes.

o Kerberos authentication: A downside of using the Kerberos v5 authentication protocol is

that the identity of the computer remains unencrypted up to the point that the whole

payload is encrypted at authentication.

o Pre-shared keys; should be used when none of the former authentication methods can

be used.

Anti-replay ensures that the authentication data cannot be interpreted as it is sent over thenetwork. In addition to authentication, IPSec can provide nonrepudiation. With nonrepudiation,

the sender of the data cannot at a later stage deny actually sending the data.

Data integrity: Data integrity deals with ensuring that the data received at the recipient has not

been tampered with. A hashing algorithm is used to ensure that the data is not modified as it is

passed over the network. The hashing algorithms which can be used by IPSec are:

7


8/52

o Message Digest ( MD5); a one-way hash that results in a 128-bit hash which is used

for integrity checking.

o Secure Hash Algorithm 1 (SHA1); a 160-bit secret key to generate a 160-bit message

digest which provides more security than MD5.

Data confidentiality: IPSec ensures data confidentiality by applying encryption algorithms to

data before it is sent over the network. If the data is intercepted, encryption ensures that the

intruder cannot interpret the data. To ensure data confidentiality, IPSec can use either of the

following encryption algorithms:

o Data Encryption Standard (DES); the default encryption algorithm used in Windows

Server 2003 which uses 56-bit encryption.

o Triple DEC (3DES); data is encrypted with one key, decrypted with another key, and

encrypted again with a different key.

o 40-bit DES; the least secure encryption algorithm.

Understanding the IPSec Modes

IPSec can operate in one of the following modes:

Tunnel mode: IPSec tunnel mode can be used to provide security for WAN and VPN

connections that use the Internet as the connection medium. In tunnel mode, IPSec encrypts

the IP header and the IP payload. With tunneling, the data contained in a packet is

encapsulated inside an additional packet. The new packet is then sent over the network.

Tunnel mode is typically used for the following configurations:

o Server to server

o Server to gateway

o Gateway to gateway

The process of communication that occurs when tunnel mode is defined as the IPSec mode is

detailed below:

o Data is transmitted using unprotected IP datagrams from a computer on the private

network.

o When the packets arrive at the router, the router encapsulates the packet using IPSec

security protocols.

o The router then forwards the packet to the router at the other end of the connection.

o This router checks the integrity of the packet.

8
http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/md5-519http://www.tech-faq.com/microsoft-ipsec/glossary-1/t/tunneling-125http://www.tech-faq.com/microsoft-ipsec/glossary-1/g/gateway-427http://www.tech-faq.com/microsoft-ipsec/glossary-1/r/router-109http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/md5-519http://www.tech-faq.com/microsoft-ipsec/glossary-1/t/tunneling-125http://www.tech-faq.com/microsoft-ipsec/glossary-1/g/gateway-427http://www.tech-faq.com/microsoft-ipsec/glossary-1/r/router-109


9/52

o The packet is decrypted.

o The data of the packet is then added to unprotected IP datagrams and sent to the

destination computer on the private network.

Transport Mode: This is the default mode of operation used by IPSec in which only the IP

payload is encrypted through the AH protocol or ESP protocol. Transport mode is used for

end-to-end communication security between two computers on the network.

IPSec Components

The primary two components installed when IPSec is deployed are:

IPSec Policy Agent: This is a service running on a computer running Windows Server 2003

that accesses IPSec policy information. The IPSec Policy Agent accesses the IPSec policy

information in either the Windows registry or in Active Directory. The main functions which the

IPSec Policy Agent provides are listed below:

o The IPSec Policy Agent passes information to the IPSec driver.

o The IPSec Policy Agent accesses IPSec policy information from the local Windows

registry when the computer does not belong to a domain.

o The IPSec Policy Agent accesses IPSec policy information from the Active Directory

when the computer is a member of a domain.

o The IPSec Policy Agent scans IPSec policies for any configuration changes.

IPSec driver: The IPSec driver performs a number of operations to enable secure network

communication, including the following:

o Creates IPSec packets

o Generates checksums.

o Initiates the IKE communication

o Adds the AH and ESP headers

o Encrypts data before it is transmitted.

o Calculates hashes and checksums for incoming packets

Understanding the IPSec Protocols

As mentioned previously, the main IPSec security protocols are the Authentication Header (AH) and

Encapsulating Security Payload (ESP) protocols. There are other IPSec protocols such as ISAKMP,

IKE, and Oakley that use the Diffie-Hellman algorithm.

9


10/52

Authentication Header (AH) Protocol

The AH protocol provides the following security services to secure data:

Authentication

Anti-replay

Data integrity

The AH protocol ensures that data is not modified as it moves over the network. It also ensures that

the data originated from the sender.

The AH protocol does not though provide data confidentiality because it does not encrypt the data

contained in the IP packets. This basically means, that if the AH protocol is used by itself; intruders

that are able to capture data would be able to read the data. They would not though be able to

change the data. The AH protocol can be used in combination with the ESP protocol if you need to

ensure data confidentiality as well.

The communication process which occurs when the AH protocol is used is shown here:

1. One computer transmits data to another computer.

2. The IP header, AH header, and the data itself is signed to ensure data integrity.

3. The AH header is inserted between the IP header and IP payload to provide authentication

and integrity.

The fields within a AH header, together with the role performed by each field is listed here:

Next Header; used to specify the type of IP payload through the IP protocol ID that exists after

this AH header.

Length; indicates the length of the AH header.

Security Parameters Index (SPI); indicates the correct security association for the

communication through a combination of the following:

o IPSec security protocol.

o Destination IP address

Sequence Number; used to provide IPSec anti-replay protection for the communication. The

sequence number commences at 1, and is incremented by 1 in each ensuing packet. Packets

that have the same sequence number and security association are discarded.

Authentication Data; holds the integrity check value ( ICV) calculated by the sending

computer to provide data integrity and authentication. The receiving computer calculates the

ICV over the IP header, AH header, and IP payload, and then compares the two ICV values.

10
http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/icv-457http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/icv-457


11/52

Encapsulating Security Payload (ESP) protocol

The ESP protocol provides the following security services to secure data:

Authentication

Anti-replay

Data integrity

Data confidentiality

The primary difference between the AH protocol and the ESP protocol is that the ESP protocol

provides all the security services provided by the AH protocol, together with data confidentiality

through encryption. ESP can be used on its own, and it can be used together with the AH protocol. In

transport mode, the ESP protocol only signs and protects the IP payload. The IP header is not

protected. If the ESP protocol is used together with the AH protocol, then the entire packet is signed.

ESP inserts an ESP header and ESP trailer, which basically encloses the payload of the IP datagram.

All data after the ESP header to the point of the ESP trailer, and the actual ESP trailer is encrypted.

The fields within an ESP header, together with the role performed by each field are listed here:

Security Parameters Index (SPI); indicates the correct security association for the

communication through a combination of the following:

o IPSec security protocol.

o Destination IP address

Sequence Number; used to provide IPSec anti-replay protection for the communication. The

sequence number commences at 1, and is incremented by 1 in each ensuing packet. Packets

that have the same sequence number and security association are discarded.

The fields within an ESP trailer, together with the role performed by each field are listed here:

Padding; required by the encryption algorithm to ensure that byte boundaries are present.

Padding Length; indicates the length (bytes) of the padding which was used in the Padding

field.

Next Header; used to specify the type of IP payload through the IP protocol ID.

Authentication Data; holds the integrity check value (ICV) calculated by the sending computer

to provide data integrity and authentication. The receiving computer calculates the ICV over

the IP header, AH header, and IP payload, and then compares the two ICV values.

Understanding IPSec Security Filters, Security Methods, and SecurityPolicies

11


12/52

Security filters basically match security protocols to a specific network address. IPSec filters can be

used to filter out unauthorized traffic. The filter contains the following information:

Source and destination IP address

Protocol used

Source and destination ports

Each IP address contains a network ID portion and a host ID portion. Through security filters, you can

filter traffic according to the following:

Traffic allowed to pass through

Traffic to secure

Traffic to block

Security filters can be grouped into a filter list. There is no limit to the number of filters which can be

included in a filter list. IPSec policies uses IP filters to ascertain whether an IP security rule should beused in a packet.

You can use a security method to specify the manner in which an IPSec policy should deal with traffic

matching an IP filter. Security methods are also referred to as filter actions. The filter actions result in

either of the following events:

Drops traffic

Allows Traffic

Negotiates security.

To apply security in your network, IPSec policies are used. The IPSec policies define when and how

data should be secured. The IPSec policies also determine which security methods to use when

securing data at the different levels in your network. You can configure IPSec policies so that different

types of traffic are affected by each individual policy.

IPSec policies can be applied at the following levels within a network:

Active Directory domain

Active Directory site

Active Directory organizational unit

Computers

Applications

12


13/52

The different components of an IPSec policy are listed here:

IP filter; informs the IPSec driver on the type of inbound traffic and outbound traffic which

should be secured.

IP filter list; used to group multiple IP filters into a single list in order to isolate a specific set of

network traffic.

Filter action; used to define how the IPSec driver should secure traffic.

Security method; refers to security types and algorithms used for the key exchange process

and for authentication.

Connection type: identifies the type of connection which the IPSec policy impacts.

Tunnel setting; the tunnel endpoint's IP address/DNS name.

Rule; a grouping of the following components to secure a specific subset of traffic in a

particular manner:

o IP filter

o Filter action.

o Security method

o Connection type

o Tunnel setting.

Monitoring IPSec

Using the IP Security MonitorSnap-In to Monitor IPSec

The IP Security Monitor snap-in, a new feature in Windows Server 2003, can be used to monitor and

troubleshoot IPSec activity. The IP Security Monitor snap-in provides enhanced IPSec security

monitoring. As long as the IPSec policy is active, you can monitor how the IPSec policy is functioning

within your networking environment through the IP Security Monitor.

The main administrative activities which you can perform through the IP Security Monitor snap-in are

listed here:

Customize the IP Security Monitor display

Monitor IPSec information on the local computer. Monitor IPSec information on remote computers.

View IPSec statistics. View information on IPSec policies

View security associations information. View generic filters

13
http://www.topbits.com/what-is-dns.htmlhttp://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/monitor-533http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/networking-554http://www.topbits.com/what-is-dns.htmlhttp://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/monitor-533http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/networking-554


14/52

View specific filters

Search for specific filters based on IP address

By default, the computer which is listed in the IP Security Monitor snap-in is the local computer. You

can though add another computer(s) which you want to monitor to the IP Security Monitor snap-in.

To add other computers to the IP Security Monitor snap-in,

1. Open the IP Security Monitor.

2. In the left pane, right-click the IP Security Monitor nodeand then click Add Computer on the shortcutmenu.

The information which is displayed in the IP Security Monitor snap-in is categorized into the following

three nodes:

Active Policy node Main Mode node

Quick Mode node

IPSec information, on which IPSec policy is assigned, is displayed under the Active Policy node

within the IP Security Monitor tool. This includes the following IPSec policy information:

Policy Name Policy Description

Policy Last Modified. Policy Store

Policy Patch Organization Unit Group Policy Object Name

For the Main Mode and Quick Mode nodes, you can view IP Security statistics by clicking the

Statistics node contained within the Main Mode node and Quick Mode node. It is this Statistics nodewhich should be used to monitor IPSec activity:

The Statistics node located under the Main Mode node can be used to obtain information on Phase 1 ofthe IPSec negotiations.

The Statistics node located under the Quick Mode node can be used to obtain information on Phase 2of the IPSec negotiations.

The various Main mode statistics, together with a brief description on what each statistic tracks are

listed here:

Active Acquire; indicates and tracks the number of IKE requests needed to start an IKE negotiationso that an SA can be established between two computers running IPSec. The figure displayed for thisstatistic includes the current IKE negotiation request and all requests which are queued by the IKE

process. Acquire Failures; indicates the number of requests to establish SAs between IPSec computers that

have failed since the last time the IPSec service started. Receive Failures;indicates the number of errors which took place at the time of receiving IKE messages

since the last time the IPSec service started. Send Failures; indicates the number of errors which took place at the time of sending IKE messages

since the last time the IPSec service started.

14
http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-address-68http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/node-259http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/node-259http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ike-460http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-address-68http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/node-259http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ike-460


15/52

Acquire Heap Size; indicates the number of queued outbound requests for SAs between IPSeccomputers.

Receive Heap Size; indicates the number of incoming IKE messages which were successful. Authentication Failures;indicates the number of authentication failures that have occurred since the last

time the IPSec service started. Authentication failures are typically caused by mismatchedauthentication methods and authentication configuration errors.

Negotiation Filures; indicates the number of negotiation failures that have occurred since the last timethe IPSec service started. Negotiation failures are typically caused by mismatched authentication

methods, authentication configuration errors, and mismatched security methods and security settings.

Invalid Cookies Received; indicates the number of cookies which was left unmatched to a Main

mode SA. Total Acquire; indicates the total number of requests which was sent to IKE to establish a Main mode

SA since the last time that the IPSec service started. Total Get SPI;indicates the number of requests to the IPSec driver for a Security Parameters Index

(SPI). Key Additions;indicates the number of outbound Quick mode SAs which were added to the IPSec

driver. Key Updates; indicates the number of inbound Quick mode SAs which were added to the IPSec driver.

Get SPI Failures;indicates the number of failed requests to the IPSec driver for a Security ParametersIndex (SPI).

Key Addition Failures; indicates the number of failed outbound Quick mode SAs which were added to

the IPSec driver. Key Update Failures;indicates the number of failed inbound Quick mode SAs which were added to the

IPSec driver. ISADB List Size; indicates the total number of successful Main mode entries. This includes all queued

Main mode negotiations and failed Main mode negotiations. Connection List Size; indicates the queued Quick mode negotiations.

IKE Main Mode; indicates the total number of successful SAs which have been created during Mainmode since the last time that the IPSec service started.

IKE Quick Mode; indicates the total number of successful SAs which have been created during Quickmode since the last time that the IPSec service started.

Soft Associations;indicates the total number of negotiations with computers not running IPSec whichcreated unencrypted soft SAs.

Invalid Packets Received; indicates the number of IKE messages that was received but was invalid.Typically caused by mismatched preshared keys.

The various Quick mode statistics, together with a brief description on what each statistic tracks are

listed here:

Active Security Associations;indicates the number of active Quick mode SAs. Offloaded Security Associations;indicates the number of active Quick mode SAs accelerated by certain

hardware such as network adapters that can accelerate IPSec processing. Pending Key Operations;indicates the current number of IPSec key exchange operations which are in

queue or in progress that still have to complete.

Key Additions;indicates the number of successful Quick mode SAs added from when the computer waslast started.

Key Deletions;indicates the number of successful Quick mode SAs deleted from when the computerwas last started.

Rekeys; indicates the total number of rekey operations for Quick mode SAs from when the computerwas last started.

Active Tunnels; indicates the number of active IPSec tunnels. Bad SPI Packets; indicates the total number of packets which have been impacted by an incorrect or

bad Security Parameter Index (SPI) from when the computer was last started.

15
http://www.tech-faq.com/microsoft-ipsec/glossary-1/c/cookies-346http://www.tech-faq.com/microsoft-ipsec/glossary-1/c/cookies-346


16/52

Packets Not Decrypted; indicates the number of packets that could not be decrypted from when thecomputer was last started.

Packets Not Authenticated; indicates the number of packets for which the source could not beauthenticated or verified.

Packets With Replay Detection; indicates the total number of packets which included an invalidsequence number from when the computer was last started.

Confidential Bytes Sent;indicates the total number of encrypted bytes sent which were encrypted

through the Encapsulating Security Payload (ESP) protocol, from when the computer was last

started. Confidential Bytes Received; indicates the total number of encrypted bytes received which were

encrypted through the Encapsulating Security Payload (ESP) protocol, from when the computer waslast started.

Authenticated Bytes Sent; indicates the total number of authenticated bytes sent through theAuthentication Header (AH) protocol or the Encapsulating Security Payload (ESP) protocol, from when

the computer was last started. Authenticated Bytes Received; indicates the total number of authenticated bytes received through the

Authentication Header (AH) protocol or the Encapsulating Security Payload (ESP) protocol, from whenthe computer was last started.

Transport Bytes Sent; indicates the total number of bytes sent through Transport mode from when thecomputer was last started.

Transport Bytes Received; indicates the total number of bytes received through Transport mode from

when the computer was last started. Bytes Sent In Tunnels; indicates the total number of bytes sent through Tunnel mode from when the

computer was last started. Bytes Received In Tunnels; indicates the total number of bytes received through Tunnel mode from

when the computer was last started. Offloaded Bytes Sent;indicates the total number of bytes sent through IPSec hardware offload from

when the computer was last started. Offloaded Bytes Received; indicates the total number of bytes received through IPSec hardware offload

from when the computer was last started.

How to monitor IPSec with the Security Monitor

1. Click Start, click Run, type mmc in the Run dialog box, and then click OK.2. Click the File Menu item and select Add/Remove Snap-in.3. The Add/Remove Snap-in dialog box opens. Click Add.

4. The Add Standalone Snap-In dialog box opens.5. In the Available Standalone Snap-ins list, select IP Security Monitor, and then click Add.

6. The Select Computer Or Domain dialog box opens.7. Click the Local Computer option.

8. Click Finish.

9. Click Close to close the Add Standalone Snap-in dialog box.10. Click OK to close the Add/Remove Snap-in dialog box.

11. To add another computer to the IP Security Monitor console, right-click IP Security Monitor and then

select Add Computer from the shortcut menu.12. To view active policy information, double-click the Active Policy node.13. To view IP Security statistics for Main mode, expand the Main Mode node in the left pane and then click

Statistics.14. To view IP Security statistics for Quick mode, expand the Quick Mode node in the left pane and then

click Statistics.

Using the Netsh command command-line utility to Monitor IPSec

16
http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248


17/52

The Netsh command-line utility can be used to view information on IPSec policies and to monitor

IPSec on computers running Windows Server 2003. If you use the Netsh command-line utility to

monitor IPSec, you can find and view exactly the same the information which is available for IPSec in

the IP Security Monitor snap-in.

The netsh diag command with the additional diagnostics switches which you can use at the command

prompt to monitor IPSec are listed here:

netsh diag connect; to connect to proxy servers, mail serves, and news servers.

netsh diag dump; to display a script used for configuration. netsh diag show; for displaying the following information:

o Operating systeminformation.

o Computer information.

o Network information.

o Proxy server information

o News information.

o Mail information

netsh diag gui; for displaying diagnostics on a Web page

Using Event Viewer to Monitor IPSec

If you configure IPSec to add events to the event logs, you can use the Event Viewer tool, located in

the Administrative Tools folder, to monitor IPSec activity. Event Viewer stores events that are logged

in the system log, application log, and security log.

IPSec can add events for the following:

Successful IPSec negotiations Unsuccessful IPSec negotiations

Dropped packets

If you want to log an event whenever a change is made to an IPSec policy, you can enable the Audit

Policy Change policy.

A few IPSec event log messages are listed here:

Event ID 541 (Success audit); added whenever a Main mode SA or an IPSec SA is successfully

negotiated. Event ID 542 (Success audit); added whenever an IPSec SA is successfully deleted by IKE. Event ID 543 (success audit); added whenever a Main mode SA is successfully deleted by IKE.

Event ID 544 (failure audit); logged whenever the IKE negotiation process terminates due to either ofthe following reasons:

o Certificate trust failure.o Authentication failure.

Event ID 545 (failure audit); logged whenever the IKE negotiation process terminates due to thefollowing reason:

o Validation failure of the computer certificate signature.

Event ID 546 (failure audit); logged whenever an SA is not created due to an invalid IKE proposal froman IPSec-enabled computer.

Event ID 547 (failure audit); logged whenever an SA negotiation process fails, and no SA was created.

Using Network Monitor to Monitor IPSec Activity

17
http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/proxy-247http://www.tech-faq.com/microsoft-ipsec/glossary-1/o/operating-system-572http://www.tech-faq.com/microsoft-ipsec/glossary-1/o/operating-system-572http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/proxy-247http://www.tech-faq.com/microsoft-ipsec/glossary-1/o/operating-system-572


18/52

You can use Network Monitor to monitor network traffic, and to troubleshoot network issues or

problems. Network Monitor shipped with Windows Server 2003 allow you to monitor network activity

and use the gathered information to manage and optimize traffic, identify unnecessary protocols, and

to detect problems with network applications and services.

In order to capture frames, you have to install the Network Monitor application and the Network

Monitor driver on the server where you are going to run Network Monitor. The Network Monitor driver

makes it possible for Network Monitor to receive frames from the network adapter.

The two versions of Network Monitor are:

The Network Monitor version included with Windows Server 2003: With this version of Network Monitor

you can monitor network activity only on the local computer running Network Monitor.

The Network Monitor version (full) included with Microsoft Systems Management Server ( SMS):With this version, you can monitor network activity on all devices on a network segment. You can

capture frames from a remote computer, resolve device names to MAC addresses, and determine

the user and protocol that is consuming the most bandwidth.

Because of these features, you canuse Network Monitor to monitor and troubleshoot IPSec traffic.

To install Network Monitor

1. Click Start, and then click Control Panel.

2. Click Add Or Remove Programs to open the Add Or Remove programs dialog box.

3. Click Add/Remove Windows Components.4. Select Management and Monitoring Tools and click the Details button.

5. On the Management and Monitoring Tools dialog box, select the Network Monitor Tools checkbox andclick OK.

6. Click Next when you are returned to the Windows Components Wizard.

7. If prompted during the installation process for additional files, place the Windows Server 2003 CD-

ROM into the CD-ROM drive.

8. Click Finish on the Completing the Windows Components Wizard page.

To start a Network Monitor capture

1. Click Start, click Administrative Tools, and then click Network Monitor.2. If you need to specify a network connection, expand Local Computer and then select Local Area

Connection. Click OK.

3. Click the Start command on the Action menu.4. If You want to examine captured data during the capture, select Stop And View from the Capture menu

How to monitor IPSec logon activity

1. Click Start, click Run, type mmc in the Run dialog box, and then click OK.2. Click the File Menu item and select Add/Remove Snap-in.

3. The Add/Remove Snap-in dialog box opens. Click Add.4. The Add Standalone Snap-In dialog box opens.

5. In the Available Standalone Snap-ins list, select Group Policy Object Editor, and then click Add.6. The Select Computer Or Domain dialog box opens.

7. Click the Local Computer option.8. Click Finish.

9. Click Close to close the Add Standalone Snap-in dialog box.

18
http://www.tech-faq.com/microsoft-ipsec/glossary-1/s/sms-646http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/mac-513http://www.tech-faq.com/microsoft-ipsec/glossary-1/b/bandwidth-8http://www.tech-faq.com/microsoft-ipsec/glossary-1/c/control-panel-345http://www.tech-faq.com/microsoft-ipsec/glossary-1/c/control-panel-345http://www.tech-faq.com/microsoft-ipsec/glossary-1/r/rom-619http://www.tech-faq.com/microsoft-ipsec/glossary-1/s/sms-646http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/mac-513http://www.tech-faq.com/microsoft-ipsec/glossary-1/b/bandwidth-8http://www.tech-faq.com/microsoft-ipsec/glossary-1/c/control-panel-345http://www.tech-faq.com/microsoft-ipsec/glossary-1/r/rom-619


19/52

10. Click OK to close the Add/Remove Snap-in dialog box.11. Navigate to the Audit Policy node.

12. Double-click Audit Logon Events.13. The Local Security Policy Setting dialog box opens.

14. Enable the Success checkbox and the Failure checkbox and then click OK.15. Double-click Audit Object Access.

16. Enable the Success checkbox and the Failure checkbox.17. Click OK.

18. You can now view the event log to determine whether IPSec negotiations were successful or not.

IPSec Security Considerations

Securing the Network

When planning for and implementing network security, the activities which you should be

performing would typically involve the following:

Planning how the network infrastructure will be secured from both internal and external threats.

Defining and creating internal and external security boundaries.

Implementing network security technologies and mechanisms that can assist the organization

in meeting its security requirements.

Implementing server security technologies and mechanisms.

Implementing application security technologies and mechanisms.

Implementing user security technologies and mechanisms.

Planning and implementing an auditing strategy.

Implementing network monitoring.

A few methods of securing your network infrastructure are listed here:

Physically securing all mission-critical network servers.

Using the NTFS file system and its security features.

Using the Encrypting File System (EFS).

Securing network access points.

Enforcing user authentication.

Securing network access.

Enforcing the use of strong passwords.

19
http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/network-security-552http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/ntfs-559http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/network-security-552http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/ntfs-559


20/52

Securing confidential network service data as it moves over the network.

Securing confidential application data as it moves over the network.

Securing confidential user data as it moves over the network.

IPSec is a framework of open standards which can be used for encrypting TCP/ IP traffic within

networking environments. IPSec works by encrypting the information contained in IP datagrams

through encapsulating. This in turn provides network level data integrity, data confidentiality, data

origin authentication, and replay protection. To secure data moving over the intranet, extranet, and

Internet, IPSec can be used. IPSec can also be used to secure remote access connections.

A few security features provided by IPSec are listed here:

Authentication; a digital signature is used to verify the identity of the sender of the information.

IPSec can use Kerberos, a preshared key, or digital certificates for authentication.

Data integrity; a hash algorithm is used to ensure that data is not tampered with. A

checksum called a hash message authentication code (HMAC) is calculated for the data of the

packet.

Data privacy; encryption algorithms are utilized to ensure that data being transmitted is

undecipherable.

Anti-replay; prevents an attacker from resending packets in an attempt to gain access to the

private network.

Nonrepudiation; public key digital signatures are used to prove message origin.

Dynamic rekeying; keys can be created during data sending to protect segments of the

communication with different keys.

Key generation; the Diffie-Hellman key agreement algorithm is used to enable two computers

to exchange a shared encryption key.

IP Packet filtering; the packet filtering capability of IPSec can be used to filter and block

specific types of traffic, based on either of the following elements or on a combination of them:

o IP addresses

o Protocols

o Ports

Considering all the security features provided by IPSec, it makes sense that you need to fist

determine which security methods you need to implement when you deploy IPSec security.

Determining the Encryption Algorithm to use

20
http://www.tech-faq.com/microsoft-ipsec/glossary-1/f/framework-422http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/networking-554http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/intranet-467http://www.tech-faq.com/microsoft-ipsec/glossary-1/k/kerberos-493http://www.tech-faq.com/microsoft-ipsec/glossary-1/a/algorithm-138http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/packet-257http://www.tech-faq.com/microsoft-ipsec/glossary-1/e/encryption-48http://www.tech-faq.com/microsoft-ipsec/glossary-1/f/framework-422http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/networking-554http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/intranet-467http://www.tech-faq.com/microsoft-ipsec/glossary-1/k/kerberos-493http://www.tech-faq.com/microsoft-ipsec/glossary-1/a/algorithm-138http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/packet-257http://www.tech-faq.com/microsoft-ipsec/glossary-1/e/encryption-48


21/52


22/52

Authentication deals with verifying the identity of the computer sending the data, or the identity of the

computer receiving the data. The methods which IPSec can use to authenticate the sender or

receiver of data are:

Digital certificates: Provides the most secure means of authenticating identities. Certificate

authorities (CAs) such as Netscape, Entrust, VeriSign, and Microsoft provide certificates which

can be used for authentication purposes.

Kerberos authentication: A downside of using the Kerberos authentication protocol is that

the identity of the computer remains unencrypted up to the point that the whole payload is

encrypted at authentication.

Preshared keys: You should only use preshared keys when none of the former authentication

methods can be used.

Because preshared keys is considered the least secure supported authentication method, you should

only use preshared keys when you cannot use the digital certificates or the Kerberos v5

authentication protocol. Preshared keys should really only be used in testing environments.

You can define more than one authentication method and then set the order of precedence for the

authentication methods.

IPSec Policies

IPSec Policies Overview

IPSec encrypts data information contained in IP datagrams through encapsulation to provide data

integrity, data confidentiality, data origin authentication, and replay protection. The two main IPSeccomponents that are installed when you install IPSec are the IPSec Policy Agent and the IPSec

driver. The IPSec Policy Agent is a service running on a Windows Server 2003 computer that

accesses IPSec policy information. The IPSec Policy Agent accesses the IPSec policy information in

the local Windows registry or in Active Directory. The IPSec Policy Agent then passes this information

to the IPSec driver. The IPSec driver performs a number of operations to enable secure network

communications such as initiating IKE communication, creating IPSec packets, encrypts data, and

calculates hashes.

IPSec policies are used to apply security in your network. The IPSec policies define when and how

data should be secured. The IPSec policies also determine which security methods to use whensecuring data at the different levels in your network. You can configure IPSec policies so that different

types of traffic are affected by each individual policy.

The different components of an IPSec policyare listed here:

IP filter; informs the IPSec driver on the type of inbound traffic and outbound traffic which

should be secured.

22
http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ike-460http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ike-460


23/52

IP filter list; used to group multiple IP filters into a single list in order to isolate a specific set of

network traffic.

Filter action; used to define how the IPSec driver should secure traffic.

Security method; refers to security types and algorithms used for the key exchange process

and for authentication.

Connection type: identifies the type of connection which the IPSec policy impacts.

Tunnel setting; the tunnel endpoint's IP address/DNS name.

Rule; a grouping of components such as filters and filter actions to secure a specific subset of

traffic in a particular manner:

IPSec policies can be applied at the following levels within a network:

Active Directory domain

Active Directory site

Active Directory organizational unit

Computers

Applications

When you configure and manage IPSec, you would basically be configuring the following aspects of

IPSec policies:

Assign the predefined default IPSec policies.

Create customized IPSec policies that include customized rules and filters.

Control how IPSec policies are applied.

Apply IPSec policies at different levels on the network.

To configure IPSec policies, you can use either of the following methods:

You can use the IP Security Policy Management snap-in to configure IP security policies onthe local computer. To create a new IPSec policy, you have to right-click the IP Security

Policies node in the IP Security Policy Management snap-in, and then click Create IP

Security Policy.

You can use the Group Policy Object Editor snap-in to change local and domain GPOs. To

create a new IPSec policy, you have to right-click the IP Security Policies node in the Group

Policy Object Editor and then click Create IP Security Policy.

23
http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-address-68http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-address-68http://www.topbits.com/what-is-dns.htmlhttp://www.tech-faq.com/microsoft-ipsec/glossary-1/n/node-259http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-address-68http://www.topbits.com/what-is-dns.htmlhttp://www.tech-faq.com/microsoft-ipsec/glossary-1/n/node-259


24/52

The IP Security Policy Management snap-in is used to manage IPSec with respect to:

Create IPSec policies

Edit existing IPSec policies

Assign IPSec policies

Add and remove filters which are applied to IPSec policies.

When you install the IPSec IP Security Policy Management snap-in, you need to select which IPSec

policy you want to manage, and on what network level you want to manage IPSec. You can select

either of the following options:

Manage a local IPSec policy on the computer.

Manage the local IPSec policy a different computer.

Manage the default policy for the domain in which the computer resides.

Manage the default policy for a different domain.

Understanding Default IPSec Policies

Windows Server 2003 IPSec deployments include predefined IPSec rules, filter lists, filter actions,

and three default IPSec policies. Each default IPSec policy contains a set of predefined rules, filter

lists and filter actions.

Each IPSec policy is based on number of rules. An IPSec policy can contain a single rule, or a set of

rules. It is these rules that enable secure connections, based on the following factors:

Source address

Destination address

Type of traffic

An IPSec rule contains the following components:

A filter list.

A filter action.

An authentication method.

A connection type.

A tunnel configuration.

24


25/52

The three default IPSec policies and their predefined configuration are described below:

Client (Respond Only): The Client (Respond Only) default IPSec policy is the least secure

default policy. With this default IPSec policy, the computer assigned the policy never initiates

secure data communication. The computer only responds to IPSec requests from other

computers who request it. The Client (Respond Only) default IPSec policy contains the default

response rule that creates dynamic IPSec filters for inbound and outbound traffic based on the

protocol and port which was requested. The predefined policy settings for the Client

(Respond Only) default IPSec policy are listed here:

o IP Filter List; All

o Filter Action; None

o Authentication; Kerberos

o Tunnel Setting; None

o Connection Type; All

Secure Server (Request Security): With the Secure Server (Request Security) default IPSec

policy, the computer prefers and initiates secure data communication. If the other computer

supports IPSec, secure data communication will take place. If the other computer does not

support IPSec, the computer will allow unsecured communication with that computer. The

Secure Server (Request Security) default IPSec policy contains three rules, and predefined

policy settings:

The predefined policy settings for Rule 1 are:

o IP Filter List; All IP Traffic

o Filter Action; Request Security (Optional)





o IP Filter List; All ICMP Traffic

o Filter Action; Permit

o Authentication; N/a



25
http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248http://www.tech-faq.com/microsoft-ipsec/glossary-1/k/kerberos-493http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/icmp-455http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248http://www.tech-faq.com/microsoft-ipsec/glossary-1/k/kerberos-493http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/icmp-455


26/52


o IP Filter List; Dynamic

o Filter Action; Default Response




Secure Server (Require Security): With the Secure Server (Require Security) default IPSec

policy only secure data communication is allowed. If the other computer does not support

IPSec, the connection is not established. The Secure Server (Require Security) default IPSec

policy contains three rules, and predefined policy settings:


o IP Filter List; All IP Traffic

o Filter Action; Require Security

o Authentication; N/a




o IP Filter List; All ICMP Traffic

o Filter Action; Permit





o IP Filter List; Dynamic

o Filter Action; Default Response



26


27/52


You can also create customized IPSec policies that include customized rules and filters that suit

specific security requirements of the organization. You can also create your own IPSec policy by

using the IP Security Wizard which you can initiate from within the IP Security Policy Management

MMC.

For filter actions, you can select between the filter actions listed below. Remember that the filter

action which is defined determines how IPSec responds to computers matching a filter list, and it

determines which security methods is used:

Permit action (pass through action); used to allow traffic to pass through without applying any

security rules and modifying the traffic. The traffic is simply allowed. Typically used for data

that is considered non-sensitive.

Block action; used to block all traffic.

Allow Unsecured Communication With Non-IPSec Aware Computers; when used unsecuredconnections will be accepted by your computers. Generally recommended that you do not

utilize this option.

Accept Unsecured Communication, But Always Respond Using IPSec action; when used the

computer will always request IPSec before allowing any connections, but it will allow

unsecured connections. Secured connections will though always be requested. This option

therefore allows for secured connections and unsecured connections.

Use These Security Settings action; used to specify custom security methods which should be

applied for connections matching the filter.

How to view default IPSec policies

1. Click Start, click Run, type mmc in the Run dialog box, and then click OK.

2. Click the File Menu item, and select Add/Remove Snap-in.

3. The Add/Remove Snap-in dialog box opens. Click Add.

4. The Add Standalone Snap-In dialog box opens.

5. Select Group Policy Object Editor, and then click Add.

6. Select the Local Computer default option.

7. Click Finish.


27


28/52

9. Click OK to close the Add/Remove Snap-in dialog box.

10.Expand Local Computer Policy, expand Computer Configuration, expand Windows Settings,

expand Security Settings, expand IP Security Policies on Active Directory.

11.The details pane displays the default IPSec policies.

12.Right-click the initial default IPSec policy displayed, which should be the Server (Request

Security) policy, and then click Properties to open the Server (Request Security) default

policy's Properties dialog box.

13.Click the General tab. The configuration settings on the General tab are listed here:

o The name of the policy is listed in the Name text box.

o A description of the policy appears in the Description text box.

o The Check For Policy Changes Every box contains the interval for which clients usingthis specific policy checks for policy updates.

14.Clicking the Settings button on the General tab opens the Key Exchange Settings dialog box.

On the Exchange Settings dialog box you can specify when new keys are generated for the

policy.

15. Clicking the Methods button opens the Key Exchange Security Methods dialog box. You

change the IKE settings and security preference methods on this dialog box. This is where is

you can change encryption, integrity, and Diffie-Hellman Group settings.

16.To close the Key Exchange Security Methods dialog box, click Cancel.

17.To close the Key Exchange Settings dialog box, click Cancel.

18.The Server (Request Security) default IPSec policy's Properties dialog box should be

displayed once more.

19.Click the Rules tab.

20.The three IPSec rules described in this Article earlier are defined on the Rules tab.

21.Each IPSec rule has IP filter list, Filter Action, Authentication, Tunnel Endpoint, and

Connection Type settings.

22.To view the settings of a rule, click the Edit button.

23.The Edit Rule Propertie dialog box opens.

28
http://www.tech-faq.com/microsoft-ipsec/glossary-1/e/encryption-48http://www.tech-faq.com/microsoft-ipsec/glossary-1/e/encryption-48


29/52

24.The Edit Rule Properties dialog box contains the following tabs which you can use to set

configuration settings for the IPSec rule:

o IP Filter List tab; used to add, remove, and configure the filter lists for the rule. All

currently configured filter lists are displayed in the IP Filter Lists list.

o Filter Action tab; used to configure filter actions for the rule. The current filter actions

defined for the rule are listed in the Filter Actions list. The Edit, Add, and Remove

buttons can be used to change, add and remove filter actions for the rule. You can also

specify whether the IP Security Filter Action Wizard should be initiated when a new filter

action is added by enabling the Use Add Wizard checkbox.

o Authentication Methods tab; used to set the authentication method(s) which should be

used for the rule. Options include Kerberos, digital certificates, or preshared keys. If you

define more than one authentication method, you can set the order of precedence for

the authentication methods.

o Tunnel Setting tab; used to configure whether the rule should establish an IPSec tunnelwith another end system.

o Connection Type tab; used to set the connection type for the rule:

All Network Connections option

Local Area Network option.

Remote Access option.

25.To close the Edit Rule Properties dialog box dialog box, click Cancel.

26.To close the Server (Request Security) Properties dialog box of the Default IPSec policy, click

Cancel.

Understanding How IPSec Policy is Applied

Whenever a computer starts, the IPSec Policy Agent service starts automatically too. The IPSec

Policy Agent service running on the computer accesses IPSec policy information in either the

Windows registry or in Active Directory.

The main functions which the IPSec Policy Agent provides are listed below:

The IPSec Policy Agent accesses IPSec policy information from the local Windows registry

when the computer does not belong to a domain.

29


30/52

The IPSec Policy Agent accesses IPSec policy information from the Active Directory when the

computer is a member of a domain.

The IPSec Policy Agent scans IPSec policies for any configuration changes.

The IPSec Policy Agent passes information to the IPSec driver.

IPSec policies are accessed when the computer starts, and at the specific interval defined in the

particular IPSec policy. For computers that belong to a domain in Active Directory but are however

disconnected from the domain, then cached IPSec policy information is used.

As mentioned previously, the IPSec Policy Agent passes information to the IPSec driver. The IPSec

driver performs a number of operations to enable secure network communication. The IPSec driver

checks inbound and outbound packets to determine whether a packet matches criteria for secured

communication. The IPSec driver checks the IP Filter List of the IPSec policy to determine this

information. If a match is found, the IPSec driver uses the filter list and filter actions to determine how

security should be applied.

A few functions performed by the IPSec driver are listed here:

Creates IPSec packets.

Generates checksums.

Initiates the IKE communication.

Adds the AH and ESP headers.

Encrypts data before it is transmitted.

Calculates hashes and checksums for incoming packets.

The IKE protocol is used by computers to create a security association (SA) and to exchange

information to generate Diffie-Hellman keys. IKE manages and exchanges cryptographic keys so that

computers can have a common set of security settings. Negotiation occurs on which authentication

method, and encryption algorithm and hashing algorithm the computers will use. The computers

negotiate and agree on a number of factors, including the following:

Determine whether the Authentication Header (AH) IPSec protocol should be used for the

connection.

Determine whether the Encapsulating Security Payload (ESP) IPSec protocol should be used

for the connection.

The connections that should be authenticated.

The encryption algorithm that should be used.

The algorithm that should be used to verify message integrity.

30
http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/packet-257http://www.tech-faq.com/microsoft-ipsec/glossary-1/a/algorithm-138http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/packet-257http://www.tech-faq.com/microsoft-ipsec/glossary-1/a/algorithm-138


31/52

Understanding How the IPSec driver operates

The IPSec driver operates in the following three modes:

Computer startup mode: When the computer starts, the IPSec driver is loaded and the IPSec

Policy Agent puts the IPSec driver in operational mode.

In the Computer Startup mode, the IPSec driver can operate in any of the following modes:

o Permit; the default mode if there are no IPSec policies defined for the computer. In

Permit mode all traffic is allowed because no packets are filtered.

o Stateful; the default mode if IPSec policy is applied for the computer. In this mode,

outbound traffic is allowed. Unicast, multicast and broadcast inbound packets are

dropped.

o Block; only IP packets which match those filters defined to be used in this mode, and all

DHCP-specific traffic is allowed.

The configuration of the startup type of the IPSec service determines the mode in which the

IPSec driver starts. The IPSec driver can start in one of the following modes:

o Disabled; when the IPSec driver starts in Disabled mode, the following occurs:

The IPSec driver loads in Permit mode.

No packet filtering occurs.

No IPSec security occurs.

o Manual; when the IPSec driver starts in Manual mode, the following occurs:

The IPSec driver loads in Permit mode.



o Automatic; when the IPSec driver starts in Automatic mode, the following occurs:

The IPSec driver loads in the mode which was defined by the IPSec policy agent.

The IPSec driver loads in Stateful mode if there is IPSec policy applied.

The IPSec driver loads in Permit mode if there is no IPSec policy applied.

31
http://www.tech-faq.com/microsoft-ipsec/glossary-1/d/dhcp-372http://www.tech-faq.com/microsoft-ipsec/glossary-1/d/dhcp-372


32/52

Operational mode: After the IPSec service has started, the IPSec driver moves to either of the

following operational modes:

o Secure; when the IPSec driver runs in Secure mode, the following occurs:

If no IPSec policy is assigned, then no IPSec security is applied.

The IPSec policy filters are applied for normal IPSec operations if IPSec policy is

assigned.

IPSec security is applied after persistent policies are applied but before local

policies and Active Directory policies are applied.

If there are no persistent policies, then IPSec security is applied after local

policies and Active Directory policies are applied.

o Permit; the IPSec driver runs in the Permit mode if the IPSec service was manually

stopped on the computer. In Permit mode, the following occurs:



o Block; when the IPSec driver runs in the Block mode, the following occurs:

No inbound traffic is allowed

No outbound traffic is allowed.

Diagnostic mode: used to log inbound and outbound packet drop events when the IPSec driver

runs in Startup mode and Operational mode. You first though have to enable logging because

it is disabled by default. It is strongly recommended that you do not enable logging for a log

time frame because the System log file can become full in a short period of time.

Configuring and Assigning IPSec Policy

You can use the IP Security Policy Management snap-in to manage IPSec policy, create IPSec

policies edit existing IPSec policies, and assign IPSec policies. You can use the tool to addand

remove filters which are applied to IPSec policies. If you are planning a Windows Server 2003 IPSec

implementation, then you have to use the Windows Server 2003 IPSec Policy Management MMC

snap-in if you want to use the latest IPSec features.

You can also configure IPSec using the Netsh command-line utility. The netsh command-line utility

replaces the previously used Ipsecpol.exe command-line utility. The netsh command-line utility can

be used to view information on IPSec policies, configure startup security for computers, and enable

IPSec driver event logging and to troubleshoot IPSec configuration.

You can assign IPSec policy at the following levels within Active Directory. You can though only apply

a single IPSec policy at a specific level in Active Directory:

32


33/52

Domain

Site

Organizational unit (OU)

An IPSec policy that is assigned for a domain in Active Directory has precedence over a locally

applied IPSec policy. With Active Directory, organizational units (OUs) automatically inherit the IPSec

policy of their associated parent OU in Active Directory. IPSec policy assigned for an organizational

unit (OU) has precedence over domain level policies for members of the specific OU. An IPSec policy

that is assigned to the lowest level organizational unit has precedence over an IPSec policy which is

assigned to the higher level organizational units.

How to create an MMC console for the IP Security IP Security PolicyManagement snap-in





5. In the Available Standalone Snap-ins list, select IP Security Policy Management, and then click

the Add button.

6. The Select Computer Or Domain dialog box opens.

7. Click the Local Computer option.

8. Click Finish.


10.Click OK to close the Add/Remove Snap-in dialog box.

How to create a new IPSec policy

1. Open the IP Security Policy Management console.

2. Right-click IP Security Policies and then select Create IP Security Policy from the shortcut

menu.

3. The IP Security Policy Wizard initiates.

4. Click Next on the IP Security Policy Wizard Welcome page.

33


34/52

5. On the IP Security Policy Name page, provide a name and a description for the new IPSec

policy, and then click Next.

6. On the Requests for Secure Communication page, you can leave the Activate the default

response rule option selected, or you can deselect the option. Click Next.

7. On the Default Rule Authentication Method page, set the authentication method for the

security rule, and then click Next.

8. On the Completing the IP Security Policy Wizard page, select the Edit properties option, and

then click Finish.

9. The IP Security Policy Properties dialog box for the new policy opens so that you can change

the properties of the policy, and change any security rules.

10.Click Edit on the IP Security Policy Properties dialog box.

11.When the Edit Rule Properties dialog box opens, you can add and remove security methods,

modify existing security methods, set the order of precedence for security methods, and

specify the utilization of session key perfect forward secrecy (PFS).

12.Click the Authentication tab. This is where you add and remove authentication methods, and

set the order of precedence for authentication methods.

13.Click OK to close the Edit Rule Properties dialog box.

14.Before you assign the IPSec policy, first ensure that the IPSec service is running.

15.In the IP Security Policy Management console, right-click the new policy name that you want to

assign, and then click Assign from the shortcut menu.

How to assign IPSec policy for a Active Directory domain





5. Select Group Policy Object Editor, and then click Add.

6. The Select Group Policy Object dialog box opens. Click Browse

7. The Browse For A Group Policy Object dialog box opens.

8. Select Default Domain Policy, and then click OK.

34


35/52


36/52

11.On the Tunnel Endpoint page, select The Tunnel Endpoint Is Specified By The Following IP

Address option, and then enter the IP address of the other machine. Click Next.

12. On the Network Type page, select the Local Area Network ( LAN) option and then click Next

13.Specify the All IP Traffic option and then click Next.

14.On the Filter Action page, specify the Request Security (Optional) option and then click Next.

15.On the Authentication Method page, specify the Active Directory Default (Kerberos V5

protocol) option and then click Next.

16.Click Finish and then click OK.

17.Repeat the process on the other machine.

Configuring and Managing IPSec

IPSec Review

IPSec is a framework of open standards for encrypting TCP/ IP traffic within networking

environments. IPSec works by encrypting the information contained in IP datagrams through

encapsulating to provide data integrity, data confidentiality, data origin authentication, and replay

protection.

IPSec uses cryptography to provide authentication, data integrity, and data confidentiality services.

Authentication deals with verifying the identity of the computer sending the data, or the identity of the

computer receiving the data. IPSec can use digital certificates, the Kerberos v5 authentication

protocol, or pre-shared keys as an authentication method.Anti-replayensures that the authentication

data cannot be interpreted as it is sent over the network. IPSec can provide non-repudiation. Withnon-repudiation, the sender of the data cannot at a later stage deny actually sending the data. Data

integritydeals with ensuring that the data received at the recipient has not been tampered with. A

hashing algorithm is used to ensure that the data is not modified as it is passed over the network.

The hashing algorithms which can be used by IPSec are Message Digest ( MD5) ad Secure Hash

Algorithm 1 (SHA1). Data confidentialityensures that data is kept private by applying encryption

algorithms to data before it is sent over the network. IPSec uses encryption algorithms such as Data

Encryption Standard (DES), Triple DEC (3DES), or 40-bit DES to provide data confidentiality.

IPSec uses the Authentication Header (AH) protocol and Encapsulating Security Payload (ESP)

protocol to provide data security on client computers, domain servers, corporate workgroups, LANs,WANs and remote offices. TheAuthentication Header (AH) protocolprovides data authentication and

integrity, and can therefore be used on its own when data integrity and authentication are important to

the organization but confidentiality is not. The AH protocol does not provide for encryption, and

therefore cannot provide data confidentiality. The Encapsulating Security Payload (ESP) protocol

ensures data confidentiality through encryption, data integrity, data authentication, and other features

that support optional anti-replay services. To ensure data confidentiality, a number of encryption

algorithms are used. The main difference between the AH protocol and the ESP protocol is that the

36
http://www.tech-faq.com/microsoft-ipsec/glossary-1/l/lan-497http://www.tech-faq.com/microsoft-ipsec/glossary-1/f/framework-422http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/networking-554http://www.tech-faq.com/microsoft-ipsec/glossary-1/k/kerberos-493http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248http://www.tech-faq.com/microsoft-ipsec/glossary-1/a/algorithm-138http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/md5-519http://www.tech-faq.com/microsoft-ipsec/glossary-1/e/encryption-48http://www.tech-faq.com/microsoft-ipsec/glossary-1/l/lan-497http://www.tech-faq.com/microsoft-ipsec/glossary-1/f/framework-422http://www.tech-faq.com/microsoft-ipsec/glossary-1/i/ip-469http://www.tech-faq.com/microsoft-ipsec/glossary-1/n/networking-554http://www.tech-faq.com/microsoft-ipsec/glossary-1/k/kerberos-493http://www.tech-faq.com/microsoft-ipsec/glossary-1/p/protocol-248http://www.tech-faq.com/microsoft-ipsec/glossary-1/a/algorithm-138http://www.tech-faq.com/microsoft-ipsec/glossary-1/m/md5-519http://www.tech-faq.com/microsoft-ipsec/glossary-1/e/encryption-48


37/52

ESP protocol provides all the security services provided by the AH protocol, together with data

confidentiality through encryption.

When you install IPSec, the two main IPSec components which are installed are the IPSec Policy

Agent and the IPSec driver. The IPSec Policy Agentis a service running on a Windows Server 2003

computer that accesses IPSec policy information. The IPSec Policy Agent accesses the IPSec policy

information in the local Windows registry or in Active Directory. The IPSec Policy Agent then passes

information to the IPSec driver. The IPSec driverperforms a number of operations to enable secure

network communications such as initiating IKE communication, creating IPSec packets, encrypts

data, and calculates hashes.

IPSec can operate in either Tunnel mode or in Transport mode. IPSec Tunnelmode should be used

to provide security for WAN and VPN connections that use the Internet. In tunnel mode, IPSec

encrypts the IP header and the IP payload. With tunneling, the data contained in a packet is

encapsulated inside an additional packet. The new packet is then sent over the network. In Transport

Mode, the default mode of operation used by IPSec, only the IP payload is encrypted. Transport

mode is used for end-to-end communication security between two computers on the network.

IPSec policies are used to apply security in your network. The IPSec policies define when and how

data should be secured. The IPSec policies also determine which security methods to use whensecuring data at the different levels inyour network. You can configure IPSec policies so that different

types of traffic are affected by each individual policy. IPSec policies can be applied at the Active

Directory domain level, site level, OU level, and it can be applied on computers and applications. You

can use

ipsec tech - faq

Documents