best practices - logging

7/28/2019 Best Practices - Logging

1/16

TechNet Home > Products & Technologies > Servers > ISA Server TechCenter Home > ISA Server 2004 >Technical Library > Configuration and Administration

Best Practices: LoggingPublished: December 9, 2005

On This Page

Introduction

This guide is designed to provide you with essential information about logging for Microsoft Internet Security

and Acceleration (ISA) Server 2004 Standard Edition and ISA Server 2004 Enterprise Edition. The guide

reviews the logging formats, describes specific logging maintenance considerations, details capacity

guidelines, and outlines special considerations when logging to a Microsoft SQL Server database.

This guide focuses explicitly on best practices to follow when configuring logging as part of your ISA Server

deployment. You should use this guide as part of your overall deployment strategy for ISA Server 2004.

Specifically, this guide provides detailed answers to the following questions:

Top of page

Log Storage Format

You can use the ISA Server 2004 log viewer to monitor and analyze traffic and troubleshoot network activity.

The log viewer can display log entries as they occur (live). In this case, each time an event is logged, it is

displayed in the log viewer.

ISA Server creates the following logs:

The fields that can be logged in these files are detailed in online Help.

ISA Server log information can be viewed in a log viewer, directly from ISA Server Management. In addition,

the log information can be stored in one of the following formats:

Download

Get Office File Viewers

Logging-best-practices.doc

487 KBMicrosoft Word file

Introduction

Log Storage Format

General Logging Best Practices

SQL Logging

MSDE Logging

Additional Information

What is the most appropriate logging format for my specific deployment?

How should I optimally secure the logs?

How should I maintain the logs?

What special considerations are there when logging to an SQL database?

What special considerations are there when logging to a Microsoft SQL Server 2000 Desktop Engine (MSDE2000) database?

Firewall log

Web Proxy log

SMTP Message Screener log

File

Page 1 of 16Microsoft TechNet: Best Practices: Logging

22. 1. 2007http://www.microsoft.com/technet/isa/2004/plan/logging-best-practices.mspx?pf=true


2/16

Selecting Log Format

Each log format supported by ISA Server features different advantages. Use the table that follows to select

the optimal log format, based on your specific deployment.

File

You can save ISA Server logs to a file, in one of the following formats:

MSDE database

SQL database

Issues File MSDE SQL

Format Two modes: Internet

Information Services (IIS)

and World Wide Web

Consortium (W3C)

standardized text formats

Format used to store

Firewall and Web

Proxy log entries

Format used to store Firewall and

Web Proxy log entries

Network

bandwidth

consumption

Because logging is local,

no network bandwidth

consumption

Because logging is

local, no network

bandwidth

consumption

Because logging is to remote

server, sufficient network

bandwidth is required, preferably

1 gigabyte (GB) connectivity

between ISA Server and

computers running SQL Server

Log size Limited to 2 GB and

switched automatically

Limited to 1.5 GB

and switched

automatically

No limit, and configured by the

user, based on retention and

maintenance policy

Maintenance Log maintenance feature

enforces log size and

cleans out log, as

appropriate

Log maintenance

feature enforces log

size and cleans out

log, as appropriate

Database administrator

responsible for maintenance

Security Log failure stops Firewall

service

Log failure stops

Firewall service

MSDE runs on theISA Server computer

MSDE instance can

only be accessed

locally

Log failure stops Firewall service

Account used for logging must

have permissions on the computerrunning SQL Server

Data is encrypted on the

connection to the computer

running SQL Server

SQL Server and ISA Server are

mutually authenticated

Historical or

offline log viewer

Not supported Supported Supported (ISA Server Enterprise

Edition only)

Online log viewer Supported Supported Supported

Performance Best Good Depends on the following:

Number of ISA Server computers

logging

SQL Server settings

Bandwidth allocation

Centralized

logging (ISA

Server Enterprise

Edition only)

Central log for all array

members

Central log for all

array members

Central log for all arrays in the

enterprise

World Wide Web Consortium (W3C) format




3/16

The SMTP Message Screener log information is saved by default in file format. It cannot be saved to a

database.

Log files are limited to 2 GB. When a file exceeds this limit, ISA Server automatically creates a new file.

Similarly, a new log file is created at the beginning of every day.

W3C logs contain both data and directives, describing the version, date, and logged fields. Because the fieldsare described in the file, unselected fields are not logged. The tab character is used as a delimiter. Date and

time are in Coordinated Universal Time (UTC).

ISA Server format contains only data with no directives. All fields are always logged. Unselected fields are

logged with a dash, to indicate that they are empty. The comma character is used as a delimiter. The date

and time fields are in local time.

By default, the log information for log files is stored in the ISALogs folder, under the ISA Server installation

folder. You can change the location. If you specify a relative directory, the log is saved in the ISALogs folder,

under the ISA Server installation folder. If you specify an absolute path, the actual log folder may be

different on every server.

MSDE DatabaseMSDE 2000 logs are limited to 2 GB. When a log exceeds this limit, ISA Server automatically creates a new

database. Similarly, a new log is created at the beginning of every day. The log viewer, however, displays all

the data as if it were in a single database.

When you select to save the logs to an MSDE 2000 database, logs are saved in databases named

ISALOG_yyyymmdd_xxx_nnn where:

For each log database, two files are created: ISALOG_yyyymmdd_xxx_nnn.mdf and

ISALOG_yyyymmdd_xxx_nnn.ldf.

ISA Server prepares, in advance, log databases for the next day. When you save logs to MSDE 2000, a

database that refers to the next day always exists.

By default, the log information for MSDE 2000 logs and log files is stored in the ISALogs folder, under the

ISA Server installation folder. You can change the location. If you specify a relative directory, the log is

saved in the ISALogs folder, under the ISA Server installation folder. If you specify an absolute path, the

actual log folder may be different on every server.

SQL Database

You can save log information to an SQL database. Saving the log information to an SQL database is useful

for remote logging.

When you configure logging to an SQL database, you specify the database connection parameters, and

credential information.

The system policy rule named Allow remote logging using NetBIOS transport to trusted servers must

be enabled to log to an SQL database.

Important

ISA Server format

yyyyrepresents the year that the log database refers to.

mm represents the month that the log database refers to.

ddrepresents the day that the log database refers to.

xxxrepresents the type that the log database refers to. This can be one of the following:

FWS. Represents the Firewall log.

WEB. Represents the Web Proxy log.

EML. Represents the e-mail (SMTP) log.

nnn is a number that distinguishes between log databases that refer to the same day.




4/16

For maximum security and functionality, we strongly recommend consulting with a SQL Server database

administrator when using SQL logging.

Top of page

General Logging Best Practices

This section details recommended general best practices to follow when using the ISA Server logs. It also

describes techniques to implement in case of log failure or connectivity failure.

General Security Best Practices

Follow these guidelines to help secure ISA Server logs:

General Capacity Planning Guidelines

Regardless of log format, we recommend that you allocate 8 GB for logging. Depending on your specific

logging capacity, in addition to the 8 GB, we recommend that you further allocate enough space for an

additional day and a half of logging. The amount of space required for a day and a half of logging depends on

your specific logging requirements.

Log Failure

Because ISA Server is deployed to secure your network, it is critical that logging information is always

available and accurate. You should carefully monitor alerts and verify that their activity is always being

logged. Check for alerts that indicate failure to log for a variety of reasons, including disk space, SQL Server

connectivity issues, and others.

If log information cannot be saved for any reason, the ISA Server computer should be locked down. For this

reason, a preconfigured alert for the Log Failure event stops the Microsoft Firewall service.

By default, if ISA Server cannot log activity, the Microsoft Firewall service is stopped. You can change the

default behavior by configuring the log failure alert to not stop ISA Server services.

If the ISA Server computer fails, the last log records may be lost.

Maintaining Logs

After you select and configure a specific logging mechanism, follow the best practices listed in this section to

maintain the logs.

Reviewing Logs

Review the logs regularly and carefully, checking for suspicious access and usage of network resources.

Log Maintenance

ISA Server has a log maintenance feature, which you can configure so that log files do not exceed specific

space requirements. Use the log maintenance feature to ensure that the disk on which log information is

stored does not become full.

When you log to an MSDE 2000 database or to a file, you can configure how long log information should be

stored on the local disk, and how much disk space should be allocated for logging.

Note

To configure the Log Storage Limits alert definition to stop the ISA Server services, perform the

following steps

Save the logs to a separate NTFS disk partition for maximum security. Only administrators of the ISAServer computer should have access to the logs.

If you are logging the information to a remote database, configure encryption and data signature for thelog information being copied to the remote database.

You cannot set log limits for SQL database logs.

ISA Server checks every ten minutes that logs do not exceed the specified limits. For up to a period of tenminutes, logs might exceed the limits.

The log maintenance feature does not apply to the SMTP filter log.




5/16

1. In the console tree ofISA Server Management, click Monitoring:

For ISA Server 2004 Enterprise Edition, expand Microsoft Internet Security and Acceleration Server2004, expand Arrays, expandArray_Name, and then click Monitoring.

For ISA Server 2004 Standard Edition, expand Microsoft Internet Security and Acceleration Server2004, expand Server_Name, and then click Monitoring.

2. In the details pane, click the Alerts tab.

3. On the Tasks tab, click Configure Alert Definitions.

4. In Alert Definitions, click Log storage limits, and then click Edit.

5. On the General tab, select Enable.

6. On the Actions tab, click Stop selected services, and then click Select.




6/16

7. In Services, select Microsoft Firewall and Microsoft ISA Server Job Scheduler.

To configure log storage limits, perform the following steps

1. In the console tree of ISA Server Management, click Monitoring:

For ISA Server Enterprise Edition, expand Microsoft Internet Security and Acceleration Server2004, expandArrays, expandArray_Name, and then click Monitoring.

For ISA Server Standard Edition, expand Microsoft Internet Security and Acceleration Server 2004,expand Server_Name, and then click Monitoring.

2. In the details pane, click the Logging tab.

3. On the Tasks tab, select the appropriate task:

Configure Firewall Logging. Used to configure the firewall log limits.

Configure Web Proxy Logging. Used to configure the Web Proxy log limits.

4. On the Log tab, select File or MSDE Database, and then click the Options button.




7/16

5. To limit the size of the logs, select Limit total size of log files (GB). Then, type the maximum log

size.

6. To maintain a specified amount of free disk space on the disk where the logs are stored, select

Maintain free disk space (MB). Then, type how much free disk space to maintain.




8/16

7. If you selected either Limit total size of log files (GB) or Maintain free disk space (MB), select one of

the following:

Deleting older log files as necessary. Used to delete the oldest log files when you exceed thelimits specified previously.

Discarding new log entries. Used to stop ISA Server from adding any new log entries (while

keeping all the old log information).




9/16

Configuring Logs During Flood Attacks

A flood attack occurs when an attempt is made to deny services to legitimate users by intentionally

overloading a network. Flood attacks might occur, for example, when a worm tries to propagate outside of

your corporate network.

The first symptoms that show that ISA Server is experiencing a flood attack are a sudden surge in CPU

utilization, increased memory consumption, or very high logging rates on the ISA Server computer.

If you determine that the ISA Server computer is experiencing a flood attack, use the log viewer to

determine the source of the offending traffic. Specifically, look for the following:

Another way to detect and list the offending computers is to temporarily reconfigure the Connection Limit

alerts to be triggered every one second (instead of using the Manually Reset option). A list of alerts is

generated, each one indicating the offending IP address in the alert text. After you identify the list of

offending IP addresses, perform the procedure to log requests matching a rule (described later in this

document), to improve the performance of ISA Server during the flood.

When an attack occurs, many events will be logged to the computer running SQL Server. To continue logging

8. To delete old log information, select Delete files older than (days). Then, type how long to keep

log information.

Log entries for denied traffic. Pay special attention to traffic that is denied because the quota isexceeded, spoofed packets, and packets with corrupted CHECKSUM. These usually are indicative of amalicious client. In ISA Server 2004 Standard Edition, connections that are terminated due to exceeding

the connections limit will have a result code of 0x80074e23. In ISA Server 2004 Enterprise Edition, the

result will appear as text, which clearly indicates the connection termination reason.

Logs that indicate numerous connections that are created and then immediately closed. Thisoften indicates that a client computer is scanning an Internet Protocol (IP) address range for a specific

vulnerability.




10/16

despite the large number of events, follow these guidelines:

To log requests matching a rule, perform the following steps

By default, the log failure alert is configured to stop the Microsoft Firewall service when it is generated.Consider reconfiguring this alert to send an e-mail message to an administrator's e-mail address. Also, use

the ISA Server software development kit (SDK) to create a script that does not drop connections for which

traffic is not logged. For example, you can use the script located at the Coding Corner. For more

information about using these properties, see ISA Server SDK Help, available in the SDK folder on the ISA

Server CD.

Do not log traffic that is denied by the Default rule.

Disable logging either on the specific rule that matches the flood or altogether until the flood attack isstopped.

For example, if a large amount of data is being logged from a specific protocol or source, you can create anew rule, which applies to that type of traffic, for which requests are not logged. For example, suppose

your policy does not allow Dynamic Host Configuration Protocol (DHCP) requests, and as a result, there

are many DHCP requests that are being denied. You can create a new access rule that denies DHCP

requests, but does not log the requests.

Reconfigure the Connections Limit alerts (or any other types of alerts that may be triggered repeatedlyas a result of the specific attack) back to manually reset.

1. In the console tree of ISA Server Management, click Firewall Policy or Enterprise Policies (for

enterprise-level requests in Enterprise Edition):

For ISA Server 2004 Enterprise Edition, for a specific enterprise policy, expand Microsoft InternetSecurity and Acceleration Server 2004, expand Enterprise, expand Enterprise Policies, and

then click Enterprise_Policy.

For ISA Server 2004 Enterprise Edition, for array-level firewall policy, expand Microsoft InternetSecurity and Acceleration Server 2004, expand Arrays, expand Array_Name, and then click

Firewall Policy.

For ISA Server 2004 Standard Edition, expand Microsoft Internet Security and AccelerationServer 2004, expand Server_Name, and then click Firewall Policy.

2. In the details pane, click the rule for which logging should be enabled.

3. On the Tasks tab, click Edit Selected Rule.

4. On the Action tab, select the Log requests matching this rule check box.




11/16

Note

By disabling logging for a specific rule, you effectively reduce the load on the ISA Server computer if it is

under attack. However, note that if you disable logging on the default deny rule, ISA Server cannot detect

port scan attacks.

Connectivity Failure

Network issues, such as floods or congestion, may cause connectivity failure between the ISA Server

computer and the logging server. Such connectivity issues will cause ISA Server to enter lockdown mode. To

avoid such issues, do the following:

SQL Data and Transaction Log Location

Best practices on databases recommend separating the physical drives to be used for the data file from the

transaction log file. Doing so will improve the overall performance of the SQL-based logs.

Top of page

SQL Logging

This section describes specific guidelines for SQL logging.

Security Best Practices for Logging to SQL Server

Follow these guidelines to help secure ISA Server logs:

Use a private network between the ISA Server computer and the logging server.

Protect the logging servers from receiving traffic from untrusted sources, by allowing them to receivetraffic only from ISA Server computers and arrays.

For optimal security, configure Internet Protocol security (IPsec) for the communication between the ISAServer computer and the logging server.

When you save log information to an SQL database, use Windows authentication (and not SQL




12/16

Specifically, limit the privileges allowed to the SQL account that has access to the ISA Server log tables. Even

if the ISA Server firewall is compromised, the malicious attacker cannot delete old log entries. Additionally,allow use of the SELECT and INSERT SQL commands only when the designated SQL account is set for ISA

Server logging purposes.

Connectivity Failure

Network issues, such as floods or congestion, may cause connectivity failure between the ISA Server

computer and the logging server. Such connectivity issues will cause ISA Server to enter lockdown mode. To

avoid such issues, do the following:

The Remote Logging (SQL) system policy configuration group must be enabled to log to an SQL database.

Capacity Planning for SQL Logs

We recommend that if you are using an SQL database, you ensure that there is sufficient bandwidth

available from the ISA Server arrays to the computer running SQL Server. We also recommend that the

computer running SQL Server is configured to handle simultaneous, large requests.

If the computer running SQL Server might not have the necessary capacity to handle large requests, do one

of the following:

The following minimal network connectivity is required between the ISA Server firewall and the computer

running SQL Server:

Remote SQL Logging

You can use remote SQL logging to log all records to a centrally managed SQL database. As compared to

MSDE and file logging, Remote SQL logging consumes CPU resources somewhere in between those used by

MSDE and file logging, and practically uses no disk I/O. However, remote SQL logging introduces other

capacity requirements that must be taken into account, because all log records are written to a central

remote database:

The following table provides an estimate of the transaction rate and the bandwidth required for logging for

the Internet link bandwidths. The table shows megabits per second (Mbps) and kilobits per second (Kbps).

authentication).

Save the logs to a separate NTFS disk partition for maximum security. Only administrators of the ISAServer computer should have access to the logs.

Configure appropriate security when logging to an SQL database.

Use a private network between the ISA Server computer and the logging server.

Configure IPsec for the communication between the ISA Server computer and the logging server.

Deploy the computer running SQL Server in a separate network and configure a special firewall policy

allowing traffic only between the SQL Server network and ISA Server 2004.

Use local MSDE logging, rather than centralized SQL logging.

Use centralized SQL logging, but do not generate daily summary reports. Instead, generate reportsdirectly from the SQL database using SQL Reporting Services.

100 megabits for up to three array members

1 gigabit for four or more array members

Network connections between ISA Server and the remote SQL database must dedicate a gigabit bandwidthto accommodate the capacity of the log traffic.

Network connections between ISA Server and the remote SQL database must utilize IPsec to secure thelog records when sent to the remote SQL database.

Sufficient redundant array of independent disks (RAID) hardware must be available to support the loggingrate of several ISA Server computers.

Internet link bandwidth 1 Mbps 5 T1 (7.5 Mbps) 25 Mbps T3 (45 Mbps)




13/16

For larger bandwidths, the numbers in the table can be extrapolated linearly.

Unlike the MSDE logs, an SQL database is a central log for all of the members of various ISA Server arrays inan enterprise. ISA Server array members are preconfigured to generate the daily summary reports at 00:30

(12:30 A.M.). In a scenario with many array members, the simultaneous requests for gigabyte-sized logs

from the computer running SQL Server will generate heavy network traffic and a significant load on the

computer. The daily summary generation time can be changed, so you could stagger the summary

generation time from server to server. We recommend that you configure the time of the summary

generation when ISA Server is not busy with other tasks (for example, late at night or early in the morning).

To configure time of daily summary generation, perform the following steps

SQL transactions/sec 25 188 625 1125

SQL transaction bandwidth 92 Kbps 700 Kbps 2.3 Mbps 4.2 Mbps


For ISA Server 2004 Enterprise Edition, expand Microsoft Internet Security and Acceleration Server2004, expand Arrays, expandArray_Name, and then click Monitoring.

For ISA Server 2004 Standard Edition, expand Microsoft Internet Security and Acceleration Server2004, expand Server_Name, and then click Monitoring.

In the details pane, click the Reports tab.

On the Tasks tab, do one of the following:

For ISA Server 2004 Enterprise Edition, click Configure Log Summary and ReportPreferences.

For ISA Server 2004 Standard Edition, click Configure Log Summary.

2. On the Log Summary tab, select the Enable daily and monthly summaries check box.




14/16

Special Considerations for SQL Logging

If you select to log to an SQL database, note the following:

Data Encryption for the SQL Log

By default, ISA Server uses a Secure Sockets Layer (SSL)-encrypted connection to the computer running

SQL Server, to help secure the sensitive data in the log files. You can configure whether the connection

should be SSL-encrypted.

If you are logging the information to a remote database, we recommend that you configure encryption anddata signature for the log information being copied to the remote database.

To use data encryption when connecting to an SQL database, perform the following steps

3. In Specify the generation time, type the time of day that the report data should be generated.

Microsoft SQL Server with Service Pack 3 (SP3) must be installed. SQL Server SP3 is available at theMicrosoft Download Center.

We recommend that you do not configure the autogrow and autoshrink parameters for the SQL database.Follow the guidelines stipulated in INF: Considerations for Autogrow and Autoshrink Configuration at the

Microsoft Help and Support Web site.

If you are logging the information to a remote database, configure encryption and data signature for thelog information being copied to the remote database.


For ISA Server Enterprise Edition, expand Microsoft Internet Security and Acceleration Server2004, expandArrays, expandArray_Name, and then click Monitoring.

For ISA Server Standard Edition, expand Microsoft Internet Security and Acceleration Server2004, expand Server_Name, and then click Monitoring.

2. In the details pane, click the Logging tab.

3. On the Tasks tab, select the appropriate task:

ConfigureFirewall Logging. Used to configure the firewall log.

Configure Web Proxy Logging. Used to configure the Web Proxy log.

4. On the Log tab, select SQL Database.

5. Click Options.

6. Select Force data encryption.




15/16

Note

If you configure encryption when logging to an SQL database, you must install a certificate on the computer

running SQL Server. Then, update the trusted root authority on each array member to trust the server

certificate.

We recommend that you use Windows authentication (and not SQL authentication).

Top of page

MSDE Logging

This section describes specific guidelines for MSDE logging.

Capacity Planning for MSDE Logs

MSDE uses more system resources than file logging. Specifically, you can expect an overall 10 to 20 percent

improvement in processor utilization when switching to file logging from MSDE.

MSDE logging also consumes more disk storage resources. MSDE logging performs about two disk accesses

on every megabit. File logging will require the same amount of disk accesses for 10 megabits. One way to

improve ISA Server performance is to switch from MSDE to file logging. This is recommended only when

there is a performance problem caused by saturated processor or disk access.

Compressed Drives

Compressed drives cause severe database performance degradation and disk fragmentation. This can slow

MSDE performance by 500 percent or more, potentially causing the ISA Server firewall to lock down when

experiencing above normal traffic.

Top of page

Additional Information

Additional ISA Server 2004 documents are available at the ISA Server 2004 Guidance page.

Do you have comments about this document? Send feedback.

Top of page

Manage Your Profile




16/16

best practices - logging

Documents