Logging Cheat Sheet

Introduction

This article is focused on providing developer guidance on building application logging mechanisms. Many systems enable operating system, web server, mail server and database server logging, but often custom application event logging is missing, disabled or poorly configured. Web application (e.g. web site or web service) logging is much more than having web server logs enabled.

Purpose

Application logging should be always be included for security events. The logs are invaluable data to help investigate errors and unusual conditions, to provide additional application-specific data for incident investigation which is lacking in other log sources, and to help defend against vulnerability identification and exposure by hackers.

Application logging might also be used to record events for:

• business process monitoring (e.g sales process abandonment)
• audit trails (data addition, modification and deletion)
• performance monitoring (e.g. data load time, page timeouts)
• compliance monitoring
• subsequent requests for information (e.g. subject access, freedom of information, litigation, police and other regulatory investigations)
• other business-specific requirements

But there should always be some level of security event logging, regardless of any other type.

Design, implementation and testing

What to log

The level of security monitoring, alerting and reporting needs to be set during the requirements and design stage of projects, and should be proportional to the information security risks. This can then be used to define what should be logged. There is no one size fits all solution, and a blind checklist approach can lead to un-necessary "alarm fog" that means real problems go undetected.

Always log:

• General
  • All input validation failures
  • All server-side error and system events
    • code errors
    • database access and performance
    • web server errors
    • third party services
    • lack of storage space
• Authentication, authorisation and session management
  • All authentication successes and failures
  • All authorisation failures
• Use of higher-risk functionality
  • Administrative tasks
  • Data import and export (including screen-based reports)

Optionally consider if the following can be logged and is desirable information:

• Excessive use
• Fraud and other criminal activities
• Suspicious, unacceptable or unexpected behaviour
• Modifications to configuration

Data to exclude

The following must not be recorded directly in the logs, but should be removed, masked, sanitized or encrypted:

• Session identification cookie values
• Sensitive personal data and some forms of personally identifiable information (PII)
• Authentication passwords
• Database connection strings
• Bank account or payment cardholder data
• Data of a higher security classification than the logging system is allowed to store
• Commercially-sensitive information

Sometimes the following data can exist, and whilst useful for subsequent investigation, it may also need to be treated in some manner before the event is recorded:

• File paths
• Database connection strings
• Internal network names and addresses
• Non sensitive personal data (e.g. personal names, telephone numbers, email addresses)

Where to log

Applications most commonly write event log data to the file system or a database (SQL or NoSQL). Applications installed on desktops and on mobile devices may used local storage and databases. All types of applications may also, or instead of writing data locally, send event data to remote systems. These may be a centralised log collection and management system (e.g. SIEM or SEM) or an application on trusted systems.

• When using the file system, it is preferable to use a separate partition to the operating system, other application files and user generated content.
  • On file-based logs, apply strict permissions on which users can access the directories, and the permisisons of files within the directories
  • On a web-based application, the logs must never be exposed in web-accessible locations
• When using a database, it is preferable to use a separate database account only used for writing log data which has very restrictive database , table, function and command permissions.
• Use standard formats over secure protocols to send event data, or log files, to other systems

Consider separate files/tables for extended event information such as error stack traces or a record of HTTP request and response headers and bodies.

Event data sources

The application itself has access to a wide range of information events that should be used to generate log entries. Thus, the primary event data source is the application code itself. Other sources that could also be considered are:

• Client software e.g. actions on desktop software and mobile devices in local logs or using messaging technologies, web browser such as using Content Security Policy (CSP) reporting mechanism
• Network firewalls
• Network and host intrusion detection systems (NIDS and HIDS)
• Application firewalls e.g. XML gateways, web application firewalls (WAFs)
• Database servers e.g. automatic audit trails, trigger-based actions
• Reputation monitoring services (e.g. uptime or malware monitoring)
• Related applications

The degree of confidence in the information has to be considered when including event data from systems in a different trust zone. Data may be missing, modified, forged and could be malicious – it must always be treated as untrusted data.

Note that the application itself has the most information about the user (e.g. identity, roles, permissions) and the context of the event (target, action, outcome), and often this data is not available to network devices.

Event data properties

Each log entry nneds to include sufficient information for the intended subsequent monitoring and analysis.

The application logs must record:

• Log date and time
• Event date and time (may be different to above)
• Application identifier (e.g. name)
• Application address (e.g. host name and domain, or server IPv4 or IPv6 address, or local device identifier)
• Service (e.g. name, port and protocol)
• User identity (if authenticated or otherwise known)
• Window/form/page (e.g. entry point URL and HTTP method for a web application)
• Source address (e.g. user's device/machine identifier, user's IP address)
• Type of event
• Description
• Request HTTP headers (web applications only)
• Analytical confidence in the event detection (low, medium, high or a numeric value).
• Severity of event (e.g. 0=emergency, 1=alert, ..., 7=debug)
• Rsponses seen by the user and/or taken by the application (e.g. status code, custom text messages, session termination, administrator alerts)

unauthorised file and configuration change monitoring,

Customisable logging

It may be desirable to be able to alter the level of logging (type of events based on severity or threat level, amount of detail recorded). If this is implemented, ensure that:

• The default level must provide sufficient detail for the business's needs.
• It should not be possible to completely inactivate application logging or logging of events that are necessary for compliance requirements.
• Alterations to the level of logging must be intrinsic to the application (e.g. undertaken automatically by the application based on an algorithm) or follow change management processes (e.g. changes to configuration data)
• The logging level must be verified periodically

Denial of service

Ensure logging cannot be used to deplete system resources by for example filling up disk space or exceeding database transaction log space.

Collection:

Perform input validation on event data from other trust zones to ensure it is in the correct format (and consider alerting and not logging if there is an input validation failure) Perform sanitization on all event data to prevent log injection attacks Use parameterised interfaces (e.g. prepared statements for SQL) to send Synchroinize time across servers and devices. This is not always possible where the application is running on a device under some other party's control (e.g. on an individual's mobile phone, on a customer's workstation on their own corporate network). In these cases attempt to measure the time offset, or record a confidence level in the event timestamp.

Time of event may be different from time of logging (e.g. server logging where the client application is hosted on remote device that are only periodically or intermittently online).

Testing

Ensure the logging is working correctly and as psecified during application testing processes.

Deployment and operation

Add information about the logging mechanisms to release documentation Provide security configuration information for the logging mechanisms

Protection

The logging systems must be protected from mis-use such as tampering in transit, and unauthorised access, modification and deletion once collected.

The extraction and examination of audit trail data must be controlled:

   Logs may contain personal and other sensitive information.
   The data may give away information regarding the application's code and logic.
   Consider whether parts of the data may need to be excluded, masked, sanitised or encrypted.
   The privileges to read audit trails should be restricted and reviewed periodically.
   All access to the audit trail must be recorded and monitored by a separate manager.
   

At rest:

Build in tamper detection so you know if a record has been modified or deleted. Store, or copy, log data to read-only media as soon as possible.

In transit:

If log data is sent over untrusted networks (e.g. for collection or for dispatch elsewhere), use a secure transmission protocol. Consider whether the origin of the event data needs to be verified.

Monitoring

The logging then needs to be tied in with appropriate monitoring, alerting and reporting.

• Ensure incident response team is aware of the application logging mechanisms.
• If possible, aggregate the application log data with centralized logging and analysis sytems (e.g. SIEM).

Disposal

Log data must not be destroyed before the duration of its required data retention period, and not kept beyond this time.

Related articles

Arcsight Common Event Format (CEF)

IEEE Security & Privacy Journal How to Do Application Logging Right, Anton Chuvakin and Gunnar Peterson

IETF syslog protocol

Mitre Common Event Expression (CEE)

NIST SP 800-92 Guide to Computer Security Log Management

OWASP ESAPI Documentation

OWASP Logging Project

W3C Extended Log File Format

Authors and primary contributors

Colin Watson - colin.watson[at]owasp.org

OWASP Cheat Sheets Project Homepage

• OWASP Cheat Sheet Series

OWASP_Defenders