Revision as of 12:35, 23 April 2012

DRAFT CHEAT SHEET - WORK IN PROGRESS

Introduction

This article is focused on providing developer guidance on building application logging mechanisms. Many systems enable operating system, web server, mail server and database server logging, but often custom application event logging is missing, disabled or poorly configured. It provides much greater insight than infrastructure logging alone. Web application (e.g. web site or web service) logging is much more than having web server logs enabled.

Application logging should be consistent within the application, consistent across an organization's application portfolio and use industry standards where relevent, so the logged event data can be consumed, correlated, analyzed and managed by a wide variety of systems.

Purpose

Application logging should be always be included for security events. Application logs are invaluable data for identifying security incidents, monitoring policy violations, establishing baselines, providing information about problems and unusual conditions, contributing additional application-specific data for incident investigation which is lacking in other log sources, and helping defend against vulnerability identification and exploitation through attack detection. Application logging might also be used to record other types of events too. Thus application logging could include:

• Security events
• Business process monitoring e.g. sales process abandonment
• Audit trails e.g. data addition, modification and deletion
• Performance monitoring e.g. data load time, page timeouts
• Compliance monitoring
• Subsequent requests for information e.g. subject access, freedom of information, litigation, police and other regulatory investigations
• Other business-specific requirements

Design, implementation and testing

Event data sources

The application itself has access to a wide range of information events that should be used to generate log entries. Thus, the primary event data source is the application code itself. Other sources that could also be considered are:

• Client software e.g. actions on desktop software and mobile devices in local logs or using messaging technologies, web browser such as using Content Security Policy (CSP) reporting mechanism
• Network firewalls
• Network and host intrusion detection systems (NIDS and HIDS)
• Application firewalls e.g. filters, guards, XML gateways, web application firewalls (WAFs)
• Database servers e.g. automatic audit trails, trigger-based actions
• Reputation monitoring services e.g. uptime or malware monitoring
• Related applications e.g. fraud monitoring, CRM

The degree of confidence in the event information has to be considered when including event data from systems in a different trust zone. Data may be missing, modified, forged and could be malicious – it must always be treated as untrusted data.

Note that the application itself has the most information about the user (e.g. identity, roles, permissions) and the context of the event (target, action, outcome), and often this data is not available to infrastructure devices.

Where to log

Applications most commonly write event log data to the file system or a database (SQL or NoSQL). Applications installed on desktops and on mobile devices may use local storage and local databases. All types of applications may send event data to remote systems (instead of or as well as more local storage). This could be a centralized log collection and management system (e.g. SIEM or SEM) or another application elsewhere.

• When using the file system, it is preferable to use a separate partition than those used by the operating system, other application files and user generated content.
  • For file-based logs, apply strict permissions concerning which users can access the directories, and the permisisons of files within the directories
  • In web applications, the logs must never be exposed in web-accessible locations
• When using a database, it is preferable to utilize a separate database account that is only used for writing log data and which has very restrictive database , table, function and command permissions.
• Use standard formats over secure protocols to send event data, or log files, to other systems

Consider separate files/tables for extended event information such as error stack traces or a record of HTTP request and response headers and bodies.

Which events to log

The level and content of security monitoring, alerting and reporting needs to be set during the requirements and design stage of projects, and should be proportional to the information security risks. This can then be used to define what should be logged. There is no one size fits all solution, and a blind checklist approach can lead to unnecessary "alarm fog" that means real problems go undetected. Always log:

• Input validation failures e.g. protocol violations, unacceptable encodings, invalid parameter names and values
• Output validation failures e.g. database record set mismatch, invalid data encoding
• Authentication successes and failures
• Authorization failures
• Session management failures e.g. cookie session identification value modification
• Application errors and system events e.g. code errors, connectivity, performance, third party services, file system, file upload virus detection
• Start-ups and shut-downs
• Use of higher-risk functionality e.g. changes to privileges, submission of user-generated content - especially file uploads, administrative tasks, data import and export, including screen-based reports

Optionally consider if the following events can be logged and whether it is desirable information:

• Sequencing failure
• Excessive use
• Data changes
• Fraud and other criminal activities
• Suspicious, unacceptable or unexpected behaviour
• Modifications to configuration
• Application code file and/or memory changes

Event properties

Each log entry needs to include sufficient information for the intended subsequent monitoring and analysis. The application logs must record:

• Log date and time
• Event date and time - the event timestamp may be different to the time of logging e.g. server logging where the client application is hosted on remote device that is only periodically or intermittently online
• Application identifier e.g. name
• Application address e.g. host name and domain, or server IPv4 or IPv6 address, or local device identifier
• Service e.g. name, port and protocol
• Window/form/page e.g. entry point URL and HTTP method for a web application
• Interaction identifier §
• Source address e.g. user's device/machine identifier, user's IP address
• User identity (if authenticated or otherwise known)
• Type of event %
• Description
• Request HTTP headers (web applications only)
• Analytical confidence in the event detection % e.g. low, medium, high or a numeric value
• Severity of event % e.g. 0=emergency, 1=alert, ..., 7=debug
• Responses seen by the user and/or taken by the application e.g. status code, custom text messages, session termination, administrator alerts

§ The "Interaction identifier" is a method of linking all (relevant) events for a single user interaction (e.g. desktop application form submission, web page request, mobile app button click, web service call). The application knows all these events relate to the same interaction, and this should be recorded instead of losing the information and forcing subsequent correlation techniques to re-construct the separate events. For example a single SOAP request may have multiple input validation failures and they may span a small range of times. As another example, an output validation failure may occur much later than the input submission for a long-running "saga request" submitted by the application to a database server.

% Each organisation should ensure it has a consistent, and documented, approach to classification of events (type, confidence, severity), the syntax of descriptions, and field lengths & data types including the format used for dates/times.

Data to exclude

The following should not usually be recorded directly in the logs, but instead should be removed, masked, sanitized or encrypted:

• Session identification cookie values
• Sensitive personal data and some forms of personally identifiable information (PII)
• Authentication passwords
• Database connection strings
• Bank account or payment cardholder data
• Data of a higher security classification than the logging system is allowed to store
• Commercially-sensitive information

Sometimes the following data can also exist, and whilst useful for subsequent investigation, it may also need to be treated in some special manner before the event is recorded:

• File paths
• Database connection strings
• Internal network names and addresses
• Non sensitive personal data (e.g. personal names, telephone numbers, email addresses)

In some system, sanitization can be undertaken pot log collection, and prior to log display.

Customizable logging

It may be desirable to be able to alter the level of logging (type of events based on severity or threat level, amount of detail recorded). If this is implemented, ensure that:

• The default level must provide sufficient detail for business needs
• It should not be possible to completely inactivate application logging or logging of events that are necessary for compliance requirements
• Alterations to the level/extent of logging must be intrinsic to the application (e.g. undertaken automatically by the application based on an approved algorithm) or follow change management processes (e.g. changes to configuration data, modification of source code)
• The logging level must be verified periodically

Event collection

Implement an application-wide log handler which can be called from other modules/components. Document the interface referencing the organisation-specific event classification and description syntax requirements.

• Perform input validation on event data from other trust zones to ensure it is in the correct format (and consider alerting and not logging if there is an input validation failure)
• Perform sanitization on all event data to prevent log injection attacks (and optionally to remove sensitive data)
• If writing to databases read, understand and apply the SQL injection cheat sheet
• Synchronize time across all servers and devices &

& This is not always possible where the application is running on a device under some other party's control (e.g. on an individual's mobile phone, on a customer's workstation on their own corporate network). In these cases attempt to measure the time offset, or record a confidence level in the event timestamp.

Testing

Logging functionality and systems must be including in application testing:

• Ensure the logging is working correctly and as specified
• Check events are being classified consistently and the field names, types and lengths are correctly defined to an agreed standard.
• Ensure logging cannot be used to deplete system resources, for example by filling up disk space or exceeding database transaction log space, leading to denial of service

Deployment and operation

Add information about the logging mechanisms to release documentation Provide security configuration information for the logging mechanisms

Protection

The logging systems must be protected from mis-use such as tampering in transit, and unauthorised access, modification and deletion once collected.

The extraction and examination of audit trail data must be controlled:

   Logs may contain personal and other sensitive information.
   The data may give away information regarding the application's code and logic.
   Consider whether parts of the data may need to be excluded, masked, sanitised or encrypted.
   The privileges to read audit trails should be restricted and reviewed periodically.
   All access to the audit trail must be recorded and monitored by a separate manager.
   

At rest:

Build in tamper detection so you know if a record has been modified or deleted. Store, or copy, log data to read-only media as soon as possible.

In transit:

If log data is sent over untrusted networks (e.g. for collection or for dispatch elsewhere), use a secure transmission protocol. Consider whether the origin of the event data needs to be verified.

Monitoring

The logging then needs to be tied in with appropriate monitoring, alerting and reporting.

• Ensure incident response team is aware of the application logging mechanisms.
• If possible, aggregate the application log data with centralized logging and analysis sytems (e.g. SIEM).

Disposal

Log data must not be destroyed before the duration of its required data retention period, and not kept beyond this time.

Related articles

OWASP ESAPI Documentation

OWASP Logging Project

Arcsight Common Event Format (CEF)

IEEE Security & Privacy Journal How to Do Application Logging Right, Anton Chuvakin and Gunnar Peterson

IETF syslog protocol

Mitre Common Event Expression (CEE)

NIST SP 800-92 Guide to Computer Security Log Management

W3C Extended Log File Format

Authors and primary contributors

Colin Watson - colin.watson[at]owasp.org

OWASP Cheat Sheets Project Homepage

• OWASP Cheat Sheet Series