OWASP - User contributions [en]

Canonicalization, locale and Unicode

2007-08-15T19:02:25Z

Dkaplan: /* How to determine if you are vulnerable */

[[Guide Table of Contents]]__TOC__

==Objective ==

To ensure the application is robust when subjected to encoded, internationalized and Unicode input.

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS11.9 – Data processing integrity

==Description ==

Applications are rarely tested for Unicode exploits, and yet many are vulnerable due to the same sort of issues which allows HTTP Request Smuggling to work – every browser, web server, web application firewall or HTTP inspection agent, and other device treats user locale handling in different (and usually confusing) manner.

Canonicalization deals with the way in which systems convert data from one form to another. Canonical means the simplest or most standard form of something. Canonicalization is the process of converting something from one representation to the simplest form.

Web applications have to deal with lots of canonicalization issues from URL encoding to IP address translation. When security decisions are made based on less than perfectly canonicalized data, the application itself must be able to deal with unexpected input safely.

'''NB: To be secure against canocalization attacks does not mean that every application has to be internationalized, but all applications should be safe when Unicode and malformed representations is entered. '''

==Unicode ==

Unicode Encoding is a method for storing characters with multiple bytes. Wherever input data is allowed, data can be entered using Unicode to disguise malicious code and permit a variety of attacks. RFC 2279 references many ways that text can be encoded.

Unicode was developed to allow a Universal Character Set (UCS) that encompasses most of the world's writing systems. Multi-octet characters, however, are not compatible with many current applications and protocols, and this has led to the development of a few UCS transformation formats (UTF) with varying characteristics. UTF-8 has the characteristic of preserving the full US-ASCII range. It is compatible with file systems, parsers and other software relying on US-ASCII values, but it is transparent to other values.

The importance of UTF-8 representation stems from the fact that web-servers/applications perform several steps on their input of this format. The order of the steps is sometimes critical to the security of the application. Basically, the steps are "URL decoding" potentially followed by "UTF-8 decoding", and intermingled with them are various security checks, which are also processing steps.

If, for example, one of the security checks is searching for "..", and it is carried out before UTF-8 decoding takes place, it is possible to inject ".." in their overlong UTF-8 format. Even if the security checks recognize some of the non-canonical format for dots, it may still be that not all formats are known to it.

Consider the ASCII character "." (dot). Its canonical representation is a dot (ASCII 2E). Yet if we think of it as a character in the second UTF-8 range (2 bytes), we get an overlong representation of it, as C0 AE. Likewise, there are more overlong representations: E0 80 AE, F0 80 80 AE, F8 80 80 80 AE and FC 80 80 80 80 AE.

{| border=1

|| UCS-4 Range || UTF-8 Encoding

|-

|| ''0x00000000-0x0000007F'' || ''0xxxxxxx''

|-

|| ''0x00000080 - 0x000007FF'' || ''110xxxxx 10xxxxxx''

|-

|| ''0x00000800-0x0000FFFF'' || ''1110xxxx 10xxxxxx 10xxxxxx''

|-

|| ''0x00010000-0x001FFFFF'' || ''11110xxx 10xxxxxx 10xxxxxx 10xxxxxx''

|-

|| ''0x00200000-0x03FFFFFF'' || ''111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx''

|-

|| ''0x04000000-0x7FFFFFFF'' || ''1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx''

|-

|}

Consider the representation C0 AE of a ".". Like UTF-8 encoding requires, the second octet has "10" as its two most significant bits. Now, it is possible to define 3 variants for it, by enumerating the rest of the possible 2 bit combinations ("00", "01" and "11"). Some UTF-8 decoders would treat these variants as identical to the original symbol (they simply use the least significant 6 bits, disregarding the most significant 2 bits). Thus, the 3 variants are C0 2E, C0 5E and C0 FE.

It is thus possible to form illegal UTF-8 encodings, in two senses:

* A UTF-8 sequence for a given symbol may be longer than necessary for representing the symbol.

* A UTF-8 sequence may contain octets that are in incorrect format (i.e. do not comply with the above 6 formats).

To further "complicate" things, each representation can be sent over HTTP in several ways:

'''In the raw'''. That is, without URL encoding at all. This usually results in sending non-ASCII octets in the path, query or body, which violates the HTTP standards. Nevertheless, most HTTP servers do get along just fine with non-ASCII characters.

'''Valid URL encoding'''. Each non-ASCII character (more precisely, all characters that require URL encoding - a superset of non ASCII characters) is URL-encoded. This results in sending, say, %C0%AE.

'''Invalid URL encoding'''. This is a variant of valid URL encoding, wherein some hexadecimal digits are replaced with non-hexadecimal digits, yet the result is still interpreted as identical to the original, under some decoding algorithms. For example, %C0 is interpreted as character number ('C'-'A'+10)*16+('0'-'0') = 192. Applying the same algorithm to %M0 yields ('M'-'A'+10)*16+('0'-'0') = 448, which, when forced into a single byte, yields (8 least significant bits) 192, just like the original. So, if the algorithm is willing to accept non-hexadecimal digits (such as 'M'), then it is possible to have variants for %C0 such as %M0 and %BG.

It should be kept in mind that these techniques are not directly related to Unicode, and they can be used in non-Unicode attacks as well.

http://www.example.com/cgi-bin/bad.cgi?foo=../../bin/ls%20-al

URL Encoding of the example attack:

http://www.example.com/cgi-bin/bad.cgi?foo=..%2F../bin/ls%20-al

Unicode encoding of the example attack:

http://www.example.com/cgi-bin/bad.cgi?foo=..%c0%af../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%c1%9c../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%c1%pc../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%c0%9v../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%c0%qf../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%c1%8s../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%c1%1c../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%c1%9c../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%c1%af../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%e0%80%af../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%f0%80%80%af../bin/ls%20-al

http://www.example.com/cgi-bin/bad.cgi?foo=..%f8%80%80%80%af../bin/ls%20-al

===How to protect yourself ===

A suitable canonical form should be chosen and all user input canonicalized into that form before any authorization decisions are performed. Security checks should be carried out after UTF-8 decoding is completed. Moreover, it is recommended to check that the UTF-8 encoding is a valid canonical encoding for the symbol it represents.

http://www.ietf.org/rfc/rfc2279.txt?number=2279 

==Input Formats ==

Web applications usually operate internally as one of ASCII, ISO 8859-1, or Unicode (Java programs are UTF-16 example). Your users may be using another locale, and attackers can choose their locale and character set with impunity.

===How to determine if you are vulnerable ===

Investigate the web application to determine if it asserts an internal code page, locale or culture.

If the default character set, locale is not asserted it will be one of the following:

* HTTP Posts. Interesting tidbit: All HTTP posts are required to be ISO 8859-1, which will lose data for most double byte character sets. You must test your application with your supported browsers to determine if they pass in fully encoded double byte characters safely

* HTTP Gets. Depends on the previously rendered page and per-browser implementations, but URL encoding is not properly defined for double byte character sets. IE can be optionally forced to do all submits as UTF-8 which is then properly canonicalized on the server

* .NET: Unicode (little endian)

* JSP implementations, such as Tomcat: UTF8 - see “javaEncoding” in web.xml by many servlet containers

* Java: Unicode (UTF-16, big endian, '''''or''''' depends on the OS during JVM startup)

* PHP: Set in php.ini, ISO 8859-1.

'''NB: Many PHP functions make (invalid) assumptions as to character set and may not work properly when changed to another character set. Test your application with the new character set thoroughly!'''

===How to protect yourself ===

* Determine your application’s needs, and set both the asserted language locale and character set appropriately.

==Locale assertion ==

The web server should always assert a locale and preferably a country code, such as “en_US”, “fr_FR”, “zh_CN”

===How to determine if you are vulnerable ===

Use a HTTP header sniffer or even just telnet against your web server:

''HEAD / HTTP1.0''

Should display something like this:

''HTTP/1.1 200 OK''

''Date: Sun, 24 Jul 2005 08:13:17 GMT''

''Server: Apache/1.3.29''

''Connection: close''

''Content-Type: text/html;''''' charset=iso-8859-1'''

===How to protect yourself ===

Review and implement these guidelines:

http://www.w3.org/International/technique-index

At a minimum, select the correct output locale and character set.

==Double (or n-) encoding ==

Most web applications only check once to determine if the input is has been de-encoded into the correct Unicode values. However, an attacker may have doubly encoded the attack string.

===How to determine if you are vulnerable ===

* Use XSS Cheat Sheet double encoder utility to double encode a XSS string

http://ha.ckers.org/xss.html

* If the resultant injection is a successful XSS output, then your application is vulnerable

* This attack may also work against:

# Filenames
# Non-obvious items like report types, and language selectors
# Theme names

===How to protect yourself ===

* Assert the correct locale and character set for your application

* Use HTML entities, URL encoding and so on to prevent Unicode characters being treated improperly by the many divergent browser, server and application combinations

* Test your code and overall solution extensively

==HTTP Request Smuggling ==

HTTP Request Smuggling (HRS) is an issue detailed in depth by Klein, Linhart, Heled, and Orrin in a whitepaper found in the references section. The basics of HTTP Request Smuggling is that many larger solutions use many components to provide a web application. The differences between the firewall, web application firewall, load balancers, SSL accelerators, reverse proxies, and web servers allow a specially crafted attack to bypass all the controls in the front-end systems and directly attack the web server.

The types of attack they describe are:

* Web cache poisoning

* Firewall/IDS/IPS evasion

* Forward and backward HRS techniques

* Request hijacking

* Request credential hijacking

Since the whitepaper, several examples of real life HRS have been discovered.

===How to determine if you are vulnerable ===

* Review the whitepaper

* Review your infrastructure for vulnerable components

===How to protect yourself ===

* Minimize the total number of components that may interpret the inbound HTTP request

* Keep your infrastructure up to date with patches

==Further Reading ==

* IDS Evasion using Unicode

http://online.securityfocus.com/print/infocus/1232

* W3C Internationalization Home Page

http://www.w3.org/International/

* HTTP Request Smuggling

http://www.watchfire.com/resources/HTTP-Request-Smuggling.pdf

* XSS Cheat Sheet

http://ha.ckers.org/xss.html

[[Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Canonicalization]]
[[Category:Encoding]]

Data Validation

2007-08-14T20:07:23Z

Dkaplan: /* Per-User Data */

[[Guide Table of Contents]]__TOC__

==Objective ==

To ensure that the application is robust against all forms of input data, whether obtained from the user, infrastructure, external entities or database systems

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS11 – Manage Data. All sections should be reviewed

==Description ==

The most common web application security weakness is the failure to properly validate input from the client or environment. This weakness leads to almost all of the major vulnerabilities in applications, such as interpreter injection, locale/Unicode attacks, file system attacks and buffer overflows. Data from the client should never be trusted for the client has every possibility to tamper with the data.

In many cases, [[Encoding]] has the potential to defuse attacks that rely on lack of input validation. For example, if you use HTML entity encoding on user input before it is sent to a browser, it will prevent most [[XSS]] attacks. However, simply preventing attacks is not enough - you must perform [[Intrusion Detection]] in your applications. Otherwise, you are allowing attackers to repeatedly attack your application until they find a vulnerability that you haven't protected against. Detecting attempts to find these weaknesses is a critical protection mechanism.

==Definitions ==

These definitions are used within this document:

* '''Integrity checks'''

Ensure that the data has not been tampered with and is the same as before

* '''Validation'''

Ensure that the data is strongly typed, correct syntax, within length boundaries, contains only permitted characters, or that numbers are correctly signed and within range boundaries

* '''Business rules'''

Ensure that data is not only validated, but business rule correct. For example, interest rates fall within permitted boundaries.

Some documentation and references interchangeably use the various meanings, which is very confusing to all concerned. This confusion directly causes continuing financial loss to the organization.

==Where to include integrity checks ==

Integrity checks must be included wherever data passes from a trusted to a less trusted boundary, such as from the application to the user's browser in a hidden field, or to a third party payment gateway, such as a transaction ID used internally upon return.

The type of integrity control (checksum, HMAC, encryption, digital signature) should be directly related to the risk of the data transiting the trust boundary.

==Where to include validation ==

Validation must be performed on every tier. However, validation should be performed as per the function of the server executing the code. For example, the web / presentation tier should validate for web related issues, persistence layers should validate for persistence issues such as SQL / HQL injection, directory lookups should check for LDAP injection, and so on.

==Where to include business rule validation ==

Business rules are known during design, and they influence implementation. However, there are bad, good and "best" approaches. Often the best approach is the simplest in terms of code.

===Example - Scenario ===

* You are to populate a list with accounts provided by the back-end system
* The user will choose an account, choose a biller, and press next

===Wrong way===

The account select option is read directly and provided in a message back to the backend system without validating the account number if one of the accounts provided by the backend system.

===Why this is bad===

An attacker can change the HTML in any way they choose:

* The lack of validation requires a round-trip to the backend to provide an error message that the front end code could easily have eliminated

* The back end may not be able to cope with the data payload the front-end code could have easily eliminated. For example, buffer overflows, XML injection, or similar.

===Acceptable Method ===

The account select option parameter ("payee_id") is read by the code, and compared to an already-known list.

<code>
<pre>
if (account.hasPayee( session.getParameter("payee_id") )) {
backend.performTransfer( session.getParameter("payee_id") );
}
</pre>
</code>

This prevents parameter tampering, but requires the list of possible payee_id's to be to be calculated beforehand.

===Best Method ===

The original code emitted indexes <option value="1" ... > rather than account names.

''int payeeLstId = session.getParameter('payeelstid');''

''accountFrom = account.getAcctNumberByIndex(payeeLstId);''

Not only is this easier to render in HTML, it makes validation and business rule validation trivial. The field cannot be tampered with.



===Conclusion ===

To provide defense in depth and to prevent attack payloads from trust boundaries, such as backend hosts, which are probably incapable of handling arbitrary input data, business rule validation is to be performed (preferably in workflow or command patterns), even if it is known that the back end code performs business rule validation.

This is not to say that the entire set of business rules need be applied - it means that the fundamentals are performed to prevent unnecessary round trips to the backend and to prevent the backend from receiving most tampered data.

==Data Validation Strategies ==

There are four strategies for validating data, and they should be used in this order:

===Accept known good===

This strategy is also known as "whitelist" or "positive" validation. The idea is that you should check that the data is one of a set of tightly constrained known good values. Any data that doesn't match should be rejected. Data should be:

* Strongly typed at all times
* Length checked and fields length minimized
* Range checked if a numeric
* Unsigned unless required to be signed
* Syntax or grammar should be checked prior to first use or inspection

If you expect a postcode, validate for a postcode (type, length and syntax):

<code>
<pre>
public String isPostcode(String postcode) {
return (postcode != null && Pattern.matches("^(((2|8|9)\d{2})|((02|08|09)\d{2})|([1-9]\d{3}))$", postcode)) ? postcode : "";
}
</pre>
</code>

Coding guidelines should use some form of visible tainting on input from the client or untrusted sources, such as third party connectors to make it obvious that the input is unsafe:

<code>
<pre>
String taintPostcode = request.getParameter("postcode");
ValidationEngine validator = new ValidationEngine();
boolean isValidPostcode = validator.isPostcode(taintPostcode);
</pre>
</code>

===Reject known bad===

This strategy, also known as "negative" or "blacklist" validation is a weak alternative to positive validation. Essentially, if you don't expect to see characters such as %3f or JavaScript or similar, reject strings containing them. This is a dangerous strategy, because the set of possible bad data is potentially infinite. Adopting this strategy means that you will have to maintain the list of "known bad" characters and patterns forever, and you will by definition have incomplete protection.

''public String removeJavascript(String input) {''

'' Pattern p = Pattern.compile("javascript", CASE_INSENSITIVE);''

'' p.matcher(input);''

'' return (!p.matches()) ? input : '';''

''}''

It can take upwards of 90 regular expressions (see the CSS Cheat Sheet in the Guide 2.0) to eliminate known malicious software, and each regex needs to be run over every field. Obviously, this is slow and not secure. Just rejecting "current known bad" (which is at the time of writing hundreds of strings and literally millions of combinations) is insufficient if the input is a string. This strategy is directly akin to anti-virus pattern updates. Unless the business will allow updating "bad" regexes on a daily basis and support someone to research new attacks regularly, this approach will be obviated before long.

===Sanitize===

Eliminate or translate characters (such as to HTML entities or to remove quotes) in an effort to make the input "safe".
Like blacklists, this approach requires maintenance and is usually incomplete. As most fields have a particular grammar, it is simpler, faster, and more secure to simply validate a single correct positive test than to try to include complex and slow sanitization routines for all current and future attacks.

<code>
<pre>
public String quoteApostrophe(String input) {
if (input != null)
return input.replaceAll("[\']", "&rsquo;");
else
return null;
}
</pre>
</code>

===No validation===

This is inherently unsafe and strongly discouraged. The business must sign off each and every example of no validation as the lack of validation usually leads to direct obviation of application, host and network security controls.

''account.setAcctId(getParameter('formAcctNo'));''
''...''

''public setAcctId(String acctId) {''
'' cAcctId = acctId;''
''}''

==Prevent parameter tampering ==

There are many input sources:

* HTTP headers, such as REMOTE_ADDR, PROXY_VIA or similar

* Environment variables, such as getenv() or via server properties

* All GET, POST and Cookie data

This includes supposedly tamper resistant fields such as radio buttons, drop downs, etc - any client side HTML can be re-written to suit the attacker

Configuration data (mistakes happen :))

External systems (via any form of input mechanism, such as XML input, RMI, web services, etc)

All of these data sources supply untrusted input. Data received from untrusted data sources must be properly checked before first use.

==Hidden fields ==

Hidden fields are a simple way to avoid storing state on the server. Their use is particularly prevalent in "wizard-style" multi-page forms. However, their use exposes the inner workings of your application, and exposes data to trivial tampering, replay, and validation attacks. In general, only use hidden fields for page sequence.

If you have to use hidden fields, there are some rules:

* Secrets, such as passwords, should never be sent in the clear

* Hidden fields need to have integrity checks and preferably encrypted using non-constant initialization vectors (i.e. different users at different times have different yet cryptographically strong random IVs)

* Encrypted hidden fields must be robust against replay attacks, which means some form of temporal keying

* Data sent to the user must be validated on the server once the last page has been received, even if it has been previously validated on the server - this helps reduce the risk from replay attacks.

The preferred integrity control should be at least a HMAC using SHA-256 or preferably digitally signed or encrypted using PGP. IBMJCE supports SHA-256, but PGP JCE support require the inclusion of the Legion of the Bouncy Castle (http://www.bouncycastle.org/) JCE classes.

It is simpler to store this data temporarily in the session object. Using the session object is the safest option as data is never visible to the user, requires (far) less code, nearly no CPU, disk or I/O utilization, less memory (particularly on large multi-page forms), and less network consumption.

In the case of the session object being backed by a database, large session objects may become too large for the inbuilt handler. In this case, the recommended strategy is to store the validated data in the database, but mark the transaction as "incomplete". Each page will update the incomplete transaction until it is ready for submission. This minimizes the database load, session size, and activity between the users whilst remaining tamperproof.

Code containing hidden fields should be rejected during code reviews.

==ASP.NET Viewstate ==

ASP.NET sends form data back to the client in a hidden “Viewstate” field. Despite looking forbidding, this “encryption” is simply plain-text equivalent (base64 encoding) and has no data integrity without further action on your behalf in ASP.NET 1.0. In ASP.NET 1.1 and 2.0, tamper proofing, called "enableViewStateMAC" is on by default using a SHA/1 hash.

Any application framework with a similar mechanism might be at fault – you should investigate your application framework’s support for sending data back to the user. Preferably it should not round trip.

===How to determine if you are vulnerable ===

These configurations are set hierarchically in the .NET framework. The machine.config file contains the global configuration; each web directory may contain a web.config file further specifying or overriding configuration; each page may contain @page directives specifying same configuration or overrides; you must check all three locations:

* If the enableViewStateMac is not set to “true”, you are at risk if your viewstate contains authorization state

* If the viewStateEncryptionMode is not set to “always”, you are at risk if your viewstate contains secrets such as credentials

* If you share a host with many other customers, you all share the same machine key by default in ASP.NET 1.1. In ASP.NET 2.0, it is possible to configure unique viewstate keys per application

===How to protect yourself ===

* If your application relies on data returning from the viewstate without being tampered with, you should turn on viewstate integrity checks at the least, and strongly consider:

* Encrypt viewstate if any of the data is application sensitive

* Upgrade to ASP.NET 2.0 as soon as practical if you are on a shared hosting arrangement

* Move truly sensitive viewstate data to the session variable instead

===Selects, radio buttons, and checkboxes ===

It is commonly held belief that the value settings for these items cannot be easily tampered. This is wrong. In the following example, actual account numbers are used, which can lead to compromise:

''<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">''

''<bean:message key="msg.card.name" arg0="<%=acct.getCardName(1).toString( )%>" />''

''<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">''

''<bean:message key="msg.card.name" arg0="<%=acct.getCardName(2).toString( )%>" />''

This produces (for example):

''<input type="radio" name="acctNo" value="455712341234">Gold Card''

''<input type="radio" name="acctNo" value="455712341235">Platinum Card''

If the value is retrieved and then used directly in a SQL query, an interesting form of SQL injection may occur: authorization tampering leading to information disclosure. As the connection pool connects to the database using a single user, it may be possible to see other user's accounts if the SQL looks something like this:

''String acctNo = getParameter('acctNo');''

''String sql = "SELECT acctBal FROM accounts WHERE acctNo = '?'";''

''PreparedStatement st = conn.prepareStatement(sql);''

''st.setString(1, acctNo);''

''ResultSet rs = st.executeQuery();''

This should be re-written to retrieve the account number via index, and include the client's unique ID to ensure that other valid account numbers are exposed:

''String acctNo = acct.getCardNumber(getParameter('acctIndex'));''

''String sql = "SELECT acctBal FROM accounts WHERE acct_id = '?' AND acctNo = '?'";''

''PreparedStatement st = conn.prepareStatement(sql);''

''st.setString(1, acct.getID());''

''st.setString(2, acctNo);''

''ResultSet rs = st.executeQuery();''

This approach requires rendering input values from 1 to ... x, and assuming accounts are stored in a Collection which can be iterated using logic:iterate:

''<logic:iterate id="loopVar" name="MyForm" property="values">''

'' <html:radio property="acctIndex" idName="loopVar" value="value"/> ''

'' <bean:write name="loopVar" property="name"/> ''

''</logic:iterate>''

The code will emit HTML with the values "1" .. "x" as per the collection's content.

''<input type="radio" name="acctIndex" value="1" />Gold Credit Card''

''<input type="radio" name="acctIndex" value="2" />Platinum Credit Card''

This approach should be used for any input type that allows a value to be set: radio buttons, checkboxes, and particularly select / option lists.

===Per-User Data ===

In fully normalized databases, the aim is to minimize the amount of repeated data. However, some data is inferred. For example, users can see messages that are stored in a messages table. Some messages are private to the user. However, in a fully normalized database, the list of message IDs are kept within another table:

<pre>
+------------------------+
| MESSAGES |
+------------------------+
| msgid | message |
+------------------------+
</pre>

If a user marks a message for deletion, the usual way is to recover the message ID from the user, and delete that:

''DELETE FROM message WHERE msgid='frmMsgId'''

However, how do you know if the user is eligible to delete that message ID? Such tables need to be denormalized slightly to include a user ID or make it easy to perform a single query to delete the message safely. For example, by adding back an (optional) uid column, the delete is now made reasonably safe:

''DELETE FROM message WHERE uid='session.myUserID' and msgid='frmMsgId';''

Where the data is potentially both a private resource and a public resource (for example, in the secure message service, broadcast messages are just a special type of private message), additional precautions need to be taken to prevent users from deleting public resources without authorization. This can be done using role based checks, as well as using SQL statements to discriminate by message type:

''DELETE FROM message ''

''WHERE''

''uid='session.myUserID' AND''

''msgid='frmMsgId' AND''

''broadcastFlag = false;''

==URL encoding ==

Data sent via the URL, which is strongly discouraged, should be URL encoded and decoded. This reduces the likelihood of cross-site scripting attacks from working.

In general, do not send data via GET request unless for navigational purposes.

==HTML encoding ==

Data sent to the user needs to be safe for the user to view. This can be done using <bean:write ...> and friends. Do not use <%=var%> unless it is used to supply an argument for <bean:write...> or similar.

HTML encoding translates a range of characters into their HTML entities. For example, > becomes &gt; This will still display as > on the user's browser, but it is a safe alternative.

==Encoded strings ==

Some strings may be received in encoded form. It is essential to send the correct locale to the user so that the web server and application server can provide a single level of canoncalization prior to the first use.

Do not use getReader() or getInputStream() as these input methods do not decode encoded strings. If you need to use these constructs, you must decanoncalize data by hand.

==Data Validation and Interpreter Injection ==

This section focuses on preventing injection in ColdFusion. Interpreter Injection involves manipulating application parameters to execute malicious code on the system. The most prevalent of these is SQL injection but it also includes other injection techniques, including LDAP, ORM, User Agent, XML, etc. – see the Interpreter Injection chapter of this document for greater details. As a developer you should assume that all input is malicious. Before processing any input coming from a user, data source, component, or data service it should be validated for type, length, and/or range. ColdFusion includes support for Regular Expressions and CFML tags that can be used to validate input.

'''SQL Injection'''

SQL Injection involves sending extraneous SQL queries as variables. ColdFusion provides the <cfqueryparam> and <cfprocparam> tags for validating database parameters. These tags nests inside <cfquery> and <cfstoredproc>, respectively. For dynamic SQL submitted in <cfquery>, use the CFSQLTYPE attribute of the <cfqueryparam> to validate variables against the expected database datatype. Similarly, use the CFSQLTYPE attribute of <cfprocparam> to validate the datatypes of stored procedure parameters passed through <cfstoredproc>.

You can also strengthen your systems against SQL Injection by disabling the Allowed SQL operations for individual data sources. See the '''Configuration''' section below for more information.

'''LDAP Injection'''

ColdFusion uses the <cfldap> tag to communicate with LDAP servers. This tag has an ACTION attribute which dictates the query performed against the LDAP. The valid values for this attribute are: add, delete, query (default), modify, and modifyDN. <cfldap> calls are turned into JNDI (Java Naming And Directory Interface) lookups. However, because <cfldap> wraps the calls, it will throw syntax errors if native JNDI code is passed to its attributes making LDAP injection more difficult.

'''XML Injection'''

Two parsers exist for XML data – SAX and DOM. ColdFusion uses DOM which reads the entire XML document into the server’s memory. This requires the administrator to restrict the size of the JVM containing ColdFusion. ColdFusion is built on Java therefore by default, entity references are expanded during parsing. To prevent unbounded entity expansion, before a string is converted to an XML DOM, filter out DOCTYPES elements.

After the DOM has been read, to reduce the risk of XML, Injection use the ColdFusion XML decision functions: isXML(), isXmlAttribute(), isXmlElement(), isXmlNode(), and isXmlRoot(). The isXML() function determines if a string is well-formed XML. The other functions determine whether or not the passed parameter is a valid part of an XML document. Use the xmlValidate() function to validate external XML documents against a Document Type Definition (DTD) or XML Schema.

'''Event Gateway, IM, and SMS Injection'''

ColdFusion MX 7 enables Event Gateways, instant messaging (IM), and SMS (short message service) for interacting with external systems. Event Gateways are ColdFusion components that respond asynchronously to non-HTTP requests – e.g. instant messages, SMS text from wireless devices, etc. ColdFusion provides Lotus Sametime and XMPP (Extensible Messaging and Presence Protocol) gateways for instant messaging. It also provides an event gateway for interacting with SMS text messages.

Injection along these gateways can happen when end users (and/or systems) send malicious code to execute on the server. These gateways all utilize ColdFusion Components (CFCs) for processing. Use standard ColdFusion functions, tags, and validation techniques to protect against malicious code injection. Sanitize all input strings and do not allow un-validated code to access backend systems.

'''Best Practices'''

Use the XML functions to validate XML input.

Before performing XPath searches and transformations in ColdFusion, validate the source before executing.

Use ColdFusion validation techniques to sanitize strings passed to xmlSearch for performing XPath queries.

When performing XML transformations only use a trusted source for the XSL stylesheet.

Ensure that the memory size of the Java Sandbox containing ColdFusion can handle large XML documents without adversely affecting server resources.

Set the memory value to less than the amount of RAM on the server (-Xmx)

Remove DOCTYPE elements from the XML string before converting it to an XML object.

Using scriptProtect can be used to thwart most attempts of cross-site scripting. Set scriptProtect to All in the Application.cfc

Use <cfparam> or <cfargument> to instantiate variables in ColdFusion. Use this tag with the name and type attributes. If the value is not of the specified type, ColdFusion returns an error.

To handle untyped variables use IsValid() to validate its value against any legal object type that ColdFusion supports.

Use <cfqueryparam> and <cfprocparam> to valid dynamic SQL variables against database datatypes

Use CFLDAP for accessing LDAP servers. Avoid allowing native JNDI calls to connect to LDAP

'''Best Practice in Action'''

The sample code below shows a database authentication function using some of the input validation techniques discussed in this section.

{| border=1

||
* <cffunction name="dblogin" access="private" output="false" returntype="struct">

* <cfargument name="strUserName" required="true" type="string">

* <cfargument name="strPassword" required="true" type="string">

* <cfset var retargs = StructNew()>

* <cfif IsValid("regex", uUserName, "[A-Za-z0-9%]*") AND IsValid("regex", uPassword, "[A-Za-z0-9%]*")>

* <cfquery name="loginQuery" dataSource="#Application.DB#" >

* SELECT hashed_password, salt

* FROM UserTable

* WHERE UserName =

* <cfqueryparam value="#strUserName#" cfsqltype="CF_SQL_VARCHAR" maxlength="25">

* </cfquery>

* <cfif loginQuery.hashed_password EQ Hash(strPassword & loginQuery.salt, "SHA-256" )>

* <cfset retargs.authenticated="YES">

* <cfset Session.UserName = strUserName>

* 

* <cfelse>

* <cfset retargs.authenticated="NO">

* </cfif>

* <cfelse>

* <cfset retargs.authenticated="NO">

* </cfif>

* <cfreturn retargs>

* </cffunction>

|-

|}

==Delimiter and special characters ==

There are many characters that mean something special to various programs. If you followed the advice only to accept characters that are considered good, it is very likely that only a few delimiters will catch you out.

Here are the usual suspects:

* NULL (zero) %00

* LF - ANSI chr(10) "\r"

* CR - ANSI chr(13) "\n"

* CRLF - "\n\r"

* CR - EBCDIC 0x0f

* Quotes " '

* Commas, slashes spaces and tabs and other white space - used in CSV, tab delimited output, and other specialist formats

* <> - XML and HTML tag markers, redirection characters

* ; & - Unix and NT file system continuance

* @ - used for e-mail addresses

* 0xff

* ... more

Whenever you code to a particular technology, you should determine which characters are "special" and prevent them appearing in input, or properly escaping them.

==Further Reading ==

* ASP.NET 2.0 Viewstate

http://channel9.msdn.com/wiki/default.aspx/Channel9.HowToConfigureTheMachineKeyInASPNET2

[[Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Validation]]
[[Category:Encoding]]

Data Validation

2007-08-14T19:20:11Z

Dkaplan: /* Best Method */

[[Guide Table of Contents]]__TOC__

==Objective ==

To ensure that the application is robust against all forms of input data, whether obtained from the user, infrastructure, external entities or database systems

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS11 – Manage Data. All sections should be reviewed

==Description ==

The most common web application security weakness is the failure to properly validate input from the client or environment. This weakness leads to almost all of the major vulnerabilities in applications, such as interpreter injection, locale/Unicode attacks, file system attacks and buffer overflows. Data from the client should never be trusted for the client has every possibility to tamper with the data.

In many cases, [[Encoding]] has the potential to defuse attacks that rely on lack of input validation. For example, if you use HTML entity encoding on user input before it is sent to a browser, it will prevent most [[XSS]] attacks. However, simply preventing attacks is not enough - you must perform [[Intrusion Detection]] in your applications. Otherwise, you are allowing attackers to repeatedly attack your application until they find a vulnerability that you haven't protected against. Detecting attempts to find these weaknesses is a critical protection mechanism.

==Definitions ==

These definitions are used within this document:

* '''Integrity checks'''

Ensure that the data has not been tampered with and is the same as before

* '''Validation'''

Ensure that the data is strongly typed, correct syntax, within length boundaries, contains only permitted characters, or that numbers are correctly signed and within range boundaries

* '''Business rules'''

Ensure that data is not only validated, but business rule correct. For example, interest rates fall within permitted boundaries.

Some documentation and references interchangeably use the various meanings, which is very confusing to all concerned. This confusion directly causes continuing financial loss to the organization.

==Where to include integrity checks ==

Integrity checks must be included wherever data passes from a trusted to a less trusted boundary, such as from the application to the user's browser in a hidden field, or to a third party payment gateway, such as a transaction ID used internally upon return.

The type of integrity control (checksum, HMAC, encryption, digital signature) should be directly related to the risk of the data transiting the trust boundary.

==Where to include validation ==

Validation must be performed on every tier. However, validation should be performed as per the function of the server executing the code. For example, the web / presentation tier should validate for web related issues, persistence layers should validate for persistence issues such as SQL / HQL injection, directory lookups should check for LDAP injection, and so on.

==Where to include business rule validation ==

Business rules are known during design, and they influence implementation. However, there are bad, good and "best" approaches. Often the best approach is the simplest in terms of code.

===Example - Scenario ===

* You are to populate a list with accounts provided by the back-end system
* The user will choose an account, choose a biller, and press next

===Wrong way===

The account select option is read directly and provided in a message back to the backend system without validating the account number if one of the accounts provided by the backend system.

===Why this is bad===

An attacker can change the HTML in any way they choose:

* The lack of validation requires a round-trip to the backend to provide an error message that the front end code could easily have eliminated

* The back end may not be able to cope with the data payload the front-end code could have easily eliminated. For example, buffer overflows, XML injection, or similar.

===Acceptable Method ===

The account select option parameter ("payee_id") is read by the code, and compared to an already-known list.

<code>
<pre>
if (account.hasPayee( session.getParameter("payee_id") )) {
backend.performTransfer( session.getParameter("payee_id") );
}
</pre>
</code>

This prevents parameter tampering, but requires the list of possible payee_id's to be to be calculated beforehand.

===Best Method ===

The original code emitted indexes <option value="1" ... > rather than account names.

''int payeeLstId = session.getParameter('payeelstid');''

''accountFrom = account.getAcctNumberByIndex(payeeLstId);''

Not only is this easier to render in HTML, it makes validation and business rule validation trivial. The field cannot be tampered with.



===Conclusion ===

To provide defense in depth and to prevent attack payloads from trust boundaries, such as backend hosts, which are probably incapable of handling arbitrary input data, business rule validation is to be performed (preferably in workflow or command patterns), even if it is known that the back end code performs business rule validation.

This is not to say that the entire set of business rules need be applied - it means that the fundamentals are performed to prevent unnecessary round trips to the backend and to prevent the backend from receiving most tampered data.

==Data Validation Strategies ==

There are four strategies for validating data, and they should be used in this order:

===Accept known good===

This strategy is also known as "whitelist" or "positive" validation. The idea is that you should check that the data is one of a set of tightly constrained known good values. Any data that doesn't match should be rejected. Data should be:

* Strongly typed at all times
* Length checked and fields length minimized
* Range checked if a numeric
* Unsigned unless required to be signed
* Syntax or grammar should be checked prior to first use or inspection

If you expect a postcode, validate for a postcode (type, length and syntax):

<code>
<pre>
public String isPostcode(String postcode) {
return (postcode != null && Pattern.matches("^(((2|8|9)\d{2})|((02|08|09)\d{2})|([1-9]\d{3}))$", postcode)) ? postcode : "";
}
</pre>
</code>

Coding guidelines should use some form of visible tainting on input from the client or untrusted sources, such as third party connectors to make it obvious that the input is unsafe:

<code>
<pre>
String taintPostcode = request.getParameter("postcode");
ValidationEngine validator = new ValidationEngine();
boolean isValidPostcode = validator.isPostcode(taintPostcode);
</pre>
</code>

===Reject known bad===

This strategy, also known as "negative" or "blacklist" validation is a weak alternative to positive validation. Essentially, if you don't expect to see characters such as %3f or JavaScript or similar, reject strings containing them. This is a dangerous strategy, because the set of possible bad data is potentially infinite. Adopting this strategy means that you will have to maintain the list of "known bad" characters and patterns forever, and you will by definition have incomplete protection.

''public String removeJavascript(String input) {''

'' Pattern p = Pattern.compile("javascript", CASE_INSENSITIVE);''

'' p.matcher(input);''

'' return (!p.matches()) ? input : '';''

''}''

It can take upwards of 90 regular expressions (see the CSS Cheat Sheet in the Guide 2.0) to eliminate known malicious software, and each regex needs to be run over every field. Obviously, this is slow and not secure. Just rejecting "current known bad" (which is at the time of writing hundreds of strings and literally millions of combinations) is insufficient if the input is a string. This strategy is directly akin to anti-virus pattern updates. Unless the business will allow updating "bad" regexes on a daily basis and support someone to research new attacks regularly, this approach will be obviated before long.

===Sanitize===

Eliminate or translate characters (such as to HTML entities or to remove quotes) in an effort to make the input "safe".
Like blacklists, this approach requires maintenance and is usually incomplete. As most fields have a particular grammar, it is simpler, faster, and more secure to simply validate a single correct positive test than to try to include complex and slow sanitization routines for all current and future attacks.

<code>
<pre>
public String quoteApostrophe(String input) {
if (input != null)
return input.replaceAll("[\']", "&rsquo;");
else
return null;
}
</pre>
</code>

===No validation===

This is inherently unsafe and strongly discouraged. The business must sign off each and every example of no validation as the lack of validation usually leads to direct obviation of application, host and network security controls.

''account.setAcctId(getParameter('formAcctNo'));''
''...''

''public setAcctId(String acctId) {''
'' cAcctId = acctId;''
''}''

==Prevent parameter tampering ==

There are many input sources:

* HTTP headers, such as REMOTE_ADDR, PROXY_VIA or similar

* Environment variables, such as getenv() or via server properties

* All GET, POST and Cookie data

This includes supposedly tamper resistant fields such as radio buttons, drop downs, etc - any client side HTML can be re-written to suit the attacker

Configuration data (mistakes happen :))

External systems (via any form of input mechanism, such as XML input, RMI, web services, etc)

All of these data sources supply untrusted input. Data received from untrusted data sources must be properly checked before first use.

==Hidden fields ==

Hidden fields are a simple way to avoid storing state on the server. Their use is particularly prevalent in "wizard-style" multi-page forms. However, their use exposes the inner workings of your application, and exposes data to trivial tampering, replay, and validation attacks. In general, only use hidden fields for page sequence.

If you have to use hidden fields, there are some rules:

* Secrets, such as passwords, should never be sent in the clear

* Hidden fields need to have integrity checks and preferably encrypted using non-constant initialization vectors (i.e. different users at different times have different yet cryptographically strong random IVs)

* Encrypted hidden fields must be robust against replay attacks, which means some form of temporal keying

* Data sent to the user must be validated on the server once the last page has been received, even if it has been previously validated on the server - this helps reduce the risk from replay attacks.

The preferred integrity control should be at least a HMAC using SHA-256 or preferably digitally signed or encrypted using PGP. IBMJCE supports SHA-256, but PGP JCE support require the inclusion of the Legion of the Bouncy Castle (http://www.bouncycastle.org/) JCE classes.

It is simpler to store this data temporarily in the session object. Using the session object is the safest option as data is never visible to the user, requires (far) less code, nearly no CPU, disk or I/O utilization, less memory (particularly on large multi-page forms), and less network consumption.

In the case of the session object being backed by a database, large session objects may become too large for the inbuilt handler. In this case, the recommended strategy is to store the validated data in the database, but mark the transaction as "incomplete". Each page will update the incomplete transaction until it is ready for submission. This minimizes the database load, session size, and activity between the users whilst remaining tamperproof.

Code containing hidden fields should be rejected during code reviews.

==ASP.NET Viewstate ==

ASP.NET sends form data back to the client in a hidden “Viewstate” field. Despite looking forbidding, this “encryption” is simply plain-text equivalent (base64 encoding) and has no data integrity without further action on your behalf in ASP.NET 1.0. In ASP.NET 1.1 and 2.0, tamper proofing, called "enableViewStateMAC" is on by default using a SHA/1 hash.

Any application framework with a similar mechanism might be at fault – you should investigate your application framework’s support for sending data back to the user. Preferably it should not round trip.

===How to determine if you are vulnerable ===

These configurations are set hierarchically in the .NET framework. The machine.config file contains the global configuration; each web directory may contain a web.config file further specifying or overriding configuration; each page may contain @page directives specifying same configuration or overrides; you must check all three locations:

* If the enableViewStateMac is not set to “true”, you are at risk if your viewstate contains authorization state

* If the viewStateEncryptionMode is not set to “always”, you are at risk if your viewstate contains secrets such as credentials

* If you share a host with many other customers, you all share the same machine key by default in ASP.NET 1.1. In ASP.NET 2.0, it is possible to configure unique viewstate keys per application

===How to protect yourself ===

* If your application relies on data returning from the viewstate without being tampered with, you should turn on viewstate integrity checks at the least, and strongly consider:

* Encrypt viewstate if any of the data is application sensitive

* Upgrade to ASP.NET 2.0 as soon as practical if you are on a shared hosting arrangement

* Move truly sensitive viewstate data to the session variable instead

===Selects, radio buttons, and checkboxes ===

It is commonly held belief that the value settings for these items cannot be easily tampered. This is wrong. In the following example, actual account numbers are used, which can lead to compromise:

''<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">''

''<bean:message key="msg.card.name" arg0="<%=acct.getCardName(1).toString( )%>" />''

''<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">''

''<bean:message key="msg.card.name" arg0="<%=acct.getCardName(2).toString( )%>" />''

This produces (for example):

''<input type="radio" name="acctNo" value="455712341234">Gold Card''

''<input type="radio" name="acctNo" value="455712341235">Platinum Card''

If the value is retrieved and then used directly in a SQL query, an interesting form of SQL injection may occur: authorization tampering leading to information disclosure. As the connection pool connects to the database using a single user, it may be possible to see other user's accounts if the SQL looks something like this:

''String acctNo = getParameter('acctNo');''

''String sql = "SELECT acctBal FROM accounts WHERE acctNo = '?'";''

''PreparedStatement st = conn.prepareStatement(sql);''

''st.setString(1, acctNo);''

''ResultSet rs = st.executeQuery();''

This should be re-written to retrieve the account number via index, and include the client's unique ID to ensure that other valid account numbers are exposed:

''String acctNo = acct.getCardNumber(getParameter('acctIndex'));''

''String sql = "SELECT acctBal FROM accounts WHERE acct_id = '?' AND acctNo = '?'";''

''PreparedStatement st = conn.prepareStatement(sql);''

''st.setString(1, acct.getID());''

''st.setString(2, acctNo);''

''ResultSet rs = st.executeQuery();''

This approach requires rendering input values from 1 to ... x, and assuming accounts are stored in a Collection which can be iterated using logic:iterate:

''<logic:iterate id="loopVar" name="MyForm" property="values">''

'' <html:radio property="acctIndex" idName="loopVar" value="value"/> ''

'' <bean:write name="loopVar" property="name"/> ''

''</logic:iterate>''

The code will emit HTML with the values "1" .. "x" as per the collection's content.

''<input type="radio" name="acctIndex" value="1" />Gold Credit Card''

''<input type="radio" name="acctIndex" value="2" />Platinum Credit Card''

This approach should be used for any input type that allows a value to be set: radio buttons, checkboxes, and particularly select / option lists.

===Per-User Data ===

In fully normalized databases, the aim is to minimize the amount of repeated data. However, some data is inferred. For example, users can see messages that are stored in a messages table. Some messages are private to the user. However, in a fully normalized database, the list of message IDs are kept within another table:

If a user marks a message for deletion, the usual way is to recover the message ID from the user, and delete that:

''DELETE FROM message WHERE msgid='frmMsgId'''

However, how do you know if the user is eligible to delete that message ID? Such tables need to be denormalized slightly to include a user ID or make it easy to perform a single query to delete the message safely. For example, by adding back an (optional) uid column, the delete is now made reasonably safe:

''DELETE FROM message WHERE uid='session.myUserID' and msgid='frmMsgId';''

Where the data is potentially both a private resource and a public resource (for example, in the secure message service, broadcast messages are just a special type of private message), additional precautions need to be taken to prevent users from deleting public resources without authorization. This can be done using role based checks, as well as using SQL statements to discriminate by message type:

''DELETE FROM message ''

''WHERE''

''uid='session.myUserID' AND''

''msgid='frmMsgId' AND''

''broadcastFlag = false;''

==URL encoding ==

Data sent via the URL, which is strongly discouraged, should be URL encoded and decoded. This reduces the likelihood of cross-site scripting attacks from working.

In general, do not send data via GET request unless for navigational purposes.

==HTML encoding ==

Data sent to the user needs to be safe for the user to view. This can be done using <bean:write ...> and friends. Do not use <%=var%> unless it is used to supply an argument for <bean:write...> or similar.

HTML encoding translates a range of characters into their HTML entities. For example, > becomes &gt; This will still display as > on the user's browser, but it is a safe alternative.

==Encoded strings ==

Some strings may be received in encoded form. It is essential to send the correct locale to the user so that the web server and application server can provide a single level of canoncalization prior to the first use.

Do not use getReader() or getInputStream() as these input methods do not decode encoded strings. If you need to use these constructs, you must decanoncalize data by hand.

==Data Validation and Interpreter Injection ==

This section focuses on preventing injection in ColdFusion. Interpreter Injection involves manipulating application parameters to execute malicious code on the system. The most prevalent of these is SQL injection but it also includes other injection techniques, including LDAP, ORM, User Agent, XML, etc. – see the Interpreter Injection chapter of this document for greater details. As a developer you should assume that all input is malicious. Before processing any input coming from a user, data source, component, or data service it should be validated for type, length, and/or range. ColdFusion includes support for Regular Expressions and CFML tags that can be used to validate input.

'''SQL Injection'''

SQL Injection involves sending extraneous SQL queries as variables. ColdFusion provides the <cfqueryparam> and <cfprocparam> tags for validating database parameters. These tags nests inside <cfquery> and <cfstoredproc>, respectively. For dynamic SQL submitted in <cfquery>, use the CFSQLTYPE attribute of the <cfqueryparam> to validate variables against the expected database datatype. Similarly, use the CFSQLTYPE attribute of <cfprocparam> to validate the datatypes of stored procedure parameters passed through <cfstoredproc>.

You can also strengthen your systems against SQL Injection by disabling the Allowed SQL operations for individual data sources. See the '''Configuration''' section below for more information.

'''LDAP Injection'''

ColdFusion uses the <cfldap> tag to communicate with LDAP servers. This tag has an ACTION attribute which dictates the query performed against the LDAP. The valid values for this attribute are: add, delete, query (default), modify, and modifyDN. <cfldap> calls are turned into JNDI (Java Naming And Directory Interface) lookups. However, because <cfldap> wraps the calls, it will throw syntax errors if native JNDI code is passed to its attributes making LDAP injection more difficult.

'''XML Injection'''

Two parsers exist for XML data – SAX and DOM. ColdFusion uses DOM which reads the entire XML document into the server’s memory. This requires the administrator to restrict the size of the JVM containing ColdFusion. ColdFusion is built on Java therefore by default, entity references are expanded during parsing. To prevent unbounded entity expansion, before a string is converted to an XML DOM, filter out DOCTYPES elements.

After the DOM has been read, to reduce the risk of XML, Injection use the ColdFusion XML decision functions: isXML(), isXmlAttribute(), isXmlElement(), isXmlNode(), and isXmlRoot(). The isXML() function determines if a string is well-formed XML. The other functions determine whether or not the passed parameter is a valid part of an XML document. Use the xmlValidate() function to validate external XML documents against a Document Type Definition (DTD) or XML Schema.

'''Event Gateway, IM, and SMS Injection'''

ColdFusion MX 7 enables Event Gateways, instant messaging (IM), and SMS (short message service) for interacting with external systems. Event Gateways are ColdFusion components that respond asynchronously to non-HTTP requests – e.g. instant messages, SMS text from wireless devices, etc. ColdFusion provides Lotus Sametime and XMPP (Extensible Messaging and Presence Protocol) gateways for instant messaging. It also provides an event gateway for interacting with SMS text messages.

Injection along these gateways can happen when end users (and/or systems) send malicious code to execute on the server. These gateways all utilize ColdFusion Components (CFCs) for processing. Use standard ColdFusion functions, tags, and validation techniques to protect against malicious code injection. Sanitize all input strings and do not allow un-validated code to access backend systems.

'''Best Practices'''

Use the XML functions to validate XML input.

Before performing XPath searches and transformations in ColdFusion, validate the source before executing.

Use ColdFusion validation techniques to sanitize strings passed to xmlSearch for performing XPath queries.

When performing XML transformations only use a trusted source for the XSL stylesheet.

Ensure that the memory size of the Java Sandbox containing ColdFusion can handle large XML documents without adversely affecting server resources.

Set the memory value to less than the amount of RAM on the server (-Xmx)

Remove DOCTYPE elements from the XML string before converting it to an XML object.

Using scriptProtect can be used to thwart most attempts of cross-site scripting. Set scriptProtect to All in the Application.cfc

Use <cfparam> or <cfargument> to instantiate variables in ColdFusion. Use this tag with the name and type attributes. If the value is not of the specified type, ColdFusion returns an error.

To handle untyped variables use IsValid() to validate its value against any legal object type that ColdFusion supports.

Use <cfqueryparam> and <cfprocparam> to valid dynamic SQL variables against database datatypes

Use CFLDAP for accessing LDAP servers. Avoid allowing native JNDI calls to connect to LDAP

'''Best Practice in Action'''

The sample code below shows a database authentication function using some of the input validation techniques discussed in this section.

{| border=1

||
* <cffunction name="dblogin" access="private" output="false" returntype="struct">

* <cfargument name="strUserName" required="true" type="string">

* <cfargument name="strPassword" required="true" type="string">

* <cfset var retargs = StructNew()>

* <cfif IsValid("regex", uUserName, "[A-Za-z0-9%]*") AND IsValid("regex", uPassword, "[A-Za-z0-9%]*")>

* <cfquery name="loginQuery" dataSource="#Application.DB#" >

* SELECT hashed_password, salt

* FROM UserTable

* WHERE UserName =

* <cfqueryparam value="#strUserName#" cfsqltype="CF_SQL_VARCHAR" maxlength="25">

* </cfquery>

* <cfif loginQuery.hashed_password EQ Hash(strPassword & loginQuery.salt, "SHA-256" )>

* <cfset retargs.authenticated="YES">

* <cfset Session.UserName = strUserName>

* 

* <cfelse>

* <cfset retargs.authenticated="NO">

* </cfif>

* <cfelse>

* <cfset retargs.authenticated="NO">

* </cfif>

* <cfreturn retargs>

* </cffunction>

|-

|}

==Delimiter and special characters ==

There are many characters that mean something special to various programs. If you followed the advice only to accept characters that are considered good, it is very likely that only a few delimiters will catch you out.

Here are the usual suspects:

* NULL (zero) %00

* LF - ANSI chr(10) "\r"

* CR - ANSI chr(13) "\n"

* CRLF - "\n\r"

* CR - EBCDIC 0x0f

* Quotes " '

* Commas, slashes spaces and tabs and other white space - used in CSV, tab delimited output, and other specialist formats

* <> - XML and HTML tag markers, redirection characters

* ; & - Unix and NT file system continuance

* @ - used for e-mail addresses

* 0xff

* ... more

Whenever you code to a particular technology, you should determine which characters are "special" and prevent them appearing in input, or properly escaping them.

==Further Reading ==

* ASP.NET 2.0 Viewstate

http://channel9.msdn.com/wiki/default.aspx/Channel9.HowToConfigureTheMachineKeyInASPNET2

[[Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Validation]]
[[Category:Encoding]]

Data Validation

2007-08-14T19:05:07Z

Dkaplan: /* Wrong way */

[[Guide Table of Contents]]__TOC__

==Objective ==

To ensure that the application is robust against all forms of input data, whether obtained from the user, infrastructure, external entities or database systems

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS11 – Manage Data. All sections should be reviewed

==Description ==

The most common web application security weakness is the failure to properly validate input from the client or environment. This weakness leads to almost all of the major vulnerabilities in applications, such as interpreter injection, locale/Unicode attacks, file system attacks and buffer overflows. Data from the client should never be trusted for the client has every possibility to tamper with the data.

In many cases, [[Encoding]] has the potential to defuse attacks that rely on lack of input validation. For example, if you use HTML entity encoding on user input before it is sent to a browser, it will prevent most [[XSS]] attacks. However, simply preventing attacks is not enough - you must perform [[Intrusion Detection]] in your applications. Otherwise, you are allowing attackers to repeatedly attack your application until they find a vulnerability that you haven't protected against. Detecting attempts to find these weaknesses is a critical protection mechanism.

==Definitions ==

These definitions are used within this document:

* '''Integrity checks'''

Ensure that the data has not been tampered with and is the same as before

* '''Validation'''

Ensure that the data is strongly typed, correct syntax, within length boundaries, contains only permitted characters, or that numbers are correctly signed and within range boundaries

* '''Business rules'''

Ensure that data is not only validated, but business rule correct. For example, interest rates fall within permitted boundaries.

Some documentation and references interchangeably use the various meanings, which is very confusing to all concerned. This confusion directly causes continuing financial loss to the organization.

==Where to include integrity checks ==

Integrity checks must be included wherever data passes from a trusted to a less trusted boundary, such as from the application to the user's browser in a hidden field, or to a third party payment gateway, such as a transaction ID used internally upon return.

The type of integrity control (checksum, HMAC, encryption, digital signature) should be directly related to the risk of the data transiting the trust boundary.

==Where to include validation ==

Validation must be performed on every tier. However, validation should be performed as per the function of the server executing the code. For example, the web / presentation tier should validate for web related issues, persistence layers should validate for persistence issues such as SQL / HQL injection, directory lookups should check for LDAP injection, and so on.

==Where to include business rule validation ==

Business rules are known during design, and they influence implementation. However, there are bad, good and "best" approaches. Often the best approach is the simplest in terms of code.

===Example - Scenario ===

* You are to populate a list with accounts provided by the back-end system
* The user will choose an account, choose a biller, and press next

===Wrong way===

The account select option is read directly and provided in a message back to the backend system without validating the account number if one of the accounts provided by the backend system.

===Why this is bad===

An attacker can change the HTML in any way they choose:

* The lack of validation requires a round-trip to the backend to provide an error message that the front end code could easily have eliminated

* The back end may not be able to cope with the data payload the front-end code could have easily eliminated. For example, buffer overflows, XML injection, or similar.

===Acceptable Method ===

The account select option parameter ("payee_id") is read by the code, and compared to an already-known list.

<code>
<pre>
if (account.hasPayee( session.getParameter("payee_id") )) {
backend.performTransfer( session.getParameter("payee_id") );
}
</pre>
</code>

This prevents parameter tampering, but requires the list of possible payee_id's to be to be calculated beforehand.

===Best Method ===

The original code emitted indexes <option value="1" ... > rather than account names.

''int payeeLstId = session.getParameter('payeelstid');''
''accountFrom = account.getAcctNumberByIndex(payeeLstId);''

Not only is this easier to render in HTML, it makes validation and business rule validation trivial. The field cannot be tampered with.

===Conclusion ===

To provide defense in depth and to prevent attack payloads from trust boundaries, such as backend hosts, which are probably incapable of handling arbitrary input data, business rule validation is to be performed (preferably in workflow or command patterns), even if it is known that the back end code performs business rule validation.

This is not to say that the entire set of business rules need be applied - it means that the fundamentals are performed to prevent unnecessary round trips to the backend and to prevent the backend from receiving most tampered data.

==Data Validation Strategies ==

There are four strategies for validating data, and they should be used in this order:

===Accept known good===

This strategy is also known as "whitelist" or "positive" validation. The idea is that you should check that the data is one of a set of tightly constrained known good values. Any data that doesn't match should be rejected. Data should be:

* Strongly typed at all times
* Length checked and fields length minimized
* Range checked if a numeric
* Unsigned unless required to be signed
* Syntax or grammar should be checked prior to first use or inspection

If you expect a postcode, validate for a postcode (type, length and syntax):

<code>
<pre>
public String isPostcode(String postcode) {
return (postcode != null && Pattern.matches("^(((2|8|9)\d{2})|((02|08|09)\d{2})|([1-9]\d{3}))$", postcode)) ? postcode : "";
}
</pre>
</code>

Coding guidelines should use some form of visible tainting on input from the client or untrusted sources, such as third party connectors to make it obvious that the input is unsafe:

<code>
<pre>
String taintPostcode = request.getParameter("postcode");
ValidationEngine validator = new ValidationEngine();
boolean isValidPostcode = validator.isPostcode(taintPostcode);
</pre>
</code>

===Reject known bad===

This strategy, also known as "negative" or "blacklist" validation is a weak alternative to positive validation. Essentially, if you don't expect to see characters such as %3f or JavaScript or similar, reject strings containing them. This is a dangerous strategy, because the set of possible bad data is potentially infinite. Adopting this strategy means that you will have to maintain the list of "known bad" characters and patterns forever, and you will by definition have incomplete protection.

''public String removeJavascript(String input) {''

'' Pattern p = Pattern.compile("javascript", CASE_INSENSITIVE);''

'' p.matcher(input);''

'' return (!p.matches()) ? input : '';''

''}''

It can take upwards of 90 regular expressions (see the CSS Cheat Sheet in the Guide 2.0) to eliminate known malicious software, and each regex needs to be run over every field. Obviously, this is slow and not secure. Just rejecting "current known bad" (which is at the time of writing hundreds of strings and literally millions of combinations) is insufficient if the input is a string. This strategy is directly akin to anti-virus pattern updates. Unless the business will allow updating "bad" regexes on a daily basis and support someone to research new attacks regularly, this approach will be obviated before long.

===Sanitize===

Eliminate or translate characters (such as to HTML entities or to remove quotes) in an effort to make the input "safe".
Like blacklists, this approach requires maintenance and is usually incomplete. As most fields have a particular grammar, it is simpler, faster, and more secure to simply validate a single correct positive test than to try to include complex and slow sanitization routines for all current and future attacks.

<code>
<pre>
public String quoteApostrophe(String input) {
if (input != null)
return input.replaceAll("[\']", "&rsquo;");
else
return null;
}
</pre>
</code>

===No validation===

This is inherently unsafe and strongly discouraged. The business must sign off each and every example of no validation as the lack of validation usually leads to direct obviation of application, host and network security controls.

''account.setAcctId(getParameter('formAcctNo'));''
''...''

''public setAcctId(String acctId) {''
'' cAcctId = acctId;''
''}''

==Prevent parameter tampering ==

There are many input sources:

* HTTP headers, such as REMOTE_ADDR, PROXY_VIA or similar

* Environment variables, such as getenv() or via server properties

* All GET, POST and Cookie data

This includes supposedly tamper resistant fields such as radio buttons, drop downs, etc - any client side HTML can be re-written to suit the attacker

Configuration data (mistakes happen :))

External systems (via any form of input mechanism, such as XML input, RMI, web services, etc)

All of these data sources supply untrusted input. Data received from untrusted data sources must be properly checked before first use.

==Hidden fields ==

Hidden fields are a simple way to avoid storing state on the server. Their use is particularly prevalent in "wizard-style" multi-page forms. However, their use exposes the inner workings of your application, and exposes data to trivial tampering, replay, and validation attacks. In general, only use hidden fields for page sequence.

If you have to use hidden fields, there are some rules:

* Secrets, such as passwords, should never be sent in the clear

* Hidden fields need to have integrity checks and preferably encrypted using non-constant initialization vectors (i.e. different users at different times have different yet cryptographically strong random IVs)

* Encrypted hidden fields must be robust against replay attacks, which means some form of temporal keying

* Data sent to the user must be validated on the server once the last page has been received, even if it has been previously validated on the server - this helps reduce the risk from replay attacks.

The preferred integrity control should be at least a HMAC using SHA-256 or preferably digitally signed or encrypted using PGP. IBMJCE supports SHA-256, but PGP JCE support require the inclusion of the Legion of the Bouncy Castle (http://www.bouncycastle.org/) JCE classes.

It is simpler to store this data temporarily in the session object. Using the session object is the safest option as data is never visible to the user, requires (far) less code, nearly no CPU, disk or I/O utilization, less memory (particularly on large multi-page forms), and less network consumption.

In the case of the session object being backed by a database, large session objects may become too large for the inbuilt handler. In this case, the recommended strategy is to store the validated data in the database, but mark the transaction as "incomplete". Each page will update the incomplete transaction until it is ready for submission. This minimizes the database load, session size, and activity between the users whilst remaining tamperproof.

Code containing hidden fields should be rejected during code reviews.

==ASP.NET Viewstate ==

ASP.NET sends form data back to the client in a hidden “Viewstate” field. Despite looking forbidding, this “encryption” is simply plain-text equivalent (base64 encoding) and has no data integrity without further action on your behalf in ASP.NET 1.0. In ASP.NET 1.1 and 2.0, tamper proofing, called "enableViewStateMAC" is on by default using a SHA/1 hash.

Any application framework with a similar mechanism might be at fault – you should investigate your application framework’s support for sending data back to the user. Preferably it should not round trip.

===How to determine if you are vulnerable ===

These configurations are set hierarchically in the .NET framework. The machine.config file contains the global configuration; each web directory may contain a web.config file further specifying or overriding configuration; each page may contain @page directives specifying same configuration or overrides; you must check all three locations:

* If the enableViewStateMac is not set to “true”, you are at risk if your viewstate contains authorization state

* If the viewStateEncryptionMode is not set to “always”, you are at risk if your viewstate contains secrets such as credentials

* If you share a host with many other customers, you all share the same machine key by default in ASP.NET 1.1. In ASP.NET 2.0, it is possible to configure unique viewstate keys per application

===How to protect yourself ===

* If your application relies on data returning from the viewstate without being tampered with, you should turn on viewstate integrity checks at the least, and strongly consider:

* Encrypt viewstate if any of the data is application sensitive

* Upgrade to ASP.NET 2.0 as soon as practical if you are on a shared hosting arrangement

* Move truly sensitive viewstate data to the session variable instead

===Selects, radio buttons, and checkboxes ===

It is commonly held belief that the value settings for these items cannot be easily tampered. This is wrong. In the following example, actual account numbers are used, which can lead to compromise:

''<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">''

''<bean:message key="msg.card.name" arg0="<%=acct.getCardName(1).toString( )%>" />''

''<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">''

''<bean:message key="msg.card.name" arg0="<%=acct.getCardName(2).toString( )%>" />''

This produces (for example):

''<input type="radio" name="acctNo" value="455712341234">Gold Card''

''<input type="radio" name="acctNo" value="455712341235">Platinum Card''

If the value is retrieved and then used directly in a SQL query, an interesting form of SQL injection may occur: authorization tampering leading to information disclosure. As the connection pool connects to the database using a single user, it may be possible to see other user's accounts if the SQL looks something like this:

''String acctNo = getParameter('acctNo');''

''String sql = "SELECT acctBal FROM accounts WHERE acctNo = '?'";''

''PreparedStatement st = conn.prepareStatement(sql);''

''st.setString(1, acctNo);''

''ResultSet rs = st.executeQuery();''

This should be re-written to retrieve the account number via index, and include the client's unique ID to ensure that other valid account numbers are exposed:

''String acctNo = acct.getCardNumber(getParameter('acctIndex'));''

''String sql = "SELECT acctBal FROM accounts WHERE acct_id = '?' AND acctNo = '?'";''

''PreparedStatement st = conn.prepareStatement(sql);''

''st.setString(1, acct.getID());''

''st.setString(2, acctNo);''

''ResultSet rs = st.executeQuery();''

This approach requires rendering input values from 1 to ... x, and assuming accounts are stored in a Collection which can be iterated using logic:iterate:

''<logic:iterate id="loopVar" name="MyForm" property="values">''

'' <html:radio property="acctIndex" idName="loopVar" value="value"/> ''

'' <bean:write name="loopVar" property="name"/> ''

''</logic:iterate>''

The code will emit HTML with the values "1" .. "x" as per the collection's content.

''<input type="radio" name="acctIndex" value="1" />Gold Credit Card''

''<input type="radio" name="acctIndex" value="2" />Platinum Credit Card''

This approach should be used for any input type that allows a value to be set: radio buttons, checkboxes, and particularly select / option lists.

===Per-User Data ===

In fully normalized databases, the aim is to minimize the amount of repeated data. However, some data is inferred. For example, users can see messages that are stored in a messages table. Some messages are private to the user. However, in a fully normalized database, the list of message IDs are kept within another table:

If a user marks a message for deletion, the usual way is to recover the message ID from the user, and delete that:

''DELETE FROM message WHERE msgid='frmMsgId'''

However, how do you know if the user is eligible to delete that message ID? Such tables need to be denormalized slightly to include a user ID or make it easy to perform a single query to delete the message safely. For example, by adding back an (optional) uid column, the delete is now made reasonably safe:

''DELETE FROM message WHERE uid='session.myUserID' and msgid='frmMsgId';''

Where the data is potentially both a private resource and a public resource (for example, in the secure message service, broadcast messages are just a special type of private message), additional precautions need to be taken to prevent users from deleting public resources without authorization. This can be done using role based checks, as well as using SQL statements to discriminate by message type:

''DELETE FROM message ''

''WHERE''

''uid='session.myUserID' AND''

''msgid='frmMsgId' AND''

''broadcastFlag = false;''

==URL encoding ==

Data sent via the URL, which is strongly discouraged, should be URL encoded and decoded. This reduces the likelihood of cross-site scripting attacks from working.

In general, do not send data via GET request unless for navigational purposes.

==HTML encoding ==

Data sent to the user needs to be safe for the user to view. This can be done using <bean:write ...> and friends. Do not use <%=var%> unless it is used to supply an argument for <bean:write...> or similar.

HTML encoding translates a range of characters into their HTML entities. For example, > becomes &gt; This will still display as > on the user's browser, but it is a safe alternative.

==Encoded strings ==

Some strings may be received in encoded form. It is essential to send the correct locale to the user so that the web server and application server can provide a single level of canoncalization prior to the first use.

Do not use getReader() or getInputStream() as these input methods do not decode encoded strings. If you need to use these constructs, you must decanoncalize data by hand.

==Data Validation and Interpreter Injection ==

This section focuses on preventing injection in ColdFusion. Interpreter Injection involves manipulating application parameters to execute malicious code on the system. The most prevalent of these is SQL injection but it also includes other injection techniques, including LDAP, ORM, User Agent, XML, etc. – see the Interpreter Injection chapter of this document for greater details. As a developer you should assume that all input is malicious. Before processing any input coming from a user, data source, component, or data service it should be validated for type, length, and/or range. ColdFusion includes support for Regular Expressions and CFML tags that can be used to validate input.

'''SQL Injection'''

SQL Injection involves sending extraneous SQL queries as variables. ColdFusion provides the <cfqueryparam> and <cfprocparam> tags for validating database parameters. These tags nests inside <cfquery> and <cfstoredproc>, respectively. For dynamic SQL submitted in <cfquery>, use the CFSQLTYPE attribute of the <cfqueryparam> to validate variables against the expected database datatype. Similarly, use the CFSQLTYPE attribute of <cfprocparam> to validate the datatypes of stored procedure parameters passed through <cfstoredproc>.

You can also strengthen your systems against SQL Injection by disabling the Allowed SQL operations for individual data sources. See the '''Configuration''' section below for more information.

'''LDAP Injection'''

ColdFusion uses the <cfldap> tag to communicate with LDAP servers. This tag has an ACTION attribute which dictates the query performed against the LDAP. The valid values for this attribute are: add, delete, query (default), modify, and modifyDN. <cfldap> calls are turned into JNDI (Java Naming And Directory Interface) lookups. However, because <cfldap> wraps the calls, it will throw syntax errors if native JNDI code is passed to its attributes making LDAP injection more difficult.

'''XML Injection'''

Two parsers exist for XML data – SAX and DOM. ColdFusion uses DOM which reads the entire XML document into the server’s memory. This requires the administrator to restrict the size of the JVM containing ColdFusion. ColdFusion is built on Java therefore by default, entity references are expanded during parsing. To prevent unbounded entity expansion, before a string is converted to an XML DOM, filter out DOCTYPES elements.

After the DOM has been read, to reduce the risk of XML, Injection use the ColdFusion XML decision functions: isXML(), isXmlAttribute(), isXmlElement(), isXmlNode(), and isXmlRoot(). The isXML() function determines if a string is well-formed XML. The other functions determine whether or not the passed parameter is a valid part of an XML document. Use the xmlValidate() function to validate external XML documents against a Document Type Definition (DTD) or XML Schema.

'''Event Gateway, IM, and SMS Injection'''

ColdFusion MX 7 enables Event Gateways, instant messaging (IM), and SMS (short message service) for interacting with external systems. Event Gateways are ColdFusion components that respond asynchronously to non-HTTP requests – e.g. instant messages, SMS text from wireless devices, etc. ColdFusion provides Lotus Sametime and XMPP (Extensible Messaging and Presence Protocol) gateways for instant messaging. It also provides an event gateway for interacting with SMS text messages.

Injection along these gateways can happen when end users (and/or systems) send malicious code to execute on the server. These gateways all utilize ColdFusion Components (CFCs) for processing. Use standard ColdFusion functions, tags, and validation techniques to protect against malicious code injection. Sanitize all input strings and do not allow un-validated code to access backend systems.

'''Best Practices'''

Use the XML functions to validate XML input.

Before performing XPath searches and transformations in ColdFusion, validate the source before executing.

Use ColdFusion validation techniques to sanitize strings passed to xmlSearch for performing XPath queries.

When performing XML transformations only use a trusted source for the XSL stylesheet.

Ensure that the memory size of the Java Sandbox containing ColdFusion can handle large XML documents without adversely affecting server resources.

Set the memory value to less than the amount of RAM on the server (-Xmx)

Remove DOCTYPE elements from the XML string before converting it to an XML object.

Using scriptProtect can be used to thwart most attempts of cross-site scripting. Set scriptProtect to All in the Application.cfc

Use <cfparam> or <cfargument> to instantiate variables in ColdFusion. Use this tag with the name and type attributes. If the value is not of the specified type, ColdFusion returns an error.

To handle untyped variables use IsValid() to validate its value against any legal object type that ColdFusion supports.

Use <cfqueryparam> and <cfprocparam> to valid dynamic SQL variables against database datatypes

Use CFLDAP for accessing LDAP servers. Avoid allowing native JNDI calls to connect to LDAP

'''Best Practice in Action'''

The sample code below shows a database authentication function using some of the input validation techniques discussed in this section.

{| border=1

||
* <cffunction name="dblogin" access="private" output="false" returntype="struct">

* <cfargument name="strUserName" required="true" type="string">

* <cfargument name="strPassword" required="true" type="string">

* <cfset var retargs = StructNew()>

* <cfif IsValid("regex", uUserName, "[A-Za-z0-9%]*") AND IsValid("regex", uPassword, "[A-Za-z0-9%]*")>

* <cfquery name="loginQuery" dataSource="#Application.DB#" >

* SELECT hashed_password, salt

* FROM UserTable

* WHERE UserName =

* <cfqueryparam value="#strUserName#" cfsqltype="CF_SQL_VARCHAR" maxlength="25">

* </cfquery>

* <cfif loginQuery.hashed_password EQ Hash(strPassword & loginQuery.salt, "SHA-256" )>

* <cfset retargs.authenticated="YES">

* <cfset Session.UserName = strUserName>

* 

* <cfelse>

* <cfset retargs.authenticated="NO">

* </cfif>

* <cfelse>

* <cfset retargs.authenticated="NO">

* </cfif>

* <cfreturn retargs>

* </cffunction>

|-

|}

==Delimiter and special characters ==

There are many characters that mean something special to various programs. If you followed the advice only to accept characters that are considered good, it is very likely that only a few delimiters will catch you out.

Here are the usual suspects:

* NULL (zero) %00

* LF - ANSI chr(10) "\r"

* CR - ANSI chr(13) "\n"

* CRLF - "\n\r"

* CR - EBCDIC 0x0f

* Quotes " '

* Commas, slashes spaces and tabs and other white space - used in CSV, tab delimited output, and other specialist formats

* <> - XML and HTML tag markers, redirection characters

* ; & - Unix and NT file system continuance

* @ - used for e-mail addresses

* 0xff

* ... more

Whenever you code to a particular technology, you should determine which characters are "special" and prevent them appearing in input, or properly escaping them.

==Further Reading ==

* ASP.NET 2.0 Viewstate

http://channel9.msdn.com/wiki/default.aspx/Channel9.HowToConfigureTheMachineKeyInASPNET2

[[Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Validation]]
[[Category:Encoding]]

Web Services

2007-08-07T19:00:20Z

Dkaplan: /* Communication Protection Mechanisms */

[[Guide Table of Contents]]
__TOC__

This section of the Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need a XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations, which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Service Interoperability group (WS-I, see http://www.ws-i.org/) was formed to promote general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and develop so-called profiles, or set of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional token’s attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. Example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' ''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body>''

'' </soap:Body>''

''</soap:Envelope>''

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi... ''

'' </wsse:BinarySecurityToken>''

'' <xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>''

'' <dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference> ''

'' </dsig:KeyInfo>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>Nb0Mf...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' <xenc:ReferenceList>''

'' <xenc:DataReference URI="#aDNa2iD"/>''

'' </xenc:ReferenceList>''

'' </xenc:EncryptedKey>''

'' <wsse:SecurityTokenReference wsu:Id="aZG0sG">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

'' <dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">''

'' <dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">''

'' ...''

'' <dsig:Reference URI="#s91397860">''

'' ... ''

'' <dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>''

'' </dsig:Reference>''

'' </dsig:SignedInfo>''

'' <dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>''

'' <dsig:KeyInfo>''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference>''

'' </dsig:KeyInfo>''

'' </dsig:Signature>''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">''

'' <xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>XFM4J6C...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' </xenc:EncryptedData>''

'' </soap:Body>''

''</soap:Envelope>''

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

''Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )''

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">''

'' MIICtzCCAi...''

'' </wsse:BinarySecurityToken>''

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.

Typically, a XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, a SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here is just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained all time while client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, DataPower provides a nice XS40 XML Security Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of this standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, can not always be used as an optimal solution, as they can not be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices

==Reference
[[Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2007-08-07T18:02:09Z

Dkaplan: /* XML security specifications (XML-dsig & Encryption) */

[[Guide Table of Contents]]
__TOC__

This section of the Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however,in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need a XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations, which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Service Interoperability group (WS-I, see http://www.ws-i.org/) was formed to promote general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and develop so-called profiles, or set of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional token’s attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. Example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' ''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body>''

'' </soap:Body>''

''</soap:Envelope>''

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi... ''

'' </wsse:BinarySecurityToken>''

'' <xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>''

'' <dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference> ''

'' </dsig:KeyInfo>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>Nb0Mf...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' <xenc:ReferenceList>''

'' <xenc:DataReference URI="#aDNa2iD"/>''

'' </xenc:ReferenceList>''

'' </xenc:EncryptedKey>''

'' <wsse:SecurityTokenReference wsu:Id="aZG0sG">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

'' <dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">''

'' <dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">''

'' ...''

'' <dsig:Reference URI="#s91397860">''

'' ... ''

'' <dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>''

'' </dsig:Reference>''

'' </dsig:SignedInfo>''

'' <dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>''

'' <dsig:KeyInfo>''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference>''

'' </dsig:KeyInfo>''

'' </dsig:Signature>''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">''

'' <xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>XFM4J6C...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' </xenc:EncryptedData>''

'' </soap:Body>''

''</soap:Envelope>''

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

''Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )''

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">''

'' MIICtzCCAi...''

'' </wsse:BinarySecurityToken>''

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues, facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.

Typically, a XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, a SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here is just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained all time while client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, DataPower provides a nice XS40 XML Security Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of this standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, can not always be used as an optimal solution, as they can not be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices

==Reference
[[Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2007-08-07T17:53:17Z

Dkaplan: /* Standards committees */

[[Guide Table of Contents]]
__TOC__

This section of the Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need a XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations, which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Service Interoperability group (WS-I, see http://www.ws-i.org/) was formed to promote general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and develop so-called profiles, or set of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where do look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional token’s attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. Example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' ''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body>''

'' </soap:Body>''

''</soap:Envelope>''

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi... ''

'' </wsse:BinarySecurityToken>''

'' <xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>''

'' <dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference> ''

'' </dsig:KeyInfo>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>Nb0Mf...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' <xenc:ReferenceList>''

'' <xenc:DataReference URI="#aDNa2iD"/>''

'' </xenc:ReferenceList>''

'' </xenc:EncryptedKey>''

'' <wsse:SecurityTokenReference wsu:Id="aZG0sG">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

'' <dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">''

'' <dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">''

'' ...''

'' <dsig:Reference URI="#s91397860">''

'' ... ''

'' <dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>''

'' </dsig:Reference>''

'' </dsig:SignedInfo>''

'' <dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>''

'' <dsig:KeyInfo>''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference>''

'' </dsig:KeyInfo>''

'' </dsig:Signature>''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">''

'' <xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>XFM4J6C...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' </xenc:EncryptedData>''

'' </soap:Body>''

''</soap:Envelope>''

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

''Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )''

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">''

'' MIICtzCCAi...''

'' </wsse:BinarySecurityToken>''

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues, facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.

Typically, a XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, a SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here is just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained all time while client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, DataPower provides a nice XS40 XML Security Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of this standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, can not always be used as an optimal solution, as they can not be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices

==Reference
[[Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2007-08-07T07:44:53Z

Dkaplan: /*added What are Web Services*/

[[Guide Table of Contents]]
__TOC__

This section of the Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need a XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations, which are developing and promoting them. There are quite a few industry-wide groups and consortiums, working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Service Interoperability group (WS-I, see http://www.ws-i.org/) was formed to promote general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and develop so-called profiles, or set of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where do look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional token’s attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. Example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' ''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body>''

'' </soap:Body>''

''</soap:Envelope>''

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi... ''

'' </wsse:BinarySecurityToken>''

'' <xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>''

'' <dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference> ''

'' </dsig:KeyInfo>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>Nb0Mf...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' <xenc:ReferenceList>''

'' <xenc:DataReference URI="#aDNa2iD"/>''

'' </xenc:ReferenceList>''

'' </xenc:EncryptedKey>''

'' <wsse:SecurityTokenReference wsu:Id="aZG0sG">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

'' <dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">''

'' <dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">''

'' ...''

'' <dsig:Reference URI="#s91397860">''

'' ... ''

'' <dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>''

'' </dsig:Reference>''

'' </dsig:SignedInfo>''

'' <dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>''

'' <dsig:KeyInfo>''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference>''

'' </dsig:KeyInfo>''

'' </dsig:Signature>''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">''

'' <xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>XFM4J6C...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' </xenc:EncryptedData>''

'' </soap:Body>''

''</soap:Envelope>''

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

''Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )''

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">''

'' MIICtzCCAi...''

'' </wsse:BinarySecurityToken>''

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues, facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.

Typically, a XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, a SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here is just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained all time while client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, DataPower provides a nice XS40 XML Security Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of this standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, can not always be used as an optimal solution, as they can not be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices

==Reference
[[Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

What are web applications?

2007-08-07T07:05:26Z

Dkaplan: /* Large scale applications */

[[Guide Table of Contents]]

__TOC__

In the early days of the web, web sites consisted of static pages, which severely limited interaction with the user. In the early 1990’s, this limitation was removed when web servers were modified to allow communication with server-side custom scripts. No longer were applications just static brochure-ware, edited only by those who knew the arcane mysteries of HTML; with this single change, normal users could interact with the application for the first time.

[[Image:What are web applications.JPG]]

This is a huge and fundamental step towards the web as we know it today. Without interactivity, there would be no e-commerce (such as Amazon), no web e-mail (Hotmail or GMail), no Internet Banking, no blogs, no online share trading, and no web forums or communities like Orkut or Friendster. The static Internet would have been vastly different to today.

The trend towards increased interactivity has continued apace, with the advent of “Web 2.0”, a term that encompasses many existing technologies, but heavily features highly interactive, user centric, web-aware applications.

==Technologies ==

Initially, it was quite difficult to write sophisticated applications. The first generation web applications were primitive, usually little more than form submissions and search applications. Even these basic applications took quite a great deal of skill to craft.

Over time, the arcane knowledge required to write applications has been reduced. Today, it is relatively easy to write sophisticated applications with modern platforms and simpler languages, like PHP or VB.NET.

However, this push to make applications as easy to write as possible has a downside – many entry-level programmers are completely unaware of the security implications of their code. This is discussed further in the “Security Principles” chapter.

Let’s look at the various generations of web application technology.

==First generation – CGI ==

Common Gateway Interface (CGI) reigned supreme from approximately 1993 through to the late 1990’s when scripting languages took over in a big way.

CGI works by encapsulating user supplied data in environment variables. These are inherited by the custom written scripts or programs, usually developed in Perl or C. The custom programs process the supplied user data, and send fully formed HTML to the “standard out” (stdout), which is captured by the web server and passed back to the user.

Examples of complex CGI include Hotmail, which was essentially Perl scripts running on top of FreeBSD boxes and Slashdot, again a large Perl script running under Linux

As few sites today write new CGI applications, the techniques to secure CGI applications are not discussed within the Guide. However, many of the techniques discussed can be used with few or no changes.

==Filters ==

Filters can be used to control access to a web site, implement a different web application framework (such as Perl, PHP, or ASP), or to provide a security check.
Because filters live within the execution context of the web server itself, they can be high performance. Typical examples of a filter interface include Apache web server modules, SunONE’s NSAPI, and Microsoft’s ISAPI. Filters can be written in C/C++ in order to integrate with the web/app server at a low-level. However, more recently, filters are being implemented in higher level languages, using interfaces like the J2EE filter API.

==Scripting ==

CGI’s lack of session management and authorization controls hampered the development of commercially useful web applications.

Web developers turned to scripting languages, such as PHP and Ruby to solve these problems. Scripting languages run script code within the web server without being compiled.


Unlike low-level languages, scripting languages rarely suffer from buffer overflows or resource leaks. Thus, programmers who use them can avoid two of the most common security issues. However, they do have their disadvantages:

* Most scripting languages aren’t strongly typed and do not promote good programming practices

* Scripting languages are generally slower than their compiled counterparts (sometimes as much as 100 times slower)

* Scripts often lead to unmanageable code bases that perform poorly as their size grows

* It’s difficult (but not impossible) to write multi-tier large scale applications in scripting languages, because often the presentation, application and data tiers reside on the same machine, thus limiting scalability and security

* Most scripting languages do not natively support remote method or web service calls, which makes it difficult to communicate with application servers and external web services.

Despite their disadvantages, many large and useful applications have been written with scripting languages, such as eGroupWare (egroupware.org), which is written in PHP. Too, many older Internet banking sites are written in ASP.

Scripting frameworks include ASP, Perl, Cold Fusion, and PHP. However, many of these would be considered interpreted hybrids now, particularly later versions of PHP and Cold Fusion, which pre-tokenize and optimize scripts.

==Web application frameworks – J2EE and ASP.NET ==

As scripting languages reached the boundaries of performance and scalability, many larger vendors jumped on Sun’s J2EE web development platform. There are many J2EE implementations, including Tomcat from the Apache Foundation, JBoss and Glassfish.


J2EE:

* Uses the Java language to produce applications which run nearly as quickly as C++ based ones, and that do not easily suffer from buffer overflows and memory leaks

* Allowed large distributed applications to run acceptably for the first time

* Possesses good session and authorization controls

* Enabled relatively transparent multi-tier applications via various remote component invocation mechanisms, and

* Is strongly typed to prevent many common security and programming issues before the program even runs

J2EE’s downside is that it has a steep learning curve which makes it difficult for web designers and entry-level programmers to use it to write applications. While certain graphical development tools make it somewhat easier to program with J2EE, a scripting language like PHP is still much easier to use.

When Microsoft updated their ASP technology to ASP.NET. which mimics the J2EE framework in many ways, they offered several improvements on the development process. For example, .NET:

* Makes it easy for entry level programmers and web designers to whip up smaller applications

* Allows large distributed applications

* Offers good session and authorization controls

* Allows programmers to use their favorite language, which is compiled to native code for excellent performance (near C++ speeds), along with buffer overflow and resource garbage collection

* Permits transparent communication with remote and external components

* Is strongly typed to prevent many common security and programming issues before the program even runs

The choice of J2EE or ASP.NET frameworks is largely dependent upon platform. There is little reason to choose one over the other from a security perspective.

Applications targeting J2EE theoretically can run with few (if any) changes between any of the major vendors and on many platforms from Linux, to AIX, MacOS X, or Windows. (While in practice, some tweaking is required, complete re-writes are not required. )

# ''ASP.Net is primarily available for Microsoft Windows. The Mono project (http://www.go-mono.com/) can run ASP.NET applications on many platforms including Solaris, Netware, and Linux. ''

==Small to medium scale applications ==

Most applications are either small or medium scale. The usual architecture is a simple linear procedural script. This is the most common form of coding for ASP, Cold Fusion and PHP scripts, but rarer (but not impossible) for ASP.NET and J2EE applications.

The reason for this architecture is that it is easy to write, and few skills are required to maintain the code. For smaller applications, any perceived performance benefit from moving to a more scalable architecture will never be recovered in the development time for those applications. For example, if it takes an additional three weeks of developer time to re-factor the scripts into an MVC approach, the three weeks will never be recovered (or noticed by end users) from the improvements in scalability.

It is typical to find many security issues in such applications, including dynamic database queries constructed from insufficiently validated data input, poor error handling and weak authorization controls.

This Guide provides advice throughout to help improve the security of these applications.

==Large scale applications ==

Larger applications need a different architecture to that of a simple survey or feedback form. As applications get larger, it becomes ever more difficult to implement and maintain features and to keep scalability high. Using scalable application architectures becomes a necessity rather than a luxury when an application needs more than about three database tables or presents more than approximately 20 - 50 functions to a user.

Scalable application architecture is often divided into tiers, and if design patterns are used, often broken down into re-usable chunks using specific guidelines to enforce modularity, interface requirements and object re-use. Breaking the application into tiers allows the application to be distributed to various servers, thus improving the scalability of the application at the expense of complexity.

One of the most common web application architectures is model-view-controller (MVC).  The main concept of MVC is to keep the code that displays content (the view) completely separate from the data and business logic (the model). The controller exists to handle user input. Lets use an order form as an example to solidify these terms. The html of the form, its layout and anything you *see* that is not data should be generated by the view. You type your information into the form and click the "purchase" button. Behind the scenes, the controller receives this information and passes it on to the model. The model either accepts the data or rejects it (if, for example, you forgot to enter your name). If the data is accepted, the controller forwards you to the next view (perhaps a view that says you successfully ordered the product). If the data is rejected, the controller may forward you back to the order form view with error messages on the page saying you need to enter your name.

MVC is typical of most Apache Foundation Jakarta Struts applications, and the code-behind ASP.NET can be considered a partial implementation of this approach. For PHP, the WACT project (http://wact.sourceforge.net) aims to implement the MVC paradigm in a PHP friendly fashion.

==View ==

The front-end rendering code, often called the presentation tier, should aim to produce the HTML output for the user with little to no application logic.

As many applications will be internationalized (i.e. contain no localized strings or culture information in the presentation layer), they must use calls into the model (application logic) to obtain the data required to render useful information to the user in their preferred language and culture, script direction, and units.

All user input is directed back to controllers in the application logic.

==Controller ==

The controller (or application logic) takes input from the users and gates it through various workflows that call on the application’s model objects to retrieve, process, or store the data.

Well written controllers centrally server-side validate input data against common security issues before passing the data to the model for processing, and ensure that output is safe or in a ready form for safe output by the view code.

As the application is likely to be internationalized and accessible, the data needs to be in the local language and culture. For example, dates cannot only be in different orders, but an entirely different calendar could be in use. Applications need to be flexible about presenting and storing data. Simply displaying “7/4/2005” is ambiguous to anyone outside a few countries.

==Model ==

Models encapsulate functionality, such as “Account” or “User”. A good model should be transparent to the caller, and provide a method to deal with high-level business processes rather than a thin shim to the data store. For example, a good model will allow pseudo code such as this to exist in the controller:

<pre>
oAccount->TransferFunds(fromAcct, ToAcct, Amount)
</pre>

rather than writing it such as this:

<pre>
if ( oAccount->isMyAcct(fromAcct) &&
amount < oAccount->getMaxTransferLimit() &&
oAccount->getBalance(fromAcct) > amount &&
oAccount->ToAccountExists(ToAcct) )
then
if oAccount->withdraw(fromAcct, Amount) == OK
then oAccount->deposit(ToAcct, Amount)
end if
end if
</pre>

The idea is to encapsulate the actual dirty work into the model code, rather than exposing primitives. If the controller and model are on different machines, the performance difference will be staggering, so it is important for the model to be useful at a high level.

The model is responsible for checking data against business rules, and any residual risks unique to the data store in use. For example, if a model stores data in a flat file, the code needs to check for OS injection commands if the flat files are named by the user. If the model stores data in an interpreted language, such as SQL, then the model is responsible for preventing SQL injection. If it uses a message queue interface to a mainframe, the message queue data format (typically XML) needs to be well formed and compliant with a DTD.

The contract between the controller and the model needs to be carefully considered to ensure that data is strongly typed, with reasonable structure (syntax), and appropriate length, whilst allowing flexibility to allow for internationalization and future needs.

Calls by the model to the data store should be through the most secure method possible. Often the weakest possibility is dynamic queries, where a string is built up from unverified user input. This leads directly to SQL injection and is frowned upon. For more information, see the Interpreter Injections chapter.

The best performance and highest security is often obtained through parameterized stored procedures, followed by parameterized queries (also known as prepared statements) with strong typing of the parameters and schema. The major reason for using stored procedures is to minimize network traffic for a multi-stage transaction or to remove security sensitive information from traversing the network.

Stored procedures are not always a good idea – they tie you to a particular database vendor and many implementations are not fast for numeric computation. If you use the 80/20 rule for optimization and move the slow and high-risk transactions to stored procedures, the wins can be worthwhile from a security and performance perspective.

==Conclusion ==

Web applications can be written in many different ways, and in many different languages. Although the Guide concentrates upon three common choices for its examples (PHP, J2EE and ASP.NET), the Guide can be used with any web application technology.

[[Guide Table of Contents]]

[[Category:OWASP_Guide_Project]]

What are web applications?

2007-08-06T23:56:17Z

Dkaplan: /* Large scale applications */

[[Guide Table of Contents]]

__TOC__

In the early days of the web, web sites consisted of static pages, which severely limited interaction with the user. In the early 1990’s, this limitation was removed when web servers were modified to allow communication with server-side custom scripts. No longer were applications just static brochure-ware, edited only by those who knew the arcane mysteries of HTML; with this single change, normal users could interact with the application for the first time.

[[Image:What are web applications.JPG]]

This is a huge and fundamental step towards the web as we know it today. Without interactivity, there would be no e-commerce (such as Amazon), no web e-mail (Hotmail or GMail), no Internet Banking, no blogs, no online share trading, and no web forums or communities like Orkut or Friendster. The static Internet would have been vastly different to today.

The trend towards increased interactivity has continued apace, with the advent of “Web 2.0”, a term that encompasses many existing technologies, but heavily features highly interactive, user centric, web-aware applications.

==Technologies ==

Initially, it was quite difficult to write sophisticated applications. The first generation web applications were primitive, usually little more than form submissions and search applications. Even these basic applications took quite a great deal of skill to craft.

Over time, the arcane knowledge required to write applications has been reduced. Today, it is relatively easy to write sophisticated applications with modern platforms and simpler languages, like PHP or VB.NET.

However, this push to make applications as easy to write as possible has a downside – many entry-level programmers are completely unaware of the security implications of their code. This is discussed further in the “Security Principles” chapter.

Let’s look at the various generations of web application technology.

==First generation – CGI ==

Common Gateway Interface (CGI) reigned supreme from approximately 1993 through to the late 1990’s when scripting languages took over in a big way.

CGI works by encapsulating user supplied data in environment variables. These are inherited by the custom written scripts or programs, usually developed in Perl or C. The custom programs process the supplied user data, and send fully formed HTML to the “standard out” (stdout), which is captured by the web server and passed back to the user.

Examples of complex CGI include Hotmail, which was essentially Perl scripts running on top of FreeBSD boxes and Slashdot, again a large Perl script running under Linux

As few sites today write new CGI applications, the techniques to secure CGI applications are not discussed within the Guide. However, many of the techniques discussed can be used with few or no changes.

==Filters ==

Filters can be used to control access to a web site, implement a different web application framework (such as Perl, PHP, or ASP), or to provide a security check.
Because filters live within the execution context of the web server itself, they can be high performance. Typical examples of a filter interface include Apache web server modules, SunONE’s NSAPI, and Microsoft’s ISAPI. Filters can be written in C/C++ in order to integrate with the web/app server at a low-level. However, more recently, filters are being implemented in higher level languages, using interfaces like the J2EE filter API.

==Scripting ==

CGI’s lack of session management and authorization controls hampered the development of commercially useful web applications.

Web developers turned to scripting languages, such as PHP and Ruby to solve these problems. Scripting languages run script code within the web server without being compiled.


Unlike low-level languages, scripting languages rarely suffer from buffer overflows or resource leaks. Thus, programmers who use them can avoid two of the most common security issues. However, they do have their disadvantages:

* Most scripting languages aren’t strongly typed and do not promote good programming practices

* Scripting languages are generally slower than their compiled counterparts (sometimes as much as 100 times slower)

* Scripts often lead to unmanageable code bases that perform poorly as their size grows

* It’s difficult (but not impossible) to write multi-tier large scale applications in scripting languages, because often the presentation, application and data tiers reside on the same machine, thus limiting scalability and security

* Most scripting languages do not natively support remote method or web service calls, which makes it difficult to communicate with application servers and external web services.

Despite their disadvantages, many large and useful applications have been written with scripting languages, such as eGroupWare (egroupware.org), which is written in PHP. Too, many older Internet banking sites are written in ASP.

Scripting frameworks include ASP, Perl, Cold Fusion, and PHP. However, many of these would be considered interpreted hybrids now, particularly later versions of PHP and Cold Fusion, which pre-tokenize and optimize scripts.

==Web application frameworks – J2EE and ASP.NET ==

As scripting languages reached the boundaries of performance and scalability, many larger vendors jumped on Sun’s J2EE web development platform. There are many J2EE implementations, including Tomcat from the Apache Foundation, JBoss and Glassfish.


J2EE:

* Uses the Java language to produce applications which run nearly as quickly as C++ based ones, and that do not easily suffer from buffer overflows and memory leaks

* Allowed large distributed applications to run acceptably for the first time

* Possesses good session and authorization controls

* Enabled relatively transparent multi-tier applications via various remote component invocation mechanisms, and

* Is strongly typed to prevent many common security and programming issues before the program even runs

J2EE’s downside is that it has a steep learning curve which makes it difficult for web designers and entry-level programmers to use it to write applications. While certain graphical development tools make it somewhat easier to program with J2EE, a scripting language like PHP is still much easier to use.

When Microsoft updated their ASP technology to ASP.NET. which mimics the J2EE framework in many ways, they offered several improvements on the development process. For example, .NET:

* Makes it easy for entry level programmers and web designers to whip up smaller applications

* Allows large distributed applications

* Offers good session and authorization controls

* Allows programmers to use their favorite language, which is compiled to native code for excellent performance (near C++ speeds), along with buffer overflow and resource garbage collection

* Permits transparent communication with remote and external components

* Is strongly typed to prevent many common security and programming issues before the program even runs

The choice of J2EE or ASP.NET frameworks is largely dependent upon platform. There is little reason to choose one over the other from a security perspective.

Applications targeting J2EE theoretically can run with few (if any) changes between any of the major vendors and on many platforms from Linux, to AIX, MacOS X, or Windows. (While in practice, some tweaking is required, complete re-writes are not required. )

# ''ASP.Net is primarily available for Microsoft Windows. The Mono project (http://www.go-mono.com/) can run ASP.NET applications on many platforms including Solaris, Netware, and Linux. ''

==Small to medium scale applications ==

Most applications are either small or medium scale. The usual architecture is a simple linear procedural script. This is the most common form of coding for ASP, Cold Fusion and PHP scripts, but rarer (but not impossible) for ASP.NET and J2EE applications.

The reason for this architecture is that it is easy to write, and few skills are required to maintain the code. For smaller applications, any perceived performance benefit from moving to a more scalable architecture will never be recovered in the development time for those applications. For example, if it takes an additional three weeks of developer time to re-factor the scripts into an MVC approach, the three weeks will never be recovered (or noticed by end users) from the improvements in scalability.

It is typical to find many security issues in such applications, including dynamic database queries constructed from insufficiently validated data input, poor error handling and weak authorization controls.

This Guide provides advice throughout to help improve the security of these applications.

==Large scale applications ==

Larger applications need a different architecture to that of a simple survey or feedback form. As applications get larger, it becomes ever more difficult to implement and maintain features and to keep scalability high. Using scalable application architectures becomes a necessity rather than a luxury when an application needs more than about three database tables or presents more than approximately 20 - 50 functions to a user.

Scalable application architecture is often divided into tiers, and if design patterns are used, often broken down into re-usable chunks using specific guidelines to enforce modularity, interface requirements and object re-use. Breaking the application into tiers allows the application to be distributed to various servers, thus improving the scalability of the application at the expense of complexity.

One of the most common web application architectures is model-view-controller (MVC).  The main concept of MVC is to keep the code that displays content (the view) completely separate from the data and business logic (the model). The controller exists to handle user input. Lets use an order form as an example to solidify these terms. The html of the form, its layout and anything you *see* should be generated by the view. You type your information into the form and click the "purchase" button. Behind the scenes, the controller receives this information and passes it on to the model. The model either accepts the data or rejects it (if, for example, you forgot to enter your name). If the data is accepted, the controller forwards you to the next view (perhaps a view that says you successfully ordered the product). If the data is rejected, the controller may forward you back to the order form view with error messages on the page saying you need to enter your name.

MVC is typical of most Apache Foundation Jakarta Struts applications, and the code-behind ASP.NET can be considered a partial implementation of this approach. For PHP, the WACT project (http://wact.sourceforge.net) aims to implement the MVC paradigm in a PHP friendly fashion.

==View ==

The front-end rendering code, often called the presentation tier, should aim to produce the HTML output for the user with little to no application logic.

As many applications will be internationalized (i.e. contain no localized strings or culture information in the presentation layer), they must use calls into the model (application logic) to obtain the data required to render useful information to the user in their preferred language and culture, script direction, and units.

All user input is directed back to controllers in the application logic.

==Controller ==

The controller (or application logic) takes input from the users andgates it through various workflows that call on the application’s model objects to retrieve, process, or store the data.

Well written controllers centrally server-side validate input data against common security issues before passing the data to the model for processing, and ensure that output is safe or in a ready form for safe output by the view code.

As the application is likely to be internationalized and accessible, the data needs to be in the local language and culture. For example, dates cannot only be in different orders, but an entirely different calendar could be in use. Applications need to be flexible about presenting and storing data. Simply displaying “7/4/2005” is ambiguous to anyone outside a few countries.

==Model ==

Models encapsulate functionality, such as “Account” or “User”. A good model should be transparent to the caller, and provide a method to deal with high-level business processes rather than a thin shim to the data store. For example, a good model will allow pseudo code such as this to exist in the controller:

<pre>
oAccount->TransferFunds(fromAcct, ToAcct, Amount)
</pre>

rather than writing it such as this:

<pre>
if ( oAccount->isMyAcct(fromAcct) &&
amount < oAccount->getMaxTransferLimit() &&
oAccount->getBalance(fromAcct) > amount &&
oAccount->ToAccountExists(ToAcct) )
then
if oAccount->withdraw(fromAcct, Amount) == OK
then oAccount->deposit(ToAcct, Amount)
end if
end if
</pre>

The idea is to encapsulate the actual dirty work into the model code, rather than exposing primitives. If the controller and model are on different machines, the performance difference will be staggering, so it is important for the model to be useful at a high level.

The model is responsible for checking data against business rules, and any residual risks unique to the data store in use. For example, if a model stores data in a flat file, the code needs to check for OS injection commands if the flat files are named by the user. If the model stores data in an interpreted language, such as SQL, then the model is responsible for preventing SQL injection. If it uses a message queue interface to a mainframe, the message queue data format (typically XML) needs to be well formed and compliant with a DTD.

The contract between the controller and the model needs to be carefully considered to ensure that data is strongly typed, with reasonable structure (syntax), and appropriate length, whilst allowing flexibility to allow for internationalization and future needs.

Calls by the model to the data store should be through the most secure method possible. Often the weakest possibility is dynamic queries, where a string is built up from unverified user input. This leads directly to SQL injection and is frowned upon. For more information, see the Interpreter Injections chapter.

The best performance and highest security is often obtained through parameterized stored procedures, followed by parameterized queries (also known as prepared statements) with strong typing of the parameters and schema. The major reason for using stored procedures is to minimize network traffic for a multi-stage transaction or to remove security sensitive information from traversing the network.

Stored procedures are not always a good idea – they tie you to a particular database vendor and many implementations are not fast for numeric computation. If you use the 80/20 rule for optimization and move the slow and high-risk transactions to stored procedures, the wins can be worthwhile from a security and performance perspective.

==Conclusion ==

Web applications can be written in many different ways, and in many different languages. Although the Guide concentrates upon three common choices for its examples (PHP, J2EE and ASP.NET), the Guide can be used with any web application technology.

[[Guide Table of Contents]]

[[Category:OWASP_Guide_Project]]

What are web applications?

2007-08-06T23:38:05Z

Dkaplan: /* Small to medium scale applications */

[[Guide Table of Contents]]

__TOC__

In the early days of the web, web sites consisted of static pages, which severely limited interaction with the user. In the early 1990’s, this limitation was removed when web servers were modified to allow communication with server-side custom scripts. No longer were applications just static brochure-ware, edited only by those who knew the arcane mysteries of HTML; with this single change, normal users could interact with the application for the first time.

[[Image:What are web applications.JPG]]

This is a huge and fundamental step towards the web as we know it today. Without interactivity, there would be no e-commerce (such as Amazon), no web e-mail (Hotmail or GMail), no Internet Banking, no blogs, no online share trading, and no web forums or communities like Orkut or Friendster. The static Internet would have been vastly different to today.

The trend towards increased interactivity has continued apace, with the advent of “Web 2.0”, a term that encompasses many existing technologies, but heavily features highly interactive, user centric, web-aware applications.

==Technologies ==

Initially, it was quite difficult to write sophisticated applications. The first generation web applications were primitive, usually little more than form submissions and search applications. Even these basic applications took quite a great deal of skill to craft.

Over time, the arcane knowledge required to write applications has been reduced. Today, it is relatively easy to write sophisticated applications with modern platforms and simpler languages, like PHP or VB.NET.

However, this push to make applications as easy to write as possible has a downside – many entry-level programmers are completely unaware of the security implications of their code. This is discussed further in the “Security Principles” chapter.

Let’s look at the various generations of web application technology.

==First generation – CGI ==

Common Gateway Interface (CGI) reigned supreme from approximately 1993 through to the late 1990’s when scripting languages took over in a big way.

CGI works by encapsulating user supplied data in environment variables. These are inherited by the custom written scripts or programs, usually developed in Perl or C. The custom programs process the supplied user data, and send fully formed HTML to the “standard out” (stdout), which is captured by the web server and passed back to the user.

Examples of complex CGI include Hotmail, which was essentially Perl scripts running on top of FreeBSD boxes and Slashdot, again a large Perl script running under Linux

As few sites today write new CGI applications, the techniques to secure CGI applications are not discussed within the Guide. However, many of the techniques discussed can be used with few or no changes.

==Filters ==

Filters can be used to control access to a web site, implement a different web application framework (such as Perl, PHP, or ASP), or to provide a security check.
Because filters live within the execution context of the webserver itself, they can be high performance. Typical examples of a filter interface include Apache web server modules, SunONE’s NSAPI, and Microsoft’s ISAPI. Filters can be written in C/C++ in order to integrate with the web/app server at a low-level. However, more recently, filters are being implemented in higher level languages, using interfaces like the J2EE filter API.

==Scripting ==

CGI’s lack of session management and authorization controls hampered the development of commercially useful web applications.

Web developers turned to scripting languages, such as PHP and Ruby to solve these problems. Scripting languages run script code within the web server without being compiled.


Unlike low-level languages, scripting languages rarely suffer from buffer overflows or resource leaks. Thus, programmers who use them can avoid two of the most common security issues. However, they do have their disadvantages:

* Most scripting languages aren’t strongly typed and do not promote good programming practices

* Scripting languages are generally slower than their compiled counterparts (sometimes as much as 100 times slower)

* Scripts often lead to unmanageable code bases that perform poorly as their size grows

* It’s difficult (but not impossible) to write multi-tier large scale applications in scripting languages, because often the presentation, application and data tiers reside on the same machine, thus limiting scalability and security

* Most scripting languages do not natively support remote method or web service calls, which makes it difficult to communicate with application servers and external web services.

Despite their disadvantages, many large and useful applications have been written with scripting languages, such as eGroupWare (egroupware.org), which is written in PHP. Too, many older Internet banking sites are written in ASP.

Scripting frameworks include ASP, Perl, Cold Fusion, and PHP. However, many of these would be considered interpreted hybrids now, particularly later versions of PHP and Cold Fusion, which pre-tokenize and optimize scripts.

==Web application frameworks – J2EE and ASP.NET ==

As scripting languages reached the boundaries of performance and scalability, many larger vendors jumped on Sun’s J2EE web development platform. There are many J2EE implementations, including Tomcat from the Apache Foundation, JBoss and Glassfish.


J2EE:

* Uses the Java language to produce applications which run nearly as quickly as C++ based ones, and that do not easily suffer from buffer overflows and memory leaks

* Allowed large distributed applications to run acceptably for the first time

* Possesses good session and authorization controls

* Enabled relatively transparent multi-tier applications via various remote component invocation mechanisms, and

* Is strongly typed to prevent many common security and programming issues before the program even runs

J2EE’s downside is that it has a steep learning curve which makes it difficult for web designers and entry-level programmers to use it to write applications. While certain graphical development tools make it somewhat easier to program with J2EE, a scripting language like PHP is still much easier to use.

When Microsoft updated their ASP technology to ASP.NET. which mimics the J2EE framework in many ways, they offered several improvements on the development process. For example, .NET:

* Makes it easy for entry level programmers and web designers to whip up smaller applications

* Allows large distributed applications

* Offers good session and authorization controls

* Allows programmers to use their favorite language, which is compiled to native code for excellent performance (near C++ speeds), along with buffer overflow and resource garbage collection

* Permits transparent communication with remote and external components

* Is strongly typed to prevent many common security and programming issues before the program even runs

The choice of J2EE or ASP.NET frameworks is largely dependent upon platform. There is little reason to choose one over the other from a security perspective.

Applications targeting J2EE theoretically can run with few (if any) changes between any of the major vendors and on many platforms from Linux, to AIX, MacOS X, or Windows. (While in practice, some tweaking is required, complete re-writes are not required. )

# ''ASP.Net is primarily available for Microsoft Windows. The Mono project (http://www.go-mono.com/) can run ASP.NET applications on many platforms including Solaris, Netware, and Linux. ''

==Small to medium scale applications ==

Most applications are either small or medium scale. The usual architecture is a simple linear procedural script. This is the most common form of coding for ASP, Cold Fusion and PHP scripts, but rarer (but not impossible) for ASP.NET and J2EE applications.

The reason for this architecture is that it is easy to write, and few skills are required to maintain the code. For smaller applications, any perceived performance benefit from moving to a more scalable architecture will never be recovered in the development time for those applications. For example, if it takes an additional three weeks of developer time to re-factor the scripts into an MVC approach, the three weeks will never be recovered (or noticed by end users) from the improvements in scalability.

It is typical to find many security issues in such applications, including dynamic database queries constructed from insufficiently validated data input, poor error handling and weak authorization controls.

This Guide provides advice throughout to help improve the security of these applications.

==Large scale applications ==

Larger applications need a different architecture to that of a simple survey or feedback form. As applications get larger, it becomes ever more difficult to implement and maintain features and to keep scalability high. Using scalable application architectures becomes a necessity rather than a luxury when an application needs more than about three database tables or presents more than approximately 20 - 50 functions to a user.

Scalable application architecture is often divided into tiers, and if design patterns are used, often broken down into re-usable chunks using specific guidelines to enforce modularity, interface requirements and object re-use. Breaking the application into tiers allows the application to be distributed to various servers, thus improving the scalability of the application at the expense of complexity.

One of the most common web application architectures is model-view-controller (MVC), which implements the Smalltalk 80 application architecture. MVC is typical of most Apache Foundation Jakarta Struts J2EE applications, and the code-behinds of ASP.NET can be considered a partial implementation of this approach. For PHP, the WACT project (http://wact.sourceforge.net) aims to implement the MVC paradigm in a PHP friendly fashion.

==View ==

The front-end rendering code, often called the presentation tier, should aim to produce the HTML output for the user with little to no application logic.

As many applications will be internationalized (i.e. contain no localized strings or culture information in the presentation layer), they must use calls into the model (application logic) to obtain the data required to render useful information to the user in their preferred language and culture, script direction, and units.

All user input is directed back to controllers in the application logic.

==Controller ==

The controller (or application logic) takes input from the users and gates it through various workflows that call on the application’s model objects to retrieve, process, or store the data.

Well written controllers centrally server-side validate input data against common security issues before passing the data to the model for processing, and ensure that output is safe or in a ready form for safe output by the view code.

As the application is likely to be internationalized and accessible, the data needs to be in the local language and culture. For example, dates cannot only be in different orders, but an entirely different calendar could be in use. Applications need to be flexible about presenting and storing data. Simply displaying “7/4/2005” is ambiguous to anyone outside a few countries.

==Model ==

Models encapsulate functionality, such as “Account” or “User”. A good model should be transparent to the caller, and provide a method to deal with high-level business processes rather than a thin shim to the data store. For example, a good model will allow pseudo code such as this to exist in the controller:

<pre>
oAccount->TransferFunds(fromAcct, ToAcct, Amount)
</pre>

rather than writing it such as this:

<pre>
if ( oAccount->isMyAcct(fromAcct) &&
amount < oAccount->getMaxTransferLimit() &&
oAccount->getBalance(fromAcct) > amount &&
oAccount->ToAccountExists(ToAcct) )
then
if oAccount->withdraw(fromAcct, Amount) == OK
then oAccount->deposit(ToAcct, Amount)
end if
end if
</pre>

The idea is to encapsulate the actual dirty work into the model code, rather than exposing primitives. If the controller and model are on different machines, the performance difference will be staggering, so it is important for the model to be useful at a high level.

The model is responsible for checking data against business rules, and any residual risks unique to the data store in use. For example, if a model stores data in a flat file, the code needs to check for OS injection commands if the flat files are named by the user. If the model stores data in an interpreted language, such as SQL, then the model is responsible for preventing SQL injection. If it uses a message queue interface to a mainframe, the message queue data format (typically XML) needs to be well formed and compliant with a DTD.

The contract between the controller and the model needs to be carefully considered to ensure that data is strongly typed, with reasonable structure (syntax), and appropriate length, whilst allowing flexibility to allow for internationalization and future needs.

Calls by the model to the data store should be through the most secure method possible. Often the weakest possibility is dynamic queries, where a string is built up from unverified user input. This leads directly to SQL injection and is frowned upon. For more information, see the Interpreter Injections chapter.

The best performance and highest security is often obtained through parameterized stored procedures, followed by parameterized queries (also known as prepared statements) with strong typing of the parameters and schema. The major reason for using stored procedures is to minimize network traffic for a multi-stage transaction or to remove security sensitive information from traversing the network.

Stored procedures are not always a good idea – they tie you to a particular database vendor and many implementations are not fast for numeric computation. If you use the 80/20 rule for optimization and move the slow and high-risk transactions to stored procedures, the wins can be worthwhile from a security and performance perspective.

==Conclusion ==

Web applications can be written in many different ways, and in many different languages. Although the Guide concentrates upon three common choices for its examples (PHP, J2EE and ASP.NET), the Guide can be used with any web application technology.

[[Guide Table of Contents]]

[[Category:OWASP_Guide_Project]]

What are web applications?

2007-08-06T23:35:18Z

Dkaplan: /* Web application frameworks – J2EE and ASP.NET */

[[Guide Table of Contents]]

__TOC__

In the early days of the web, web sites consisted of static pages, which severely limited interaction with the user. In the early 1990’s, this limitation was removed when web servers were modified to allow communication with server-side custom scripts. No longer were applications just static brochure-ware, edited only by those who knew the arcane mysteries of HTML; with this single change, normal users could interact with the application for the first time.

[[Image:What are web applications.JPG]]

This is a huge and fundamental step towards the web as we know it today. Without interactivity, there would be no e-commerce (such as Amazon), no web e-mail (Hotmail or GMail), no Internet Banking, no blogs, no online share trading, and no web forums or communities like Orkut or Friendster. The static Internet would have been vastly different to today.

The trend towards increased interactivity has continued apace, with the advent of “Web 2.0”, a term that encompasses many existing technologies, but heavily features highly interactive, user centric, web-aware applications.

==Technologies ==

Initially, it was quite difficult to write sophisticated applications. The first generation web applications were primitive, usually little more than form submissions and search applications. Even these basic applications took quite a great deal of skill to craft.

Over time, the arcane knowledge required to write applications has been reduced. Today, it is relatively easy to write sophisticated applications with modern platforms and simpler languages, like PHP or VB.NET.

However, this push to make applications as easy to write as possible has a downside – many entry-level programmers are completely unaware of the security implications of their code. This is discussed further in the “Security Principles” chapter.

Let’s look at the various generations of web application technology.

==First generation – CGI ==

Common Gateway Interface (CGI) reigned supreme from approximately 1993 through to the late 1990’s when scripting languages took over in a big way.

CGI works by encapsulating user supplied data in environment variables. These are inherited by the custom written scripts or programs, usually developed in Perl or C. The custom programs process the supplied user data, and send fully formed HTML to the “standard out” (stdout), which is captured by the web server and passed back to the user.

Examples of complex CGI include Hotmail, which was essentially Perl scripts running on top of FreeBSD boxes and Slashdot, again a large Perl script running under Linux

As few sites today write new CGI applications, the techniques to secure CGI applications are not discussed within the Guide. However, many of the techniques discussed can be used with few or no changes.

==Filters ==

Filters can be used to control access to a web site, implement a different web application framework (such as Perl, PHP, or ASP), or to provide a security check.
Because filters live within the execution context of the web server itself, they can be high performance. Typical examples of a filter interface include Apache web server modules, SunONE’s NSAPI, and Microsoft’s ISAPI. Filters can be written in C/C++ in order to integrate with the web/app server at a low-level. However, more recently, filters are being implemented in higher level languages, using interfaces like the J2EE filter API.

==Scripting ==

CGI’s lack of session management and authorization controls hampered the development of commercially useful web applications.

Web developers turned to scripting languages, such as PHP and Ruby to solve these problems. Scripting languages run script code within the web server without being compiled.


Unlike low-level languages, scripting languages rarely suffer from buffer overflows or resource leaks. Thus, programmers who use them can avoid two of the most common security issues. However, they do have their disadvantages:

* Most scripting languages aren’t strongly typed and do not promote good programming practices

* Scripting languages are generally slower than their compiled counterparts (sometimes as much as 100 times slower)

* Scripts often lead to unmanageable code bases that perform poorly as their size grows

* It’s difficult (but not impossible) to write multi-tier large scale applications in scripting languages, because often the presentation, application and data tiers reside on the same machine, thus limiting scalability and security

* Most scripting languages do not natively support remote method or web service calls, which makes it difficult to communicate with application servers and external web services.

Despite their disadvantages, many large and useful applications have been written with scripting languages, such as eGroupWare (egroupware.org), which is written in PHP. Too, many older Internet banking sites are written in ASP.

Scripting frameworks include ASP, Perl, Cold Fusion, and PHP. However, many of these would be considered interpreted hybrids now, particularly later versions of PHP and Cold Fusion, which pre-tokenize and optimize scripts.

==Web application frameworks – J2EE and ASP.NET ==

As scripting languages reached the boundaries of performance and scalability, many larger vendors jumped on Sun’s J2EE web development platform. There are many J2EE implementations, including Tomcat from the Apache Foundation, JBoss and Glassfish.


J2EE:

* Uses the Java language to produce applications which run nearly as quickly as C++ based ones, and that do not easily suffer from buffer overflows and memory leaks

* Allowed large distributed applications to run acceptably for the first time

* Possesses good session and authorization controls

* Enabled relatively transparent multi-tier applications via various remote component invocation mechanisms, and

* Is strongly typed to prevent many common security and programming issues before the program even runs

J2EE’s downside is that it has a steep learning curve which makes it difficult for web designers and entry-level programmers to use it to write applications. While certain graphical development tools make it somewhat easier to program with J2EE, a scripting language like PHP is still much easier to use.

When Microsoft updated their ASP technology to ASP.NET. which mimics the J2EE framework in many ways, they offered several improvements on the development process. For example, .NET:

* Makes it easy for entry level programmers and web designers to whip up smaller applications

* Allows large distributed applications

* Offers good session and authorization controls

* Allows programmers to use their favorite language, which is compiled to native code for excellent performance (near C++ speeds), along with buffer overflow and resource garbage collection

* Permits transparent communication with remote and external components

* Is strongly typed to prevent many common security and programming issues before the program even runs

The choice of J2EE or ASP.NET frameworks is largely dependent upon platform. There is little reason to choose one over the other from a security perspective.

Applications targeting J2EE theoretically can run with few (if any) changes between any of the major vendors and on many platforms from Linux, to AIX, MacOS X, or Windows. (While in practice, some tweaking is required, complete re-writes are not required. )

# ''ASP.Net is primarily available for Microsoft Windows. The Mono project (http://www.go-mono.com/) can run ASP.NET applications on many platforms including Solaris, Netware, and Linux. ''

==Small to medium scale applications ==

Most applications are either small or medium scale. The usual architecture is a simple linear procedural script. This is the most common form of coding for ASP, Cold Fusion and PHP scripts, but rarer (but not impossible) for ASP.NET and J2EE applications.

The reason for this architecture is that it is easy to write, and few skills are required to maintain the code. For smaller applications, any perceived performance benefit from moving to a more scalable architecture will never be recovered in the runtime for those applications. For example, if it takes an additional three weeks of developer time to re-factor the scripts into an MVC approach, the three weeks will never be recovered (or noticed by end users) from the improvements in scalability.

It is typical to find many security issues in such applications, including dynamic database queries constructed from insufficiently validated data input, poor error handling and weak authorization controls.

This Guide provides advice throughout to help improve the security of these applications.

==Large scale applications ==

Larger applications need a different architecture to that of a simple survey or feedback form. As applications get larger, it becomes ever more difficult to implement and maintain features and to keep scalability high. Using scalable application architectures becomes a necessity rather than a luxury when an application needs more than about three database tables or presents more than approximately 20 - 50 functions to a user.

Scalable application architecture is often divided into tiers, and if design patterns are used, often broken down into re-usable chunks using specific guidelines to enforce modularity, interface requirements and object re-use. Breaking the application into tiers allows the application to be distributed to various servers, thus improving the scalability of the application at the expense of complexity.

One of the most common web application architectures is model-view-controller (MVC), which implements the Smalltalk 80 application architecture. MVC is typical of most Apache Foundation Jakarta Struts J2EE applications, and the code-behinds of ASP.NET can be considered a partial implementation of this approach. For PHP, the WACT project (http://wact.sourceforge.net) aims to implement the MVC paradigm in a PHP friendly fashion.

==View ==

The front-end rendering code, often called the presentation tier, should aim to produce the HTML output for the user with little to no application logic.

As many applications will be internationalized (i.e. contain no localized strings or culture information in the presentation layer), they must use calls into the model (application logic) to obtain the data required to render useful information to the user in their preferred language and culture, script direction, and units.

All user input is directed back to controllers in the application logic.

==Controller ==

The controller (or application logic) takes input from the users and gates it through various workflows that call on the application’s model objects to retrieve, process, or store the data.

Well written controllers centrally server-side validate input data against common security issues before passing the data to the model for processing, and ensure that output is safe or in a ready form for safe output by the view code.

As the application is likely to be internationalized and accessible, the data needs to be in the local language and culture. For example, dates cannot only be in different orders, but an entirely different calendar could be in use. Applications need to be flexible about presenting and storing data. Simply displaying “7/4/2005” is ambiguous to anyone outside a few countries.

==Model ==

Models encapsulate functionality, such as “Account” or “User”. A good model should be transparent to the caller, and provide a method to deal with high-level business processes rather than a thin shim to the data store. For example, a good model will allow pseudo code such as this to exist in the controller:

<pre>
oAccount->TransferFunds(fromAcct, ToAcct, Amount)
</pre>

rather than writing it such as this:

<pre>
if ( oAccount->isMyAcct(fromAcct) &&
amount < oAccount->getMaxTransferLimit() &&
oAccount->getBalance(fromAcct) > amount &&
oAccount->ToAccountExists(ToAcct) )
then
if oAccount->withdraw(fromAcct, Amount) == OK
then oAccount->deposit(ToAcct, Amount)
end if
end if
</pre>

The idea is to encapsulate the actual dirty work into the model code, rather than exposing primitives. If the controller and model are on different machines, the performance difference will be staggering, so it is important for the model to be useful at a high level.

The model is responsible for checking data against business rules, and any residual risks unique to the data store in use. For example, if a model stores data in a flat file, the code needs to check for OS injection commands if the flat files are named by the user. If the model stores data in an interpreted language, such as SQL, then the model is responsible for preventing SQL injection. If it uses a message queue interface to a mainframe, the message queue data format (typically XML) needs to be well formed and compliant with a DTD.

The contract between the controller and the model needs to be carefully considered to ensure that data is strongly typed, with reasonable structure (syntax), and appropriate length, whilst allowing flexibility to allow for internationalization and future needs.

Calls by the model to the data store should be through the most secure method possible. Often the weakest possibility is dynamic queries, where a string is built up from unverified user input. This leads directly to SQL injection and is frowned upon. For more information, see the Interpreter Injections chapter.

The best performance and highest security is often obtained through parameterized stored procedures, followed by parameterized queries (also known as prepared statements) with strong typing of the parameters and schema. The major reason for using stored procedures is to minimize network traffic for a multi-stage transaction or to remove security sensitive information from traversing the network.

Stored procedures are not always a good idea – they tie you to a particular database vendor and many implementations are not fast for numeric computation. If you use the 80/20 rule for optimization and move the slow and high-risk transactions to stored procedures, the wins can be worthwhile from a security and performance perspective.

==Conclusion ==

Web applications can be written in many different ways, and in many different languages. Although the Guide concentrates upon three common choices for its examples (PHP, J2EE and ASP.NET), the Guide can be used with any web application technology.

[[Guide Table of Contents]]

[[Category:OWASP_Guide_Project]]

What are web applications?

2007-08-06T23:27:45Z

Dkaplan: /* Scripting */

[[Guide Table of Contents]]

__TOC__

In the early days of the web, web sites consisted of static pages, which severely limited interaction with the user. In the early 1990’s, this limitation was removed when web servers were modified to allow communication with server-side custom scripts. No longer were applications just static brochure-ware, edited only by those who knew the arcane mysteries of HTML; with this single change, normal users could interact with the application for the first time.

[[Image:What are web applications.JPG]]

This is a huge and fundamental step towards the web as we know it today. Without interactivity, there would be no e-commerce (such as Amazon), no web e-mail (Hotmail or GMail), no Internet Banking, no blogs, no online share trading, and no web forums or communities like Orkut or Friendster. The static Internet would have been vastly different to today.

The trend towards increased interactivity has continued apace, with the advent of “Web 2.0”, a term that encompasses many existing technologies, but heavily features highly interactive, user centric, web-aware applications.

==Technologies ==

Initially, it was quite difficult to write sophisticated applications. The first generation web applications were primitive, usually little more than form submissions and search applications. Even these basic applications took quite a great deal of skill to craft.

Over time, the arcane knowledge required to write applications has been reduced. Today, it is relatively easy to write sophisticated applications with modern platforms and simpler languages, like PHP or VB.NET.

However, this push to make applications as easy to write as possible has a downside – many entry-level programmers are completely unaware of the security implications of their code. This is discussed further in the “Security Principles” chapter.

Let’s look at the various generations of web application technology.

==First generation – CGI ==

Common Gateway Interface (CGI) reigned supreme from approximately 1993 through to the late 1990’s when scripting languages took over in a big way.

CGI works by encapsulating user supplied data in environment variables. These are inherited by the custom written scripts or programs, usually developed in Perl or C. The custom programs process the supplied user data, and send fully formed HTML to the “standard out” (stdout), which is captured by the web server and passed back to the user.

Examples of complex CGI include Hotmail, which was essentially Perl scripts running on top of FreeBSD boxes and Slashdot, again a large Perl script running under Linux

As few sites today write new CGI applications, the techniques to secure CGI applications are not discussed within the Guide. However, many of the techniques discussed can be used with few or no changes.

==Filters ==

Filters can be used to control access to a web site, implement a different web application framework (such as Perl, PHP, or ASP), or to provide a security check.
Because filters live within the execution context of the web server itself, they can be high performance. Typical examples of a filter interface include Apache web server modules, SunONE’s NSAPI, and Microsoft’s ISAPI. Filters can be written in C/C++ in order to integrate with the web/app server at a low-level. However, more recently, filters are being implemented in higher level languages, using interfaces like the J2EE filter API.

==Scripting ==

CGI’s lack of session management and authorization controls hampered the development of commercially useful web applications.

Web developers turned to scripting languages, such as PHP and Ruby to solve these problems. Scripting languages run script code within the web server without being compiled.


Unlike low-level languages, scripting languages rarely suffer from buffer overflows or resource leaks. Thus, programmers who use them can avoid two of the most common security issues. However, they do have their disadvantages:

* Most scripting languages aren’t strongly typed and do not promote good programming practices

* Scripting languages are generally slower than their compiled counterparts (sometimes as much as 100 times slower)

* Scripts often lead to unmanageable code bases that perform poorly as their size grows

* It’s difficult (but not impossible) to write multi-tier large scale applications in scripting languages, because often the presentation, application and data tiers reside on the same machine, thus limiting scalability and security

* Most scripting languages do not natively support remote method or web service calls, which makes it difficult to communicate with application servers and external web services.

Despite their disadvantages, many large and useful applications have been written with scripting languages, such as eGroupWare (egroupware.org), which is written in PHP. Too, many older Internet banking sites are written in ASP.

Scripting frameworks include ASP, Perl, Cold Fusion, and PHP. However, many of these would be considered interpreted hybrids now, particularly later versions of PHP and Cold Fusion, which pre-tokenize and optimize scripts.

==Web application frameworks – J2EE and ASP.NET ==

As scripting languages reached the boundaries of performance and scalability, many larger vendors jumped on Sun’s J2EE web development platform. There are many J2EE implementations, including Tomcat from the Apache Foundation and _______.

J2EE:

* Uses the Java language to produce applications which run nearly as quickly as C++ based ones, and that do not easily suffer from buffer overflows and memory leaks

* Allowed large distributed applications to run acceptably for the first time

* Possesses good session and authorization controls

* Enabled relatively transparent multi-tier applications via various remote component invocation mechanisms, and

* Is strongly typed to prevent many common security and programming issues before the program even runs

J2EE’s downside is that it has a steep learning curve which makes it difficult for web designers and entry-level programmers to use it to write applications. While certain graphical development tools make it somewhat easier to program with J2EE, a scripting language like PHP is still much easier to use.

When Microsoft updated their ASP technology to ASP.NET. which mimics the J2EE framework in many ways, they offered several improvements on the development process. For example, .NET:

* Makes it easy for entry level programmers and web designers to whip up smaller applications

* Allows large distributed applications

* Offers good session and authorization controls

* Allows programmers to use their favorite language, which is compiled to native code for excellent performance (near C++ speeds), along with buffer overflow and resource garbage collection

* Permits transparent communication with remote and external components

* Is strongly typed to prevent many common security and programming issues before the program even runs

The choice of J2EE or ASP.NET frameworks is largely dependent upon platform. There is little reason to choose one over the other from a security perspective.

Applications targeting J2EE theoretically can run with few (if any) changes between any of the major vendors and on many platforms from Linux, to AIX, MacOS X, or Windows. (While in practice, some tweaking is required, complete re-writes are not required. )

# ''ASP.Net is primarily available for Microsoft Windows. The Mono project (http://www.go-mono.com/) can run ASP.NET applications on many platforms including Solaris, Netware, and Linux. ''

==Small to medium scale applications ==

Most applications are either small or medium scale. The usual architecture is a simple linear procedural script. This is the most common form of coding for ASP, Cold Fusion and PHP scripts, but rarer (but not impossible) for ASP.NET and J2EE applications.

The reason for this architecture is that it is easy to write, and few skills are required to maintain the code. For smaller applications, any perceived performance benefit from moving to a more scalable architecture will never be recovered in the runtime for those applications. For example, if it takes an additional three weeks of developer time to re-factor the scripts into an MVC approach, the three weeks will never be recovered (or noticed by end users) from the improvements in scalability.

It is typical to find many security issues in such applications, including dynamic database queries constructed from insufficiently validated data input, poor error handling and weak authorization controls.

This Guide provides advice throughout to help improve the security of these applications.

==Large scale applications ==

Larger applications need a different architecture to that of a simple survey or feedback form. As applications get larger, it becomes ever more difficult to implement and maintain features and to keep scalability high. Using scalable application architectures becomes a necessity rather than a luxury when an application needs more than about three database tables or presents more than approximately 20 - 50 functions to a user.

Scalable application architecture is often divided into tiers, and if design patterns are used, often broken down into re-usable chunks using specific guidelines to enforce modularity, interface requirements and object re-use. Breaking the application into tiers allows the application to be distributed to various servers, thus improving the scalability of the application at the expense of complexity.

One of the most common web application architectures is model-view-controller (MVC), which implements the Smalltalk 80 application architecture. MVC is typical of most Apache Foundation Jakarta Struts J2EE applications, and the code-behinds of ASP.NET can be considered a partial implementation of this approach. For PHP, the WACT project (http://wact.sourceforge.net) aims to implement the MVC paradigm in a PHP friendly fashion.

==View ==

The front-end rendering code, often called the presentation tier, should aim to produce the HTML output for the user with little to no application logic.

As many applications will be internationalized (i.e. contain no localized strings or culture information in the presentation layer), they must use calls into the model (application logic) to obtain the data required to render useful information to the user in their preferred language and culture, script direction, and units.

All user input is directed back to controllers in the application logic.

==Controller ==

The controller (or application logic) takes input from the users and gates it through various workflows that call on the application’s model objects to retrieve, process, or store the data.

Well written controllers centrally server-side validate input data against common security issues before passing the data to the model for processing, and ensure that output is safe or in a ready form for safe output by the view code.

As the application is likely to be internationalized and accessible, the data needs to be in the local language and culture. For example, dates cannot only be in different orders, but an entirely different calendar could be in use. Applications need to be flexible about presenting and storing data. Simply displaying “7/4/2005” is ambiguous to anyone outside a few countries.

==Model ==

Models encapsulate functionality, such as “Account” or “User”. A good model should be transparent to the caller, and provide a method to deal with high-level business processes rather than a thin shim to the data store. For example, a good model will allow pseudo code such as this to exist in the controller:

<pre>
oAccount->TransferFunds(fromAcct, ToAcct, Amount)
</pre>

rather than writing it such as this:

<pre>
if ( oAccount->isMyAcct(fromAcct) &&
amount < oAccount->getMaxTransferLimit() &&
oAccount->getBalance(fromAcct) > amount &&
oAccount->ToAccountExists(ToAcct) )
then
if oAccount->withdraw(fromAcct, Amount) == OK
then oAccount->deposit(ToAcct, Amount)
end if
end if
</pre>

The idea is to encapsulate the actual dirty work into the model code, rather than exposing primitives. If the controller and model are on different machines, the performance difference will be staggering, so it is important for the model to be useful at a high level.

The model is responsible for checking data against business rules, and any residual risks unique to the data store in use. For example, if a model stores data in a flat file, the code needs to check for OS injection commands if the flat files are named by the user. If the model stores data in an interpreted language, such as SQL, then the model is responsible for preventing SQL injection. If it uses a message queue interface to a mainframe, the message queue data format (typically XML) needs to be well formed and compliant with a DTD.

The contract between the controller and the model needs to be carefully considered to ensure that data is strongly typed, with reasonable structure (syntax), and appropriate length, whilst allowing flexibility to allow for internationalization and future needs.

Calls by the model to the data store should be through the most secure method possible. Often the weakest possibility is dynamic queries, where a string is built up from unverified user input. This leads directly to SQL injection and is frowned upon. For more information, see the Interpreter Injections chapter.

The best performance and highest security is often obtained through parameterized stored procedures, followed by parameterized queries (also known as prepared statements) with strong typing of the parameters and schema. The major reason for using stored procedures is to minimize network traffic for a multi-stage transaction or to remove security sensitive information from traversing the network.

Stored procedures are not always a good idea – they tie you to a particular database vendor and many implementations are not fast for numeric computation. If you use the 80/20 rule for optimization and move the slow and high-risk transactions to stored procedures, the wins can be worthwhile from a security and performance perspective.

==Conclusion ==

Web applications can be written in many different ways, and in many different languages. Although the Guide concentrates upon three common choices for its examples (PHP, J2EE and ASP.NET), the Guide can be used with any web application technology.

[[Guide Table of Contents]]

[[Category:OWASP_Guide_Project]]