OWASP - User contributions [en]

Fingerprint Web Application Framework (OTG-INFO-008)

2014-03-22T05:11:13Z

Aditya Balapure:

{{Template:OWASP Testing Guide v4}}
== Summary ==
Web framework[*] fingerprinting is an important subtask of the information gathering process. Knowing the type of framework can automatically give a great advantage if such a framework has already been tested by the penetration tester. It is not only the known vulnerabilities in unpatched versions but specific misconfigurations in the framework and known file structure which makes the fingerprinting process so important.

Several different vendors and versions of web frameworks are widely used.
Information about it significantly helps in the testing process, and can also help in changing the course of the test.
Such information can be derived by careful analysis of certain common locations. Most of the web frameworks have several markers in those locations which help an attacker to spot them.
This is basically what all automatic tools do, they look for a marker from a predefined location
and then compare it to the database of known signatures. For better accuracy several markers are usually used.

[*] Please note that in this article we make no difference between Web Application Frameworks (WAF) and Content Management Systems (CMS).
This has been done to make it convenient to fingerprint both of them in one chapter.Further we reference both of the categories as web frameworks.

== Test Objectives ==
To define type of used web framework so to have a better understanding of the penetration testing methodology.

== How to Test ==

=== Black Box testing ===
There are several most common locations to look in in order to define the current framework:
*HTTP headers
*Cookies
*HTML source code
*Specific files and folders

Let's look closer at those approaches.

==== HTTP headers ====
The most basic form of identifying a web framework is to look at the ''X-Powered-By'' field in the HTTP response header.
Many tools can be used to fingerprint a target. The simplest one is netcat utility.
Consider the following HTTP Request-Response:
<pre>
$ nc 127.0.0.1 80
HEAD / HTTP/1.0

HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Mono
</pre>

From the ''X-Powered-By'' field, we understand that the web application framework is likely to be Mono.

However, although simplicity and quickness of such an approach, this methodology doesn't work in 100% of cases. It is possible to easily disable ''X-Powered-By'' header by a proper configuration. There are also several techniques that allow a web site to obfuscate HTTP headers (see an example in [[#Remediation]] chapter)
So in the same example we could either miss the ''X-Powered-By'' header or obtain an answer like the following:
<pre>
HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Blood, sweat and tears
</pre>

Sometimes there are more HTTP-headers which point at the certain web framework. In the following example according to the information from HTTP-request one can see that ''X-Powered-By'' header contains PHP version. However, ''X-Generator'' header points out the used framework is actually Swiftlet, which helps a penetration tester to expand his attack vectors. When performing fingerprinting, always carefully inspect every HTTP-header for such leaks.
<pre>
HTTP/1.1 200 OK
Server: nginx/1.4.1
Date: Sat, 07 Sep 2013 09:22:52 GMT
Content-Type: text/html
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/5.4.16-1~dotdeb.1
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Generator: Swiftlet
</pre>

==== Cookies ====
Another similar and somehow more reliable way to determine the current web framework are framework-specific cookies.
Consider the following HTTP-request:

[[Image:Cakephp_cookie.png]]

As we can see, cookie ''CAKEPHP'' has automatically been set, which gives us the information about the framework being used. List of common cookies names is presented in chapter [[#Cookies_2]]. Limitations are the same - it is possible to change the name of the cookie. For example, for the selected ''CakePHP'' framework this could be done by the following configuration (excerpt from core.php):

<pre>
/**
* The name of CakePHP's session cookie.
*
* Note the guidelines for Session names states: "The session name references
* the session id in cookies and URLs. It should contain only alphanumeric
* characters."
* @link http://php.net/session_name
*/
Configure::write('Session.cookie', 'CAKEPHP');
</pre>

However from the experience, these changes are less likely to be made than, for example, ''X-Powered-By'' header so this approach can be considered as more reliable one.

==== HTML source code ====
This technique is based on finding certain patterns in the HTML page source code. Often one can find much information which helps a tester to recognize a specific web framework.
Ones of the common markers are HTML comments which directly lead to framework disclosure. More often certain framework-specific paths can be found, i.e. links to framework-specific css and/or js folders. Finally, specific script variables might also point to a certain framework.
From the screenshot below one can easily learn the used framework and its version by the mentioned markers: comment, specific paths and script variables can all help an attacker to quickly determine an instance of ZK framework.

[[Image:Zk_html_source.png]]

More frequently such information is placed between <head></head>, in <meta> tags or at the end of the page. Nevertheless, it is recommended to check the whole document since it can be useful for other purposes such as inspection of other useful comments and hidden fields.
Sometimes, however, web developers do not care much about hiding tracks of the used framework. It's still possible to stumble upon something like that at the bottom of the page:

[[Image:banshee_bottom_page.png]]

==== Specific files and folders ====
Apart from information gathered from HTML sources, there is another approach which greatly helps an attacker to determine the framework with high accuracy.
Every framework has its own specific file and folder structure on the server. We pointed out that one can see the specific path from the HTML page source but sometimes they are not explicitly presented there and still reside on the server. In order to uncover them a technique known as dirbusting is used. Dirbusting is brute forcing target with predictable folder and file names and monitoring HTTP-responses thus enumerating server contents. This information can be used both for finding default files and attacking them in next stages and for fingerprinting the web framework.
Dirbusting can be done in several ways, example below shows successful dirbusting attack against a WordPress-powered target with the help of defined list and Intruder functionality of Burp Suite.

[[Image:Wordpress_dirbusting.png]]

We can see that for some WordPress-specific folders (for instance, /wp-includes/, /wp-admin/ and /wp-content/) HTTP-reponses are 403 (Forbidden), 302 (Found, redirection to ''wp-login.php'') and 200 (OK) respectively. This is a good indicator that the target is WordPress-powered. The same way it is possible to dirbust different framework plugin folders and their versions. On the screenshot below one can see a typical CHANGELOG file of a Drupal plugin, which provides information on the framework being used and discloses vulnerable plugin version.

[[Image:Drupal_botcha_disclosure.png]]

Tip: before starting dirbusting (which is a useful operation anyway), it is recommended to check robots.txt file first. Sometimes framework specific folders and other sensitive information can be found there as well! Example of such robots file is presented on a screenshot below.

[[Image:Robots-info-disclosure.png]]

=== Gray Box testing ===
Please address yourself to watching Monty Python's Flying Circus, I'm sure you'll like it!

== Common frameworks ==
=== Cookies ===

{| class="wikitable"
|-
! Framework !! Cookie name
|-
| Zope || zope3
|-
| CakePHP || cakephp
|-
| Kohana || kohanasession
|-
| Laravel || laravel_session
|-
| 1C-Bitrix || BITRIX_
|-
| AMPcms || AMP
|-
|-
| Django CMS || django
|-
| DotNetNuke || DotNetNukeAnonymous
|-
| e107 || e107_tz
|-
| EPiServer || EPiTrace, EPiServer
|-
| Graffiti CMS || graffitibot
|-
| Hotaru CMS || hotaru_mobile
|-
| ImpressCMS || ICMSession
|-
| Indico || MAKACSESSION
|-
| InstantCMS || InstantCMS[logdate]
|-
| Kentico CMS || CMSPreferredCulture
|-
| MODx || SN4[12symb]
|-
| TYPO3 || fe_typo_user
|-
| Dynamicweb || Dynamicweb
|-
| LEPTON || lep[some_numeric_value]+sessionid
|-
| Wix || Domain=.wix.com
|-
| VIVVO || VivvoSessionId
|}

=== HTML source code ===
==== General markers ====
{| class="wikitable"
|-
| %framework_name%
|-
| powered by
|-
| built upon
|-
| running
|}

==== Specific markers ====
{| class="wikitable"
|-
! Framework !! Keyword
|-
| Adobe ColdFusion || 
|-
| Indexhibit || ndxz-studio
|}

=== Specific files and folders ===
These are different for each specific framework. It is recommended to install the corresponding framework during penetration tests in order to have better understanding of what infrastructure is presented and which files might be left on the server. However, several good pre-made lists already exist, one good example of them - FuzzDB wordlists of predictable files/folders (http://code.google.com/p/fuzzdb/)

== Tools ==
A list of general and well-known tools is presented below. There are also a lot of other utilities, as well as framework-based fingerprinting tools.

=== WhatWeb ===
Website: http://www.morningstarsecurity.com/research/whatweb 
Currently one of the best fingerprinting tools on the market. Included in a default [[Kali Linux]] build.
Language: Ruby
Matches for fingerprinting are made with:
* Text strings (case sensitive)
* Regular expressions
* Google Hack Database queries (limited set of keywords)
* MD5 hashes
* URL recognition
* HTML tag patterns
* Custom ruby code for passive and aggressive operations

Sample output is presented on a screenshot below:

[[Image:whatweb-sample.png]]

=== BlindElephant ===
Website: https://community.qualys.com/community/blindelephant 
This great tool works on the principle of static file checksum based version difference thus providing a very high quality of fingerprinting.
Language: Python

Sample output of a successful fingerprint:
<pre>
pentester$ python BlindElephant.py http://my_target drupal
Loaded /Library/Python/2.7/site-packages/blindelephant/dbs/drupal.pkl with 145 versions, 478 differentiating paths, and 434 version groups.
Starting BlindElephant fingerprint for version of drupal at http://my_target

Hit http://my_target/CHANGELOG.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 527b085a3717bd691d47713dff74acf4

Hit http://my_target/INSTALL.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 14dfc133e4101be6f0ef5c64566da4a4

Hit http://my_target/misc/drupal.js
Possible versions based on result: 7.12, 7.13, 7.14

Hit http://my_target/MAINTAINERS.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 36b740941a19912f3fdbfcca7caa08ca

Hit http://my_target/themes/garland/style.css
Possible versions based on result: 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 7.10, 7.11, 7.12, 7.13, 7.14

...

Fingerprinting resulted in:
7.14

Best Guess: 7.14
</pre>

=== Wappalyzer ===
Website: http://wappalyzer.com 
Wapplyzer is a Firefox/Chrome plug-in. It works only on regular expression matching and doesn't need anything other than the page to be loaded on browser. It works completely at the browser level and gives results in the form of icons. Although sometimes it has false positives, this is very handy to have notion of what technologies were used to construct a target website immediately after browsing a page.
Sample output of a plug-in is presented on a screenshot below.

[[Image:Owasp-wappalyzer.png]]

== Vulnerability References ==
'''Whitepapers''' 
* Saumil Shah: "An Introduction to HTTP fingerprinting" - http://www.net-square.com/httprint_paper.html
* Anant Shrivastava : "Web Application Finger Printing" - http://anantshri.info/articles/web_app_finger_printing.html

== Remediation ==
General advice: use several tools described above and check scan logs in order to better understand what exactly helps an attacker to disclose your web framework. By performing multiple scans after changes you've done to hide framework tracks, it's possible to achieve better security level and to make sure of undetectability by automatic scans.
Below you may find specific recommendations by framework marker location and some additional interesting approaches.

==== HTTP headers ====
Check your configuration and disable/obfuscate all HTTP-headers which disclose information about used technologies.
An interesting article about HTTP-headers obfuscation using Netscaler:
http://grahamhosking.blogspot.ru/2013/07/obfuscating-http-header-using-netscaler.html

==== Cookies ====
It is recommended to change cookie names by making proper changes in corresponding config files.

==== HTML source code ====
Check manually the contents of your HTML code and remove everything what explicitly points to the framework.
General guidelines:
*Make sure you didn't leave any visual markers disclosing used framework
*Remove any unnecessary comments (copyrights, bug information, specific framework comments)
*Remove META and generator tags
*Use your own css/js and do not store those in a framework-specific folders
*Make sure you're not using default scripts on the page; if using them is needed, obfuscate them.

==== Specific files and folders ====
*Remove any unnecessary/unused files on the server. This implies text files disclosing information about versions and installation too.
*Restrict access to other files in order to achieve 404-response when accessing them from outside. This can be done, for example, by modifying htaccess file and adding RewriteCond / RewriteRule there. Example of such restriction for two common WordPress folders is presented below.
<pre>
RewriteCond %{REQUEST_URI} /wp-login\.php$ [OR]
RewriteCond %{REQUEST_URI} /wp-admin/$
RewriteRule $ /http://your_website [R=404,L]
</pre>
However, these are not the only ways to restrict access. In order to automate this process, certain framework-specific plugins exist. One example for WordPress is StealthLogin (http://wordpress.org/plugins/stealth-login-page)

==== Additional approaches ====
*Checksum management
*:Purpose of this approach is to beat checksum-based scanners and don't let them to disclose files by their hashes. Generally, there are two approaches in checksum management:
*:*Change the location of where those files are placed (i.e. move them to another folder, or rename existing)
*:*Modify their contents - even slight modification results in a completely different hash sum, so adding a single byte in the end of the file should not be a big problem.
*Controlled chaos
*:A funny and effective method which essense is adding bogus files and folders from other frameworks in order to fool scanners and confuse an attacker. But be careful not to overwrite exisiting files and folders and to break the current framework!

Fingerprint Web Application Framework (OTG-INFO-008)

2014-03-22T04:25:44Z

Aditya Balapure:

{{Template:OWASP Testing Guide v4}}
== Summary ==
Web framework[*] fingerprinting is an important subtask of the information gathering process. Knowing the type of framework can automatically give a great advantage if such a framework has already been tested by the penetration tester. It is not only the known vulnerabilities in unpatched versions but specific misconfigurations in the framework and known file structure which makes the fingerprinting process so important.

Several different vendors and versions of web frameworks are widely used.
Information about it significantly helps in the testing process, and can also help in changing the course of the test.
Such information can be derived by careful analysis of certain common locations. Most of the web frameworks have several markers in those locations which help an attacker to spot them.
This is basically what all automatic tools do, they look for a marker from a predefined location
and then compare it to the database of known signatures. For better accuracy several markers are usually used.

[*] Please note that in this article we make no difference between Web Application Frameworks (WAF) and Content Management Systems (CMS).
This has been done to make it convenient to fingerprint both of them in one chapter.Further we reference both of the categories as web frameworks.

== Test Objectives ==
To define type of used web framework so to have a better understanding of the penetration testing methodology.

== How to Test ==

=== Black Box testing ===
There are several most common locations to look in in order to define the current framework:
*HTTP headers
*Cookies
*HTML source code
*Specific files and folders

Let's look closer at those approaches.

==== HTTP headers ====
The most basic form of identifying a web framework is to look at the ''X-Powered-By'' field in the HTTP response header.
Many tools can be used to fingerprint a target. The simplest one is netcat utility.
Consider the following HTTP Request-Response:
<pre>
$ nc 127.0.0.1 80
HEAD / HTTP/1.0

HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Mono
</pre>

From the ''X-Powered-By'' field, we understand that the web application framework is likely to be Mono.

However, although simplicity and quickness of such an approach, this methodology doesn't work in 100% of cases. It is possible to easily disable ''X-Powered-By'' header by a proper configuration. There are also several techniques that allow a web site to obfuscate HTTP headers (see an example in [[#Remediation]] chapter)
So in the same example we could either miss the ''X-Powered-By'' header or obtain an answer like the following:
<pre>
HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Blood, sweat and tears
</pre>

Sometimes there are more HTTP-headers which point at the certain web framework. In the following example according to the information from HTTP-request one can see that ''X-Powered-By'' header contains PHP version. However, ''X-Generator'' header points out the used framework is actually Swiftlet, which helps a penetration tester to expand his attack vectors. When performing fingerprinting, always carefully inspect every HTTP-header for such leaks.
<pre>
HTTP/1.1 200 OK
Server: nginx/1.4.1
Date: Sat, 07 Sep 2013 09:22:52 GMT
Content-Type: text/html
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/5.4.16-1~dotdeb.1
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Generator: Swiftlet
</pre>

==== Cookies ====
Another similar and somehow more reliable way to determine the current web framework are framework-specific cookies.
Consider the following HTTP-request:

[[Image:Cakephp_cookie.png]]

As we can see, cookie ''CAKEPHP'' has automatically been set, which gives us the information about the framework being used. List of common cookies names is presented in chapter [[#Cookies_2]]. Limitations are the same - it is possible to change the name of the cookie. For example, for the selected ''CakePHP'' framework this could be done by the following configuration (excerpt from core.php):

<pre>
/**
* The name of CakePHP's session cookie.
*
* Note the guidelines for Session names states: "The session name references
* the session id in cookies and URLs. It should contain only alphanumeric
* characters."
* @link http://php.net/session_name
*/
Configure::write('Session.cookie', 'CAKEPHP');
</pre>

However from the experience, these changes are less likely to be made than, for example, ''X-Powered-By'' header so this approach can be considered as more reliable one.

==== HTML source code ====
This technique is based on finding certain patterns in the HTML page source code. Often one can find much information which helps a tester to recognize a specific web framework.
Ones of the common markers are HTML comments which directly lead to framework disclosure. More often certain framework-specific paths can be found, i.e. links to framework-specific css and/or js folders. Finally, specific script variables might also point to a certain framework.
From the screenshot below one can easily learn the used framework and its version by the mentioned markers: comment, specific paths and script variables can all help an attacker to quickly determine an instance of ZK framework.

[[Image:Zk_html_source.png]]

More frequently such information is placed between <head></head>, in <meta> tags or at the end of the page. Nevertheless, it is recommended to check the whole document since it can be useful for other purposes such as inspection of other useful comments and hidden fields.
Sometimes, however, web developers do not care much about hiding tracks of the used framework. It's still possible to stumble upon something like that at the bottom of the page:

[[Image:banshee_bottom_page.png]]

==== Specific files and folders ====
Apart from information gathered from HTML sources, there is another approach which greatly helps an attacker to determine the framework with high accuracy.
Every framework has its own specific file and folder structure on the server. We pointed out that one can see the specific path from the HTML page source but sometimes they are not explicitly presented there and still reside on the server. In order to uncover them a technique known as dirbusting is used. Dirbusting is brute forcing target with predictable folder and file names and monitoring HTTP-responses thus enumerating server contents. This information can be used both for finding default files and attacking them in next stages and for fingerprinting the web framework.
Dirbusting can be done in several ways, example below shows successful dirbusting attack against a WordPress-powered target with the help of defined list and Intruder functionality of Burp Suite.

[[Image:Wordpress_dirbusting.png]]

We can see that for some WordPress-specific folders (for instance, /wp-includes/, /wp-admin/ and /wp-content/) HTTP-reponses are 403 (Forbidden), 302 (Found, redirection to ''wp-login.php'') and 200 (OK) respectively. This is a good indicator that the target is WordPress-powered. The same way it is possible to dirbust different framework plugin folders and their versions. On the screenshot below one can see a typical CHANGELOG file of a Drupal plugin, which provides information on the framework being used and discloses vulnerable plugin version.

[[Image:Drupal_botcha_disclosure.png]]

Tip: before starting dirbusting (which is a useful operation anyway), it is recommended to check robots.txt file first. Sometimes framework specific folders and other sensitive information can be found there as well! Example of such robots file is presented on a screenshot below.

[[Image:Robots-info-disclosure.png]]

=== Gray Box testing ===
Please address yourself to watching Monty Python's Flying Circus, I'm sure you'll like it!

== Common frameworks ==
=== Cookies ===

{| class="wikitable"
|-
! Framework !! Cookie name
|-
| Zope || zope3
|-
| CakePHP || cakephp
|-
| Kohana || kohanasession
|-
| Laravel || laravel_session
|-
| 1C-Bitrix || BITRIX_
|-
| AMPcms || AMP
|-
|-
| Django CMS || django
|-
| DotNetNuke || DotNetNukeAnonymous
|-
| e107 || e107_tz
|-
| EPiServer || EPiTrace, EPiServer
|-
| Graffiti CMS || graffitibot
|-
| Hotaru CMS || hotaru_mobile
|-
| ImpressCMS || ICMSession
|-
| Indico || MAKACSESSION
|-
| InstantCMS || InstantCMS[logdate]
|-
| Kentico CMS || CMSPreferredCulture
|-
| MODx || SN4[12symb]
|-
| TYPO3 || fe_typo_user
|-
| Dynamicweb || Dynamicweb
|-
| LEPTON || lep[some_numeric_value]+sessionid
|-
| Wix || Domain=.wix.com
|-
| VIVVO || VivvoSessionId
|}

=== HTML source code ===
==== General markers ====
{| class="wikitable"
|-
| %framework_name%
|-
| powered by
|-
| built upon
|-
| running
|}

==== Specific markers ====
{| class="wikitable"
|-
! Framework !! Keyword
|-
| Adobe ColdFusion || 
|-
| Indexhibit || ndxz-studio
|}

=== Specific files and folders ===
These are different for each specific framework. It is recommended to install the corresponding framework during penetration tests in order to have better understanding of what infrastructure is presented and which files might be left on the server. However, several good pre-made lists already exist, one good example of them - FuzzDB wordlists of predictable files/folders (http://code.google.com/p/fuzzdb/)

== Tools ==
A list of general and well-known tools is presented below. There are also a lot of other utilities, as well as framework-based fingerprinting tools.

=== WhatWeb ===
Website: http://www.morningstarsecurity.com/research/whatweb 
Currently one of the best fingerprinting tools on the market. Included in a default [[Kali Linux]] build.
Language: Ruby
Matches for fingerprinting are made with:
* Text strings (case sensitive)
* Regular expressions
* Google Hack Database queries (limited set of keywords)
* MD5 hashes
* URL recognition
* HTML tag patterns
* Custom ruby code for passive and aggressive operations

Sample output is presented on a screenshot below:

[[Image:whatweb-sample.png]]

=== BlindElephant ===
Website: https://community.qualys.com/community/blindelephant 
This great tool works on the principle of static file checksum based version difference thus providing a very high quality of fingerprinting.
Language: Python

Sample output of a successful fingerprint:
<pre>
pentester$ python BlindElephant.py http://my_target drupal
Loaded /Library/Python/2.7/site-packages/blindelephant/dbs/drupal.pkl with 145 versions, 478 differentiating paths, and 434 version groups.
Starting BlindElephant fingerprint for version of drupal at http://my_target

Hit http://my_target/CHANGELOG.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 527b085a3717bd691d47713dff74acf4

Hit http://my_target/INSTALL.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 14dfc133e4101be6f0ef5c64566da4a4

Hit http://my_target/misc/drupal.js
Possible versions based on result: 7.12, 7.13, 7.14

Hit http://my_target/MAINTAINERS.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 36b740941a19912f3fdbfcca7caa08ca

Hit http://my_target/themes/garland/style.css
Possible versions based on result: 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 7.10, 7.11, 7.12, 7.13, 7.14

...

Fingerprinting resulted in:
7.14

Best Guess: 7.14
</pre>

=== Wappalyzer ===
Website: http://wappalyzer.com 
Wapplyzer is a Firefox/Chrome plug-in. It works on only regular expression matching and doesn't need anything other than the page to be loaded on browser. It works completely at the browser level and give results in form of icons. Although sometimes it has false positives, this is very handy to have notion of what technologies were used to construct a target website immediately after browsing a page.
Sample output of a plug-in is presented on a screenshot below.

[[Image:Owasp-wappalyzer.png]]

== Vulnerability References ==
'''Whitepapers''' 
* Saumil Shah: "An Introduction to HTTP fingerprinting" - http://www.net-square.com/httprint_paper.html
* Anant Shrivastava : "Web Application Finger Printing" - http://anantshri.info/articles/web_app_finger_printing.html

== Remediation ==
General advice: use several tools described above and check scan logs in order to better understand what exactly helps an attacker to disclose your web framework. By performing multiple scans after changes you've done to hide framework tracks, it's possible to achieve better security level and to make sure of undetectability by automatic scans.
Below you may find specific recommendations by framework marker location and some additional interesting approaches.

==== HTTP headers ====
Check your configuration and disable/obfuscate all HTTP-headers which disclose information about used technologies.
An interesting article about HTTP-headers obfuscation using Netscaler:
http://grahamhosking.blogspot.ru/2013/07/obfuscating-http-header-using-netscaler.html

==== Cookies ====
It is recommended to change cookie names by making proper changes in corresponding config files.

==== HTML source code ====
Check manually the contents of your HTML code and remove everything what explicitly points to the framework.
General guidelines:
*Make sure you didn't leave any visual markers disclosing used framework
*Remove any unnecessary comments (copyrights, bug information, specific framework comments)
*Remove META and generator tags
*Use your own css/js and do not store those in a framework-specific folders
*Make sure you're not using default scripts on the page; if using them is needed, obfuscate them.

==== Specific files and folders ====
*Remove any unnecessary/unused files on the server. This implies text files disclosing information about versions and installation too.
*Restrict access to other files in order to achieve 404-response when accessing them from outside. This can be done, for example, by modifying htaccess file and adding RewriteCond / RewriteRule there. Example of such restriction for two common WordPress folders is presented below.
<pre>
RewriteCond %{REQUEST_URI} /wp-login\.php$ [OR]
RewriteCond %{REQUEST_URI} /wp-admin/$
RewriteRule $ /http://your_website [R=404,L]
</pre>
However, these are not the only ways to restrict access. In order to automate this process, certain framework-specific plugins exist. One example for WordPress is StealthLogin (http://wordpress.org/plugins/stealth-login-page)

==== Additional approaches ====
*Checksum management
*:Purpose of this approach is to beat checksum-based scanners and don't let them to disclose files by their hashes. Generally, there are two approaches in checksum management:
*:*Change the location of where those files are placed (i.e. move them to another folder, or rename existing)
*:*Modify their contents - even slight modification results in a completely different hash sum, so adding a single byte in the end of the file should not be a big problem.
*Controlled chaos
*:A funny and effective method which essense is adding bogus files and folders from other frameworks in order to fool scanners and confuse an attacker. But be careful not to overwrite exisiting files and folders and to break the current framework!

Fingerprint Web Application Framework (OTG-INFO-008)

2014-03-18T22:03:43Z

Aditya Balapure:

{{Template:OWASP Testing Guide v4}}
== Summary ==
Web framework[*] fingerprinting is an important subtask of the information gathering process. Knowing the type of framework can automatically give a great advantage if such a framework has already been tested by the penetration tester. It is not only the known vulnerabilities in unpatched versions but specific misconfigurations in the framework and known file structure which makes the fingerprinting process so important.

Several different vendors and versions of web frameworks are widely used.
Information about it significantly helps in the testing process, and can also help in changing the course of the test.
Such information can be derived by careful analysis of certain common locations. Most of the web frameworks have several markers in those locations which help an attacker to spot them.
This is basically what all automatic tools do, they look for a marker from a predefined location
and then compare it to the database of known signatures. For better accuracy several markers are usually used.

[*] Please note that in this article we make no difference between Web Application Frameworks (WAF) and Content Management Systems (CMS).
This has been done to make it convenient to fingerprint both of them in one chapter.Further we reference both of the categories as web frameworks.

== Test Objectives ==
To define type of used web framework so to have a better understanding of the penetration testing methodology.

== How to Test ==

=== Black Box testing ===
There are several most common locations to look in in order to define the current framework:
*HTTP headers
*Cookies
*HTML source code
*Specific files and folders

Let's look closer at those approaches.

==== HTTP headers ====
The most basic form of identifying a web framework is to look at the ''X-Powered-By'' field in the HTTP response header.
Many tools can be used to fingerprint a target. The simplest one is netcat utility.
Consider the following HTTP Request-Response:
<pre>
$ nc 127.0.0.1 80
HEAD / HTTP/1.0

HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Mono
</pre>

From the ''X-Powered-By'' field, we understand that the web application framework is likely to be Mono.

However, although simplicity and quickness of such an approach, this methodology doesn't work in 100% of cases. It is possible to easily disable ''X-Powered-By'' header by a proper configuration. There are also several techniques that allow a web site to obfuscate HTTP headers (see an example in [[#Remediation]] chapter)
So in the same example we could either miss the ''X-Powered-By'' header or obtain an answer like the following:
<pre>
HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Blood, sweat and tears
</pre>

Sometimes there are more HTTP-headers which point at the certain web framework. In the following example according to the information from HTTP-request one can see that ''X-Powered-By'' header contains PHP version. However, ''X-Generator'' header points out the used framework is actually Swiftlet, which helps a penetration tester to expand his attack vectors. When performing fingerprinting, always carefully inspect every HTTP-header for such leaks.
<pre>
HTTP/1.1 200 OK
Server: nginx/1.4.1
Date: Sat, 07 Sep 2013 09:22:52 GMT
Content-Type: text/html
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/5.4.16-1~dotdeb.1
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Generator: Swiftlet
</pre>

==== Cookies ====
Another similar and somehow more reliable way to determine the current web framework are framework-specific cookies.
Consider the following HTTP-request:

[[Image:Cakephp_cookie.png]]

As we can see, cookie ''CAKEPHP'' has automatically been set, which gives us the information about the framework being used. List of common cookies names is presented in chapter [[#Cookies_2]]. Limitations are the same - it is possible to change the name of the cookie. For example, for the selected ''CakePHP'' framework this could be done by the following configuration (excerpt from core.php):

<pre>
/**
* The name of CakePHP's session cookie.
*
* Note the guidelines for Session names states: "The session name references
* the session id in cookies and URLs. It should contain only alphanumeric
* characters."
* @link http://php.net/session_name
*/
Configure::write('Session.cookie', 'CAKEPHP');
</pre>

However from the experience, these changes are less likely to be made than, for example, ''X-Powered-By'' header so this approach can be considered as more reliable one.

==== HTML source code ====
This technique is based on finding certain patterns in the HTML page source code. Often one can find much information which helps a tester to recognize a specific web framework.
Ones of the common markers are HTML comments which directly lead to framework disclosure. More often certain framework-specific paths can be found, i.e. links to framework-specific css and/or js folders. Finally, specific script variables might also point to a certain framework.
From the screenshot below one can easily learn the used framework and its version by the mentioned markers: comment, specific paths and script variables can all help an attacker to quickly determine an instance of ZK framework.

[[Image:Zk_html_source.png]]

More frequently such information is placed between <head></head>, in <meta> tags or at the end of the page. Nevertheless, it is recommended to check the whole document since it can be useful for other purposes such as inspection of other useful comments and hidden fields.
Sometimes, however, web developers do not care much about hiding tracks of the used framework. It's still possible to stumble upon something like that at the bottom of the page:

[[Image:banshee_bottom_page.png]]

==== Specific files and folders ====
Apart from information gathered from HTML sources, there is another approach which greatly helps an attacker to determine the framework with high accuracy.
Every framework has its own specific file and folder structure on the server. We pointed out that one can see the specific path from the HTML page source but sometimes they are not explicitly presented there and still reside on the server. In order to uncover them such technique as dirbusting is used. Dirbusting essense is fuzzing target with predictable folder- and filenames and monitoring HTTP-responses thus enumerating server contents. This information can be used both for finding default files and attacking them in next stages and for fingerprinting the web framework.
Dirbusting can be done in several ways, example below shows successful dirbusting attack against a WordPress-powered target with the help of defined list and Intruder functionality of Burp Suite.

[[Image:Wordpress_dirbusting.png]]

We can see that for some WordPress-specific folders (for instance, /wp-includes/, /wp-admin/ and /wp-content/) HTTP-reponses are 403 (Forbidden), 302 (Found, redirection to ''wp-login.php'') and 200 (OK) respectively. This is a good indicator that the target is WordPress-powered. The same way it is possible to dirbust different framework plugin folders and their versions. On a screenshot below one can see a typical CHANGELOG file of a Drupal plugin, which both points out on the used framework and discloses vulnerable plugin version.

[[Image:Drupal_botcha_disclosure.png]]

Tip: before starting dirbusting (which is a useful operation anyway), it is recommended to check robots.txt file first. Sometimes framework specific folders and other sensitive information can be found there as well! Example of such robots file is presented on a screenshot below.

[[Image:Robots-info-disclosure.png]]

=== Gray Box testing ===
Please address yourself to watching Monty Python's Flying Circus, I'm sure you'll like it!

== Common frameworks ==
=== Cookies ===

{| class="wikitable"
|-
! Framework !! Cookie name
|-
| Zope || zope3
|-
| CakePHP || cakephp
|-
| Kohana || kohanasession
|-
| Laravel || laravel_session
|-
| 1C-Bitrix || BITRIX_
|-
| AMPcms || AMP
|-
|-
| Django CMS || django
|-
| DotNetNuke || DotNetNukeAnonymous
|-
| e107 || e107_tz
|-
| EPiServer || EPiTrace, EPiServer
|-
| Graffiti CMS || graffitibot
|-
| Hotaru CMS || hotaru_mobile
|-
| ImpressCMS || ICMSession
|-
| Indico || MAKACSESSION
|-
| InstantCMS || InstantCMS[logdate]
|-
| Kentico CMS || CMSPreferredCulture
|-
| MODx || SN4[12symb]
|-
| TYPO3 || fe_typo_user
|-
| Dynamicweb || Dynamicweb
|-
| LEPTON || lep[some_numeric_value]+sessionid
|-
| Wix || Domain=.wix.com
|-
| VIVVO || VivvoSessionId
|}

=== HTML source code ===
==== General markers ====
{| class="wikitable"
|-
| %framework_name%
|-
| powered by
|-
| built upon
|-
| running
|}

==== Specific markers ====
{| class="wikitable"
|-
! Framework !! Keyword
|-
| Adobe ColdFusion || 
|-
| Indexhibit || ndxz-studio
|}

=== Specific files and folders ===
Different for each specific framework. It is recommended to install the corresponding framework during penetration tests in order to have better understanding of what infrastructure is presented and which files might be left on the server. However, several good pre-made lists already exist, one good example of them - FuzzDB wordlists of predictable files/folders (http://code.google.com/p/fuzzdb/)

== Tools ==
A list of general and well-known tools is presented below. There are also a lot of other utilities, as well as framework-based fingerprinting tools.

=== WhatWeb ===
Website: http://www.morningstarsecurity.com/research/whatweb 
Currently one of the best fingerprinting tools on the market. Included in a default [[Kali Linux]] build.
Language: Ruby
Matches for fingerprinting are made with:
* Text strings (case sensitive)
* Regular expressions
* Google Hack Database queries (limited set of keywords)
* MD5 hashes
* URL recognition
* HTML tag patterns
* Custom ruby code for passive and aggressive operations

Sample output is presented on a screenshot below:

[[Image:whatweb-sample.png]]

=== BlindElephant ===
Website: https://community.qualys.com/community/blindelephant 
This great tool works on the principle of static file checksum based version difference thus providing a very high quality of fingerprinting.
Language: Python

Sample output of a successful fingerprint:
<pre>
pentester$ python BlindElephant.py http://my_target drupal
Loaded /Library/Python/2.7/site-packages/blindelephant/dbs/drupal.pkl with 145 versions, 478 differentiating paths, and 434 version groups.
Starting BlindElephant fingerprint for version of drupal at http://my_target

Hit http://my_target/CHANGELOG.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 527b085a3717bd691d47713dff74acf4

Hit http://my_target/INSTALL.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 14dfc133e4101be6f0ef5c64566da4a4

Hit http://my_target/misc/drupal.js
Possible versions based on result: 7.12, 7.13, 7.14

Hit http://my_target/MAINTAINERS.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 36b740941a19912f3fdbfcca7caa08ca

Hit http://my_target/themes/garland/style.css
Possible versions based on result: 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 7.10, 7.11, 7.12, 7.13, 7.14

...

Fingerprinting resulted in:
7.14

Best Guess: 7.14
</pre>

=== Wappalyzer ===
Website: http://wappalyzer.com 
Wapplyzer is a Firefox/Chrome plug-in. It works on only regular expression matching and doesn't need anything other than the page to be loaded on browser. It works completely at the browser level and give results in form of icons. Although sometimes it has false positives, this is very handy to have notion of what technologies were used to construct a target website immediately after browsing a page.
Sample output of a plug-in is presented on a screenshot below.

[[Image:Owasp-wappalyzer.png]]

== Vulnerability References ==
'''Whitepapers''' 
* Saumil Shah: "An Introduction to HTTP fingerprinting" - http://www.net-square.com/httprint_paper.html
* Anant Shrivastava : "Web Application Finger Printing" - http://anantshri.info/articles/web_app_finger_printing.html

== Remediation ==
General advice: use several tools described above and check scan logs in order to better understand what exactly helps an attacker to disclose your web framework. By performing multiple scans after changes you've done to hide framework tracks, it's possible to achieve better security level and to make sure of undetectability by automatic scans.
Below you may find specific recommendations by framework marker location and some additional interesting approaches.

==== HTTP headers ====
Check your configuration and disable/obfuscate all HTTP-headers which disclose information about used technologies.
An interesting article about HTTP-headers obfuscation using Netscaler:
http://grahamhosking.blogspot.ru/2013/07/obfuscating-http-header-using-netscaler.html

==== Cookies ====
It is recommended to change cookie names by making proper changes in corresponding config files.

==== HTML source code ====
Check manually the contents of your HTML code and remove everything what explicitly points to the framework.
General guidelines:
*Make sure you didn't leave any visual markers disclosing used framework
*Remove any unnecessary comments (copyrights, bug information, specific framework comments)
*Remove META and generator tags
*Use your own css/js and do not store those in a framework-specific folders
*Make sure you're not using default scripts on the page; if using them is needed, obfuscate them.

==== Specific files and folders ====
*Remove any unnecessary/unused files on the server. This implies text files disclosing information about versions and installation too.
*Restrict access to other files in order to achieve 404-response when accessing them from outside. This can be done, for example, by modifying htaccess file and adding RewriteCond / RewriteRule there. Example of such restriction for two common WordPress folders is presented below.
<pre>
RewriteCond %{REQUEST_URI} /wp-login\.php$ [OR]
RewriteCond %{REQUEST_URI} /wp-admin/$
RewriteRule $ /http://your_website [R=404,L]
</pre>
However, these are not the only ways to restrict access. In order to automate this process, certain framework-specific plugins exist. One example for WordPress is StealthLogin (http://wordpress.org/plugins/stealth-login-page)

==== Additional approaches ====
*Checksum management
*:Purpose of this approach is to beat checksum-based scanners and don't let them to disclose files by their hashes. Generally, there are two approaches in checksum management:
*:*Change the location of where those files are placed (i.e. move them to another folder, or rename existing)
*:*Modify their contents - even slight modification results in a completely different hash sum, so adding a single byte in the end of the file should not be a big problem.
*Controlled chaos
*:A funny and effective method which essense is adding bogus files and folders from other frameworks in order to fool scanners and confuse an attacker. But be careful not to overwrite exisiting files and folders and to break the current framework!

Fingerprint Web Application Framework (OTG-INFO-008)

2014-03-18T21:45:21Z

Aditya Balapure:

{{Template:OWASP Testing Guide v4}}
== Summary ==
Web framework[*] fingerprinting is an important subtask of the information gathering process. Knowing the type of framework can automatically give a great advantage if such a framework has already been tested by the penetration tester. It is not only the known vulnerabilities in unpatched versions but specific misconfigurations in the framework and known file structure which makes the fingerprinting process so important.

Several different vendors and versions of web frameworks are widely used.
Information about it significantly helps in the testing process, and can also help in changing the course of the test.
Such information can be derived by careful analysis of certain common locations. Most of the web frameworks have several markers in those locations which help an attacker to spot them.
This is basically what all automatic tools do, they look for a marker from a predefined location
and then compare it to the database of known signatures. For better accuracy several markers are usually used.

[*] Please note that in this article we make no difference between Web Application Frameworks (WAF) and Content Management Systems (CMS).
This has been done to make it convenient to fingerprint both of them in one chapter.Further we reference both of the categories as web frameworks.

== Test Objectives ==
To define type of used web framework so to have a better understanding of the penetration testing methodology.

== How to Test ==

=== Black Box testing ===
There are several most common locations to look in in order to define the current framework:
*HTTP headers
*Cookies
*HTML source code
*Specific files and folders

Let's look closer at those approaches.

==== HTTP headers ====
The most basic form of identifying a web framework is to look at the ''X-Powered-By'' field in the HTTP response header.
Many tools can be used to fingerprint a target. The simplest one is netcat utility.
Consider the following HTTP Request-Response:
<pre>
$ nc 127.0.0.1 80
HEAD / HTTP/1.0

HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Mono
</pre>

From the ''X-Powered-By'' field, we understand that the web application framework is likely to be Mono.

However, although simplicity and quickness of such an approach, this methodology doesn't work in 100% of cases. It is possible to easily disable ''X-Powered-By'' header by a proper configuration. There are also several techniques that allow a web site to obfuscate HTTP headers (see an example in [[#Remediation]] chapter)
So in the same example we could either miss the ''X-Powered-By'' header or obtain an answer like the following:
<pre>
HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Blood, sweat and tears
</pre>

Sometimes there are more HTTP-headers which point at the certain web framework. In the following example according to the information from HTTP-request one can see that ''X-Powered-By'' header contains PHP version. However, ''X-Generator'' header points out the used framework is actually Swiftlet, which helps a penetration tester to expand his attack vectors. When performing fingerprinting, always carefully inspect every HTTP-header for such leaks.
<pre>
HTTP/1.1 200 OK
Server: nginx/1.4.1
Date: Sat, 07 Sep 2013 09:22:52 GMT
Content-Type: text/html
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/5.4.16-1~dotdeb.1
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Generator: Swiftlet
</pre>

==== Cookies ====
Another similar and somehow more reliable way to determine the current web framework are framework-specific cookies.
Consider the following HTTP-request:

[[Image:Cakephp_cookie.png]]

As we can see, cookie ''CAKEPHP'' has automatically been set, which gives us the information about the framework being used. List of common cookies names is presented in chapter [[#Cookies_2]]. Limitations are the same - it is possible to change the name of the cookie. For example, for the selected ''CakePHP'' framework this could be done by the following configuration (excerpt from core.php):

<pre>
/**
* The name of CakePHP's session cookie.
*
* Note the guidelines for Session names states: "The session name references
* the session id in cookies and URLs. It should contain only alphanumeric
* characters."
* @link http://php.net/session_name
*/
Configure::write('Session.cookie', 'CAKEPHP');
</pre>

However from the experience, these changes are less likely to be made than, for example, ''X-Powered-By'' header so this approach can be considered as more reliable one.

==== HTML source code ====
This technique is based on finding certain patterns in the HTML page source code. Often one can find much information which helps a tester to recognize a specific web framework.
Ones of the common markers are HTML comments which directly lead to framework disclosure. More often certain framework-specific paths can be found, i.e. links to framework-specific css and/or js folders. Finally, specific script variables might also point to a certain framework.
From the screenshot below one can easily learn the used framework and its version by the mentioned markers: comment, specific paths and script variables can all help an attacker to quickly determine an instance of ZK framework.

[[Image:Zk_html_source.png]]

More frequently such information is placed between <head></head>, in <meta> tags or at the end of the page. Nevertheless, it is recommended to check the whole document since it can be useful for other purposes such as inspection of other useful comments and hidden fields.
Sometimes, however, web developers do not care much about hiding tracks of the used framework. It's still possible to stumble upon something like that at the bottom of the page:

[[Image:banshee_bottom_page.png]]

==== Specific files and folders ====
Apart from information, gathered from HTML sources, there is another approach which greatly helps an attacker to determine the framework with high accuracy.
Every framework has its own specific file and folder structure on the server. We pointed out that one can see the specific path from the HTML page source but sometimes they are not explicitly presented there and still reside on the server. In order to uncover them such technique as dirbusting is used. Dirbusting essense is fuzzing target with predictable folder- and filenames and monitoring HTTP-responses thus enumerating server contents. This information can be used both for finding default files and attacking them in next stages and for fingerprinting the web framework.
Dirbusting can be done in several ways, example below shows successful dirbusting attack against a WordPress-powered target with the help of defined list and Intruder functionality of Burp Suite.

[[Image:Wordpress_dirbusting.png]]

We can see that for some WordPress-specific folders (for instance, /wp-includes/, /wp-admin/ and /wp-content/) HTTP-reponses are 403 (Forbidden), 302 (Found, redirection to ''wp-login.php'') and 200 (OK) respectively. This is a good indicator that the target is WordPress-powered. The same way it is possible to dirbust different framework plugin folders and their versions. On a screenshot below one can see a typical CHANGELOG file of a Drupal plugin, which both points out on the used framework and discloses vulnerable plugin version.

[[Image:Drupal_botcha_disclosure.png]]

Tip: before starting dirbusting (which is a useful operation anyway), it is recommended to check robots.txt file first. Sometimes framework specific folders and other sensitive information can be found there as well! Example of such robots file is presented on a screenshot below.

[[Image:Robots-info-disclosure.png]]

=== Gray Box testing ===
Please address yourself to watching Monty Python's Flying Circus, I'm sure you'll like it!

== Common frameworks ==
=== Cookies ===

{| class="wikitable"
|-
! Framework !! Cookie name
|-
| Zope || zope3
|-
| CakePHP || cakephp
|-
| Kohana || kohanasession
|-
| Laravel || laravel_session
|-
| 1C-Bitrix || BITRIX_
|-
| AMPcms || AMP
|-
|-
| Django CMS || django
|-
| DotNetNuke || DotNetNukeAnonymous
|-
| e107 || e107_tz
|-
| EPiServer || EPiTrace, EPiServer
|-
| Graffiti CMS || graffitibot
|-
| Hotaru CMS || hotaru_mobile
|-
| ImpressCMS || ICMSession
|-
| Indico || MAKACSESSION
|-
| InstantCMS || InstantCMS[logdate]
|-
| Kentico CMS || CMSPreferredCulture
|-
| MODx || SN4[12symb]
|-
| TYPO3 || fe_typo_user
|-
| Dynamicweb || Dynamicweb
|-
| LEPTON || lep[some_numeric_value]+sessionid
|-
| Wix || Domain=.wix.com
|-
| VIVVO || VivvoSessionId
|}

=== HTML source code ===
==== General markers ====
{| class="wikitable"
|-
| %framework_name%
|-
| powered by
|-
| built upon
|-
| running
|}

==== Specific markers ====
{| class="wikitable"
|-
! Framework !! Keyword
|-
| Adobe ColdFusion || 
|-
| Indexhibit || ndxz-studio
|}

=== Specific files and folders ===
Different for each specific framework. It is recommended to install the corresponding framework during penetration tests in order to have better understanding of what infrastructure is presented and which files might be left on the server. However, several good pre-made lists already exist, one good example of them - FuzzDB wordlists of predictable files/folders (http://code.google.com/p/fuzzdb/)

== Tools ==
A list of general and well-known tools is presented below. There are also a lot of other utilities, as well as framework-based fingerprinting tools.

=== WhatWeb ===
Website: http://www.morningstarsecurity.com/research/whatweb 
Currently one of the best fingerprinting tools on the market. Included in a default [[Kali Linux]] build.
Language: Ruby
Matches for fingerprinting are made with:
* Text strings (case sensitive)
* Regular expressions
* Google Hack Database queries (limited set of keywords)
* MD5 hashes
* URL recognition
* HTML tag patterns
* Custom ruby code for passive and aggressive operations

Sample output is presented on a screenshot below:

[[Image:whatweb-sample.png]]

=== BlindElephant ===
Website: https://community.qualys.com/community/blindelephant 
This great tool works on the principle of static file checksum based version difference thus providing a very high quality of fingerprinting.
Language: Python

Sample output of a successful fingerprint:
<pre>
pentester$ python BlindElephant.py http://my_target drupal
Loaded /Library/Python/2.7/site-packages/blindelephant/dbs/drupal.pkl with 145 versions, 478 differentiating paths, and 434 version groups.
Starting BlindElephant fingerprint for version of drupal at http://my_target

Hit http://my_target/CHANGELOG.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 527b085a3717bd691d47713dff74acf4

Hit http://my_target/INSTALL.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 14dfc133e4101be6f0ef5c64566da4a4

Hit http://my_target/misc/drupal.js
Possible versions based on result: 7.12, 7.13, 7.14

Hit http://my_target/MAINTAINERS.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 36b740941a19912f3fdbfcca7caa08ca

Hit http://my_target/themes/garland/style.css
Possible versions based on result: 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 7.10, 7.11, 7.12, 7.13, 7.14

...

Fingerprinting resulted in:
7.14

Best Guess: 7.14
</pre>

=== Wappalyzer ===
Website: http://wappalyzer.com 
Wapplyzer is a Firefox/Chrome plug-in. It works on only regular expression matching and doesn't need anything other than the page to be loaded on browser. It works completely at the browser level and give results in form of icons. Although sometimes it has false positives, this is very handy to have notion of what technologies were used to construct a target website immediately after browsing a page.
Sample output of a plug-in is presented on a screenshot below.

[[Image:Owasp-wappalyzer.png]]

== Vulnerability References ==
'''Whitepapers''' 
* Saumil Shah: "An Introduction to HTTP fingerprinting" - http://www.net-square.com/httprint_paper.html
* Anant Shrivastava : "Web Application Finger Printing" - http://anantshri.info/articles/web_app_finger_printing.html

== Remediation ==
General advice: use several tools described above and check scan logs in order to better understand what exactly helps an attacker to disclose your web framework. By performing multiple scans after changes you've done to hide framework tracks, it's possible to achieve better security level and to make sure of undetectability by automatic scans.
Below you may find specific recommendations by framework marker location and some additional interesting approaches.

==== HTTP headers ====
Check your configuration and disable/obfuscate all HTTP-headers which disclose information about used technologies.
An interesting article about HTTP-headers obfuscation using Netscaler:
http://grahamhosking.blogspot.ru/2013/07/obfuscating-http-header-using-netscaler.html

==== Cookies ====
It is recommended to change cookie names by making proper changes in corresponding config files.

==== HTML source code ====
Check manually the contents of your HTML code and remove everything what explicitly points to the framework.
General guidelines:
*Make sure you didn't leave any visual markers disclosing used framework
*Remove any unnecessary comments (copyrights, bug information, specific framework comments)
*Remove META and generator tags
*Use your own css/js and do not store those in a framework-specific folders
*Make sure you're not using default scripts on the page; if using them is needed, obfuscate them.

==== Specific files and folders ====
*Remove any unnecessary/unused files on the server. This implies text files disclosing information about versions and installation too.
*Restrict access to other files in order to achieve 404-response when accessing them from outside. This can be done, for example, by modifying htaccess file and adding RewriteCond / RewriteRule there. Example of such restriction for two common WordPress folders is presented below.
<pre>
RewriteCond %{REQUEST_URI} /wp-login\.php$ [OR]
RewriteCond %{REQUEST_URI} /wp-admin/$
RewriteRule $ /http://your_website [R=404,L]
</pre>
However, these are not the only ways to restrict access. In order to automate this process, certain framework-specific plugins exist. One example for WordPress is StealthLogin (http://wordpress.org/plugins/stealth-login-page)

==== Additional approaches ====
*Checksum management
*:Purpose of this approach is to beat checksum-based scanners and don't let them to disclose files by their hashes. Generally, there are two approaches in checksum management:
*:*Change the location of where those files are placed (i.e. move them to another folder, or rename existing)
*:*Modify their contents - even slight modification results in a completely different hash sum, so adding a single byte in the end of the file should not be a big problem.
*Controlled chaos
*:A funny and effective method which essense is adding bogus files and folders from other frameworks in order to fool scanners and confuse an attacker. But be careful not to overwrite exisiting files and folders and to break the current framework!

Fingerprint Web Application Framework (OTG-INFO-008)

2014-03-18T21:39:43Z

Aditya Balapure:

{{Template:OWASP Testing Guide v4}}
== Summary ==
Web framework[*] fingerprinting is an important subtask of the information gathering process. Knowing the type of framework can automatically give a great advantage if such a framework has already been tested by the penetration tester. It is not only the known vulnerabilities in unpatched versions but specific misconfigurations in the framework and known file structure which makes the fingerprinting process so important.

Several different vendors and versions of web frameworks are widely used.
Information about it significantly helps in the testing process, and can also help in changing the course of the test.
Such information can be derived by careful analysis of certain common locations. Most of the web frameworks have several markers in those locations which help an attacker to spot them.
This is basically what all automatic tools do, they look for a marker from a predefined location
and then compare it to the database of known signatures. For better accuracy several markers are usually used.

[*] Please note that in this article we make no difference between Web Application Frameworks (WAF) and Content Management Systems (CMS).
This has been done to make it convenient to fingerprint both of them in one chapter.Further we reference both of the categories as web frameworks.

== Test Objectives ==
To define type of used web framework so to have a better understanding of the penetration testing methodology.

== How to Test ==

=== Black Box testing ===
There are several most common locations to look in in order to define the current framework:
*HTTP headers
*Cookies
*HTML source code
*Specific files and folders

Let's look closer at those approaches.

==== HTTP headers ====
The most basic form of identifying a web framework is to look at the ''X-Powered-By'' field in the HTTP response header.
Many tools can be used to fingerprint a target. The simplest one is netcat utility.
Consider the following HTTP Request-Response:
<pre>
$ nc 127.0.0.1 80
HEAD / HTTP/1.0

HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Mono
</pre>

From the ''X-Powered-By'' field, we understand that the web application framework is likely to be Mono.

However, although simplicity and quickness of such an approach, this methodology doesn't work in 100% of cases. It is possible to easily disable ''X-Powered-By'' header by a proper configuration. There are also several techniques that allow a web site to obfuscate HTTP headers (see an example in [[#Remediation]] chapter)
So in the same example we could either miss the ''X-Powered-By'' header or obtain an answer like the following:
<pre>
HTTP/1.1 200 OK
Server: nginx/1.0.14
Date: Sat, 07 Sep 2013 08:19:15 GMT
Content-Type: text/html;charset=ISO-8859-1
Connection: close
Vary: Accept-Encoding
X-Powered-By: Blood, sweat and tears
</pre>

Sometimes there are more HTTP-headers which point at the certain web framework. In the following example according to the information from HTTP-request one can see that ''X-Powered-By'' header contains PHP version. However, ''X-Generator'' header points out the used framework is actually Swiftlet, which helps a penetration tester to expand his attack vectors. When performing fingerprinting, always carefully inspect every HTTP-header for such leaks.
<pre>
HTTP/1.1 200 OK
Server: nginx/1.4.1
Date: Sat, 07 Sep 2013 09:22:52 GMT
Content-Type: text/html
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/5.4.16-1~dotdeb.1
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Generator: Swiftlet
</pre>

==== Cookies ====
Another similar and somehow more reliable way to determine the current web framework are framework-specific cookies.
Consider the following HTTP-request:

[[Image:Cakephp_cookie.png]]

As we can see, cookie ''CAKEPHP'' has automatically been set, which gives us the information about the framework being used. List of common cookies names is presented in chapter [[#Cookies_2]]. Limitations are the same - it is possible to change the name of the cookie. For example, for the selected ''CakePHP'' framework this could be done by the following configuration (excerpt from core.php):

<pre>
/**
* The name of CakePHP's session cookie.
*
* Note the guidelines for Session names states: "The session name references
* the session id in cookies and URLs. It should contain only alphanumeric
* characters."
* @link http://php.net/session_name
*/
Configure::write('Session.cookie', 'CAKEPHP');
</pre>

However from the experience, these changes are less likely to be made than, for example, ''X-Powered-By'' header so this approach can be considered as more reliable one.

==== HTML source code ====
This technique is based on finding certain patterns in the HTML page source code. Often one can find much information which helps a tester to recognize a specific web framework.
Ones of the common markers are HTML comments which directly lead to framework disclosure. More often certain framework-specific paths can be found, i.e. links to framework-specific css and/or js folders. Finally, specific script variables might also point to a certain framework.
From the screenshot below one can easily learn the used framework and its version by the mentioned markers: comment, specific paths and script variables can all help an attacker to quickly determine an instance of ZK framework.

[[Image:Zk_html_source.png]]

The most frequently such information is placed either between <head></head> and in <meta> tags or in the end of the page. Nevertheless, it is recommended to check the whole document - it can be useful for other purposes (such as inspection of other useful comments and hidden fields)
Sometimes, however, web developers don't care much about hiding tracks of a framework. It's still possible to stumble upon something like that at the bottom of the page:

[[Image:banshee_bottom_page.png]]

==== Specific files and folders ====
Apart from information, gathered from HTML sources, there is another approach which greatly helps an attacker to determine the framework with high accuracy.
Every framework has its own specific file and folder structure on the server. We pointed out that one can see the specific path from the HTML page source but sometimes they are not explicitly presented there and still reside on the server. In order to uncover them such technique as dirbusting is used. Dirbusting essense is fuzzing target with predictable folder- and filenames and monitoring HTTP-responses thus enumerating server contents. This information can be used both for finding default files and attacking them in next stages and for fingerprinting the web framework.
Dirbusting can be done in several ways, example below shows successful dirbusting attack against a WordPress-powered target with the help of defined list and Intruder functionality of Burp Suite.

[[Image:Wordpress_dirbusting.png]]

We can see that for some WordPress-specific folders (for instance, /wp-includes/, /wp-admin/ and /wp-content/) HTTP-reponses are 403 (Forbidden), 302 (Found, redirection to ''wp-login.php'') and 200 (OK) respectively. This is a good indicator that the target is WordPress-powered. The same way it is possible to dirbust different framework plugin folders and their versions. On a screenshot below one can see a typical CHANGELOG file of a Drupal plugin, which both points out on the used framework and discloses vulnerable plugin version.

[[Image:Drupal_botcha_disclosure.png]]

Tip: before starting dirbusting (which is a useful operation anyway), it is recommended to check robots.txt file first. Sometimes framework specific folders and other sensitive information can be found there as well! Example of such robots file is presented on a screenshot below.

[[Image:Robots-info-disclosure.png]]

=== Gray Box testing ===
Please address yourself to watching Monty Python's Flying Circus, I'm sure you'll like it!

== Common frameworks ==
=== Cookies ===

{| class="wikitable"
|-
! Framework !! Cookie name
|-
| Zope || zope3
|-
| CakePHP || cakephp
|-
| Kohana || kohanasession
|-
| Laravel || laravel_session
|-
| 1C-Bitrix || BITRIX_
|-
| AMPcms || AMP
|-
|-
| Django CMS || django
|-
| DotNetNuke || DotNetNukeAnonymous
|-
| e107 || e107_tz
|-
| EPiServer || EPiTrace, EPiServer
|-
| Graffiti CMS || graffitibot
|-
| Hotaru CMS || hotaru_mobile
|-
| ImpressCMS || ICMSession
|-
| Indico || MAKACSESSION
|-
| InstantCMS || InstantCMS[logdate]
|-
| Kentico CMS || CMSPreferredCulture
|-
| MODx || SN4[12symb]
|-
| TYPO3 || fe_typo_user
|-
| Dynamicweb || Dynamicweb
|-
| LEPTON || lep[some_numeric_value]+sessionid
|-
| Wix || Domain=.wix.com
|-
| VIVVO || VivvoSessionId
|}

=== HTML source code ===
==== General markers ====
{| class="wikitable"
|-
| %framework_name%
|-
| powered by
|-
| built upon
|-
| running
|}

==== Specific markers ====
{| class="wikitable"
|-
! Framework !! Keyword
|-
| Adobe ColdFusion || 
|-
| Indexhibit || ndxz-studio
|}

=== Specific files and folders ===
Different for each specific framework. It is recommended to install the corresponding framework during penetration tests in order to have better understanding of what infrastructure is presented and which files might be left on the server. However, several good pre-made lists already exist, one good example of them - FuzzDB wordlists of predictable files/folders (http://code.google.com/p/fuzzdb/)

== Tools ==
A list of general and well-known tools is presented below. There are also a lot of other utilities, as well as framework-based fingerprinting tools.

=== WhatWeb ===
Website: http://www.morningstarsecurity.com/research/whatweb 
Currently one of the best fingerprinting tools on the market. Included in a default [[Kali Linux]] build.
Language: Ruby
Matches for fingerprinting are made with:
* Text strings (case sensitive)
* Regular expressions
* Google Hack Database queries (limited set of keywords)
* MD5 hashes
* URL recognition
* HTML tag patterns
* Custom ruby code for passive and aggressive operations

Sample output is presented on a screenshot below:

[[Image:whatweb-sample.png]]

=== BlindElephant ===
Website: https://community.qualys.com/community/blindelephant 
This great tool works on the principle of static file checksum based version difference thus providing a very high quality of fingerprinting.
Language: Python

Sample output of a successful fingerprint:
<pre>
pentester$ python BlindElephant.py http://my_target drupal
Loaded /Library/Python/2.7/site-packages/blindelephant/dbs/drupal.pkl with 145 versions, 478 differentiating paths, and 434 version groups.
Starting BlindElephant fingerprint for version of drupal at http://my_target

Hit http://my_target/CHANGELOG.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 527b085a3717bd691d47713dff74acf4

Hit http://my_target/INSTALL.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 14dfc133e4101be6f0ef5c64566da4a4

Hit http://my_target/misc/drupal.js
Possible versions based on result: 7.12, 7.13, 7.14

Hit http://my_target/MAINTAINERS.txt
File produced no match. Error: Retrieved file doesn't match known fingerprint. 36b740941a19912f3fdbfcca7caa08ca

Hit http://my_target/themes/garland/style.css
Possible versions based on result: 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 7.10, 7.11, 7.12, 7.13, 7.14

...

Fingerprinting resulted in:
7.14

Best Guess: 7.14
</pre>

=== Wappalyzer ===
Website: http://wappalyzer.com 
Wapplyzer is a Firefox/Chrome plug-in. It works on only regular expression matching and doesn't need anything other than the page to be loaded on browser. It works completely at the browser level and give results in form of icons. Although sometimes it has false positives, this is very handy to have notion of what technologies were used to construct a target website immediately after browsing a page.
Sample output of a plug-in is presented on a screenshot below.

[[Image:Owasp-wappalyzer.png]]

== Vulnerability References ==
'''Whitepapers''' 
* Saumil Shah: "An Introduction to HTTP fingerprinting" - http://www.net-square.com/httprint_paper.html
* Anant Shrivastava : "Web Application Finger Printing" - http://anantshri.info/articles/web_app_finger_printing.html

== Remediation ==
General advice: use several tools described above and check scan logs in order to better understand what exactly helps an attacker to disclose your web framework. By performing multiple scans after changes you've done to hide framework tracks, it's possible to achieve better security level and to make sure of undetectability by automatic scans.
Below you may find specific recommendations by framework marker location and some additional interesting approaches.

==== HTTP headers ====
Check your configuration and disable/obfuscate all HTTP-headers which disclose information about used technologies.
An interesting article about HTTP-headers obfuscation using Netscaler:
http://grahamhosking.blogspot.ru/2013/07/obfuscating-http-header-using-netscaler.html

==== Cookies ====
It is recommended to change cookie names by making proper changes in corresponding config files.

==== HTML source code ====
Check manually the contents of your HTML code and remove everything what explicitly points to the framework.
General guidelines:
*Make sure you didn't leave any visual markers disclosing used framework
*Remove any unnecessary comments (copyrights, bug information, specific framework comments)
*Remove META and generator tags
*Use your own css/js and do not store those in a framework-specific folders
*Make sure you're not using default scripts on the page; if using them is needed, obfuscate them.

==== Specific files and folders ====
*Remove any unnecessary/unused files on the server. This implies text files disclosing information about versions and installation too.
*Restrict access to other files in order to achieve 404-response when accessing them from outside. This can be done, for example, by modifying htaccess file and adding RewriteCond / RewriteRule there. Example of such restriction for two common WordPress folders is presented below.
<pre>
RewriteCond %{REQUEST_URI} /wp-login\.php$ [OR]
RewriteCond %{REQUEST_URI} /wp-admin/$
RewriteRule $ /http://your_website [R=404,L]
</pre>
However, these are not the only ways to restrict access. In order to automate this process, certain framework-specific plugins exist. One example for WordPress is StealthLogin (http://wordpress.org/plugins/stealth-login-page)

==== Additional approaches ====
*Checksum management
*:Purpose of this approach is to beat checksum-based scanners and don't let them to disclose files by their hashes. Generally, there are two approaches in checksum management:
*:*Change the location of where those files are placed (i.e. move them to another folder, or rename existing)
*:*Modify their contents - even slight modification results in a completely different hash sum, so adding a single byte in the end of the file should not be a big problem.
*Controlled chaos
*:A funny and effective method which essense is adding bogus files and folders from other frameworks in order to fool scanners and confuse an attacker. But be careful not to overwrite exisiting files and folders and to break the current framework!

Map execution paths through application (OTG-INFO-007)

2014-03-18T21:15:43Z

Aditya Balapure:

{{Template:OWASP Testing Guide v4}}

== Summary ==

Before commencing security testing, understanding the structure of the application is paramount. Without a thorough understanding of the layout of the application, it is unlkely that it will be tested thoroughly.

== Test Objectives ==

Map the target application and understand the principal workflows.

== How to Test ==

In black box testing it is extremely difficult to test the entire code base. Not just because the tester has no view of the code paths through the application, but even if they did, to test all code paths would be very time consuming. One way to reconcile this is to document what code paths were discovered and tested.

There are several ways to approach the testing and measurement of code coverage:

* '''Path''' - test each of the paths through an application which includes combinatorial and boundary value analysis testing for each decision path. While this approach offers thoroughness, the number of testable paths grows exponentially with each decision branch.
* '''Data flow (or taint analysis)''' - tests the assignment of variables via external interaction (normally users). Focuses on mapping the flow, transformation and use of data throughout an application.
* '''Race''' - tests multiple concurrent instances of the application manipulating the same data.

The trade off as to what method is used and to what degree each method is used should be negotiated with the application owner. Simpler approaches could also be adopted, including asking the application owner what functions or code sections they are particularly concerned about and how those code segments can be reached.

''' Black Box testing '''

To demonstrate code coverage to the application owner, the tester can start with a spreadsheet and document all the links discovered by spidering the application (either manually or automatically). Then the tester can look more closely at decision points in the application and investigate how many significant code paths are discovered. These should then be documented in the spreadsheet with URLs, prose and screenshot descriptions of the paths discovered.

''' Gray/White Box testing '''

Ensuring sufficient code coverage for the application owner is far easier with the gray and white box approach to testing. Information solicited by and provided to the tester will ensure the minimum requirements for code coverage are met.

=== Example ===

''' Automatic Spidering '''

The automatic spider is a tool used to automatically discover new resources (URLs) on a particular website. It begins with a list of URLs to visit, called the seeds, which depends on how the Spider is started. While there are a lot of Spidering tools, we will use the [https://code.google.com/p/zaproxy/ Zed Attack Proxy (ZAP)] in the following example

[[File:OWASPZAPSP.png |1050px|]]

[https://code.google.com/p/zaproxy/ ZAP] offers the following automatic spidering features, which can be selected based on the tester needs

*Spider Site - The seed list contains all the existing URIs already found for the selected site.
*Spider Subtree - The seed list contains all the existing URIs already found and present in the subtree of the selected node.
*Spider URL - The seed list contains only the URI corresponding to the selected node (in the Site Tree).
*Spider all in Scope - The seed list contains all the URIs the user has selected as being 'In Scope'.

== Tools ==

* [https://code.google.com/p/zaproxy/ Zed Attack Proxy (ZAP)]

* [http://en.wikipedia.org/wiki/List_of_spreadsheet_software List of spreadsheet software]

* [http://en.wikipedia.org/wiki/Diagramming_software Diagramming software]

== Vulnerability References ==

'''Whitepapers'''

[1] http://en.wikipedia.org/wiki/Code_coverage

== Remediation ==

Identify application entry points (OTG-INFO-006)

2014-03-17T21:40:09Z

Aditya Balapure:

{{Template:OWASP Testing Guide v4}}

== Summary ==
Enumerating the application and its attack surface is a key precursor before any thorough testing can be undertaken, as it allows the tester to identify likely areas of weakness. This section aims to help identify and map out areas within the application that should be investigated once enumeration and mapping have been completed.

== Test Objectives ==

Understand how requests are formed and typical responses from the application

== How to Test ==

Before any testing begins, always get a good understanding of the application and how the user/browser communicates with it. As you walk through the application, pay special attention to all HTTP requests (GET and POST Methods, also known as Verbs), as well as every parameter and form field that is passed to the application. In addition, pay attention to when GET requests are used and when POST requests are used to pass parameters to the application. It is very common that GET requests are used, but when sensitive information is passed, it is often done within the body of a POST request. Note that to see the parameters sent in a POST request, you will need to use a tool such as an intercepting proxy (for example, OWASP: [[OWASP_Zed_Attack_Proxy_Project| Zed Attack Proxy (ZAP)]]) or a browser plug-in. Within the POST request, also make special note of any hidden form fields that are being passed to the application, as these usually contain sensitive information, such as state information, quantity of items, the price of items, that the developer never intended for you to see or change.

In the author's experience, it has been very useful to use an intercepting proxy and a spreadsheet for this stage of the testing. The proxy will keep track of every request and response between you and the application as you walk through it. Additionally, at this point, testers usually trap every request and response so that they can see exactly every header, parameter, etc. that is being passed to the application and what is being returned. This can be quite tedious at times, especially on large interactive sites (think of a banking application). However, experience will teach you what to look for, and, therefore, this phase can be significantly reduced. As you walk through the application, take note of any interesting parameters in the URL, custom headers, or body of the requests/responses, and save them in your spreadsheet. The spreadsheet should include the page you requested (it might be good to also add the request number from the proxy, for future reference), the interesting parameters, the type of request (POST/GET), if access is authenticated/unauthenticated, if SSL is used, if it's part of a multi-step process, and any other relevant notes. Once you have every area of the application mapped out, then you can go through the application and test each of the areas that you have identified and make notes for what worked and what didn't work. The rest of this guide will identify how to test each of these areas of interest, but this section must be undertaken before any of the actual testing can commence.

Below are some points of interests for all requests and responses. Within the requests section, focus on the GET and POST methods, as these appear the majority of the requests. Note that other methods, such as PUT and DELETE, can be used. Often, these more rare requests, if allowed, can expose vulnerabilities. There is a special section in this guide dedicated for testing these HTTP methods.

'''Requests:'''
* Identify where GETs are used and where POSTs are used.
* Identify all parameters used in a POST request (these are in the body of the request).
* Within the POST request, pay special attention to any hidden parameters. When a POST is sent all the form fields (including hidden parameters) will be sent in the body of the HTTP message to the application. These typically aren't seen unless you are using a proxy or view the HTML source code. In addition, the next page you see, its data, and your access can all be different depending on the value of the hidden parameter(s).
* Identify all parameters used in a GET request (i.e., URL), in particular the query string (usually after a ? mark).
* Identify all the parameters of the query string. These usually are in a pair format, such as foo=bar. Also note that many parameters can be in one query string such as separated by a &, ~, :, or any other special character or encoding.
* A special note when it comes to identifying multiple parameters in one string or within a POST request is that some or all of the parameters will be needed to execute your attacks. You need to identify all of the parameters (even if encoded or encrypted) and identify which ones are processed by the application. Later sections of the guide will identify how to test these parameters. At this point, just make sure you identify each one of them.
* Also pay attention to any additional or custom type headers not typically seen (such as debug=False).

'''Responses:'''
*Identify where new cookies are set (Set-Cookie header), modified, or added to.
*Identify where there are any redirects (3xx HTTP status code), 400 status codes, in particular 403 Forbidden, and 500 internal server errors during normal responses (i.e., unmodified requests).
*Also note where any interesting headers are used. For example, "Server: BIG-IP" indicates that the site is load balanced. Thus, if a site is load balanced and one server is incorrectly configured, then you might have to make multiple requests to access the vulnerable server, depending on the type of load balancing used.

=== Black Box testing and example ===
'''Testing for application entry points:''' 
The following are two examples on how to check for application entry points. 

====EXAMPLE 1====
This example shows a GET request that would purchase an item from an online shopping application.

GET https://x.x.x.x/shoppingApp/buyme.asp?CUSTOMERID=100&ITEM=z101a&PRICE=62.50&IP=x.x.x.x
Host: x.x.x.x
Cookie: SESSIONID=Z29vZCBqb2IgcGFkYXdhIG15IHVzZXJuYW1lIGlzIGZvbyBhbmQgcGFzc3dvcmQgaXMgYmFy

'''Result Expected:'''

Here you would note all the parameters of the request such as CUSTOMERID, ITEM, PRICE, IP, and the Cookie (which could just be encoded parameters or used for session state).

====EXAMPLE 2====
This example shows a POST request that would log you into an application.

POST https://x.x.x.x/KevinNotSoGoodApp/authenticate.asp?service=login
Host: x.x.x.x
Cookie: SESSIONID=dGhpcyBpcyBhIGJhZCBhcHAgdGhhdCBzZXRzIHByZWRpY3RhYmxlIGNvb2tpZXMgYW5kIG1pbmUgaXMgMTIzNA==
CustomCookie=00my00trusted00ip00is00x.x.x.x00

Body of the POST message:

user=admin&pass=pass123&debug=true&fromtrustIP=true

'''Result Expected:'''

In this example you would note all the parameters as you have before but notice that the parameters are passed in the body of the message and not in the URL. Additionally, note that there is a custom cookie that is being used.

=== Gray Box testing and example ===

Testing for application entry points via a Gray Box methodology would consist of everything already identified above with one caveat. This would be if there are any external sources from which the application receives data and processes it (such as SNMP traps, syslog messages, SMTP, or SOAP messages from other servers). If there are any external sources of input into the application then a meeting with the application developers could identify any functions that would accept or expect user input and how it's formatted. For example, the developer could help in understanding how to formulate a correct SOAP request that the application would accept and where the web service resides (if the web service or any other function hasn't already been identified during the black box testing).

== Tools ==

'''Intercepting Proxy:''' 

*OWASP: [[OWASP_Zed_Attack_Proxy_Project| Zed Attack Proxy (ZAP)]]
*OWASP: [[OWASP_WebScarab_Project| WebScarab]]
* [http://www.portswigger.net/burp/ Burp Suite]
* [http://www.contextis.com/research/tools/cat/ CAT]

'''Browser Plug-in:''' 

*[http://www.bayden.com/TamperIE/ TamperIE for Internet Explorer]
*[https://addons.mozilla.org/en-US/firefox/addon/966 Tamper Data for Firefox]

== References ==

'''Whitepapers''' 

*RFC 2616 – Hypertext Transfer Protocol – HTTP 1.1 -
http://tools.ietf.org/html/rfc2616

Testing: WS Information Gathering (OWASP-WS-001)

2013-07-04T09:20:04Z

Aditya Balapure:

{{Template:OWASP Testing Guide v3}}

==Brief Summary==
The first step to perform a Web Service Test is to determine the WS entry points and the communication schema: this is described in the WSDL associated with the Web Service.

==Black Box Testing and example==
'''Zero Knowledge''' 
Normally you will have a WSDL path to access the Web Service, but if you have zero knowledge about it, you will have to use UDDI to find a specific service.
Web Services have three critical building blocks – UDDI, WSDL and SOAP. There is a third intermediate player facilitating communication between the consumer and supplier, referred to as Universal Business Registry (UBR).
There are several ways to find our WSDL: the easiest one is to make a search Query in public search engine. For example, if you have to assess an example.com public WS, on google.com you can type:

inurl:wsdl site:example.com

and you will find all the public Example WSDL.
Net Square wsPawn is a useful tool that acts as Web Services Consumer and makes a query to the UBR and looks for services as per requirements. Then UBR supplies the list of available services. The Web Services Consumer chooses one or more available services. Next, Web Services Consumer requests an access point or end point for these services. UBR supplies this information. From this moment, Web Services Consumer approaches the Web Services Supplier’s Host/IP address (WSDL) and starts accessing service. 

'''WSDL endpoints''' 
When a tester accesses the WSDL, he can determine an access point and available interfaces for web services. These interfaces or methods take inputs using SOAP over HTTP/HTTPS. If these inputs are not defined well at the source code level, they can be compromised and exploited.
For example given this WSDL Endpoint:

http://www.example.com/ws/FindIP.asmx?WSDL

you can obtain the following description of the Web Services:
<pre>
<?xml version="1.0" encoding="utf-8"?>
<wsdl:definitions xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:s="http://www.w3.org/2001/XMLSchema" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:tns="http://example.com/webservices/" xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" targetNamespace="http://example.com/webservices/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/">
<wsdl:types>
<s:schema elementFormDefault="qualified" targetNamespace="http://example.com/webservices/">
<s:element name="GetURLIP">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1" name="EnterURL" type="s:string" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="GetURLIPResponse">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1" name="GetURLIPResult" type="s:string" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="string" nillable="true" type="s:string" />
</s:schema>
</wsdl:types>
<wsdl:message name="GetURLIPSoapIn">
<wsdl:part name="parameters" element="tns:GetURLIP" />
</wsdl:message>
<wsdl:message name="GetURLIPSoapOut">
<wsdl:part name="parameters" element="tns:GetURLIPResponse" />
</wsdl:message>
<wsdl:message name="GetURLIPHttpGetIn">
<wsdl:part name="EnterURL" type="s:string" />
……
</wsdl:service>
</wsdl:definitions>
</pre>
This WS simply receives in input a logical name (EnterURL) and gives in output the relative IP Address. So we have GetURLIP as method for the WS and EnterURL (string) as input.
In that manner we have identified the WS entry point and we are ready to test it.
 
 
'''Web Services Discovery''' 
Web Services consumer needs a simple and standardized way to find Web Services available from remote servers.
There are two ways to discover a Web Service, DISCO and UDDI. 
The Web Service Discovery (DISCO) is one way that we can discover the URL's WSDL descriptor and other XML documents, like Schema Definition Document (.xsd). 

For instance, with an HTTP query to a Web server:
http://myexample.com/myexampleService.asmx?DISCO

we obtain the following DISCO descriptor:
 
<pre>
<?xml version="1.0" encoding="utf-8"?>
<discovery xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.xmlsoap.org/disco/">
<contractRef ref="http://myexample.com/MyexampleService.asmx?wsdl" docRef="http://myexample.com/myexample.asmx" xmlns="http://schemas.xmlsoap.org/disco/scl/" />
<soap address="http://myexample.com/MyexampleService.asmx" xmlns:q1="http://myexample.com/terraserver/" binding="q1:myexampleServiceSoap" xmlns="http://schemas.xmlsoap.org/disco/soap/" />
</discovery>
</pre>
 
We can see in the above XML document we have a reference for a WSDL document, where we can obtain a description of Web Services available from remote Web Server.

DISCO is a Microsoft technology, UDDI (Universal Description, Discovery and Integration) is an OASIS standard .
 
'''WS Well Known Naming''' 
Common Web Services platforms have a naming convention for offering a WSDL documents: This naming convention can be used to retrieve WSDL via URIs probing or through queries to web search server.

Some URLs that we can use are, for example:

http://<webservice-host>:<port>/<servicename>
http://<webservice-host>:<port>/<servicename>.wsdl
http://<webservice-host>:<port>/<servicename>?wsdl
http://<webservice-host>:<port>/<servicename>.aspx?wsdl
 instead of .aspx extension we can also use .ascx, .asmx, .ashx extensions
 Same thing with ?disco instead of ?wsdl

http://<webservice-host>:<port>/<servicename.dll>?wsdl
http://<webservice-host>:<port>/<servicename.exe>?wsdl
http://<webservice-host>:<port>/<servicename.php>?wsdl
http://<webservice-host>:<port>/<servicename.pl>?wsdl

For Apache Axis we can try:
http://<webservice-host>:<port>/axis/services/<servicename>?wsdl
http://<webservice-host>:<port>/axis/services/<service-name>
 
 

'''Search for public Web Services''' 
The seekda Web Services Search Engine can help to find a public Web Services with related descriptions.
To find Web Services just type the keyword into seekda Web Services Search Engine. We can also browse by several other criteria such as Tag Cloud, Services by Countries, Most Used Services.
http://seekda.com
 

[[Image:seekda.jpg]]

 
Another Web Server with good links and Resources is WSindex (http://www.wsindex.org).

[[Image:wsindex.png]]
 
 
''' UDDI Browser''' 
A web server that provides a very useful UDDI on-line tool to browse and search public UDDI resources is offered on http://www.soapclient.com/uddisearch.html. 
We can see we can use two operators, Microsoft and XMethods

[[Image:uddi_browser_part.jpg]]
 

The server offers, for example, to search all UDDI with a specific string in business names, service names or service types. 

'''Advanced UDDI browsing''' 

We can search private UDDI registries using the Advanced feature of UDDI browser.
 
[[Image:uddi_browser.jpg]]
 
 
This service allows interaction with Web services dynamically. 
Soapclient offer others methods to allow you to discover web services and useful links to other resources.

 
 

'''Command line interaction'''
 
Sometimes it is useful to interact with webservices from a command line.
 
 
Simple SOAP Client - SOAPClient4XG 
SOAPClient4XG is a SOAP Client for XML which allows you to make a SOAP request from command line, for example:

java -jar SOAPClient4XG http://api.google.com/search/beta2 my_sample_search.xml 
 

CURL 
We can also use a Webservice using CURL. 
For example: 

curl --request POST --header “Content-type: text/xml 
--data @my_request.xml http://api.google.com/search/beta2 

Perl - SOAPlite 
With Perl and SOAP::lite modules we can create a scripts to automatize a SOAP request. 

'''SOAP XML File'''
 
To invoke web services from command line, we can create a SOAP request file
similar to the following one, and then use CURL to submit it to server. 

<pre>
<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">

<SOAP-ENV:Body>

<m:GetZip xmlns:m="http://namespaces.example.com">
<country>Italy</country>
<city>Roma</city>
</m:GetZip>

</SOAP-ENV:Body>

</SOAP-ENV:Envelope>
</pre>

 
Creating a malformed XML file we can test a Webservices for a typical attack as the following: 
-oversized XML Tag 
-nested or recursive declarations 
-parameter attack 
-authentication testing 
-XSS 
-SQL Injection 

==References==
* DISCO: http://msdn.microsoft.com/en-us/magazine/cc302073.aspx
* UDDI OASIS Standard: http://www.oasis-open.org/specs/index.php#uddiv3.0.2
* Understanding UDDI: http://www-128.ibm.com/developerworks/webservices/library/ws-featuddi/index.html
* WebServices Testing: http://www.aboutsecurity.net
 
'''Tools''' 
* Net Square wsPawn
* [[OWASP_WebScarab_Project|OWASP WebScarab]]: Web Services plugin
* Mac OSX Soap Client: http://www.ditchnet.org/soapclient
* Foundstone WsDigger: http://www.foundstone.com/us/resources/proddesc/wsdigger.htm
* Soaplite: http://www.soaplite.com
* Perl: http://www.perl.com
* SOAPClient4XG: http://www-128.ibm.com/developerworks/xml/library/x-soapcl/
* CURL: http://curl.haxx.se
 
'''On-line tools''' 
* Web Services Directory: http://www.wsindex.org
* Seekda: http://seekda.com/
* UDDI Browser: http://www.soapclient.com/
* Xmethods: http://www.xmethods.net
* WSIndex: http://www.wsindex.org

Testing: WS Information Gathering (OWASP-WS-001)

2013-07-04T09:15:25Z

Aditya Balapure:

{{Template:OWASP Testing Guide v3}}

==Brief Summary==
The first step to perform a Web Service Test is to determine the WS entry points and the communication schema: this is described in the WSDL associated with the Web Service.

==Black Box Testing and example==
'''Zero Knowledge''' 
Normally you will have a WSDL path to access the Web Service, but if you have zero knowledge about it, you will have to use UDDI to find a specific service.
Web Services have three critical building blocks – UDDI, WSDL and SOAP. There is a third intermediate player facilitating communication between the consumer and supplier, referred to as Universal Business Registry (UBR).
There are several ways to find our WSDL: the easiest one is to make a search Query in public search engine. For example, if you have to assess an example.com public WS, on google.com you can type:

inurl:wsdl site:example.com

and you will find all the public Example WSDL.
Net Square wsPawn is a useful tool that acts as Web Services Consumer and makes a query to the UBR and looks for services as per requirements. Then UBR supplies the list of available services. The Web Services Consumer chooses one or more available services. Next, Web Services Consumer requests an access point or end point for these services. UBR supplies this information. From this moment, Web Services Consumer approaches the Web Services Supplier’s Host/IP address (WSDL) and starts accessing service. 

'''WSDL endpoints''' 
When a tester accesses the WSDL, he can determine an access point and available interfaces for web services. These interfaces or methods take inputs using SOAP over HTTP/HTTPS. If these inputs are not defined well at the source code level, they can be compromised and exploited.
For example given this WSDL Endpoint:

http://www.example.com/ws/FindIP.asmx?WSDL

you can obtain the following description of the Web Services:
<pre>
<?xml version="1.0" encoding="utf-8"?>
<wsdl:definitions xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:s="http://www.w3.org/2001/XMLSchema" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:tns="http://example.com/webservices/" xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" targetNamespace="http://example.com/webservices/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/">
<wsdl:types>
<s:schema elementFormDefault="qualified" targetNamespace="http://example.com/webservices/">
<s:element name="GetURLIP">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1" name="EnterURL" type="s:string" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="GetURLIPResponse">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1" name="GetURLIPResult" type="s:string" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="string" nillable="true" type="s:string" />
</s:schema>
</wsdl:types>
<wsdl:message name="GetURLIPSoapIn">
<wsdl:part name="parameters" element="tns:GetURLIP" />
</wsdl:message>
<wsdl:message name="GetURLIPSoapOut">
<wsdl:part name="parameters" element="tns:GetURLIPResponse" />
</wsdl:message>
<wsdl:message name="GetURLIPHttpGetIn">
<wsdl:part name="EnterURL" type="s:string" />
……
</wsdl:service>
</wsdl:definitions>
</pre>
This WS simply receives in input a logical name (EnterURL) and gives in output the relative IP Address. So we have GetURLIP as method for the WS and EnterURL (string) as input.
In that manner we have identified the WS entry point and we are ready to test it.
 
 
'''Web Services Discovery''' 
Web Services consumer needs a simple and standardized way to find Web Services available from remote servers.
There are two ways to discover a Web Service, DISCO and UDDI. 
The Web Service Discovery (DISCO) is one way that we can discover the URL's WSDL descriptor and other XML documents, like Schema Definition Document (.xsd). 

For instance, with an HTTP query to a Web server:
http://myexample.com/myexampleService.asmx?DISCO

we obtain the following DISCO descriptor:
 
<pre>
<?xml version="1.0" encoding="utf-8"?>
<discovery xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.xmlsoap.org/disco/">
<contractRef ref="http://myexample.com/MyexampleService.asmx?wsdl" docRef="http://myexample.com/myexample.asmx" xmlns="http://schemas.xmlsoap.org/disco/scl/" />
<soap address="http://myexample.com/MyexampleService.asmx" xmlns:q1="http://myexample.com/terraserver/" binding="q1:myexampleServiceSoap" xmlns="http://schemas.xmlsoap.org/disco/soap/" />
</discovery>
</pre>
 
We can see in the above XML document we have a reference for a WSDL document, where we can obtain a description of Web Services available from remote Web Server.

DISCO is a Microsoft technology, UDDI (Universal Description, Discovery and Integration) is an OASIS standard .
 
'''WS Well Known Naming''' 
Common Web Services platforms have a naming convention for offering a WSDL documents: This naming convention can be used to retrieve WSDL via URIs probing or through queries to web search server.

Some URLs that we can use are, for example:

http://<webservice-host>:<port>/<servicename>
http://<webservice-host>:<port>/<servicename>.wsdl
http://<webservice-host>:<port>/<servicename>?wsdl
http://<webservice-host>:<port>/<servicename>.aspx?wsdl
 instead of .aspx extension we can also use .ascx, .asmx, .ashx extensions
 Same thing with ?disco instead of ?wsdl

http://<webservice-host>:<port>/<servicename.dll>?wsdl
http://<webservice-host>:<port>/<servicename.exe>?wsdl
http://<webservice-host>:<port>/<servicename.php>?wsdl
http://<webservice-host>:<port>/<servicename.pl>?wsdl

For Apache Axis we can try:
http://<webservice-host>:<port>/axis/services/<servicename>?wsdl
http://<webservice-host>:<port>/axis/services/<service-name>
 
 

'''Search for public Web Services''' 
The seekda Web Services Search Engine can help to find a public Web Services with related descriptions.
To find Web Services just type the keyword into seekda Web Services Search Engine. We can also browse by several other criteria such as Tag Cloud, Services by Countries, Most Used Services.
http://seekda.com
 

[[Image:seekda.jpg]]

 
Another Web Server with good links and Resources is WSindex (http://www.wsindex.org).

[[Image:wsindex.png]]
 
 
''' UDDI Browser''' 
A web server that provides a very useful UDDI on-line tool to browse and search public UDDI resources is offered on http://www.soapclient.com. 
We can see we can use two operators, Microsoft and XMethods

[[Image:uddi_browser_part.jpg]]
 

The server offers, for example, to search all UDDI with a specific string in business names, service names or service types. 

'''Advanced UDDI browsing''' 

We can search private UDDI registries using the Advanced feature of UDDI browser.
 
[[Image:uddi_browser.jpg]]
 
 
This service allows interaction with Web services dynamically. 
Soapclient offer others methods to allow you to discover web services and useful links to other resources.

 
 

'''Command line interaction'''
 
Sometimes it is useful to interact with webservices from a command line.
 
 
Simple SOAP Client - SOAPClient4XG 
SOAPClient4XG is a SOAP Client for XML which allows you to make a SOAP request from command line, for example:

java -jar SOAPClient4XG http://api.google.com/search/beta2 my_sample_search.xml 
 

CURL 
We can also use a Webservice using CURL. 
For example: 

curl --request POST --header “Content-type: text/xml 
--data @my_request.xml http://api.google.com/search/beta2 

Perl - SOAPlite 
With Perl and SOAP::lite modules we can create a scripts to automatize a SOAP request. 

'''SOAP XML File'''
 
To invoke web services from command line, we can create a SOAP request file
similar to the following one, and then use CURL to submit it to server. 

<pre>
<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">

<SOAP-ENV:Body>

<m:GetZip xmlns:m="http://namespaces.example.com">
<country>Italy</country>
<city>Roma</city>
</m:GetZip>

</SOAP-ENV:Body>

</SOAP-ENV:Envelope>
</pre>

 
Creating a malformed XML file we can test a Webservices for a typical attack as the following: 
-oversized XML Tag 
-nested or recursive declarations 
-parameter attack 
-authentication testing 
-XSS 
-SQL Injection 

==References==
* DISCO: http://msdn.microsoft.com/en-us/magazine/cc302073.aspx
* UDDI OASIS Standard: http://www.oasis-open.org/specs/index.php#uddiv3.0.2
* Understanding UDDI: http://www-128.ibm.com/developerworks/webservices/library/ws-featuddi/index.html
* WebServices Testing: http://www.aboutsecurity.net
 
'''Tools''' 
* Net Square wsPawn
* [[OWASP_WebScarab_Project|OWASP WebScarab]]: Web Services plugin
* Mac OSX Soap Client: http://www.ditchnet.org/soapclient
* Foundstone WsDigger: http://www.foundstone.com/us/resources/proddesc/wsdigger.htm
* Soaplite: http://www.soaplite.com
* Perl: http://www.perl.com
* SOAPClient4XG: http://www-128.ibm.com/developerworks/xml/library/x-soapcl/
* CURL: http://curl.haxx.se
 
'''On-line tools''' 
* Web Services Directory: http://www.wsindex.org
* Seekda: http://seekda.com/
* UDDI Browser: http://www.soapclient.com/
* Xmethods: http://www.xmethods.net
* WSIndex: http://www.wsindex.org

User:Aditya Balapure

2013-07-04T07:45:25Z

Aditya Balapure:

Here is a link to my public profile for all the info-
in.linkedin.com/in/adityabalapure/

http://packetstormsecurity.com/files/author/10171/

Testing: WS Information Gathering (OWASP-WS-001)

2013-07-04T07:43:04Z

Aditya Balapure:

{{Template:OWASP Testing Guide v3}}

==Brief Summary==
The first step to perform a Web Service Test is to determine the WS entry points and the communication schema: this is described in the WSDL associated with the Web Service.

==Black Box Testing and example==
'''Zero Knowledge''' 
Normally you will have a WSDL path to access the Web Service, but if you have zero knowledge about it, you will have to use UDDI to find a specific service.
Web Services have three critical building blocks – UDDI, WSDL and SOAP. There is a third intermediate player facilitating communication between the consumer and supplier, referred to as Universal Business Registry (UBR).
There are several ways to find our WSDL: the easiest one is to make a search Query in public search engine. For example, if you have to assess an example.com public WS, on google.com you can type:

inurl:wsdl site:example.com

and you will find all the public Example WSDL.
Net Square wsPawn is a useful tool that acts as Web Services Consumer and makes a query to the UBR and looks for services as per requirements. Then UBR supplies the list of available services. The Web Services Consumer chooses one or more available services. Next, Web Services Consumer requests an access point or end point for these services. UBR supplies this information. From this moment, Web Services Consumer approaches the Web Services Supplier’s Host/IP address (WSDL) and starts accessing service. 

'''WSDL endpoints''' 
When a tester accesses the WSDL, he can determine an access point and available interfaces for web services. These interfaces or methods take inputs using SOAP over HTTP/HTTPS. If these inputs are not defined well at the source code level, they can be compromised and exploited.
For example given this WSDL Endpoint:

http://www.example.com/ws/FindIP.asmx?WSDL

you can obtain the following description of the Web Services:
<pre>
<?xml version="1.0" encoding="utf-8"?>
<wsdl:definitions xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:s="http://www.w3.org/2001/XMLSchema" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:tns="http://example.com/webservices/" xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" targetNamespace="http://example.com/webservices/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/">
<wsdl:types>
<s:schema elementFormDefault="qualified" targetNamespace="http://example.com/webservices/">
<s:element name="GetURLIP">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1" name="EnterURL" type="s:string" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="GetURLIPResponse">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1" name="GetURLIPResult" type="s:string" />
</s:sequence>
</s:complexType>
</s:element>
<s:element name="string" nillable="true" type="s:string" />
</s:schema>
</wsdl:types>
<wsdl:message name="GetURLIPSoapIn">
<wsdl:part name="parameters" element="tns:GetURLIP" />
</wsdl:message>
<wsdl:message name="GetURLIPSoapOut">
<wsdl:part name="parameters" element="tns:GetURLIPResponse" />
</wsdl:message>
<wsdl:message name="GetURLIPHttpGetIn">
<wsdl:part name="EnterURL" type="s:string" />
……
</wsdl:service>
</wsdl:definitions>
</pre>
This WS simply receives in input a logical name (EnterURL) and gives in output the relative IP Address. So we have GetURLIP as method for the WS and EnterURL (string) as input.
In that manner we have identified the WS entry point and we are ready to test it.
 
 
'''Web Services Discovery''' 
Web Services consumer needs a simple and standardized way to find Web Services available from remote servers.
There are two ways to discover a Web Service, DISCO and UDDI. 
The Web Service Discovery (DISCO) is one way that we can discover the URL's WSDL descriptor and other XML documents, like Schema Definition Document (.xsd). 

For instance, with an HTTP query to a Web server:
http://myexample.com/myexampleService.asmx?DISCO

we obtain the following DISCO descriptor:
 
<pre>
<?xml version="1.0" encoding="utf-8"?>
<discovery xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.xmlsoap.org/disco/">
<contractRef ref="http://myexample.com/MyexampleService.asmx?wsdl" docRef="http://myexample.com/myexample.asmx" xmlns="http://schemas.xmlsoap.org/disco/scl/" />
<soap address="http://myexample.com/MyexampleService.asmx" xmlns:q1="http://myexample.com/terraserver/" binding="q1:myexampleServiceSoap" xmlns="http://schemas.xmlsoap.org/disco/soap/" />
</discovery>
</pre>
 
We can see in the above XML document we have a reference for a WSDL document, where we can obtain a description of Web Services available from remote Web Server.

DISCO is a Microsoft technology, UDDI (Universal Description, Discovery and Integration) is an OASIS standard .
 
'''WS Well Known Naming''' 
Common Web Services platforms have a naming convention for offering a WSDL documents: This naming convention can be used to retrieve WSDL via URIs probing or through queries to web search server.

Some URLs that we can use are, for example:

http://<webservice-host>:<port>/<servicename>
http://<webservice-host>:<port>/<servicename>.wsdl
http://<webservice-host>:<port>/<servicename>?wsdl
http://<webservice-host>:<port>/<servicename>.aspx?wsdl
 instead of .aspx extension we can also use .ascx, .asmx, .ashx extensions
 Same thing with ?disco instead of ?wsdl

http://<webservice-host>:<port>/<servicename.dll>?wsdl
http://<webservice-host>:<port>/<servicename.exe>?wsdl
http://<webservice-host>:<port>/<servicename.php>?wsdl
http://<webservice-host>:<port>/<servicename.pl>?wsdl

For Apache Axis we can try:
http://<webservice-host>:<port>/axis/services/<servicename>?wsdl
http://<webservice-host>:<port>/axis/services/<service-name>
 
 

'''Search for public Web Services''' 
The seekda Web Services Search Engine can help to find a public Web Services with related descriptions.
To find Web Services just type the keyword into seekda Web Services Search Engine. We can also browse by several other criteria such as Tag Cloud, Services by Countries, Most Used Services.
http://seekda.com
 

[[Image:seekda.jpg]]

 
Another Web Server with good links and Resources is WSindex (http://www.wsindex.org).

[[Image:wsindex.png]]
 
 
''' UDDI Browser''' 
A web server that provides a very useful UDDI on-line tool for to browse and search public UDDI resource in offered from http://www.soapclient.com. 
We can see we can use two operators, Microsoft and XMethods

[[Image:uddi_browser_part.jpg]]
 

The server offers, for example, to search all UDDI with a specific string in business names, service names or service types. 

'''Advanced UDDI browsing''' 

We can search private UDDI registries using the Advanced feature of UDDI browser.
 
[[Image:uddi_browser.jpg]]
 
 
This service allows interaction with Web services dynamically. 
Soapclient offer others methods to allow you to discover web services and useful links to other resources.

 
 

'''Command line interaction'''
 
Sometimes it is useful to interact with webservices from a command line.
 
 
Simple SOAP Client - SOAPClient4XG 
SOAPClient4XG is a SOAP Client for XML which allows you to make a SOAP request from command line, for example:

java -jar SOAPClient4XG http://api.google.com/search/beta2 my_sample_search.xml 
 

CURL 
We can also use a Webservice using CURL. 
For example: 

curl --request POST --header “Content-type: text/xml 
--data @my_request.xml http://api.google.com/search/beta2 

Perl - SOAPlite 
With Perl and SOAP::lite modules we can create a scripts to automatize a SOAP request. 

'''SOAP XML File'''
 
To invoke web services from command line, we can create a SOAP request file
similar to the following one, and then use CURL to submit it to server. 

<pre>
<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">

<SOAP-ENV:Body>

<m:GetZip xmlns:m="http://namespaces.example.com">
<country>Italy</country>
<city>Roma</city>
</m:GetZip>

</SOAP-ENV:Body>

</SOAP-ENV:Envelope>
</pre>

 
Creating a malformed XML file we can test a Webservices for a typical attack as the following: 
-oversized XML Tag 
-nested or recursive declarations 
-parameter attack 
-authentication testing 
-XSS 
-SQL Injection 

==References==
* DISCO: http://msdn.microsoft.com/en-us/magazine/cc302073.aspx
* UDDI OASIS Standard: http://www.oasis-open.org/specs/index.php#uddiv3.0.2
* Understanding UDDI: http://www-128.ibm.com/developerworks/webservices/library/ws-featuddi/index.html
* WebServices Testing: http://www.aboutsecurity.net
 
'''Tools''' 
* Net Square wsPawn
* [[OWASP_WebScarab_Project|OWASP WebScarab]]: Web Services plugin
* Mac OSX Soap Client: http://www.ditchnet.org/soapclient
* Foundstone WsDigger: http://www.foundstone.com/us/resources/proddesc/wsdigger.htm
* Soaplite: http://www.soaplite.com
* Perl: http://www.perl.com
* SOAPClient4XG: http://www-128.ibm.com/developerworks/xml/library/x-soapcl/
* CURL: http://curl.haxx.se
 
'''On-line tools''' 
* Web Services Directory: http://www.wsindex.org
* Seekda: http://seekda.com/
* UDDI Browser: http://www.soapclient.com/
* Xmethods: http://www.xmethods.net
* WSIndex: http://www.wsindex.org