Difference between revisions of "Section 4: Mitigating the WebGoat lessons"

Revision as of 08:32, 21 October 2008

Project metrics at 50% project completion

See Section 2 for the WebGoat lesson Table of Contents and an overview of the results from doing the WebGoat lessons. Appendix A contains a zip file which is made up of the lesson plans and solutions - in HTML format - which were taken from WebGoat and can be viewed stand-alone.

Out of 51 possible lessons, the following are teaching lessons, not vulnerabilities, and therefore have no context for ModSecurity rules:

• 1.1 Http Basics
• 4.1 Password Strength
• 15.3 Bypass Client Side JavaScript Validation
• 17.1 Create a SOAP Request

Therefore there is a total number of 47 lessons to do; half is 24 so that was the goal of the first 50% of project completion. The lowest hanging fruit was taken first because considerable effort was put into: (1) setup and configuration of the environment; (2) getting familiar with WebGoat and taking all of the lessons; (3) learning ModSecurity (and Remo); (4) re-learning regular expressions; (5) learning Lua script; and (6) developing an efficient work methodology.

The total number of sublessons mitigated by ModSecurity rules: 25 - thereby achieving the goal of at least 50% of sublessons mitigated.

They are:

• Sublesson 1.2
• Sublesson 2.4
• Sublessons 4.2, 4.4, 4.5
• Sublesson 6.1
• Sublessons 8.1, 8.2, 8.4, 8.5, 8.7
• Sublesson 10.1
• Sublessons 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8
• Sublesson 13.1
• Sublessons 15.1, 15.2
• Sublessons 17.3, 17.4

Project metrics at 100% project completion

Based on Reviewer feedback, three (3) of the 4 sublessons that were originally assessed as not being able to be a ModSecurity solution, actually can be. They are:

• 1.1 Http Basics
• 15.3 Bypass Client Side JavaScript Validation
• 17.1 Create a SOAP Request

Fifteen (15) sublessons were solved in the 2nd phase which makes a total of 40 sublessons solved.

Five (5) sublessons do not count because ModSecurity solutions cannot be written for them (further explanation below):

• 3. AJAX Security -> 3.3 Same Origin Policy Protection
• 5. Buffer Overflows (not implemented)
• 4. Authentication Flaws -> 4.1 Password Strength
• 8. Cross-Site Scripting (XSS) -> 8.6 HTTPOnly Test
• 14. Insecure Storage -> 14.1 Encoding Basics

The remaining 7 sublessons are either unable to be solved or can be solved but were complex solutions and weren’t completed because of lack of time. The next section gives further explanation for each sublesson that was not solved.

Therefore, 40 out of 47 WebGoat sublessons were solved, or 85%. The original goal was 90%, or 42 lessons.

See ‘Section 5, The Mitigating Solutions’ for details on each of the ModSecurity solutions new in Phase 2. In chronological order, these 15 sublessons are:

• Sublesson 12. Insecure Communication -> 12.1 Insecure Login
• Sublesson 7. Concurrency -> 7.1 Thread Safety Problem
• Sublesson 7. Concurrency -> 7.2 Shopping Cart Concurrency Flaw
• Sublesson 2. Access Control Flaws -> 2.2 Bypass a Path Based Access Control Scheme
• Sublesson 8. Cross-Site Scripting (XSS) -> 8.3 Stored XSS Attacks
• Sublesson 9. Denial of Service -> 9.1 Denial of Service from Multiple Logins
• Sublesson 3. AJAX Security -> 3.4 DOM Injection
• Sublesson 3. AJAX Security -> 3.5 XML Injection
• Sublesson 3. AJAX Security -> 3.7 Silent Transactions Attacks
• Sublesson 3. AJAX Security -> 3.1 LAB: DOM-Based cross-site scripting
• Sublesson 3. AJAX Security -> 3.6 JSON Injection
• Sublesson 3. AJAX Security -> 3.8 Dangerous Use of Eval
• Sublesson 1. Introduction -> 1.1 Http Basics
• Sublesson 15. Parameter Tampering -> 15.3 Bypass Client Side JavaScript Validation
• Sublesson 17. Web Services -> 17.1 Create a SOAP Request

Note that as the project progressed, solving ModSecurity solutions over and over resulted in standard formats and cleaner output for the solution write-ups and their artifacts (ModSecurity *.conf files, Lua scripts, HTML error files, etc.). For example, to get an example of a Lua script being used, find one later in the project because it will be more refined and clearer to understand.

Sublessons that do not count or were not solved (and why)

Sublessons that do not count:

3. AJAX Security -> 3.3 Same Origin Policy Protection

This WebGoat lesson demonstrates a browser's same origin policy protection and as such has no vulnerability with a ModSecurity solution or any other way to illustrate similar functionality within ModSecurity. Therefore, this sublesson is not part of the total count.

5. Buffer Overflows (not implemented by WebGoat)

4. Authentication Flaws -> 4.1 Password Strength

This WebGoat lesson calls a 3rd party website and shows password strength (time to crack a user-supplied password that matches the criteria); as such it has no vulnerability with a ModSecurity solution or any other way to illustrate similar functionality within ModSecurity. Therefore, this sublesson is not part of the total count.

8. Cross-Site Scripting (XSS) -> 8.6 HTTPOnly Test

This WebGoat lesson is a demonstration of the HttpOnly flag which prevents client side script from reading a cookie value if it is set; "Read Cookie" and "Write Cookie" buttons can be selected to view that particular browser's support of the HttpOnly flag.

There really is no ModSecurity solution or any other way to illustrate similar functionality within ModSecurity, especially since this takes place in the browser and ModSecurity cannot modify source code in an HTTP response.

14. Insecure Storage -> 14.1 Encoding Basics

This WebGoat lesson demonstrates encoding and decoding (e.g. Base64, md5, entity, unicode); as such it has no vulnerability with a ModSecurity solution or any other way to illustrate similar functionality within ModSecurity. Therefore, this sublesson is not part of the total count.

Note: ModSecurity has many built-in encoding/decoding transformation functions - it supports each encoding/decoding function in this lesson demonstration, and more.

Sublessons that were not solved:

2. Access Control Flaws -> 2.3 LAB: Role Based Access Control

In this WebGoat lab, each role has permission to certain resources (A-F). Each user is assigned one or more roles. Only the user with the 'Admin' role should have access to the 'F' resources but there is a vulnerability where the user doesn't have the 'Admin' role can access resource; this is exploited by intercepting a request in a web proxy and altering the Employee ID number.

The WebGoat solution is to modify the back end source code and add an 'isAuthorized' method.

A ModSecurity solution would be to Lua-persist the Role Based Access Control configuration to 2 *.data files.

One would be a list of roles with the resources that they have access to, something like this:

Entry{ 
  role = "Admin",
  resources = "A,B,D,F"
} 

Entry{ 
  role = "User",
  resources = "A,B,C"
}
...

The second *.data file would contain each user and their roles:

Entry{
  employee_id = 101,
  roles = "User, Manager"
}
...

ModSecurity would intercept the request and call a Lua script which would determine if the request to the resource was valid or not.

In the real world, a web interface would serve as a front end to the *.data files.

3. AJAX Security -> 3.2 LAB: Client Side Filtering

In this WebGoat lab, an XPath query returns name, salary, and other information about users in XML in a hidden table. The vulnerability is that user information is sent for users not within the scope of the requesting user and client side filtering is relied upon to show only the necessary users.

The WebGoat solution is to modify the source code and return only the users and their information that is allowed according to the user requesting the information.

Since ModSecurity cannot alter an HTTP response, there seems to be no ModSecurity solution to this vulnerability.

3. AJAX Security -> 3.9 Insecure Client Storage

This WebGoat lesson consists of 2 stages:

Stage 1: A decrypted coupon is used to get a discount but the decryption is done in a Javascript routine; stepping through it with FireBug or IEWatch will give away the decrypted coupon to the user who can then use it to get a discount. ModSecurity cannot provide a solution for this vulnerability; the decrypt function would have to be removed from the source code in order to begin a solution, but ModSecurity cannot alter HTTP response source code.

Stage 2: A shopping cart is displayed: the quantity of each item is editable, but the unit price, the total price of that item, and the grand total is not editable. The exploit is to remove the HttpOnly attribute from the grand total field, then change the price to 0 and make the purchase. In real life, this is not very practical because the back end would calculate the grand total based on unit price and the number of each item.

ModSecurity could provide a solution if necessary: intercept the POST request, which looks something like this (lines split for readability): PRC1=69.99&QTY1=4&TOT1=279.95&PRC2=27.99&QTY2=0&TOT2=0&PRC3=1599.99&QTY3=2& TOT3=3199.98&PRC4=299.99&QTY4=0&TOT4=0&SUBTOT=3479.93&GRANDTOT=3479.93& field2=4128+3214+0002+1999&field1=&SUBMIT=Purchase

Then call a Lua script that pulls all of the relevant POST parameters, tallies up from the individual item total what the grand total should be, and then compares it with the grand total that was submitted.

16. Session Management Flaws -> 16.1 Hijack a Session

This WebGoat lesson teaches how to hijack another user's session using brute force attacks. The vulnerability is that the session ID is not complex and random enough; the exploit is complex and is done by using WebScarab and another tool called Crowbar.

Session Management Flaws are one of the few areas where a WAF will not be of much help.

The only action that can be taken is a reactive one instead of a preventative action: issue an alarm in the event of irregularities (e.g. a changing IP) or terminate a session with changing IP. Besides that, a ModSecurity solution is not possible for this lesson.

16. Session Management Flaws -> 16.2 Spoof an Authentication Cookie

This WebGoat lesson shows how to guess another user's authentication cookie. The vulnerability is that the cookie is easy to guess even from only 2 unique user accounts and their authentication cookie; it is exploited by logging in as another user and supplying the predictable authentication cookie.

A ModSecurity solution is not possible for this lesson because it is not possible to distinguish between an authentication cookie that comes from a valid user versus one that comes from an attacker.

16. Session Management Flaws -> 16.3 Session Fixation

This WebGoat session demonstrates a session fixation attack. An attacker chooses a Session ID, sends it in a link to the victim, the victim clicks the link and logs in, then the attacker uses that session ID to log in to the same banking web site and thereafter masquerades as the victim.

A ModSecurity solution for this specific lesson would be to attach the user name to each unique session ID and persist it using Lua. Afterwards, if another user tried to log in using the same session ID, then the attacker would not be allowed to log in to the site.

17. Web Services -> 17.2 WSDL Scanning

This WebGoat lesson demonstrates WSDL Scanning to get credit card information. The vulnerability is that the attacker can call a Web Service directly; in this case, it is 'getCreditCard'. The attacker retrieves the WSDL, observes the parameters of a normal request 'id=101&SUBMIT=Submit', then intercepts and manipulates the parameters to call the function directly: id=101&field=getCreditCard&SUBMIT=Submit.

A ModSecurity solution is straightforward: don't allow a user to directly call a specific set of Web Services - in our case 'getCreditCard' - by blocking on the request if that value for the 'field' parameter appears.

Overall strategy

The intention of mitigating the vulnerabilities is to demonstrate the largest variety of mitigating solutions and features of ModSecurity as possible:

• Some lessons are solved using the easiest way possible for convenience (and to count towards achieving the goal of 50% complete!)
• Some lessons are solved by using rules from the core rulesets provided by Breach Security
• Some mitigating solutions are meant to be global, meaning being in effect at all times, like the XSS and Command Injection core rules from Breach Security
• One lesson demonstrates the use of a session variable
• Some lessons require persistence beyond what is offered by ModSecurity; Lua scripts are used to achieve this
• Some lessons require a more robust capability than ModSecurity's regular expressions, 'and/or' logical mechanisms, and goto statements (skip and skipAfter); again, Lua scripts are used to achieve this.
• One lesson uses the 'append' action to append Javascript to the end of a response body, which alters the content and behavior of the HTML page

The rulesets can be used all together or, for a specific WebGoat sublesson, the initialization file (rulefile_00_initialize.conf) plus that sublesson's ruleset can be used.

The best way to open the discussion about the overall strategy used is to show a chunk of the initialization file:

<LocationMatch "^/WebGoat/attack$">
  # Group 1: the following block pertain to pages that don't have 
  #   Screen or menu parameters
1.  SecRule &ARGS:Screen "!@eq 0" chain,skipAfter:200
2.  SecRule &ARGS:menu "!@eq 0" "t:none"

  # Group 2: set session collection if entering WebGoat; POST body parameter 
  #   is "start=Start WebGoat" (Start WebGoat submit button)
3.  SecRule &ARGS_POST:start "@eq 0" "nolog,skip:3"
4.  SecRule ARGS_POST:start "!@streq Start WebGoat" \ 
      "t:urlDecodeUni,t:htmlEntityDecode,skip:2"
5.  SecRule REQUEST_COOKIES:JSESSIONID "!^$" \ 
      "chain,log,auditlog,pass,msg:'Setting session collection'"
6.  SecAction setsid:%{REQUEST_COOKIES.JSESSIONID} 
7.  SecAction "log,setvar:session.lesson13=0,msg:'setting session.lesson13=0 \  
      initially after setsid from rulefile_00-0-initialize.conf'"

8.  SecAction "t:none,allow,id:'200'"

  # Group 3: here there should be a 'menu' parameter, so set a variable for          
  #   the menu number that's used if needed in Phase 4
9.  SecRule ARGS_GET:menu "^(.*)$" "pass,setvar:tx.menu=%{MATCHED_VAR}"

In Group 1, the two chained rules denote the major section of WebGoat after the login and start pages. The 'Screen' parameter is arbitrary, while the 'menu' parameter is the lesson key. Basically, if this condition is met, skip past Group 2.

In Group 2, the next 2 rules pertain to the start page, denoted by the 'start=Start WebGoat' POST body parameter. Nothing of importance occurs on this page so we skip to the 'allow' action on line 8.

Line 5 and 6 denote that a successful login has occurred, so the session ID is saved in a session collection for later use throughout the remaining lessons. Line 7 sets a session variable (a toggle switch) for use in 'Lesson 13: Insecure Configuration'.

At Line 9, we have to save the 'menu' value for use in Phase 4.

At this point, we know the 'menu' value: it exists throughout phase 2 as is, and we have saved the value in a variable for Phase 4.

For the rest of the ruleset files, we will filter on the 'menu' parameter and each lesson will have its own ruleset file.

For example, ruleset file 'rulefile_04_authentication-flaws.conf' has 2 sections:

'SecRule ARGS:menu "!@eq 500" "phase:2,t:none,skip:2"' says "if we aren't in Lesson 4 in Phase 2, then don't do any further processing here".

Similarly, 'SecRule TX:MENU "!@eq 500" "phase:4,t:none,pass,skip:1"' says "if we aren't in Lesson 4 in Phase 2, then don't do any further processing here".

Reviewer comments

For many lesson solutions, a project reviewer has made comments regarding the project's choice of a solution (or lack of one), and often also has given a "classic" ModSecurity solution to those lessons solved by using Lua script.

Reviewer comments are added at the end of each individual lesson solution write-up in its own section; to date, the following sublessons have been updated with reviewer comments:

Using the Lua scripting language

In some situations, ModSecurity's capability is not robust and flexible enough to prevent complex exploits. Enter Lua script, a tried-and-true programming language used mainly in the gaming industry but also used in other areas such as the MySQL Proxy project.

Some advantages are:

1. Enables ModSecurity to address business logic flaws
2. Enhances the capability of using ModSecurity as an egress filter
3. Lua persistence: Super-persistence beyond the existing Session scope and capability (the SecDataDir directory is used for storage)
4. By being able to use a programming language, it is easier to do whitelisting and implement a positive security model.
5. Flexibility for virtual patching (as in, mitigating vulnerabilities discovered in a penetration test and NOT as in porting Snort or scanner signatures).
6. Ease-of-use: The Lua scripts can be developed and tested offline as standalone programs before being used by ModSecurity.
7. A programming language that is built to be rugged; e.g. handling multi-megabyte memory buffers and files.

The Implementation section of each sublesson explains in detail how Lua script is used.

In Phase 1, Lua scripts are used in sublessons:

• 4.2: Forgot Password (OWASP_ModSecurity_Securing_WebGoat_Section4_Sublesson_04.2)
• 4.4: Multi Level Login 1, 4.5: Multi Level Login 2 (OWASP_ModSecurity_Securing_WebGoat_Section4_Sublesson_04.4_04.5)
• 8.2 LAB: Cross Site Scripting (OWASP_ModSecurity_Securing_WebGoat_Section4_Sublesson_08.2_08.4_08.5_08.7)
• 15.1 Exploit Hidden Fields, 15.2 Exploit Unchecked Email (OWASP_ModSecurity_Securing_WebGoat_Section4_Sublesson_15.1_15.2)

In Phase 1, Lua scripts are used in sublessons:

• 3. AJAX Security -> 3.4 DOM Injection

• 3. AJAX Security -> 3.5 XML Injection

• 3. AJAX Security -> 3.6 JSON Injection

• 7. Concurrency -> 7.1 Thread Safety Problem

Shows the process of developing a standalone Lua script on Windows, then integrating it with ModSecurity on Linux

• 7. Concurrency -> 7.2 Shopping Cart Concurrency Flaw

Shows how Lua can be used to handle application state; in this case, a purchase that utilizes a shopping cart.

• 8. Cross-Site Scripting (XSS) -> 8.3 Stored XSS Attacks

Shows rudimentary output sanitization using Lua.

• 9. Denial of Service -> 9.1 Denial of Service from Multiple Logins

• 12. Insecure Communication -> 12.1 Insecure Login

Shows: (1) using Javascript injection that adds an MD5 library on the front end in order to hash a password before being sent; (2) rebuilding the Lua engine and including a 3rd party MD5 library; and (3) hashing the expected password within a Lua script and comparing it with the client side password.

Using Javascript 'prepend' and 'append'

The Javascript 'prepend' and 'append' actions and their applications are shown in sublessons:

• 3. AJAX Security -> 3.1 LAB: DOM-Based cross-site scripting
• 3. AJAX Security -> 3.7 Silent Transactions Attacks
• 12. Insecure Communication -> 12.1 Insecure Login

Structure of mitigating a lesson

The following outlines the overall structure of each lesson/sublesson (they are used interchangeably throughout this project) to be used in the next section that contains the details of each mitigating solution:

Lesson overview: The path and file name of the lesson plan as taken from WebGoat. See Appendix A for more information.

Lesson solution: The path and file name of the lesson solution as taken from WebGoat. See Appendix A for more information.

Strategy (including challenges): What approach was taken to solve the lesson and why; what were the challenges? (if any)

Implementation: Details how the sublesson was mitigated.

Note that the project's solution files that are formatted using a Linux editor with a 2-space tab. For viewing the files from Windows, it's preferable to use Wordpad instead of Notepad because Notepad might add CR/LF combinations. Still, using Notepad and the files in DOS format on Linux should pose no problems but using Wordpad is recommended.

Comment(s): Any comments on the solution.

The mitigating solutions

Following are links to the mitigating solution pages. Some may link to the same page if their sublesson is solved by the same rule or by a group of rules in the same *.conf file.

1. Sublesson 1.1: HTTP Basics
2. Sublesson 1.2: HTTP Splitting
3. Sublesson 2.2: Bypass a Path Based Access Control Scheme
4. Sublesson 2.4: Remote Admin Access
5. Sublesson 3.1: LAB: DOM-Based cross-site scripting
6. Sublesson 3.4: DOM Injection
7. Sublesson 3.5: XML Injection
8. Sublesson 3.6: JSON Injection
9. Sublesson 3.7: Silent Transactions Attacks
10. Sublesson 3.8: Dangerous Use of Eval
11. Sublesson 4.2: Forgot Password
12. Sublesson 4.4: Multi Level Login 1
13. Sublesson 4.5: Multi Level Login 2
14. Sublesson 6.1: Discover Clues in the HTML
15. Sublesson 7.1: Thread Safety Problem
16. Sublesson 7.2: Shopping Cart Concurrency Flaw
17. Sublesson 8.3: Stored XSS Attacks
18. Sublesson 8.1: Phishing with XSS
19. Sublesson 8.2: LAB: Cross Site Scripting
20. Sublesson 8.4: Reflected XSS Attacks
21. Sublesson 8.5: Cross Site Request Forgery (CSRF)
22. Sublesson 8.7: Cross Site Tracing (XST) Attacks
23. Sublesson 9.1: Denial of Service from Multiple Logins
24. Sublesson 10.1: Fail Open Authentication Scheme
25. Sublesson 11.1: Command Injection
26. Sublesson 11.2: Blind SQL Injection
27. Sublesson 11.3: Numeric SQL Injection
28. Sublesson 11.5: XPATH Injection
29. Sublesson 11.6: String SQL Injection
30. Sublesson 11.7: LAB: SQL Injection
31. Sublesson 11.8: Database Backdoors
32. Sublesson 12.1: Insecure Login
33. Sublesson 13.1: Forced Browsing
34. Sublesson 15.1: Exploit Hidden Fields
35. Sublesson 15.2: Exploit Unchecked Email
36. Sublesson 15.3: Bypass Client Side JavaScript Validation
37. Sublesson 17.1: Create a SOAP Request
38. Sublesson 17.3: Web Service SAX Injection
39. Sublesson 17.4: Web Service SQL Injection