This site is the archived OWASP Foundation Wiki and is no longer accepting Account Requests.
To view the new OWASP Foundation website, please visit https://owasp.org

Difference between revisions of "OWASP ModSec CRS Paranoia Mode"

From OWASP
Jump to: navigation, search
(Stricter siblings for existing rules)
m
 
(46 intermediate revisions by 6 users not shown)
Line 5: Line 5:
 
FIXME: Detailed description
 
FIXME: Detailed description
  
''Back to the [https://www.owasp.org/index.php/Category:OWASP_ModSecurity_Core_Rule_Set_Project OWASP ModSecurity Core Rules Set].''
+
''Back to the [[:Category:OWASP_ModSecurity_Core_Rule_Set_Project|OWASP ModSecurity Core Rules Set]].''
  
  
Line 12: Line 12:
  
 
* '''Status''': active (January 2016)
 
* '''Status''': active (January 2016)
* '''Schedule''': Pull request in <strike>January</strike>February  2016
+
* '''Schedule''': '''DONE''' (we're all done and Paranoia Mode made it into the CRS3 release.
 
* '''Who''': Christian Folini (dune73), Noël Zindel (zino), Franziska Bühler (franziskabuehler), Manuel Leos (Spartan), Walter Hop (lifeforms)
 
* '''Who''': Christian Folini (dune73), Noël Zindel (zino), Franziska Bühler (franziskabuehler), Manuel Leos (Spartan), Walter Hop (lifeforms)
 
* '''Documentation''': Here on the [https://www.owasp.org/index.php/OWASP_ModSec_CRS_Paranoia_Mode OWASP Wiki] / [https://www.netnea.com/cms/2016/02/04/owasp-modsecurity-core-rules-paranoia-mode-mechanics-proposal/ Mechanics Proposal]
 
* '''Documentation''': Here on the [https://www.owasp.org/index.php/OWASP_ModSec_CRS_Paranoia_Mode OWASP Wiki] / [https://www.netnea.com/cms/2016/02/04/owasp-modsecurity-core-rules-paranoia-mode-mechanics-proposal/ Mechanics Proposal]
 
* '''Discussion / Archive''': <tt>[email protected]</tt> / archive: http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/
 
* '''Discussion / Archive''': <tt>[email protected]</tt> / archive: http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/
* '''Github Link''': https://github.com/SpiderLabs/owasp-modsecurity-crs/tree/v3.0.0-rc1
+
* '''Github Link v3.0.0-rc1 (our base)''': https://github.com/SpiderLabs/owasp-modsecurity-crs/tree/v3.0.0-rc1
* '''Final Pull Request''': FIXME
+
* '''Github Link paranoia-mode''': https://github.com/dune73/owasp-modsecurity-crs/tree/paranoia-mode
 +
* '''Final Pull Request #1: Add paranoia mode mechanics''': https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/292 MERGED
 +
* '''Final Pull Request #2: Move first rules to paranoia mode''': https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/300 MERGED
 +
* '''Final Pull Request #3: Add 2.2.X rules to paranoia mode''': https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/308 MERGED
 +
* '''Final Pull Request #4: Add stricter siblings''': FIXME
  
 
==Tasks==
 
==Tasks==
Line 23: Line 27:
 
===Open Tasks===
 
===Open Tasks===
  
Please define state as follows: ''new'', ''assigned'', ''waiting'', ''closed''. When a task it is closed, it is moved to the seperate closed tasks table below.
+
Please define state as follows: ''new'', ''assigned'', ''waiting'', ''closed''. When a task is closed, it is moved to the seperate closed tasks table below.
  
 
{|- class="wikitable"
 
{|- class="wikitable"
Line 30: Line 34:
 
   | &nbsp;&nbsp;&nbsp;'''Status'''&nbsp;&nbsp;&nbsp;
 
   | &nbsp;&nbsp;&nbsp;'''Status'''&nbsp;&nbsp;&nbsp;
 
|-
 
|-
  | Nail down final list of rules which should be moved / recreated into the paranoia mode
+
   | Write pull request number 4
  | group
 
  | assigned
 
|-
 
  | Write new stricter siblings for existing rules
 
  | Noël
 
  | assigned
 
|-
 
  | Define ID-space for strict siblings
 
  | n.n.
 
  | new
 
|-
 
  | Sort out mechanics of the paranoia mode
 
  | Christian
 
  | assigned
 
|-
 
  | Define exact syntax of paranoia mode setup
 
  | Christian
 
  | waiting
 
|-
 
  | Sort out name: Is "Paranoia Mode" really the right term?
 
  | Christian
 
  | waiting
 
|-
 
   | Write pull request
 
 
   | n.n.
 
   | n.n.
 
   | new
 
   | new
 
|-
 
|-
   | Submit pull request
+
   | Submit pull request number 4
 
   | n.n.
 
   | n.n.
 
   | new
 
   | new
Line 67: Line 47:
 
|-
 
|-
 
   | Write documentation
 
   | Write documentation
   | n.n.
+
   | Christian
 
   | new
 
   | new
 
|}
 
|}
Line 101: Line 81:
 
   | group
 
   | group
 
   | closed (could be repeated more throughly)
 
   | closed (could be repeated more throughly)
 +
|-
 +
  | Nail down final list of rules which should be moved / recreated into the paranoia mode
 +
  | group
 +
  | closed
 +
|-
 +
  | Sort out mechanics of the paranoia mode
 +
  | Christian
 +
  | closed
 +
|-
 +
  | Write new stricter siblings for existing rules
 +
  | Noël
 +
  | closed
 +
|-
 +
  | Define ID-space for strict siblings
 +
  | Fraziska, group
 +
  | closed
 +
|-
 +
  | Define exact syntax of paranoia mode setup
 +
  | Christian, group
 +
  | closed
 +
|-
 +
  | Sort out name: Is "Paranoia Mode" really the right term?
 +
  | Christian, group
 +
  | closed
 
|}
 
|}
  
Line 113: Line 117:
 
We need to make sure, we do not mess things up, so let's add both IDs to the table, the old one and the new one.
 
We need to make sure, we do not mess things up, so let's add both IDs to the table, the old one and the new one.
  
Please set status as follows : ''candidate'', ''cloning-candidate'', ''unsure'', ''dropped''.  
+
Please set status as follows : ''confirmed'',''candidate'', ''cloning-confirmed'',''cloning-candidate'', ''unsure'', ''dropped''.  
* 'cloning-candidates' are rules, that could be cloned into an even stricter variant with a stricter limit in a higher paranoia setting.
+
* 'cloning-confirmed', 'cloning-candidates' are rules, that could be cloned into an even stricter variant with a stricter limit in a higher paranoia setting.
 
* If dropped, please provide reasoning in the remarks.
 
* If dropped, please provide reasoning in the remarks.
  
Line 124: Line 128:
 
   | &nbsp;&nbsp;&nbsp;'''Status'''&nbsp;&nbsp;&nbsp;
 
   | &nbsp;&nbsp;&nbsp;'''Status'''&nbsp;&nbsp;&nbsp;
 
   | &nbsp;&nbsp;&nbsp;'''Remarks'''&nbsp;&nbsp;&nbsp;
 
   | &nbsp;&nbsp;&nbsp;'''Remarks'''&nbsp;&nbsp;&nbsp;
|-
 
  | 900050
 
  | 910100
 
  | Client IP is from a HIGH Risk Country Location.
 
  | unsure
 
  | Franziska's candidate: Do we want to exlude countries? But then easy to configure.
 
 
|-
 
|-
 
   | 950001
 
   | 950001
 
   | 942150
 
   | 942150
 
   | SQL Injection Attack
 
   | SQL Injection Attack
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: @pmf file with very short function names, could match frequently.
 
   | Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: @pmf file with very short function names, could match frequently.
 
|-
 
|-
Line 140: Line 138:
 
   | 920230
 
   | 920230
 
   | Multiple URL Encoding Detected
 
   | Multiple URL Encoding Detected
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives
 
   | Christian's 2.2.X experience: frequently false positives
 
|-
 
|-
Line 146: Line 144:
 
   | 931130
 
   | 931130
 
   | Possible Remote File Inclusion (RFI) Attack: Off-Domain Reference/Link
 
   | Possible Remote File Inclusion (RFI) Attack: Off-Domain Reference/Link
   | candidate
+
   | confirmed
   | Walter's 2.2.X candidate: many FP; Chrstian: hardly any FPs, #discuss
+
   | Walter's 2.2.X candidate: many FP; Chrstian: hardly any FPs; <br/>discussion concluded, that rule should end up in paranoia mode, possibly with additional conditions to reduce FPs (scope outside of this paranoia mode project)<br/>[http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-February/001885.html Link to discussion]
 
|-
 
|-
 
   | 960335
 
   | 960335
 
   | 920380
 
   | 920380
 
   | Too many arguments in request
 
   | Too many arguments in request
   | candidate
+
   | confirmed
 
   | Walter's 2.2.X candidate: some FP (phpMyAdmin, large forms), alternatively would recommend raising <code>tx.max_num_args</code> to 1000
 
   | Walter's 2.2.X candidate: some FP (phpMyAdmin, large forms), alternatively would recommend raising <code>tx.max_num_args</code> to 1000
 
|-
 
|-
Line 158: Line 156:
 
   | 942130
 
   | 942130
 
   | SQL Injection Attack: SQL Tautology Detected.
 
   | SQL Injection Attack: SQL Tautology Detected.
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: very frequently false positives. Also Franziska's candidate: legitimate sentences could match. Walter's 2.2.x experience: many FP in natural text however the rule seems to have merit
 
   | Christian's 2.2.X experience: very frequently false positives. Also Franziska's candidate: legitimate sentences could match. Walter's 2.2.x experience: many FP in natural text however the rule seems to have merit
|-
 
  | 950907
 
  | 932100
 
  | System Command Injection
 
  | candidate
 
  | Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: false positives possible because of @pmf, file with short cmds. Maybe we should split the data file #discuss
 
 
|-
 
|-
 
   | 950916
 
   | 950916
 
   | 921170
 
   | 921170
 
   | HTTP Header Injection Attack via payload (CR/LF detected)
 
   | HTTP Header Injection Attack via payload (CR/LF detected)
   | candidate
+
   | confirmed
 
   | Franziska's candidate: change action from pass to block and move to paranoia mode.
 
   | Franziska's candidate: change action from pass to block and move to paranoia mode.
|-
 
  | 958977
 
  | 933110
 
  | PHP Injection Attack: Function Name Found
 
  | candidate
 
  | Franziska's candidate: false positives possible because of @pmf, file with short function names. Maybe we should split the data file #discuss
 
|-
 
  | 958979
 
  | 933120
 
  | PHP Injection Attack: Configuration Directive Found
 
  | candidate
 
  | Franziska's candidate: false positives possible because of @pmf, file with short configuration directives.
 
 
|-
 
|-
 
   | 959070
 
   | 959070
   | gone   
+
   | gone -> 942380  
 
   | SQL Injection Attack
 
   | SQL Injection Attack
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives
 
   | Christian's 2.2.X experience: frequently false positives
 
|-
 
|-
 
   | 959071
 
   | 959071
   | gone
+
   | gone -> 942390
 
   | SQL Injection Attack
 
   | SQL Injection Attack
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives
 
   | Christian's 2.2.X experience: frequently false positives
 
|-
 
|-
 
   | 959072
 
   | 959072
   | gone   
+
   | gone -> 942400  
 
   | SQL Injection Attack
 
   | SQL Injection Attack
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives
 
   | Christian's 2.2.X experience: frequently false positives
 
|-
 
|-
 
   | 959073
 
   | 959073
   | gone   
+
   | gone -> 942410  
 
   | SQL Injection Attack
 
   | SQL Injection Attack
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: very frequently false positives
 
   | Christian's 2.2.X experience: very frequently false positives
 
|-
 
|-
Line 212: Line 192:
 
   | 920300
 
   | 920300
 
   | Request Missing an Accept Header
 
   | Request Missing an Accept Header
   | candidate
+
   | confirmed
   | Christian's 2.2.X experience: very frequently false positives. Also Franziska's candidate: Not every legitimate client behaves correctly. Walter's experience: many FP (PHP SoapClient)
+
   | Christian's 2.2.X experience: very frequently false positives. Also Franziska's candidate: Not every legitimate client behaves correctly. Walter's experience: many FP (PHP SoapClient)<br/>Discussion concluded it's moved to paranoia mode.<br/>[http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-February/001888.html Link to discussion]<br/>Spartan: Many mobile devices do not send this header, very high FP.
|-
 
  | 960017
 
  | 920350
 
  | Host header is a numeric IP address
 
  | candidate
 
  | Christian's 2.2.X experience: very frequently false positives. Also Franziska's candidate: Not every legitimate client behaves correctly. Walter's experience: low FP (almost all are mass scans) #discuss
 
 
|-
 
|-
 
   | 960024
 
   | 960024
   | gone
+
   | gone -> 942460
 
   | Meta-Character Anomaly Detection Alert - Repetative Non-Word Characters
 
   | Meta-Character Anomaly Detection Alert - Repetative Non-Word Characters
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: very frequently false positives
 
   | Christian's 2.2.X experience: very frequently false positives
 
|-
 
|-
Line 230: Line 204:
 
   | 920440
 
   | 920440
 
   | URL file extension is restricted by policy
 
   | URL file extension is restricted by policy
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives
 
   | Christian's 2.2.X experience: frequently false positives
 
|-
 
|-
Line 236: Line 210:
 
   | 950100
 
   | 950100
 
   | The Application Returned a 500-Level Status Code
 
   | The Application Returned a 500-Level Status Code
   | candidate
+
   | confirmed
   | Franziska's candidate: too strict, too generic, no data leakage happened so far. Walter: it's useful however to prevent attacker from distinguishing between a failed SQLi attempt (403 blocked by ModSec) or a query error due to vulnerable app (500 from application) #discuss
+
   | Franziska's candidate: too strict, too generic, no data leakage happened so far. Walter: it's useful however to prevent attacker from distinguishing between a failed SQLi attempt (403 blocked by ModSec) or a query error due to vulnerable app (500 from application); <br/>Discussion resolved with move to paranoia mode. 403 will cloak a backend error, which is hard for an inexperienced admin and thus complicates things in standard installations<br/>[http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-February/001889.html Link to discussion]
 
|-
 
|-
 
   | 973300
 
   | 973300
   | gone
+
   | gone -> 941320
 
   | Possible XSS Attack Detected - HTML Tag Handler
 
   | Possible XSS Attack Detected - HTML Tag Handler
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP
 
|-
 
|-
 
   | 973332
 
   | 973332
   | gone
+
   | gone -> 941330
 
   | IE XSS Filters - Attack Detected.
 
   | IE XSS Filters - Attack Detected.
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP
 
|-
 
|-
 
   | 973333
 
   | 973333
   | gone
+
   | gone -> 941340
 
   | IE XSS Filters - Attack Detected.
 
   | IE XSS Filters - Attack Detected.
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP
 
|-
 
|-
 
   | 981172
 
   | 981172
   | gone
+
   | gone -> 942420
 
   | Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded
 
   | Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: very high FP
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: very high FP
 
|-
 
|-
 
   | 981173
 
   | 981173
   | gone
+
   | gone -> 942430
 
   | Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded
 
   | Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: very high FP
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: very high FP
 
|-
 
|-
 
   | 981231
 
   | 981231
   | gone
+
   | gone -> 942440
 
   | SQL Comment Sequence Detected.
 
   | SQL Comment Sequence Detected.
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: high FP but rule seems useful
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: high FP but rule seems useful
 
|-
 
|-
Line 278: Line 252:
 
   | 942300
 
   | 942300
 
   | Detects MySQL comments, conditions and ch(a)r injections
 
   | Detects MySQL comments, conditions and ch(a)r injections
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP
 
|-
 
|-
Line 284: Line 258:
 
   | 942330
 
   | 942330
 
   | Detects classic SQL injection probings 1/2
 
   | Detects classic SQL injection probings 1/2
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: one quote character already matches?? Walter: low FP, but seen in cookies injected by some US ISPs;  
 
   | Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: one quote character already matches?? Walter: low FP, but seen in cookies injected by some US ISPs;  
 
|-
 
|-
Line 290: Line 264:
 
   | 942370
 
   | 942370
 
   | Detects classic SQL injection probings 2/2
 
   | Detects classic SQL injection probings 2/2
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: medium FP
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: medium FP
 
|-
 
|-
Line 296: Line 270:
 
   | 942180
 
   | 942180
 
   | Detects basic SQL authentication bypass attempts 1/3
 
   | Detects basic SQL authentication bypass attempts 1/3
   | candidate
+
   | confirmed
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP, #discuss
+
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP; <br/> discussion did not bring up additional arguments. Moving to paranoia mode<br/>[http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-February/001890.html Link to discussion]
 
|-
 
|-
 
   | 981245
 
   | 981245
 
   | 942260
 
   | 942260
 
   | Detects basic SQL authentication bypass attempts 2/3
 
   | Detects basic SQL authentication bypass attempts 2/3
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Walter: medium FP
 
   | Christian's 2.2.X experience: frequently false positives. Walter: medium FP
 
|-
 
|-
Line 308: Line 282:
 
   | 942340
 
   | 942340
 
   | Detects basic SQL authentication bypass attempts 3/3
 
   | Detects basic SQL authentication bypass attempts 3/3
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Walter: medium FP
 
   | Christian's 2.2.X experience: frequently false positives. Walter: medium FP
 
|-
 
|-
Line 314: Line 288:
 
   | 942210
 
   | 942210
 
   | Detects chained SQL injection attempts 1/2
 
   | Detects chained SQL injection attempts 1/2
   | candidate
+
   | confirmed
   | Christian's 2.2.X experience: very frequently false positives. Walter: low FP, #discuss
+
   | Christian's 2.2.X experience: very frequently false positives. Walter: low FP, <br/> discussion did not bring up any additional arguments. Moving to paranoia mode<br/>[http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-February/001890.html Link to discussion]
 
|-
 
|-
 
   | 981249
 
   | 981249
 
   | 942310
 
   | 942310
 
   | Detects chained SQL injection attempts 2/2
 
   | Detects chained SQL injection attempts 2/2
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP but seen in very specific situations
 
   | Christian's 2.2.X experience: frequently false positives. Walter: low FP but seen in very specific situations
 
|-
 
|-
Line 326: Line 300:
 
   | 942200
 
   | 942200
 
   | Detects MySQL comment-/space-obfuscated injections and backtick termination
 
   | Detects MySQL comment-/space-obfuscated injections and backtick termination
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Walter: medium FP
 
   | Christian's 2.2.X experience: frequently false positives. Walter: medium FP
 
|-
 
|-
 
   | 981260
 
   | 981260
   | gone
+
   | gone -> 942450
 
   | SQL Hex Encoding Identified
 
   | SQL Hex Encoding Identified
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: high FP in long random strings
 
   | Christian's 2.2.X experience: very frequently false positives. Walter: high FP in long random strings
 
|-
 
|-
Line 338: Line 312:
 
   | 942110
 
   | 942110
 
   | SQL Injection Attack: Common Injection Testing Detected
 
   | SQL Injection Attack: Common Injection Testing Detected
   | candidate
+
   | confirmed
 
   | Franziska's candidate: one quote character at the beginning/end really not legitimate? Walter 2.2.X candidate: frequent FP
 
   | Franziska's candidate: one quote character at the beginning/end really not legitimate? Walter 2.2.X candidate: frequent FP
 
|-
 
|-
Line 344: Line 318:
 
   | 942120
 
   | 942120
 
   | SQL Injection Attack: SQL Operator Detected
 
   | SQL Injection Attack: SQL Operator Detected
   | candidate
+
   | confirmed
 
   | Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: very short operators or strings already match. Walter: some FP (WooCommerce)
 
   | Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: very short operators or strings already match. Walter: some FP (WooCommerce)
 
|-
 
|-
Line 350: Line 324:
 
   | 912100
 
   | 912100
 
   | Potential Denial of Service (DoS) Attack from ... - # of Request Bursts: ...    
 
   | Potential Denial of Service (DoS) Attack from ... - # of Request Bursts: ...    
   | cloning-candidate
+
   | cloning-confirmed
 
   | limit currently at 2; could be set to 1; now, the attacker has to exceed dos_counter_threshold twice. With full reset of counter after first hit. Source: 2.2.X->experimental rules
 
   | limit currently at 2; could be set to 1; now, the attacker has to exceed dos_counter_threshold twice. With full reset of counter after first hit. Source: 2.2.X->experimental rules
 
|-
 
|-
Line 356: Line 330:
 
   | 920270
 
   | 920270
 
   | Invalid character in request
 
   | Invalid character in request
   | cloning-candidate
+
   | cloning-confirmed
 
   | @validateByteRange 1-255; there was a conditional rule with stricter byterange 32-126 in 2.2.X as well
 
   | @validateByteRange 1-255; there was a conditional rule with stricter byterange 32-126 in 2.2.X as well
 
|-
 
|-
Line 362: Line 336:
 
   | 951100
 
   | 951100
 
   | none      
 
   | none      
   | cloning-candidate
+
   | cloning-confirmed
 
   | rule is only setting tx.sql_error_match. Could also trigger score directly
 
   | rule is only setting tx.sql_error_match. Could also trigger score directly
 
|-
 
|-
Line 368: Line 342:
 
   | 932100
 
   | 932100
 
   | Remote Command Execution (RCE) Attempt      
 
   | Remote Command Execution (RCE) Attempt      
   | cloning-candidate
+
   | cloning-confirmed
 
   | rule is only triggering in combination with chained rule. Could trigger on its on
 
   | rule is only triggering in combination with chained rule. Could trigger on its on
 
|-
 
|-
Line 374: Line 348:
 
   | 933110
 
   | 933110
 
   | PHP Injection Attack: Function Name Found      
 
   | PHP Injection Attack: Function Name Found      
   | cloning-candidate
+
   | cloning-confirmed
 
   | rule is only triggering in combination with chained rule. Could trigger on its on
 
   | rule is only triggering in combination with chained rule. Could trigger on its on
 
|-
 
|-
Line 380: Line 354:
 
   | 933120
 
   | 933120
 
   | PHP Injection Attack: Configuration Directive Found      
 
   | PHP Injection Attack: Configuration Directive Found      
  | cloning-candidate
 
  | rule is only triggering in combination with chained rule. Could trigger on its on
 
|-
 
  | 958980         
 
  | 933130
 
  | PHP Injection Attack: Variables Found      
 
 
   | cloning-candidate
 
   | cloning-candidate
 
   | rule is only triggering in combination with chained rule. Could trigger on its on
 
   | rule is only triggering in combination with chained rule. Could trigger on its on
Line 392: Line 360:
 
   | 942150
 
   | 942150
 
   | SQL Injection Attack        
 
   | SQL Injection Attack        
   | cloning-candidate
+
   | cloning-confirmed
 
   | rule is only triggering in combination with chained rule. Could trigger on its on
 
   | rule is only triggering in combination with chained rule. Could trigger on its on
 +
|-
 +
  | 950907
 +
  | 932100
 +
  | System Command Injection
 +
  | dropped
 +
  | Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: false positives possible because of @pmf, file with short cmds. Discussion evolved about splitting the file, which everybody thinks is a good idea. But that would be outside the scope of the introduction of the paranoia mode. So the rule stays in the standard set of rules for the time being and will be split in the future [http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-February/001886.html Link to discussion]
 +
|-
 +
  | 900050
 +
  | 910100
 +
  | Client IP is from a HIGH Risk Country Location.
 +
  | dropped
 +
  | Franziska's candidate: Do we want to exlude countries? But then easy to configure. Discussion pointed out this as an effective rule. We leave it in the standard rules, but provide an empty country list by default [http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-February/001951.html Link to discussion]. [https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/284 Separate pull request]
 +
|-
 +
  | 960017
 +
  | 920350
 +
  | Host header is a numeric IP address
 +
  | dropped
 +
  | Christian's 2.2.X experience: very frequently false positives. Also Franziska's candidate: Not every legitimate client behaves correctly. Walter's experience: low FP (almost all are mass scans); <br/> Discussion concluded that legitimate use of numeric IP addresses is rare. This is really mostly mass scanners. Rule will be kept in standard set of rules<br/>[http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-February/001888.html Link to discussion]
 +
|-
 +
  | 958977
 +
  | 933110
 +
  | PHP Injection Attack: Function Name Found
 +
  | dropped
 +
  | Franziska's candidate: false positives possible because of @pmf, file with short function names. Maybe we should split the data file. The discussion revealed that splitting the data file in a clean way is very difficult. Walter Hop volunteered to rework the php rules completely. Chaim might join that effort.
 +
|-
 +
  | 958979
 +
  | 933120
 +
  | PHP Injection Attack: Configuration Directive Found
 +
  | dropped
 +
  | Franziska's candidate: false positives possible because of @pmf, file with short configuration directives. Splitting file?  The discussion revealed that splitting the data file in a clean way is very difficult. Walter Hop volunteered to rework the php rules completely. Chaim might join that effort.
 
|}
 
|}
  
Line 931: Line 929:
 
   | 981004
 
   | 981004
 
   | Potential Obfuscated Javascript in Output - Excessive fromCharCode
 
   | Potential Obfuscated Javascript in Output - Excessive fromCharCode
   | Walter: many FP (Wordpress 4.4 inlined emoji javascripts)
+
   | Walter: many FP (Wordpress 4.4 inlined emoji javascripts); folinic: Problem solved in WP: https://core.trac.wordpress.org/ticket/35412
 
|-
 
|-
 
   | 981005
 
   | 981005
Line 1,091: Line 1,089:
 
Stricter Siblings are rules that are present in the CRS but could be accompanied by a stricter clone in Paranoia Mode. Adjustments can differ from rule to rule but include higher anomaly ratings or stricter triggers (e.g. regex counters). To prevent masses of false positives, rules can come with additional filters (chained rules) for common use-cases. These can either be included into Paranoia Mode or simply serve as a recommendation.
 
Stricter Siblings are rules that are present in the CRS but could be accompanied by a stricter clone in Paranoia Mode. Adjustments can differ from rule to rule but include higher anomaly ratings or stricter triggers (e.g. regex counters). To prevent masses of false positives, rules can come with additional filters (chained rules) for common use-cases. These can either be included into Paranoia Mode or simply serve as a recommendation.
  
Note: To avoid a cluttered project main-page, rule proposals are documented in their respective sub-page. When adding new proposals, make sure adding the rules original (2.2.x) ID, a quick description of what changes were made, and, if applicable, which additional filters were added.
+
Note: To avoid a cluttered project main-page, rule proposals are documented in their respective sub-page. When adding new proposals, make sure adding the rules original (2.2.x) ID, a quick description of what changes were made, and, if applicable, which additional filters were added. For every proposed rule change, we will create a [https://github.com/SpiderLabs/owasp-modsecurity-crs/issues Github issue]; after discussion a pull request will be created. For brainstorming about CRS rules, see our [[OWASP ModSecurity rule evaluation framework]].
  
 
'''Possible siblings:'''
 
'''Possible siblings:'''
  
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_981049|981049 : SQL Injection Character Anomaly Usage]]<br>
+
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_981173|981173 : SQL Injection Character Anomaly Usage]]<br>
 
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_981172|981172 : SQL Injection Character Anomaly Usage]]<br>
 
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_981172|981172 : SQL Injection Character Anomaly Usage]]<br>
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_981173|981173 : SQL Injection Character Anomaly Usage]]
+
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_981049|981049 : Potential Denial of Service (DoS)]]<br>
 +
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_970003|970003 : SQL Error Leakage]]<br>
 +
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_960901|960901 : Invalid character in request]]<br>
 +
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_958980|958980 : PHP Injection Attack: Variables Found]]<br>
 +
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_958979|958979 : PHP Injection Attack: Configuration Directive Found]]<br>
 +
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_958977|958977 : PHP Injection Attack: Function Name Found]]<br>
 +
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_950907|950907 : Remote Command Execution (RCE) Attempt]]<br>
 +
[[OWASP_ModSec_CRS_Paranoia_Mode_Sibling_950001|950001 : SQL Injection Attack]]
  
 
==Project Status==
 
==Project Status==
Line 1,160: Line 1,165:
  
 
</code>
 
</code>
 +
 +
==Git help==
 +
 +
====How to Perform Pull Request to CRS v3.0.0-rc1====
 +
 +
'''By Walter Hop'''
 +
 +
This example assumes that the Github username is 'lifeforms', replace with your own username.
 +
 +
'''Step 1. Create a Github fork of the original repository'''
 +
* a. Browse to CRS: https://github.com/SpiderLabs/owasp-modsecurity-crs
 +
* b. Click "Fork" button in the top right
 +
 +
'''Step 2. Download your Github fork, and checkout the correct branch '''<br/>
 +
<code>
 +
git clone [email protected]:lifeforms/owasp-modsecurity-crs.git <br/>
 +
cd owasp-modsecurity-crs <br/>
 +
git checkout v3.0.0-rc1 <br/>
 +
</code>
 +
 +
'''Step 3. Add the original repository as upstream, to integrate new changes easily'''<br/>
 +
<code>
 +
git remote add upstream [email protected]:SpiderLabs/owasp-modsecurity-crs.git
 +
</code>
 +
 +
'''Step 4. Make sure your forked repo is up to date with changes in the original repository'''<br/>
 +
 +
You need to re-do these steps if somebody else made changes to upstream in the meantime!
 +
 +
<code>
 +
git checkout v3.0.0-rc1 <br/>
 +
git fetch upstream <br/>
 +
git merge --ff upstream/v3.0.0-rc1 <br/>
 +
git push <br/>
 +
</code>
 +
 +
'''Step 5. Create a branch for every separate issue you'd like to fix.'''<br/>
 +
<code>
 +
git checkout -b myfeature v3.0.0-rc1<br/>
 +
git push --set-upstream origin myfeature<br/>
 +
</code>
 +
 +
'''Step 6. Make local changes and commit them'''<br/>
 +
<code>
 +
git add ...<br/>
 +
git commit<br/>
 +
</code>
 +
 +
'''Step 7. Push your local changes to your fork at Github'''<br/>
 +
<code>
 +
git push<br/>
 +
</code>
 +
 +
'''Step 8. Create the pull request'''
 +
* a. Browse to your own fork: https://github.com/lifeforms/owasp-modsecurity-crs
 +
* b. Click "New pull request" button
 +
* c. In the "base fork", ensure that the correct branch is selected: v3.0.0-rc1
 +
* d. In the "head fork", pick "myfeature" (if you used branches) or v3.0.0-rc1 (if you didn't)
 +
* e. Review the content of the pull request (should only contain your commits)
 +
* f. Press "Create pull request" button
 +
 +
'''Optional: Updating your pull request'''
 +
 +
After opening the pull request, people will review it, and probably will make some suggestions for changes before the PR can be accepted. You can keep pushing to your branch, and it will be reflected in the PR.
 +
 +
If in the meantime the code in upstream has drifted, you should re-do step 4 above.
 +
 +
Then, just change some files, commit and push.
 +
 +
<code>
 +
git add ...<br/>
 +
git commit<br/>
 +
git push<br/>
 +
</code>
 +
 +
See also:
 +
 +
* https://help.github.com/articles/fork-a-repo/
 +
* https://help.github.com/articles/syncing-a-fork/
 +
* https://help.github.com/articles/using-pull-requests/
 +
 +
====Testing someone else's pull request====
 +
To test a PR locally, use the following commands to create a branch for it.
 +
 +
In this example we check out PR #427, creating a branch <tt>chaim-headers</tt> locally (you can use any name):
 +
 +
<pre>
 +
git fetch upstream pull/427/head:chaim-headers
 +
git checkout chaim-headers
 +
</pre>
 +
 +
====Pushing to someone else's pull request====
 +
In an editable PR, the PR will say something like:
 +
<pre>
 +
Add more commits by pushing to the ExceptionSupport branch on csanders-git/owasp-modsecurity-crs.
 +
</pre>
 +
 +
To add to branch <tt>ExceptionSupport</tt> of user <tt>csanders-git</tt>, do:
 +
<pre>
 +
git remote add chaim [email protected]:csanders-git/owasp-modsecurity-crs.git
 +
git fetch chaim
 +
git co ExceptionSupport
 +
git add ...
 +
git push
 +
</pre>
 +
 +
====Purging a file completely from Git====
 +
Delete the file from the repo:
 +
<pre>
 +
git rm file
 +
git commit
 +
git push
 +
</pre>
 +
 +
Download [https://rtyley.github.io/bfg-repo-cleaner/ BFG Repo-Cleaner]
 +
 +
Make a NEW mirror:
 +
<pre>
 +
git clone --mirror [email protected]:SpiderLabs/owasp-modsecurity-crs.git
 +
cd owasp-modsecurity-crs.git
 +
</pre>
 +
 +
Purge the file:
 +
<pre>
 +
java -jar ~/Downloads/bfg-*.jar --delete-files example.png
 +
git reflog expire --expire=now --all && git gc --prune=now --aggressive
 +
git push -f
 +
</pre>
 +
 +
E-mail all users to REMOVE their copies of the repositories, and do a FRESH CLONE of the repo. If a user pulls again, the divergent histories will be merged, and there is a big risk that the dirty history (containing the purged file) will return again when they push later.
 +
 +
See also: [https://rtyley.github.io/bfg-repo-cleaner/ BFG Repo-Cleaner]
 +
 +
[[Category:OWASP ModSecurity Core Rule Set Project]]

Latest revision as of 17:45, 16 March 2018

Abstract

This is a page about the development of a paranoia mode aka bringing back the rules that used to yield a high number of false positives. This little project is aimed at inclusion into the 3.0.0 release of the OWASP ModSecurity Core Rules, where some rules have been removed in order to reduce the number of false positives with vanilla installations.

FIXME: Detailed description

Back to the OWASP ModSecurity Core Rules Set.


Sub-Project Infos

Tasks

Open Tasks

Please define state as follows: new, assigned, waiting, closed. When a task is closed, it is moved to the seperate closed tasks table below.

Task         Who           Status   
Write pull request number 4 n.n. new
Submit pull request number 4 n.n. new
Draw flowchart n.n. new
Write documentation Christian new

Closed Tasks

Task         Who           Status   
Assemble list of rules, which triggered false positives in 2.2.X frequently Christian closed
Assemble list of 2.2.x rules, which have disappeared from 3.0.0-rc1 Spartan closed
Assemble list of 3.0.0-rc1 rules, which could be accompanied with
stricter siblings in paranoia mode
(same idea of the rule, but harder limit etc.)
Christian closed
Assemble list of 3.0.0-rc1 rules, which could be moved to the paranoia mode Franziska closed
Assemble list of disappeared / missing 2.2.X base_rules, which should be brought back group closed
Assemble list of 2.2.X optional and experimental rules, which should be brought back group closed (could be repeated more throughly)
Nail down final list of rules which should be moved / recreated into the paranoia mode group closed
Sort out mechanics of the paranoia mode Christian closed
Write new stricter siblings for existing rules Noël closed
Define ID-space for strict siblings Fraziska, group closed
Define exact syntax of paranoia mode setup Christian, group closed
Sort out name: Is "Paranoia Mode" really the right term? Christian, group closed


Rules

Paranoia Mode Candidates

The 3.0.0-rc1 has all rules renumbered. Existing numbering was fairly crazy and the new numbering follows the numbering scheme of the rules files (-> 9<2-digit-rulefile><3-digit-id>) A mapping table exists [IdNumbering.csv] We need to make sure, we do not mess things up, so let's add both IDs to the table, the old one and the new one.

Please set status as follows : confirmed,candidate, cloning-confirmed,cloning-candidate, unsure, dropped.

  • 'cloning-confirmed', 'cloning-candidates' are rules, that could be cloned into an even stricter variant with a stricter limit in a higher paranoia setting.
  • If dropped, please provide reasoning in the remarks.


RuleID 2.2.x RuleID 3.0.0-rc1         msg           Status       Remarks   
950001 942150 SQL Injection Attack confirmed Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: @pmf file with very short function names, could match frequently.
950109 920230 Multiple URL Encoding Detected confirmed Christian's 2.2.X experience: frequently false positives
950120 931130 Possible Remote File Inclusion (RFI) Attack: Off-Domain Reference/Link confirmed Walter's 2.2.X candidate: many FP; Chrstian: hardly any FPs;
discussion concluded, that rule should end up in paranoia mode, possibly with additional conditions to reduce FPs (scope outside of this paranoia mode project)
Link to discussion
960335 920380 Too many arguments in request confirmed Walter's 2.2.X candidate: some FP (phpMyAdmin, large forms), alternatively would recommend raising tx.max_num_args to 1000
950901 942130 SQL Injection Attack: SQL Tautology Detected. confirmed Christian's 2.2.X experience: very frequently false positives. Also Franziska's candidate: legitimate sentences could match. Walter's 2.2.x experience: many FP in natural text however the rule seems to have merit
950916 921170 HTTP Header Injection Attack via payload (CR/LF detected) confirmed Franziska's candidate: change action from pass to block and move to paranoia mode.
959070 gone -> 942380 SQL Injection Attack confirmed Christian's 2.2.X experience: frequently false positives
959071 gone -> 942390 SQL Injection Attack confirmed Christian's 2.2.X experience: frequently false positives
959072 gone -> 942400 SQL Injection Attack confirmed Christian's 2.2.X experience: frequently false positives
959073 gone -> 942410 SQL Injection Attack confirmed Christian's 2.2.X experience: very frequently false positives
960015 920300 Request Missing an Accept Header confirmed Christian's 2.2.X experience: very frequently false positives. Also Franziska's candidate: Not every legitimate client behaves correctly. Walter's experience: many FP (PHP SoapClient)
Discussion concluded it's moved to paranoia mode.
Link to discussion
Spartan: Many mobile devices do not send this header, very high FP.
960024 gone -> 942460 Meta-Character Anomaly Detection Alert - Repetative Non-Word Characters confirmed Christian's 2.2.X experience: very frequently false positives
960035 920440 URL file extension is restricted by policy confirmed Christian's 2.2.X experience: frequently false positives
970901 950100 The Application Returned a 500-Level Status Code confirmed Franziska's candidate: too strict, too generic, no data leakage happened so far. Walter: it's useful however to prevent attacker from distinguishing between a failed SQLi attempt (403 blocked by ModSec) or a query error due to vulnerable app (500 from application);
Discussion resolved with move to paranoia mode. 403 will cloak a backend error, which is hard for an inexperienced admin and thus complicates things in standard installations
Link to discussion
973300 gone -> 941320 Possible XSS Attack Detected - HTML Tag Handler confirmed Christian's 2.2.X experience: frequently false positives. Walter: low FP
973332 gone -> 941330 IE XSS Filters - Attack Detected. confirmed Christian's 2.2.X experience: frequently false positives. Walter: low FP
973333 gone -> 941340 IE XSS Filters - Attack Detected. confirmed Christian's 2.2.X experience: frequently false positives. Walter: low FP
981172 gone -> 942420 Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded confirmed Christian's 2.2.X experience: very frequently false positives. Walter: very high FP
981173 gone -> 942430 Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded confirmed Christian's 2.2.X experience: very frequently false positives. Walter: very high FP
981231 gone -> 942440 SQL Comment Sequence Detected. confirmed Christian's 2.2.X experience: very frequently false positives. Walter: high FP but rule seems useful
981240 942300 Detects MySQL comments, conditions and ch(a)r injections confirmed Christian's 2.2.X experience: frequently false positives. Walter: low FP
981242 942330 Detects classic SQL injection probings 1/2 confirmed Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: one quote character already matches?? Walter: low FP, but seen in cookies injected by some US ISPs;
981243 942370 Detects classic SQL injection probings 2/2 confirmed Christian's 2.2.X experience: very frequently false positives. Walter: medium FP
981244 942180 Detects basic SQL authentication bypass attempts 1/3 confirmed Christian's 2.2.X experience: frequently false positives. Walter: low FP;
discussion did not bring up additional arguments. Moving to paranoia mode
Link to discussion
981245 942260 Detects basic SQL authentication bypass attempts 2/3 confirmed Christian's 2.2.X experience: frequently false positives. Walter: medium FP
981246 942340 Detects basic SQL authentication bypass attempts 3/3 confirmed Christian's 2.2.X experience: frequently false positives. Walter: medium FP
981248 942210 Detects chained SQL injection attempts 1/2 confirmed Christian's 2.2.X experience: very frequently false positives. Walter: low FP,
discussion did not bring up any additional arguments. Moving to paranoia mode
Link to discussion
981249 942310 Detects chained SQL injection attempts 2/2 confirmed Christian's 2.2.X experience: frequently false positives. Walter: low FP but seen in very specific situations
981257 942200 Detects MySQL comment-/space-obfuscated injections and backtick termination confirmed Christian's 2.2.X experience: frequently false positives. Walter: medium FP
981260 gone -> 942450 SQL Hex Encoding Identified confirmed Christian's 2.2.X experience: very frequently false positives. Walter: high FP in long random strings
981318 942110 SQL Injection Attack: Common Injection Testing Detected confirmed Franziska's candidate: one quote character at the beginning/end really not legitimate? Walter 2.2.X candidate: frequent FP
981319 942120 SQL Injection Attack: SQL Operator Detected confirmed Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: very short operators or strings already match. Walter: some FP (WooCommerce)
981049 912100 Potential Denial of Service (DoS) Attack from ... - # of Request Bursts: ... cloning-confirmed limit currently at 2; could be set to 1; now, the attacker has to exceed dos_counter_threshold twice. With full reset of counter after first hit. Source: 2.2.X->experimental rules
960901 920270 Invalid character in request cloning-confirmed @validateByteRange 1-255; there was a conditional rule with stricter byterange 32-126 in 2.2.X as well
970003 951100 none cloning-confirmed rule is only setting tx.sql_error_match. Could also trigger score directly
950907 932100 Remote Command Execution (RCE) Attempt cloning-confirmed rule is only triggering in combination with chained rule. Could trigger on its on
958977 933110 PHP Injection Attack: Function Name Found cloning-confirmed rule is only triggering in combination with chained rule. Could trigger on its on
958979 933120 PHP Injection Attack: Configuration Directive Found cloning-candidate rule is only triggering in combination with chained rule. Could trigger on its on
950001 942150 SQL Injection Attack cloning-confirmed rule is only triggering in combination with chained rule. Could trigger on its on
950907 932100 System Command Injection dropped Christian's 2.2.X experience: frequently false positives. Also Franziska's candidate: false positives possible because of @pmf, file with short cmds. Discussion evolved about splitting the file, which everybody thinks is a good idea. But that would be outside the scope of the introduction of the paranoia mode. So the rule stays in the standard set of rules for the time being and will be split in the future Link to discussion
900050 910100 Client IP is from a HIGH Risk Country Location. dropped Franziska's candidate: Do we want to exlude countries? But then easy to configure. Discussion pointed out this as an effective rule. We leave it in the standard rules, but provide an empty country list by default Link to discussion. Separate pull request
960017 920350 Host header is a numeric IP address dropped Christian's 2.2.X experience: very frequently false positives. Also Franziska's candidate: Not every legitimate client behaves correctly. Walter's experience: low FP (almost all are mass scans);
Discussion concluded that legitimate use of numeric IP addresses is rare. This is really mostly mass scanners. Rule will be kept in standard set of rules
Link to discussion
958977 933110 PHP Injection Attack: Function Name Found dropped Franziska's candidate: false positives possible because of @pmf, file with short function names. Maybe we should split the data file. The discussion revealed that splitting the data file in a clean way is very difficult. Walter Hop volunteered to rework the php rules completely. Chaim might join that effort.
958979 933120 PHP Injection Attack: Configuration Directive Found dropped Franziska's candidate: false positives possible because of @pmf, file with short configuration directives. Splitting file? The discussion revealed that splitting the data file in a clean way is very difficult. Walter Hop volunteered to rework the php rules completely. Chaim might join that effort.

Rules from 2.2.X, missing in 3.0.0-rc1

It looks as if only the base_rules made it into 3.0.0. In fact there are a few rule ids know from the optional and experimental rule folders in 2.2.X, but it is more likely, these are new 3.0.0 rules reusing old rule ids as the rules (regexes and msg) do not match at all.

When trying to generate the list below, be aware that the rule ids have been renumbered between 3.0.0-dev and 3.0.0-rc1. IdNumbering.csv in your friend.

Base rules

2.2.X rule id         msg           remarks   
950002 System Command Access
950006 System Command Injection
950007 Blind SQL Injection Attack
950008 Injection of Undocumented ColdFusion Tags
950010 LDAP Injection Attack
950011 SSI injection Attack
950018 Universal PDF XSS URL Detected. Walter: medium FP (foo.pdf#javascript)
950019 Email Injection Attack
950908 SQL Injection Attack.
950921 Backdoor access
950922 Backdoor access
958000 Cross-site Scripting (XSS) Attack
958001 Cross-site Scripting (XSS) Attack
958002 Cross-site Scripting (XSS) Attack
958003 Cross-site Scripting (XSS) Attack
958004 Cross-site Scripting (XSS) Attack
958005 Cross-site Scripting (XSS) Attack
958006 Cross-site Scripting (XSS) Attack
958007 Cross-site Scripting (XSS) Attack
958008 Cross-site Scripting (XSS) Attack
958009 Cross-site Scripting (XSS) Attack
958010 Cross-site Scripting (XSS) Attack
958011 Cross-site Scripting (XSS) Attack
958012 Cross-site Scripting (XSS) Attack
958013 Cross-site Scripting (XSS) Attack
958016 Cross-site Scripting (XSS) Attack
958017 Cross-site Scripting (XSS) Attack
958018 Cross-site Scripting (XSS) Attack
958019 Cross-site Scripting (XSS) Attack
958020 Cross-site Scripting (XSS) Attack
958022 Cross-site Scripting (XSS) Attack
958023 Cross-site Scripting (XSS) Attack
958024 Cross-site Scripting (XSS) Attack
958025 Cross-site Scripting (XSS) Attack
958026 Cross-site Scripting (XSS) Attack
958027 Cross-site Scripting (XSS) Attack
958028 Cross-site Scripting (XSS) Attack
958030 Cross-site Scripting (XSS) Attack
958031 Cross-site Scripting (XSS) Attack
958032 Cross-site Scripting (XSS) Attack
958033 Cross-site Scripting (XSS) Attack
958034 Cross-site Scripting (XSS) Attack
958036 Cross-site Scripting (XSS) Attack
958037 Cross-site Scripting (XSS) Attack
958038 Cross-site Scripting (XSS) Attack
958039 Cross-site Scripting (XSS) Attack
958040 Cross-site Scripting (XSS) Attack
958041 Cross-site Scripting (XSS) Attack
958045 Cross-site Scripting (XSS) Attack
958046 Cross-site Scripting (XSS) Attack
958047 Cross-site Scripting (XSS) Attack
958049 Cross-site Scripting (XSS) Attack
958051 Cross-site Scripting (XSS) Attack
958052 Cross-site Scripting (XSS) Attack
958054 Cross-site Scripting (XSS) Attack
958056 Cross-site Scripting (XSS) Attack
958057 Cross-site Scripting (XSS) Attack
958059 Cross-site Scripting (XSS) Attack
958291 Range: field exists and begins with 0. Walter: high FP (Chrome PDF viewer) and not useful.
958404 Cross-site Scripting (XSS) Attack
958405 Cross-site Scripting (XSS) Attack
958406 Cross-site Scripting (XSS) Attack
958407 Cross-site Scripting (XSS) Attack
958408 Cross-site Scripting (XSS) Attack
958409 Cross-site Scripting (XSS) Attack
958410 Cross-site Scripting (XSS) Attack
958411 Cross-site Scripting (XSS) Attack
958412 Cross-site Scripting (XSS) Attack
958413 Cross-site Scripting (XSS) Attack
958414 Cross-site Scripting (XSS) Attack
958415 Cross-site Scripting (XSS) Attack
958416 Cross-site Scripting (XSS) Attack
958417 Cross-site Scripting (XSS) Attack
958418 Cross-site Scripting (XSS) Attack
958419 Cross-site Scripting (XSS) Attack
958420 Cross-site Scripting (XSS) Attack
958421 Cross-site Scripting (XSS) Attack
958422 Cross-site Scripting (XSS) Attack
958423 Cross-site Scripting (XSS) Attack
958976 PHP Injection Attack
959070 SQL Injection Attack
959071 SQL Injection Attack
959072 SQL Injection Attack
959073 SQL Injection Attack
960014 Proxy access attempt
960018 Invalid character in request
960020 Pragma Header requires Cache-Control Header for HTTP/1.1 requests. Walter: some FP
960022 UNKNOWN
960024 Meta-Character Anomaly Detection Alert - Repetative Non-Word Characters Walter: many FP
960902 UNKNOWN
960913 Invalid request
970007 Zope Information Leakage
970008 Cold Fusion Information Leakage
970010 ISA server existence revealed
970011 File or Directory Names Leakage
970012 Microsoft Office document properties leakage
970016 Cold Fusion source code leakage Walter: some FP but not using this language
970018 IIS installed in default location
970021 WebLogic information disclosure
970903 ASP/JSP source code leakage Walter: some FP but not using this language
973300 Possible XSS Attack Detected - HTML Tag Handler
973301 XSS Attack Detected
973302 XSS Attack Detected
973303 XSS Attack Detected
973304 XSS Attack Detected
973305 XSS Attack Detected
973306 XSS Attack Detected
973307 XSS Attack Detected
973308 XSS Attack Detected
973309 XSS Attack Detected
973310 XSS Attack Detected
973311 XSS Attack Detected
973312 XSS Attack Detected
973313 XSS Attack Detected
973314 XSS Attack Detected
973316 IE XSS Filters - Attack Detected.
973325 IE XSS Filters - Attack Detected.
973327 IE XSS Filters - Attack Detected.
973328 IE XSS Filters - Attack Detected.
973329 IE XSS Filters - Attack Detected.
973330 IE XSS Filters - Attack Detected.
973331 IE XSS Filters - Attack Detected.
973332 IE XSS Filters - Attack Detected.
973333 IE XSS Filters - Attack Detected.
973334 IE XSS Filters - Attack Detected. Walter: many FP in text
973335 IE XSS Filters - Attack Detected. Walter: many FP in text
973347 IE XSS Filters - Attack Detected.
981000 Possibly malicious iframe tag in output Walter: medium FP
981001 Possibly malicious iframe tag in output Walter: medium FP (iframes with display:none)
981003 Malicious iframe+javascript tag in output
981004 Potential Obfuscated Javascript in Output - Excessive fromCharCode Walter: many FP (Wordpress 4.4 inlined emoji javascripts); folinic: Problem solved in WP: https://core.trac.wordpress.org/ticket/35412
981005 Potential Obfuscated Javascript in Output - Eval+Unescape
981006 Potential Obfuscated Javascript in Output - Unescape
981007 Potential Obfuscated Javascript in Output - Heap Spray
981018 UNKNOWN
981022 UNKNOWN
981133 UNKNOWN
981134 UNKNOWN
981136 UNKNOWN
981172 Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded Walter: many FP
981173 Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded Walter: many FP
981177 UNKNOWN
981178 UNKNOWN
981231 SQL Comment Sequence Detected.
981260 SQL Hex Encoding Identified
981300 UNKNOWN
981301 UNKNOWN
981302 UNKNOWN
981303 UNKNOWN
981304 UNKNOWN
981305 UNKNOWN
981306 UNKNOWN
981307 UNKNOWN
981308 UNKNOWN
981309 UNKNOWN
981310 UNKNOWN
981311 UNKNOWN
981312 UNKNOWN
981313 UNKNOWN
981314 UNKNOWN
981315 UNKNOWN
981316 SQL SELECT Statement Anomaly Detection Alert
981317 SQL SELECT Statement Anomaly Detection Alert
990012 Rogue web site crawler

Optional, experimental, slr rules

900048 Identifies Reflected XSS (optional_rules) Walter: could be very interesting candidate but have not used it in production
920021, 920022, 920023 Possible Credit Card Track 1 Data Leakage. (experimental_rules) Walter: could be interesting candidates but have not used it in production
981080, 920020, 920006 Detect CC# in output and block transaction (optional_rules) Walter: could be interesting candidate but have not used it in production
900047, 900048, 981180, 981182 Identifies Stored XSS (optional_rules) Walter: could be somewhat interesting candidate but have not used it in production

Stricter siblings for existing rules

Stricter Siblings are rules that are present in the CRS but could be accompanied by a stricter clone in Paranoia Mode. Adjustments can differ from rule to rule but include higher anomaly ratings or stricter triggers (e.g. regex counters). To prevent masses of false positives, rules can come with additional filters (chained rules) for common use-cases. These can either be included into Paranoia Mode or simply serve as a recommendation.

Note: To avoid a cluttered project main-page, rule proposals are documented in their respective sub-page. When adding new proposals, make sure adding the rules original (2.2.x) ID, a quick description of what changes were made, and, if applicable, which additional filters were added. For every proposed rule change, we will create a Github issue; after discussion a pull request will be created. For brainstorming about CRS rules, see our OWASP ModSecurity rule evaluation framework.

Possible siblings:

981173 : SQL Injection Character Anomaly Usage
981172 : SQL Injection Character Anomaly Usage
981049 : Potential Denial of Service (DoS)
970003 : SQL Error Leakage
960901 : Invalid character in request
958980 : PHP Injection Attack: Variables Found
958979 : PHP Injection Attack: Configuration Directive Found
958977 : PHP Injection Attack: Function Name Found
950907 : Remote Command Execution (RCE) Attempt
950001 : SQL Injection Attack

Project Status

Project Status January 30, 2016

Hello everybody,

It's time to do a status report of our little core rules project.

I am including Franziska Bühler and Walter Hop in this status mail. Both are experienced ModSec sysadmins. Franziska contributed to this first stage, Walter told me he does not have much time, but he was interested in participating at least in the discussions about the rules.

All in all, this is taking more time than anticipated. But we have also done things very throughly than I thought. Which is generally a good thing.

Done so far:

  • Manuel has provided us with a list of rules removed between 2.2.x and 3.0.0rc1
  • I have assembled a list of rules known to trigger false positives frequently in the 2.2.x ruleset, they are thus candidates for the paranoia mode
  • Franziska has looked through the 3.0.0rc1 rules and identified a set of rules which look like good candidates.
  • Noël has sharpened his skills by re-writing 981173 in a way that ignores innocent UUIDs. In my eyes, he found a very elegant solution.
  • With the development of 3.0.0-dev, Chaim unfortunately reused rule ids formerly used with optional and experimental rules. Now this has all been renumbered. I have pointed this out in the mailinglist and had private contact with Chaim where he confirmed the fact - and promised to resolve the issue.

We have not really looked at the disappeared rules and identified those who should be brought back and have not been picked so far. This includes the 2.2.X base_rules, but also the optional, experimental, and huge stock of slr rules. Of these three groups, only the anti-ddos rules have made it into 3.0.0. There are probably more interesting candidates.

If somebody among you wants to look into these, then that would be welcome, but I do not want to have these tasks delay us any further. After all, Old rules can also be brought back in subsequent releases if we see a benefit.

So the next real tasks are:

  • Looking through the list of candidates and cloning-candidates (the latter are those rules we might accompany with a clone with stricter limits in paranoia mode).
  • Defining the exact working of the paranoia mode.

Please sit down and look through the rule lists in the wiki and add remarks with regards to the candidate rules. If you think a rule should be included, if you think an individual rule should not be included etc.

I am also going to invite the people on the mailinglist to take look at the rules as well and add their remarks in the wiki (or respond via mail). This should allow us to nail down the list of rules which will actually be included in the paranoia mode.

As for defining the exact working of the paranoia mode, I guess I need to write down the idea I have in mind and see if it makes sense to you.

Thank you for contributing so far! It is a lot of fun to work in a team!

Christian

Git help

How to Perform Pull Request to CRS v3.0.0-rc1

By Walter Hop

This example assumes that the Github username is 'lifeforms', replace with your own username.

Step 1. Create a Github fork of the original repository

Step 2. Download your Github fork, and checkout the correct branch
git clone [email protected]:lifeforms/owasp-modsecurity-crs.git
cd owasp-modsecurity-crs
git checkout v3.0.0-rc1

Step 3. Add the original repository as upstream, to integrate new changes easily
git remote add upstream [email protected]:SpiderLabs/owasp-modsecurity-crs.git

Step 4. Make sure your forked repo is up to date with changes in the original repository

You need to re-do these steps if somebody else made changes to upstream in the meantime!

git checkout v3.0.0-rc1
git fetch upstream
git merge --ff upstream/v3.0.0-rc1
git push

Step 5. Create a branch for every separate issue you'd like to fix.
git checkout -b myfeature v3.0.0-rc1
git push --set-upstream origin myfeature

Step 6. Make local changes and commit them
git add ...
git commit

Step 7. Push your local changes to your fork at Github
git push

Step 8. Create the pull request

  • a. Browse to your own fork: https://github.com/lifeforms/owasp-modsecurity-crs
  • b. Click "New pull request" button
  • c. In the "base fork", ensure that the correct branch is selected: v3.0.0-rc1
  • d. In the "head fork", pick "myfeature" (if you used branches) or v3.0.0-rc1 (if you didn't)
  • e. Review the content of the pull request (should only contain your commits)
  • f. Press "Create pull request" button

Optional: Updating your pull request

After opening the pull request, people will review it, and probably will make some suggestions for changes before the PR can be accepted. You can keep pushing to your branch, and it will be reflected in the PR.

If in the meantime the code in upstream has drifted, you should re-do step 4 above.

Then, just change some files, commit and push.

git add ...
git commit
git push

See also:

Testing someone else's pull request

To test a PR locally, use the following commands to create a branch for it.

In this example we check out PR #427, creating a branch chaim-headers locally (you can use any name):

git fetch upstream pull/427/head:chaim-headers
git checkout chaim-headers

Pushing to someone else's pull request

In an editable PR, the PR will say something like:

Add more commits by pushing to the ExceptionSupport branch on csanders-git/owasp-modsecurity-crs.

To add to branch ExceptionSupport of user csanders-git, do:

git remote add chaim [email protected]:csanders-git/owasp-modsecurity-crs.git
git fetch chaim
git co ExceptionSupport 
git add ...
git push

Purging a file completely from Git

Delete the file from the repo:

git rm file
git commit
git push

Download BFG Repo-Cleaner

Make a NEW mirror:

git clone --mirror [email protected]:SpiderLabs/owasp-modsecurity-crs.git
cd owasp-modsecurity-crs.git

Purge the file:

java -jar ~/Downloads/bfg-*.jar --delete-files example.png
git reflog expire --expire=now --all && git gc --prune=now --aggressive
git push -f

E-mail all users to REMOVE their copies of the repositories, and do a FRESH CLONE of the repo. If a user pulls again, the divergent histories will be merged, and there is a big risk that the dirty history (containing the purged file) will return again when they push later.

See also: BFG Repo-Cleaner