Difference between revisions of "OWASP ModSec CRS Paranoia Mode"

Latest revision as of 17:45, 16 March 2018

Abstract

This is a page about the development of a paranoia mode aka bringing back the rules that used to yield a high number of false positives. This little project is aimed at inclusion into the 3.0.0 release of the OWASP ModSecurity Core Rules, where some rules have been removed in order to reduce the number of false positives with vanilla installations.

FIXME: Detailed description

Back to the OWASP ModSecurity Core Rules Set.

Sub-Project Infos

• Status: active (January 2016)
• Schedule: DONE (we're all done and Paranoia Mode made it into the CRS3 release.
• Who: Christian Folini (dune73), Noël Zindel (zino), Franziska Bühler (franziskabuehler), Manuel Leos (Spartan), Walter Hop (lifeforms)
• Documentation: Here on the OWASP Wiki / Mechanics Proposal
• Discussion / Archive: [email protected] / archive: http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/
• Github Link v3.0.0-rc1 (our base): https://github.com/SpiderLabs/owasp-modsecurity-crs/tree/v3.0.0-rc1
• Github Link paranoia-mode: https://github.com/dune73/owasp-modsecurity-crs/tree/paranoia-mode
• Final Pull Request #1: Add paranoia mode mechanics: https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/292 MERGED
• Final Pull Request #2: Move first rules to paranoia mode: https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/300 MERGED
• Final Pull Request #3: Add 2.2.X rules to paranoia mode: https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/308 MERGED
• Final Pull Request #4: Add stricter siblings: FIXME

Tasks

Open Tasks

Please define state as follows: new, assigned, waiting, closed. When a task is closed, it is moved to the seperate closed tasks table below.

Closed Tasks

Rules

Paranoia Mode Candidates

The 3.0.0-rc1 has all rules renumbered. Existing numbering was fairly crazy and the new numbering follows the numbering scheme of the rules files (-> 9<2-digit-rulefile><3-digit-id>) A mapping table exists [IdNumbering.csv] We need to make sure, we do not mess things up, so let's add both IDs to the table, the old one and the new one.

Please set status as follows : confirmed,candidate, cloning-confirmed,cloning-candidate, unsure, dropped.

• 'cloning-confirmed', 'cloning-candidates' are rules, that could be cloned into an even stricter variant with a stricter limit in a higher paranoia setting.
• If dropped, please provide reasoning in the remarks.

Rules from 2.2.X, missing in 3.0.0-rc1

It looks as if only the base_rules made it into 3.0.0. In fact there are a few rule ids know from the optional and experimental rule folders in 2.2.X, but it is more likely, these are new 3.0.0 rules reusing old rule ids as the rules (regexes and msg) do not match at all.

When trying to generate the list below, be aware that the rule ids have been renumbered between 3.0.0-dev and 3.0.0-rc1. IdNumbering.csv in your friend.

Base rules

Optional, experimental, slr rules

Stricter siblings for existing rules

Stricter Siblings are rules that are present in the CRS but could be accompanied by a stricter clone in Paranoia Mode. Adjustments can differ from rule to rule but include higher anomaly ratings or stricter triggers (e.g. regex counters). To prevent masses of false positives, rules can come with additional filters (chained rules) for common use-cases. These can either be included into Paranoia Mode or simply serve as a recommendation.

Note: To avoid a cluttered project main-page, rule proposals are documented in their respective sub-page. When adding new proposals, make sure adding the rules original (2.2.x) ID, a quick description of what changes were made, and, if applicable, which additional filters were added. For every proposed rule change, we will create a Github issue; after discussion a pull request will be created. For brainstorming about CRS rules, see our OWASP ModSecurity rule evaluation framework.

Possible siblings:

981173 : SQL Injection Character Anomaly Usage
981172 : SQL Injection Character Anomaly Usage
981049 : Potential Denial of Service (DoS)
970003 : SQL Error Leakage
960901 : Invalid character in request
958980 : PHP Injection Attack: Variables Found
958979 : PHP Injection Attack: Configuration Directive Found
958977 : PHP Injection Attack: Function Name Found
950907 : Remote Command Execution (RCE) Attempt
950001 : SQL Injection Attack

Project Status

Project Status January 30, 2016

Hello everybody,

It's time to do a status report of our little core rules project.

I am including Franziska Bühler and Walter Hop in this status mail. Both are experienced ModSec sysadmins. Franziska contributed to this first stage, Walter told me he does not have much time, but he was interested in participating at least in the discussions about the rules.

All in all, this is taking more time than anticipated. But we have also done things very throughly than I thought. Which is generally a good thing.

Done so far:

Git help

How to Perform Pull Request to CRS v3.0.0-rc1

By Walter Hop

This example assumes that the Github username is 'lifeforms', replace with your own username.

Step 1. Create a Github fork of the original repository

• a. Browse to CRS: https://github.com/SpiderLabs/owasp-modsecurity-crs
• b. Click "Fork" button in the top right

Step 2. Download your Github fork, and checkout the correct branch
git clone [email protected]:lifeforms/owasp-modsecurity-crs.git
cd owasp-modsecurity-crs
git checkout v3.0.0-rc1


Step 3. Add the original repository as upstream, to integrate new changes easily
git remote add upstream [email protected]:SpiderLabs/owasp-modsecurity-crs.git

Step 4. Make sure your forked repo is up to date with changes in the original repository

You need to re-do these steps if somebody else made changes to upstream in the meantime!

git checkout v3.0.0-rc1
git fetch upstream
git merge --ff upstream/v3.0.0-rc1
git push


Step 5. Create a branch for every separate issue you'd like to fix.
git checkout -b myfeature v3.0.0-rc1
git push --set-upstream origin myfeature


Step 6. Make local changes and commit them
git add ...
git commit


Step 7. Push your local changes to your fork at Github
git push


Step 8. Create the pull request

• a. Browse to your own fork: https://github.com/lifeforms/owasp-modsecurity-crs
• b. Click "New pull request" button
• c. In the "base fork", ensure that the correct branch is selected: v3.0.0-rc1
• d. In the "head fork", pick "myfeature" (if you used branches) or v3.0.0-rc1 (if you didn't)
• e. Review the content of the pull request (should only contain your commits)
• f. Press "Create pull request" button

Optional: Updating your pull request

After opening the pull request, people will review it, and probably will make some suggestions for changes before the PR can be accepted. You can keep pushing to your branch, and it will be reflected in the PR.

If in the meantime the code in upstream has drifted, you should re-do step 4 above.

Then, just change some files, commit and push.

git add ...
git commit
git push


See also:

• https://help.github.com/articles/fork-a-repo/
• https://help.github.com/articles/syncing-a-fork/
• https://help.github.com/articles/using-pull-requests/

Testing someone else's pull request

To test a PR locally, use the following commands to create a branch for it.

In this example we check out PR #427, creating a branch chaim-headers locally (you can use any name):

git fetch upstream pull/427/head:chaim-headers
git checkout chaim-headers

Pushing to someone else's pull request

In an editable PR, the PR will say something like:

Add more commits by pushing to the ExceptionSupport branch on csanders-git/owasp-modsecurity-crs.

To add to branch ExceptionSupport of user csanders-git, do:

git remote add chaim [email protected]:csanders-git/owasp-modsecurity-crs.git
git fetch chaim
git co ExceptionSupport 
git add ...
git push

Purging a file completely from Git

Delete the file from the repo:

git rm file
git commit
git push

Download BFG Repo-Cleaner

Make a NEW mirror:

git clone --mirror [email protected]:SpiderLabs/owasp-modsecurity-crs.git
cd owasp-modsecurity-crs.git

Purge the file:

java -jar ~/Downloads/bfg-*.jar --delete-files example.png
git reflog expire --expire=now --all && git gc --prune=now --aggressive
git push -f

E-mail all users to REMOVE their copies of the repositories, and do a FRESH CLONE of the repo. If a user pulls again, the divergent histories will be merged, and there is a big risk that the dirty history (containing the purged file) will return again when they push later.

See also: BFG Repo-Cleaner