Difference between revisions of "OWASP Web Mapper Project"

Installation

1. The easiest way to install WebMapper is by using Ruby Gems. Download the latest gem 'wmap-x.x.x.gem' into the local file system. Then install it from your shell environment:

$ gem install wmap-x.x.x.gem --no-rdoc

Dependency

You need the Ruby 1.9.2 or above in order to use this program. In addition, I developed and tested the code in a Ubantu Linux VM with Ruby 1.9.3.

• You need to setup Ruby 1.9.x environment. In my test environment, I was able to set it up with RVM. Please refer to this page for more installation information: http://www.ruby-lang.org/en/downloads/

• In addition, the following Ruby GEM dependency are needed by different components of this software:

1. require "uri"
2. require "open-uri"
3. require "open_uri_redirections"
4. require "nokogiri"
5. require "net/http"
6. require "httpclient"
7. require "net/ping"
8. require "netaddr"
9. require "socket"
10. require "openssl"
11. require "whois"
12. require "resolv"
13. require "geoip"
14. require "parallel"
15. require "dnsruby"

• Install dependency gem "uri" example

 $ gem install uri

• Ruby-whois gem patches

This software also depends on a patched version of Ruby gem ruby-whois-2.7.0 (http://www.ruby-whois.org/) for the domain whois lookup feature. For better result, you could manually add the patches into your local whois gem installation directory as shown below:

 $ cp whois_patches/* [Your_ruby_whois_gem_path]/whois/lib/whois/record/parser/

Or you can directly download the branched whois gem from this repository - https://github.com/yangsec888/whois

Before Using This Program

You need to define a scope for the program to run successful. The scope includes both your legitimate Internet domain, and your public network block in the CIDR format.

To add your Internet domain into the scope, use the build-in shell command below:

 $ trust XYZ.COM

To add your public network block into the scope:

 $ trust x.x.x.x/x

Automatic Discovery and Tracking

 $ wmap <seed file | target host | target url | target IP or network cidr>

The above utility is intelligent enough to take argument as either a seed file, or a string such as a host, an IP, a network block, or a URL. The new discoveries will be automatically tracked in the data file 'lib/wmap/data/target_sites'.

 Note: seed file - mix of url, cidr and domain seed, one entry per line.

url seed - known URL(s) for further discovery via the web crawler. cidr seed - list of known network blocks, for discovering web service via port scanning; it is also used to validate if the web service has a known IP (internal hosted). domain seed - validated internet domain to be used for DNS record brute-forcing; it is also used to validate the ownership of found web service.

Dump Out Discovery Database

You can dump out the program output by using the build-in utility 'wdump' as shown below:

 $ wdump [output file name from you]

The above utility will dump out the discovery database into a single file as program output. Currently, the supported file format is Comma-separated Value (.csv) and Extensible Markup Language (.xml)

Other Build-in Utilities

You may need to update your sset repository from time to time. For this purpose, ‘wmap’ provide three utilities ‘wcheck’, ‘wadd’ and ‘wdel’ to perform the tasks for you. For example, you can check out the status of a site, as shown below:

 $ wcheck https://www.owasp.org/
 {"ip"=>"104.130.192.89", "port"=>443, "url"=>"https://www.owasp.org/", "code"=>301, "redirection"=>"https://www.owasp.org/index.php/Main_Page", "md5"=>"d41d8cd98f00b204e9800998ecf8427e", "server"=>"Apache", "timestamp"=>2015-09-02 13:32:31 -0400, "status"=>"ext_hosted"}

You may want to use 'wadd' utility to add it, as shown below:

 $ wadd https://www.owasp.org/

You may want to use 'wdel' utility to delete it, as shown below:

 $ wdel https://www.owasp.org/

More Usage Examples

There are also some code examples under the 'demos' folder of this package. The examples show how to use the 'wmap' API to get your job done easily. Please check out the code - they should be easy and straightforward to be understood.

More document(s)

The software comes with the Ruby doc during your installation as shown above. For your convenience, the Ruby doc is also distributed with the software. You can navigate to the 'doc' folder of your local installation, and click the 'index.html' to open the start page in your favorite browser. You can also download the wmap-x.x.x.rdoc.zip documentation package alone from the hosting site, unzip and open the doc/index.html in your browser.

If you need additional documentation / information other than this README file and the Ruby document package, please be patient - as I'm still working on it.

How do I report the bugs, or maybe require some new features?

Contact the developer Yang Li directly at his email 'yang.li@owasp.org'.

How can I participate in your project?

All you have to do is make the Project Leader's aware of your available time to contribute to the project. It is also important to let the Leader's know how you would like to contribute and pitch in to help the project meet it's goals and milestones. There are many different ways you can contribute to an OWASP Project, but communication with the leads is key.

If I am not a programmer can I participate in your project?

Yes, you can certainly participate in the project if you are not a programmer or technical. The project needs different skills and expertise and different times during its development. Currently, we are looking for QA tester, front-end builder, document writers, graphic designers, and a project administrator.

Contributors

The OWASP Web Mapper Project is originally developed by Yang Li. A live update of project contributors can be found here.

The first contributors to the project are:

• Yang Li
• [Your name here]

Please refer to TODO file at code repository for additional information:

• https://github.com/yangsec888/wmap/blob/master/TODO

Involvement in the development and promotion of the OWASP Web Mapper Project is encouraged! You do not have to be a developer in order to contribute. Some of the ways you can get involved and contribute:

• Join the project Mailing List.
• Help QA test the project.
• Help build a user-friendly front-end.
• Project administration support.
• Wiki editing support.
• Writing support for the book.

If you can help, please sign up a GITHUB account, then notify me.