=Main=

OWASP Web Mapper is a pure Ruby library for the Internet web application discovery and tracking. Project is currently hosted under GitHub at:

* https://github.com/yangsec888/wmap/

<div style="width:100%;height:160px;border:0,margin:0;overflow: hidden;">[[File:OWASP_Project_Header.jpg|link=]]</div>


{| style="padding: 0;margin:0;margin-top:10px;text-align:left;" |-
| valign="top" style="border-right: 1px dotted gray;padding-right:25px;" |

==OWASP Web Mapper Project==

This project is designed for the web application asset discovery and tracking. It was originally developed to to cover the gaps of Veracode's Discovery service. Over the time it grows to be a more capable and complete replacement (IMHO).

==Description==

A pure Ruby library for the web application asset discovery and tracking. The tool is useful when you're handling a larger size organization with multiple Internet domain and network registered under the name, where manual discovery and tracking prove to be both time-consuming and problematic, if not impossible. The program will help you quickly identify the 'unknown' web application asset, and keep track of them automatically for you as a repeatable process. If you are serious about your organization's web exposure, this might be the perfect footprinting tool you're looking for.

Built as an open source project, the source code is both free and robust. You're welcome to keep building on top of the current code base, or include it as part of your larger project distribution.

==Licensing==

'''The OWASP Web Mapper Project is free to use. In fact it is encouraged!

The OWASP Security Principles are licensed under the [https://github.com/yangsec888/wmap/blob/master/LICENSE.txt Apache 2.0 license], so you can copy, distribute and transmit the work, and you can adapt it, and use it commercially, but all provided that you attribute the work and if you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one.


| valign="top" style="padding-left:25px;width:200px;border-right: 1px dotted gray;padding-right:25px;" |

== What is OWASP Web Mapper Project? ==

The goal is to help you better identify, and keep track of the web application asset under your watch.
Ideally we could document various reverse-engineering techniques using by this project and publish it through the OWASP Press. Of course, it will always remain freely available, and any money collected will go directly into the project and to the OWASP Foundation.

== Presentation ==

TBD

== Project Leader ==

* [https://www.owasp.org/index.php/User:Yang_Li Yang Li]

== Related Projects ==

* [[OWASP_Zed_Attack_Proxy_Project]]

== Openhub ==

* [https://www.openhub.net/orgs/OWASP OWASP Project Openhub]


| valign="top" style="padding-left:25px;width:200px;" |

== Quick Download ==



The home of the OWASP Web Mapper is on [https://github.com/yangsec888/wmap GitHub.] You are encouraged to fork, edit and push your changes back to the project through git or edit the project directly on GitHub.

However, if you like you may also download the master repository from the following links:
* [https://raw.githubusercontent.com/yangsec888/wmap/master/wmap-2.4.2.gem .gem file.]
* [https://github.com/yangsec888/wmap/zipball/master .zip file.]
* [https://github.com/yangsec888/wmap/tarball/master .tgz file.]

Project Code and Documents:https://github.com/yangsec888/wmap

== News and Events ==



* [August 1 2015] OWASP Web Mapper Project created.

== In Print ==

I'm working on project document. But it's far from become a book at this moment. Instead, please refer to the project site for project document.

==Classifications==

{| width="200" cellpadding="2"
|-
| align="center" valign="top" width="50%" rowspan="2"| [[File:New projects.png|100px|link=https://www.owasp.org/index.php/OWASP_Project_Stages#tab=Incubator_Projects]]
| align="center" valign="top" width="50%"| [[File:Owasp-builders-small.png|link=]]
|-
| align="center" valign="top" width="50%"| [[File:Owasp-defenders-small.png|link=]]
|-
| colspan="2" align="center" | [[File:Cc-button-y-sa-small.png|link=http://creativecommons.org/licenses/by-sa/3.0/]]
|-
| colspan="2" align="center" | [[File:Project_Type_Files_DOC.jpg|link=]]
|}

|}

=FAQs=

==Installation==
# The easiest way to install WebMapper is by using Ruby Gems. Download the latest gem 'wmap-x.x.x.gem' into the local file system. Then install it from your shell environment:
$ gem install wmap-x.x.x.gem --no-rdoc

== Dependency==
You need the Ruby 1.9.2 or above in order to use this program. In addition, I developed and tested the code in a Ubantu Linux VM with Ruby 1.9.3.
* You need to setup Ruby 1.9.x environment. In my test environment, I was able to set it up with RVM. Please refer to this page for more installation information: http://www.ruby-lang.org/en/downloads/

* In addition, the following Ruby GEM dependency are needed by different components of this software:
:# require "uri"
:# require "open-uri"
:# require "open_uri_redirections"
:#require "nokogiri"
:#require "net/http"
:#require "httpclient"
:#require "net/ping"
:#require "netaddr"
:#require "socket"
:#require "openssl"
:#require "whois"
:#require "resolv"
:#require "geoip"
:#require "parallel"
:#require "dnsruby"

* Install dependency gem "uri" example
$ gem install uri

* Ruby-whois gem patches
This software also depends on a patched version of Ruby gem ruby-whois-2.7.0 (http://www.ruby-whois.org/) for the domain whois lookup feature. For better result, you could manually add the patches into your local whois gem installation directory as shown below:
$ cp whois_patches/* [Your_ruby_whois_gem_path]/whois/lib/whois/record/parser/
Or you can directly download the branched whois gem from this repository - https://github.com/yangsec888/whois

== Before Using This Program ==
You need to define a scope for the program to run successful. The scope includes both your legitimate Internet domain, and your public
network block in the CIDR format.

To add your Internet domain into the scope, use the build-in shell command below:
$ trust XYZ.COM

To add your public network block into the scope:
$ trust x.x.x.x/x

== Automatic Discovery and Tracking ==

$ wmap <seed file | target host | target url | target IP or network cidr>

The above executable is intelligent enough to take argument as either a seed file, or a string such as a host, an IP, a network block, or a URL. The new discoveries will be automatically tracked in the data file 'lib/wmap/data/target_sites'.
Note: seed file - mix of url, cidr and domain seed, one entry per line.
url seed - known URL(s) for further discovery via the web crawler.
cidr seed - list of known network blocks, for discovering web service via port scanning; it is also used to validate if the web service has a known IP (internal hosted).
domain seed - validated internet domain to be used for DNS record brute-forcing; it is also used to validate the ownership of found web service.

== Dump Out Discovery Database ==
You can dump out the program output by using the build-in utility 'wdump' as shown below:
$ wdump [output file name from you]

The above executable will dump out the discovery database into a single file as program output. Currently, the supported file format is Comma-separated Value (.csv) and Extensible Markup Language (.xml)

== More Usage Cases ==
There are more examples under the 'demos' folder of this package. The examples show how to use the 'wmap' API to get your job done easily. Please check out the code - they should be easy and straightforward to be understood.

== More document(s):
The software comes with the Ruby doc during your installation as shown above. For your convenience, the Ruby doc is also distributed with this software. You can navigate to the 'doc' folder, and click the 'index.html' to open the start page in your favorite browser.

If you need additional documentation / information other than this README file and the Ruby document package, please be patient - as I'm still working on it :)

==How can I participate in your project?==
All you have to do is make the Project Leader's aware of your available time to contribute to the project. It is also important to let the Leader's know how you would like to contribute and pitch in to help the project meet it's goals and milestones. There are many different ways you can contribute to an OWASP Project, but communication with the leads is key.

==If I am not a programmer can I participate in your project?==
Yes, you can certainly participate in the project if you are not a programmer or technical. The project needs different skills and expertise and different times during its development. Currently, we are looking for researchers, writers, graphic designers, and a project administrator.

= Acknowledgements =

==Contributors==

The OWASP Web Mapper Project is originally developed by Yang Li. A live update of project [https://github.com/yangsec888/wmap/blob/master/CREDIT.txt contributors can be found here].

The first contributors to the project were:

* [https://www.owasp.org/index.php/User:Yang_Li Yang Li]

= Road Map and Getting Involved =


<span>
Details TBD. Please refer to TODO file at code repository for additional information:
*https://github.com/yangsec888/wmap/blob/master/TODO
</span>

Involvement in the development and promotion of the OWASP Web Mapper Project is actively encouraged!
You do not have to be a developer in order to contribute.
Some of the ways you can help:
* Help QA the project.
* Project administration support.
* Wiki editing support.
* Writing support for the book.


__NOTOC__ <headertabs />

[[Category:OWASP Project]] [[Category:OWASP_Builders]] [[Category:OWASP_Defenders]] [[Category:OWASP_Document]]

OWASP Web Mapper Project

2015-08-12T18:05:43Z

Yang Li:

=Main=

OWASP Web Mapper is a pure Ruby library for the Internet web application discovery and tracking. Project is currently hosted under GitHub at:

* https://github.com/yangsec888/wmap/

<div style="width:100%;height:160px;border:0,margin:0;overflow: hidden;">[[File:OWASP_Project_Header.jpg|link=]]</div>


{| style="padding: 0;margin:0;margin-top:10px;text-align:left;" |-
| valign="top" style="border-right: 1px dotted gray;padding-right:25px;" |

==OWASP Web Mapper Project==

This project is designed for the web application asset discovery and tracking. It was originally developed to to cover the gaps of Veracode's Discovery service. Over the time it grows to be a more capable and complete replacement (IMHO).

==Description==

A pure Ruby library for the web application asset discovery and tracking. The tool is useful when you're handling a larger size organization with multiple Internet domain and network registered under the name, where manual discovery and tracking prove to be both time-consuming and problematic, if not impossible. The program will help you quickly identify the 'unknown' web application asset, and keep track of them automatically for you as a repeatable process. If you are serious about your organization's web exposure, this might be the perfect footprinting tool you're looking for.

Built as an open source project, the source code is both free and robust. You're welcome to keep building on top of the current code base, or include it as part of your larger project distribution.

==Licensing==

'''The OWASP Web Mapper Project is free to use. In fact it is encouraged!

The OWASP Security Principles are licensed under the [https://github.com/yangsec888/wmap/blob/master/LICENSE.txt Apache 2.0 license], so you can copy, distribute and transmit the work, and you can adapt it, and use it commercially, but all provided that you attribute the work and if you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one.


| valign="top" style="padding-left:25px;width:200px;border-right: 1px dotted gray;padding-right:25px;" |

== What is OWASP Web Mapper Project? ==

The goal is to help you better identify, and keep track of the web application asset under your watch.
Ideally we could document various reverse-engineering techniques using by this project and publish it through the OWASP Press. Of course, it will always remain freely available, and any money collected will go directly into the project and to the OWASP Foundation.

== Presentation ==

TBD

== Project Leader ==

* [https://www.owasp.org/index.php/User:Yang_Li Yang Li]

== Related Projects ==

* [[OWASP_Zed_Attack_Proxy_Project]]

== Openhub ==

* [https://www.openhub.net/orgs/OWASP OWASP Project Openhub]


| valign="top" style="padding-left:25px;width:200px;" |

== Quick Download ==



The home of the OWASP Web Mapper is on [https://github.com/yangsec888/wmap GitHub.] You are encouraged to fork, edit and push your changes back to the project through git or edit the project directly on GitHub.

However, if you like you may also download the master repository from the following links:
* [https://raw.githubusercontent.com/yangsec888/wmap/master/wmap-2.4.2.gem .gem file.]
* [https://github.com/yangsec888/wmap/zipball/master .zip file.]
* [https://github.com/yangsec888/wmap/tarball/master .tgz file.]

Project Code and Documents:https://github.com/yangsec888/wmap

== News and Events ==



* [August 1 2015] OWASP Web Mapper Project created.

== In Print ==

I'm working on project document. But it's far from become a book at this moment. Instead, please refer to the project site for project document.

==Classifications==

{| width="200" cellpadding="2"
|-
| align="center" valign="top" width="50%" rowspan="2"| [[File:New projects.png|100px|link=https://www.owasp.org/index.php/OWASP_Project_Stages#tab=Incubator_Projects]]
| align="center" valign="top" width="50%"| [[File:Owasp-builders-small.png|link=]]
|-
| align="center" valign="top" width="50%"| [[File:Owasp-defenders-small.png|link=]]
|-
| colspan="2" align="center" | [[File:Cc-button-y-sa-small.png|link=http://creativecommons.org/licenses/by-sa/3.0/]]
|-
| colspan="2" align="center" | [[File:Project_Type_Files_DOC.jpg|link=]]
|}

|}

=FAQs=

==Installation==
# The easiest way to install WebMapper is by using Ruby Gems. Download the latest gem 'wmap-x.x.x.gem' into the local file system. Then install it from your shell environment:
$ gem install wmap-x.x.x.gem --no-rdoc

== Dependency==
You need the Ruby 1.9.2 or above in order to use this program. In addition, I developed and tested the code in a Ubantu Linux VM with Ruby 1.9.3.
* You need to setup Ruby 1.9.x environment. In my test environment, I was able to set it up with RVM. Please refer to this page for more installation information: http://www.ruby-lang.org/en/downloads/

* In addition, the following Ruby GEM dependency are needed by different components of this software:
:# require "uri"
:# require "open-uri"
:# require "open_uri_redirections"
:#require "nokogiri"
:#require "net/http"
:#require "httpclient"
:#require "net/ping"
:#require "netaddr"
:#require "socket"
:#require "openssl"
:#require "whois"
:#require "resolv"
:#require "geoip"
:#require "parallel"
:#require "dnsruby"

* Install dependency gem "uri" example
$ gem install uri

* Ruby-whois gem patches
This software also depends on a patched version of Ruby gem ruby-whois-2.7.0 (http://www.ruby-whois.org/) for the domain whois lookup feature. For better result, you could manually add the patches into your local whois gem installation directory as shown below:
$ cp whois_patches/* [Your_ruby_whois_gem_path]/whois/lib/whois/record/parser/
Or you can directly download the branched whois gem from this repository - https://github.com/yangsec888/whois

== Before Using This Program ==
You need to define a scope for the program to run successful. The scope includes both your legitimate Internet domain, and your public
network block in the CIDR format.

To add your Internet domain into the scope, use the build-in shell command below:
$ trust XYZ.COM

To add your public network block into the scope:
$ trust x.x.x.x/x

==How can I participate in your project?==
All you have to do is make the Project Leader's aware of your available time to contribute to the project. It is also important to let the Leader's know how you would like to contribute and pitch in to help the project meet it's goals and milestones. There are many different ways you can contribute to an OWASP Project, but communication with the leads is key.

==If I am not a programmer can I participate in your project?==
Yes, you can certainly participate in the project if you are not a programmer or technical. The project needs different skills and expertise and different times during its development. Currently, we are looking for researchers, writers, graphic designers, and a project administrator.

= Acknowledgements =

==Contributors==

The OWASP Web Mapper Project is originally developed by Yang Li. A live update of project [https://github.com/yangsec888/wmap/blob/master/CREDIT.txt contributors can be found here].

The first contributors to the project were:

* [https://www.owasp.org/index.php/User:Yang_Li Yang Li]

= Road Map and Getting Involved =


<span>
Details TBD. Please refer to TODO file at code repository for additional information:
*https://github.com/yangsec888/wmap/blob/master/TODO
</span>

Involvement in the development and promotion of the OWASP Web Mapper Project is actively encouraged!
You do not have to be a developer in order to contribute.
Some of the ways you can help:
* Help QA the project.
* Project administration support.
* Wiki editing support.
* Writing support for the book.


__NOTOC__ <headertabs />

[[Category:OWASP Project]] [[Category:OWASP_Builders]] [[Category:OWASP_Defenders]] [[Category:OWASP_Document]]

OWASP Web Mapper Project

2015-08-12T18:04:21Z

Yang Li: /* Dependency */