This site is the archived OWASP Foundation Wiki and is no longer accepting Account Requests.
To view the new OWASP Foundation website, please visit https://owasp.org

Difference between revisions of "OWASP Assimilation Project"

From OWASP
Jump to: navigation, search
(Roadmap updates.)
(Minimum Viable Product)
 
(25 intermediate revisions by the same user not shown)
Line 8: Line 8:
  
 
==OWASP Assimilation Project Summary==
 
==OWASP Assimilation Project Summary==
Many people compare securing systems against attackers as a form of warfare. In ''The Art of War'', Sun Tzu said "If you know your enemies and know yourself, you will not be imperiled in a hundred battles". The Assimilation helps you know yourself - your systems, networks, configurations in great detail, and then keeps it all that information ''continually'' up to date in a graph-based Configuration Management Database.
+
 
 +
Many people compare securing systems against attackers as a form of warfare. In ''The Art of War'', Sun Tzu said [https://en.wikiquote.org/wiki/Sun_Tzu#/media/File:Enchoen27n3200.jpg "If you know your enemies and know yourself, you will not be imperiled in a hundred battles"]. The Assimilation software helps you know yourself - your systems, networks, configurations in great detail, and then keeps it all that information ''continually'' up to date in a graph-based Configuration Management Database. This information is useful regardless of your threat model.
 +
 
 +
We then leverage this knowledge to compare your systems against hardening best practices, to validate checksums of files, look for vulnerable versions of packages, and help you triage your way to better security.
  
 
==Description==
 
==Description==
The Assimilation Project tracks many aspects of system configuration and security and compares them against best practices in quasi-real-time.
+
The Assimilation Project tracks many aspects of system configuration and security and compares them against best practices in near-real-time.
  
 
Here are a few of the kinds of things we track for you:
 
Here are a few of the kinds of things we track for you:
Line 23: Line 26:
 
This is all done in a [http://assimilationsystems.com/2015/04/14/scalability-from-doing-nothing/ highly scalable] way which cannot set off network security alarms and requires minimal human configuration.
 
This is all done in a [http://assimilationsystems.com/2015/04/14/scalability-from-doing-nothing/ highly scalable] way which cannot set off network security alarms and requires minimal human configuration.
  
In addition, we continually evaluate system configurations against best practices from the [http://ITBestPractices.info  IT Best Practices] project and compute risk scores for servers based on how they compare to security best practices. Since everything is stored in the [http://neo4j.org  Neo4J] graph database, visualizations of things like your [http://assimilationsystems.com/2016/02/22/attack-surface/ attack surface] are natural and straightforward.
+
In addition, we continually evaluate system configurations against best practices from the [http://ITBestPractices.info  IT Best Practices] project and compute risk scores for servers based on how they compare to security best practices, and evaluations of what areas and systems are at greater risk. Since everything is stored in the [http://neo4j.org  Neo4J] graph database, visualizations of things like your [http://assimilationsystems.com/2016/02/22/attack-surface/ attack surface] are natural and straightforward.
 +
 
  
 
The project includes [http://assimilationsystems.com/2015/12/07/assimilation-event-api-overview/ event APIs] and [http://assimilationsystems.com/2014/04/02/new-command-line-queries-in-the-assimilation-software/ canned queries].
 
The project includes [http://assimilationsystems.com/2015/12/07/assimilation-event-api-overview/ event APIs] and [http://assimilationsystems.com/2014/04/02/new-command-line-queries-in-the-assimilation-software/ canned queries].
Line 50: Line 54:
  
 
[http://assimilationsystems.com/category/videos/ Talk Videos]
 
[http://assimilationsystems.com/category/videos/ Talk Videos]
 +
 +
[http://assimilationsystems.com/category/getting-started/ Getting Started articles]
 +
 +
[http://assimilationsystems.com/category/how-to/ How-To articles]
 +
 +
[http://assimilationsystems.com/category/blog/ Blog]
  
 
== Project Leader ==
 
== Project Leader ==
Line 60: Line 70:
 
* [[OWASP_Code_Project_Template]]
 
* [[OWASP_Code_Project_Template]]
 
* [[OWASP_Documentation_Project_Template]]
 
* [[OWASP_Documentation_Project_Template]]
 +
* External project: [http://ITBestPractices.info/ IT Best Practices project] - provides the definitions of the best practices which we use to evaluate your systems.
  
 
==Classifications==
 
==Classifications==
Line 78: Line 89:
  
 
== News and Events ==
 
== News and Events ==
<span style="color:#ff0000">
+
* Project [http://assimilationsystems.com/assimevents/ events page].
This is where you can provide project updates, links to any events like conference presentations, Project Leader interviews, case studies on successful project implementations, and articles written about your project.  
+
* [http://lists.community.tummy.com/cgi-bin/mailman/listinfo/assimilation mailing list]
</span>
+
* IRC: #assimilation on irc.freenode.net
* [12 Feb 2013] Support for Spanish is now available with this release.
 
* [11 Jan 2014] The 1.0 stable version has been released! Thanks everyone for your feedback and code fixes that made this happen!
 
* [18 Dec 2013] 1.0 Release Candidate is available for download. This release provides final bug fixes and product stabilization. Any feedback (good or bad) in the next few weeks would be greatly appreciated.
 
* [20 Nov 2013] 1.0 Beta 2 Release is available for download. This release offers several bug fixes, a few performance improvements, and addressed all outstanding issues from a security audit of the code.
 
* [30 Sep 2013] 1.0 Beta 1 Release is available for download. This release offers the first version with all of the functionality for a minimum viable product.    
 
  
 
|}
 
|}
Line 105: Line 111:
 
==Contributors==
 
==Contributors==
  
<!-- Instructions are in RED and should be removed from your document by deleting the text with the span tags.-->
 
<span style="color:#ff0000">
 
The success of OWASP is due to a community of enthusiasts and contributors that work to make our projects great. This is also true for the success of your project.
 
Be sure to give credit where credit is due, no matter how small! This should be a brief list of the most amazing people involved in your project.
 
Be sure to provide a link to a complete list of all the amazing people in your project's community as well.
 
</span>
 
  
The OWASP Tool Project Template is developed by a worldwide team of volunteers. A live update of project  [https://github.com/OWASP/Security-Principles/graphs/contributors contributors is found here].  
+
 
 +
The OWASP Assimilation Project is developed by a worldwide team of volunteers.
  
 
The first contributors to the project were:
 
The first contributors to the project were:
  
* [https://www.owasp.org/index.php/User:Clerkendweller Colin Watson] who created the OWASP Cornucopia project that the template was derived from
+
* Alan Robertson
* [https://www.owasp.org/index.php/User:Chuck_Cooper Chuck Cooper] who edited the template to convert it from a documentation project to a Tool Project Template
+
* Carrie Oswald
* '''YOUR NAME BELONGS HERE AND YOU SHOULD REMOVE THE PRIOR 3 NAMES'''
+
* Dave Quigley
 +
* Roger Massey
  
 
= Road Map and Getting Involved =
 
= Road Map and Getting Involved =
Line 131: Line 133:
 
* Many more discovery agents. These turn out to be typically pretty simple, and can be written with very little knowledge of the project internals. They have a very shallow learning curve to get started.
 
* Many more discovery agents. These turn out to be typically pretty simple, and can be written with very little knowledge of the project internals. They have a very shallow learning curve to get started.
  
There is also a page that describes the future thinking as of August 2015 on [http://assimilationsystems.com/2015/08/10/cybersecurity-roadmap-2015/ our blog]. This does not include any thinking about security scoring.
+
There is also a page that describes the future thinking as of August 2016 [http://assimilationsystems.com/2016/08/06/assimilation-2016-security-roadmap/ on our latest roadmap blog post].
  
  
Line 160: Line 162:
  
 
===Localization===
 
===Localization===
There is significant opportunity for localization in our sister [http://ITBestPractices.info IT BestPractices] project, and a some in our [https://github.com/assimilation/assimilation-official/tree/master/queries canned queries]. All strings in the project are UTF-8.
+
There is significant opportunity for localization in our sister [http://ITBestPractices.info IT BestPractices] project, and some in our [https://github.com/assimilation/assimilation-official/tree/master/queries canned queries]. All strings in the project are UTF-8.
  
 
===Testing===
 
===Testing===
We have need both for human testers - people to do trials and provide feedback, and people to enhance our automated testing environment. Most of the automated testing is in Python, with some in the shell, and C.
+
We have need both for human testers - people to do trials and provide feedback, and people to enhance our automated testing environment. Most of the automated testing is in Python, with some in the shell, and C. We have a unique and powerful system-level testing methodology which was [http://assimilationsystems.com/2016/05/24/testing-distributed-systems-with-fuzzy-monkey-testing/ described on our blog]. The source code for these system-level tests can be found [https://github.com/assimilation/assimilation-official/tree/master/cma/systemtests in this directory] on GitHub.
 +
 
 +
===Building and Related Topics===
 +
We need help getting properly signed RPM and .deb repositories set up for the various versions of Linux we have set up. We have some basic continuous integration set up, and a good set of build procedures for 64-bit versions of Linux. But things could always be better and release production more automated.
  
 
===Feedback===
 
===Feedback===
Line 173: Line 178:
  
 
=Minimum Viable Product=
 
=Minimum Viable Product=
<!-- Instructions are in RED and should be removed from your document by deleting the text with the span tags.-->
 
<span style="color:#ff0000">
 
This page is where you should indicate what is the minimum set of functionality that is required to make this a useful product that addresses your core security concern.
 
Defining this information helps the project leader to think about what is the critical functionality that a user needs for this project to be useful, thereby helping determine what the priorities should be on the roadmap.  And it also helps reviewers who are evaluating the project to determine if the functionality sufficiently provides the critical functionality to determine if the project should be promoted to the next project category. 
 
</span>
 
  
The Tool Project Template must specify the minimum set of tabs a project should have, provide some an example layout on each tab, provide instructional text on how a project leader should modify the tab, and give some example text that illustrates how to create an actual project.
+
The OWASP Assimilation Project needs to have the following capabilities in order to be considered minimally viable:
 +
 
 +
* Discovery of IP and MAC addresses through ARP
 +
* Discovery of services offered on each system including IP:port combinations and binaries and arguments for the services
 +
* Discovery of packages and package versions installed on the system (OS, pip, GEMs, NPM, etc).
 +
* Discovery of checksums of network-facing binaries (discovered by services above)
 +
* Discovery of security-related settings
 +
* Comparision of security-related settings against best practices from the IT Best Practices project
 +
* A dozen or more database queries to retrieve the information above
 +
* Event API so that notifications can be performed as things change or are added in the system
  
It would also be ideal if the sample text was translated into different languages.
+
The project has had all these capabilities since 2015.
  
 
=Project About=
 
=Project About=
<!-- Instructions are in RED and should be removed from your document by deleting the text with the span tags.-->
 
<span style="color:#ff0000">
 
This page is where you need to place your legacy project template page if your project was created before October 2013. To edit this page you will need to edit your project information template. You can typically find this page by following this address and substituting your project name where it says "OWASP_Example_Project". When in doubt, ask the OWASP Projects Manager.
 
Example template page: https://www.owasp.org/index.php/Projects/OWASP_Example_Project
 
</span>
 
  
{{:Projects/OWASP_Example_Project_About_Page}}  
+
 
 +
{{:Projects/OWASP_Assimilation_Project_About_Page}}  
  
 
__NOTOC__ <headertabs />  
 
__NOTOC__ <headertabs />  
  
 
[[Category:OWASP Project]]  [[Category:OWASP_Builders]] [[Category:OWASP_Defenders]]  [[Category:OWASP_Tool]]
 
[[Category:OWASP Project]]  [[Category:OWASP_Builders]] [[Category:OWASP_Defenders]]  [[Category:OWASP_Tool]]

Latest revision as of 20:33, 8 August 2016

OWASP Project Header.jpg


OWASP Assimilation Project Summary

Many people compare securing systems against attackers as a form of warfare. In The Art of War, Sun Tzu said "If you know your enemies and know yourself, you will not be imperiled in a hundred battles". The Assimilation software helps you know yourself - your systems, networks, configurations in great detail, and then keeps it all that information continually up to date in a graph-based Configuration Management Database. This information is useful regardless of your threat model.

We then leverage this knowledge to compare your systems against hardening best practices, to validate checksums of files, look for vulnerable versions of packages, and help you triage your way to better security.

Description

The Assimilation Project tracks many aspects of system configuration and security and compares them against best practices in near-real-time.

Here are a few of the kinds of things we track for you:

  • IP and MAC addresses
  • Services - including details on which ports, which binaries and what arguments, user id, group id, current directory
  • Client connections (same details as above)
  • Security-sensitive configuration details
  • Versions of packages
  • Checksums of network-facing binaries, libraries, and JARs.

This is all done in a highly scalable way which cannot set off network security alarms and requires minimal human configuration.

In addition, we continually evaluate system configurations against best practices from the IT Best Practices project and compute risk scores for servers based on how they compare to security best practices, and evaluations of what areas and systems are at greater risk. Since everything is stored in the Neo4J graph database, visualizations of things like your attack surface are natural and straightforward.


The project includes event APIs and canned queries.

Licensing

This program is free software: you can redistribute it and/or modify it under the terms of the GNU GPL v3 License as published by the Free Software Foundation.

Project Resources

Universal Installation Script

Released Packages

Source Code

Release History

Project Home Page and Documentation

Issue Tracker

Slide Presentations

Talk Videos

Getting Started articles

How-To articles

Blog

Project Leader

Project leader: Alan Robertson

Related Projects

This is where you can link to other OWASP Projects that are similar to yours.

Classifications

Project Type Files TOOL.jpg
Incubator Project Owasp-builders-small.png
Owasp-defenders-small.png
Affero General Public License 3.0

News and Events

Many projects have "Frequently Asked Questions" documents or pages. However, the point of such a document is not the questions. The point of a document like this are the answers. The document contains the answers that people would otherwise find themselves giving over and over again. The idea is that rather than laboriously compose and post the same answers repeatedly, people can refer to this page with pre-prepared answers. Use this space to communicate your projects 'Frequent Answers.'

How can I participate in your project?

All you have to do is make the Project Leader's aware of your available time to contribute to the project. It is also important to let the Leader's know how you would like to contribute and pitch in to help the project meet it's goals and milestones. There are many different ways you can contribute to an OWASP Project, but communication with the leads is key.

If I am not a programmer can I participate in your project?

Yes, you can certainly participate in the project if you are not a programmer or technical. The project needs different skills and expertise and different times during its development. Currently, we are looking for researchers, writers, graphic designers, and a project administrator. See the Road Map and Getting Involved tab for more details.

Contributors

The OWASP Assimilation Project is developed by a worldwide team of volunteers.

The first contributors to the project were:

  • Alan Robertson
  • Carrie Oswald
  • Dave Quigley
  • Roger Massey

Roadmap

Future plans include:

  • Expanding the best practices we cover
  • Making it easy to see which machines are in need of security patches - take that into account with the security scores.
  • Taking better advantage of checksums - take that into account with security scores.
  • Many more discovery agents. These turn out to be typically pretty simple, and can be written with very little knowledge of the project internals. They have a very shallow learning curve to get started.

There is also a page that describes the future thinking as of August 2016 on our latest roadmap blog post.


There are a variety of roadmap-related artifacts on the project's Trello boards. You can find them here:

You should also consider contributing to our sister project the IT Best Practices project - since that's where the best practices we implement are defined.

You can also read more about the project's "current thinking" in our mailing list archives.

Getting Involved

Involvement in the development and promotion of Assimilation Project is actively encouraged! You do not have to be a security expert or a programmer to contribute. Some of the ways you can help are as follows:

Coding

The software is intended to be able to run on any POSIX system and Microsoft Windows. Currently it runs on at least a dozen different Linux versions. We are actively seeking people to do ports to other systems - particularly Windows.

The project has a contributor agreement for things that go into source control.

So far, the code is written in the following languages:

  • C - core nanoprobe code and communications libraries
  • Python - CMA code and a few utilities
  • POSIX shell - scripts to do discovery and monitoring
  • C# - one Windows discovery agent - should be rewritten in PowerShell
  • PowerShell - future Windows discovery agents

Localization

There is significant opportunity for localization in our sister IT BestPractices project, and some in our canned queries. All strings in the project are UTF-8.

Testing

We have need both for human testers - people to do trials and provide feedback, and people to enhance our automated testing environment. Most of the automated testing is in Python, with some in the shell, and C. We have a unique and powerful system-level testing methodology which was described on our blog. The source code for these system-level tests can be found in this directory on GitHub.

Building and Related Topics

We need help getting properly signed RPM and .deb repositories set up for the various versions of Linux we have set up. We have some basic continuous integration set up, and a good set of build procedures for 64-bit versions of Linux. But things could always be better and release production more automated.

Feedback

  • What do like?
  • What don't you like?
  • What features would you like to see prioritized on the roadmap?

The OWASP Assimilation Project needs to have the following capabilities in order to be considered minimally viable:

  • Discovery of IP and MAC addresses through ARP
  • Discovery of services offered on each system including IP:port combinations and binaries and arguments for the services
  • Discovery of packages and package versions installed on the system (OS, pip, GEMs, NPM, etc).
  • Discovery of checksums of network-facing binaries (discovered by services above)
  • Discovery of security-related settings
  • Comparision of security-related settings against best practices from the IT Best Practices project
  • A dozen or more database queries to retrieve the information above
  • Event API so that notifications can be performed as things change or are added in the system

The project has had all these capabilities since 2015.