OWASP Machine Learning Resources - Revision history

Puhley: Added HTTP data set

2017-07-26T00:18:10Z

Added HTTP data set

Puhley: Re-ordering

2017-07-25T23:42:34Z

Re-ordering

Puhley: Adding UCI ML data sets

2017-07-25T23:40:22Z

Adding UCI ML data sets

Puhley: Adding eXpose to data sets.

2017-07-25T22:29:32Z

Adding eXpose to data sets.

Puhley: Added link to Keras

2017-07-25T19:00:27Z

Added link to Keras

Puhley: Adding a Data Sets section.

2017-07-24T22:46:46Z

Adding a Data Sets section.

Puhley: Added a reference to Anaconda.

2017-07-12T17:56:41Z

Added a reference to Anaconda.

Puhley: Added bullets for readability.

2017-07-12T04:57:38Z

Added bullets for readability.

Puhley: Adding a few clarifications.

2017-07-12T02:29:09Z

Adding a few clarifications.

Puhley: Typo fix

2017-07-11T05:31:17Z

Typo fix

@@ Line 64: / Line 64: @@
 == Data Sets ==
 * [http://archive.ics.uci.edu/ml/datasets.html UCI ML Data Sets] University of California, Irvine has a collection of machine learning data sets across many fields. Within their collection, their are security relevant data sets if you search for topics such as spam, phishing, etc. <br/>
 * [https://github.com/joshsaxe/eXposeDeepNeuralNetwork eXpose Deep Nueral Network] This is an open-source deep neural network project that attempts to detect malicious URLs, file paths and registry keys with proper training. Data sets can be found in the data/models directory the in the sample_scores.json files. <br/>
 * [http://www.kdd.org/kdd-cup/view/kdd-cup-1999/Data KDD Cup 1999: Computer Network Intrusion Detection] The goal of the KDD Cup competition in 1999 was to learn a predictive model (i.e. a classifier) capable of distinguishing between legitimate and illegitimate connections in a computer network. This is a link to the large data set used for that competition. The other tabs on the page provide additional context on the data. <br/>

@@ Line 63: / Line 63: @@
 == Data Sets ==
-* [http://www.kdd.org/kdd-cup/view/kdd-cup-1999/Data KDD Cup 1999: Computer Network Intrusion Detection] The goal of the KDD Cup competition in 1999 was to learn a predictive model (i.e. a classifier) capable of distinguishing between legitimate and illegitimate connections in a computer network. This is a link to the large data set used for that competition. The other tabs on the page provide additional context on the data. <br/>
+* [http://archive.ics.uci.edu/ml/datasets.html UCI ML Data Sets] University of California, Irvine has a collection of machine learning data sets across many fields. Within their collection, their are security relevant data sets if you search for topics such as spam, phishing, etc. <br/>
 * [https://github.com/joshsaxe/eXposeDeepNeuralNetwork eXpose Deep Nueral Network] This is an open-source deep neural network project that attempts to detect malicious URLs, file paths and registry keys with proper training. Data sets can be found in the data/models directory the in the sample_scores.json files. <br/>
-* [http://archive.ics.uci.edu/ml/datasets.html UCI ML Data Sets] University of California, Irvine has a collection of machine learning data sets across many fields. Within their collection, their are security relevant data sets if you search for topics such as spam, phishing, etc. <br/>
+* [http://www.kdd.org/kdd-cup/view/kdd-cup-1999/Data KDD Cup 1999: Computer Network Intrusion Detection] The goal of the KDD Cup competition in 1999 was to learn a predictive model (i.e. a classifier) capable of distinguishing between legitimate and illegitimate connections in a computer network. This is a link to the large data set used for that competition. The other tabs on the page provide additional context on the data. <br/>

← Older revision		Revision as of 23:40, 25 July 2017
Line 66:		Line 66:

	* [https://github.com/joshsaxe/eXposeDeepNeuralNetwork eXpose Deep Nueral Network] This is an open-source deep neural network project that attempts to detect malicious URLs, file paths and registry keys with proper training. Data sets can be found in the data/models directory the in the sample_scores.json files. <br/>		* [https://github.com/joshsaxe/eXposeDeepNeuralNetwork eXpose Deep Nueral Network] This is an open-source deep neural network project that attempts to detect malicious URLs, file paths and registry keys with proper training. Data sets can be found in the data/models directory the in the sample_scores.json files. <br/>
		+
		+	* [http://archive.ics.uci.edu/ml/datasets.html UCI ML Data Sets] University of California, Irvine has a collection of machine learning data sets across many fields. Within their collection, their are security relevant data sets if you search for topics such as spam, phishing, etc. <br/>

← Older revision		Revision as of 22:29, 25 July 2017
Line 63:		Line 63:

	== Data Sets ==		== Data Sets ==
−	* [http://www.kdd.org/kdd-cup/view/kdd-cup-1999/Data KDD Cup 1999: Computer Network Intrusion Detection] The goal of the KDD Cup competition in 1999 was to learn a predictive model (i.e. a classifier) capable of distinguishing between legitimate and illegitimate connections in a computer network. This is a link to the large data set used for that competition. The other tabs on the page provide additional context. <br/>	+	* [http://www.kdd.org/kdd-cup/view/kdd-cup-1999/Data KDD Cup 1999: Computer Network Intrusion Detection] The goal of the KDD Cup competition in 1999 was to learn a predictive model (i.e. a classifier) capable of distinguishing between legitimate and illegitimate connections in a computer network. This is a link to the large data set used for that competition. The other tabs on the page provide additional context on the data. <br/>
		+
		+	* [https://github.com/joshsaxe/eXposeDeepNeuralNetwork eXpose Deep Nueral Network] This is an open-source deep neural network project that attempts to detect malicious URLs, file paths and registry keys with proper training. Data sets can be found in the data/models directory the in the sample_scores.json files. <br/>

@@ Line 5: / Line 5: @@
 == Introduction ==
 <br />
-This page is intended to provide machine learning security resources for security researchers who are new to the field. It includes a few introductory resources for the basics of machine learning as well as examples of machine learning applied to security problems on different platforms. While there are many resources and conference talks available on the web, this page focuses on open-source projects and information that would be of interest to security researchers as a place to start learning. Rather than reinvent existing information, the approach of this page is to maintain context and accuracy by linking to original source materials.<br />
+This page is intended to provide machine learning security resources for security researchers who are new to the field. It includes a few introductory resources for the basics of machine learning as well as examples of machine learning applied to security problems on different platforms. While there are many resources and conference talks available on the web, this page focuses on open-source projects and information that would be of interest to security researchers as a place to start learning. Rather than reinvent existing information, the approach of this page is to maintain context and accuracy by linking to original source materials. <br />
 <br />
@@ Line 17: / Line 17: @@
 == Getting Started with the Basics of Machine Learning ==
-There are many free videos and interactive demos for machine learning available on the web. These are just a few options:<br />
+There are many free videos and interactive demos for machine learning available on the web. These are just a few options: <br />
 === Videos ===
-* [https://www.youtube.com/playlist?list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal Machine Learning Recipes with Josh Gordon] - This Google Developer YouTube series focuses on TensorFlow and SciKit learn. The target audience is people who are new to machine learning and are able to read Python. - Josh Gordon, Google Developer YouTube channel, 2017.<br />
+* [https://www.youtube.com/playlist?list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal Machine Learning Recipes with Josh Gordon] - This Google Developer YouTube series focuses on TensorFlow and SciKit learn. The target audience is people who are new to machine learning and are able to read Python. - Josh Gordon, Google Developer YouTube channel, 2017. <br />
 === Hands On Learning ===
-* [http://playground.tensorflow.org/ Tensorflow Playground] - This web page allows you to play with neural networks from your web browser. The goal of the exercises are to design neural networks to match different data sets.<br />
+* [http://playground.tensorflow.org/ Tensorflow Playground] - This web page allows you to play with neural networks from your web browser. The goal of the exercises are to design neural networks to match different data sets. <br />
-* [https://gallery.cortanaintelligence.com/Experiment/Binary-Classification-Network-intrusion-detection-2 Azure Binary Classification: Network Intrusion Detection] - This lab demonstrates intrusion detection security analysis using Azure's Machine Learning environment. The lab exercise can be performed using a free account. In addition, the Cortana Intelligence Gallery has other types of anomaly detection examples from Microsoft and other contributors available from this site.<br />
+* [https://gallery.cortanaintelligence.com/Experiment/Binary-Classification-Network-intrusion-detection-2 Azure Binary Classification: Network Intrusion Detection] - This lab demonstrates intrusion detection security analysis using Azure's Machine Learning environment. The lab exercise can be performed using a free account. In addition, the Cortana Intelligence Gallery has other types of anomaly detection examples from Microsoft and other contributors available from this site. <br />
 === Cheat Sheets ===
 * [http://download.microsoft.com/download/A/6/1/A613E11E-8F9C-424A-B99D-65344785C288/microsoft-machine-learning-algorithm-cheat-sheet-v6.pdf Microsoft Azure Machine Learning: Algorithm Cheat Sheet] - "This cheat sheet helps you choose the best Azure Machine Learning Studio algorithm for your predictive analytics solution. Your decision is driven by both the nature of your data and the question you’re trying to answer." Microsoft provides this documentation on how to interpret the cheat sheet: [https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-algorithm-choice How to choose algorithms for Microsoft Azure Machine Learning] While this reference is specific to Microsoft, many of the concepts are universal. <br />
-* [https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Scikit_Learn_Cheat_Sheet_Python.pdf Python for data science cheat sheet] - A cheat sheet for the different Scikit machine learning Python modules.<br />
+* [https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Scikit_Learn_Cheat_Sheet_Python.pdf Python for data science cheat sheet] - A cheat sheet for the different Scikit machine learning Python modules. <br />
-* [http://scikit-learn.org/stable/tutorial/machine_learning_map/index.html Scikit Learn: Choosing the right estimator] - "The flowchart below is designed to give users a bit of a rough guide on how to approach problems with regard to which estimators to try on your data."<br />
+* [http://scikit-learn.org/stable/tutorial/machine_learning_map/index.html Scikit Learn: Choosing the right estimator] - "The flowchart below is designed to give users a bit of a rough guide on how to approach problems with regard to which estimators to try on your data." <br />
 <br />
@@ Line 39: / Line 39: @@
 === Videos ===
 * [https://www.youtube.com/watch?v=6Slj2FV9CLA Advances in Cloud-Scale Machine Learning for Cyberdefense] - "Picking an attacker's signals out of billions of log events in near real time from petabyte scale storage is a daunting task, but Microsoft has been using security data science at cloud scale to successfully disrupt attacker. This session presents the latest frameworks, techniques and the unconventional machine learning algorithms that Microsoft uses to protect its infrastructure and customers." - Mark Russinovich, BlueHat 2017 <br />
-* [https://www.youtube.com/watch?v=fRklX97iGIw Machine Learning and the Cloud: Disrupting Threat Detection and Prevention] - "Machine learning with large data sets gives unprecedented insights and anomaly detection capability. Learn how Microsoft uses the agility and scale of the cloud to protect its infrastructure and customers by applying data mining and machine learning algorithms and security domain learnings to the vast amounts of data and telemetry gathered by its many different systems and services." - Mark Russinovich, RSA USA 2016.<br />
+* [https://www.youtube.com/watch?v=fRklX97iGIw Machine Learning and the Cloud: Disrupting Threat Detection and Prevention] - "Machine learning with large data sets gives unprecedented insights and anomaly detection capability. Learn how Microsoft uses the agility and scale of the cloud to protect its infrastructure and customers by applying data mining and machine learning algorithms and security domain learnings to the vast amounts of data and telemetry gathered by its many different systems and services." - Mark Russinovich, RSA USA 2016. <br />
 * [https://www.youtube.com/watch?v=Wh7nvpVQ_58 Predictive Security: Using Big Data to Fortify Your Defenses] - "This session shows you how to build a predictive analytics stack on AWS, which harnesses the power of Amazon Machine Learning in conjunction with Amazon Elasticsearch Service, AWS CloudTrail, and VPC Flow Logs to perform tasks such as anomaly detection and log analysis. We also demonstrate how you can use AWS Lambda to act on this information in an automated fashion, such as performing updates to AWS WAF and security groups, leading to an improved security posture and alleviating operational burden on your security teams." - Michael Capicotto & Matt Nowina, AWS re:Invent 2016 <br />
 === Blogs ===
-* [https://blog.openai.com/adversarial-example-research/ Attacking Machine Learning with Adversarial Examples] - "In this post we’ll show how adversarial examples work across different mediums, and will discuss why securing systems against them can be difficult." - February 24, 2017<br />
+* [https://blog.openai.com/adversarial-example-research/ Attacking Machine Learning with Adversarial Examples] - "In this post we’ll show how adversarial examples work across different mediums, and will discuss why securing systems against them can be difficult." - February 24, 2017 <br />
 <br />
 == Tools ==
-* [http://www.cs.waikato.ac.nz/ml/weka/ Weka] - "Weka is a collection of machine learning algorithms for data mining tasks. The algorithms can either be applied directly to a dataset or called from your own Java code. Weka contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. It is also well-suited for developing new machine learning schemes."<br />
+* [http://www.cs.waikato.ac.nz/ml/weka/ Weka] - "Weka is a collection of machine learning algorithms for data mining tasks. The algorithms can either be applied directly to a dataset or called from your own Java code. Weka contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. It is also well-suited for developing new machine learning schemes." <br />
 <br />
 == Languages and Libraries ==
-* [http://scikit-learn.org/stable/ Scikit-Learn] - Data mining and data analysis in Python that is built on NumPy, SciPy, and matplotlib.<br />
+* [http://scikit-learn.org/stable/ Scikit-Learn] - Data mining and data analysis in Python that is built on NumPy, SciPy, and matplotlib. <br />
-* [https://www.tensorflow.org/ Tensor Flow]  - TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.<br />
+* [https://www.tensorflow.org/ Tensor Flow]  - TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. <br />
-* [https://www.r-project.org/ R Project] -  The R language is widely used among statisticians and data miners for developing statistical software and data analysis.<br />
+* [https://www.r-project.org/ R Project] -  The R language is widely used among statisticians and data miners for developing statistical software and data analysis. <br />
-* [https://www.continuum.io/downloads Anaconda] -  "The open source version of Anaconda is a high performance distribution of Python and R and includes over 100 of the most popular Python, R and Scala packages for data science. ... If you don't have time or disk space for the entire distribution, try Miniconda which contains only conda and Python. Then install just the individual packages you want through the conda command."<br />
+* [https://www.continuum.io/downloads Anaconda] -  "The open source version of Anaconda is a high performance distribution of Python and R and includes over 100 of the most popular Python, R and Scala packages for data science. ... If you don't have time or disk space for the entire distribution, try Miniconda which contains only conda and Python. Then install just the individual packages you want through the conda command." <br />
 <br />
 == Data Sets ==
 * [http://www.kdd.org/kdd-cup/view/kdd-cup-1999/Data KDD Cup 1999: Computer Network Intrusion Detection] The goal of the KDD Cup competition in 1999 was to learn a predictive model (i.e. a classifier) capable of distinguishing between legitimate and illegitimate connections in a computer network. This is a link to the large data set used for that competition. The other tabs on the page provide additional context. <br/>

← Older revision		Revision as of 22:46, 24 July 2017
Line 58:		Line 58:

	* [https://www.continuum.io/downloads Anaconda] - "The open source version of Anaconda is a high performance distribution of Python and R and includes over 100 of the most popular Python, R and Scala packages for data science. ... If you don't have time or disk space for the entire distribution, try Miniconda which contains only conda and Python. Then install just the individual packages you want through the conda command."<br />		* [https://www.continuum.io/downloads Anaconda] - "The open source version of Anaconda is a high performance distribution of Python and R and includes over 100 of the most popular Python, R and Scala packages for data science. ... If you don't have time or disk space for the entire distribution, try Miniconda which contains only conda and Python. Then install just the individual packages you want through the conda command."<br />
		+	<br />
		+
		+	== Data Sets ==
		+	* [http://www.kdd.org/kdd-cup/view/kdd-cup-1999/Data KDD Cup 1999: Computer Network Intrusion Detection] The goal of the KDD Cup competition in 1999 was to learn a predictive model (i.e. a classifier) capable of distinguishing between legitimate and illegitimate connections in a computer network. This is a link to the large data set used for that competition. The other tabs on the page provide additional context. <br/>

← Older revision		Revision as of 17:56, 12 July 2017
Line 56:		Line 56:

	* [https://www.r-project.org/ R Project] - The R language is widely used among statisticians and data miners for developing statistical software and data analysis.<br />		* [https://www.r-project.org/ R Project] - The R language is widely used among statisticians and data miners for developing statistical software and data analysis.<br />
		+
		+	* [https://www.continuum.io/downloads Anaconda] - "The open source version of Anaconda is a high performance distribution of Python and R and includes over 100 of the most popular Python, R and Scala packages for data science. ... If you don't have time or disk space for the entire distribution, try Miniconda which contains only conda and Python. Then install just the individual packages you want through the conda command."<br />

@@ Line 12: / Line 12: @@
 <br />
 # Adversarial Machine Learning - This technique attempts to confuse ML algorithms into returning the attacker's selected data instead of the expected response.
-# Machine learning for improved security analysis - Machine learning can be used in security log analysis in order to better detect or prevent attacks.
+# Machine learning for improved security analysis - Machine learning can be used in security log analysis in order to better detect or prevent attacks. Typical applications include detecting anomalies, predicting values, or classifying data into two or more categories.
 # Attacks against machine learning software - These attacks target the software that is being used for machine learning with the hopes of causing a denial-of-service attack or arbitrary code execution. These types of attacks typically result in a software patch for the impacted library or service.
 <br />
@@ Line 27: / Line 27: @@
 === Cheat Sheets ===
-[http://download.microsoft.com/download/A/6/1/A613E11E-8F9C-424A-B99D-65344785C288/microsoft-machine-learning-algorithm-cheat-sheet-v6.pdf Microsoft Azure Machine Learning: Algorithm Cheat Sheet] - "This cheat sheet helps you choose the best Azure Machine Learning Studio algorithm for your predictive analytics solution. Your decision is driven by both the nature of your data and the question you’re trying to answer." <br />
+[http://download.microsoft.com/download/A/6/1/A613E11E-8F9C-424A-B99D-65344785C288/microsoft-machine-learning-algorithm-cheat-sheet-v6.pdf Microsoft Azure Machine Learning: Algorithm Cheat Sheet] - "This cheat sheet helps you choose the best Azure Machine Learning Studio algorithm for your predictive analytics solution. Your decision is driven by both the nature of your data and the question you’re trying to answer." Microsoft provides this documentation on how to interpret the cheat sheet: [https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-algorithm-choice How to choose algorithms for Microsoft Azure Machine Learning] <br />
 [https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Scikit_Learn_Cheat_Sheet_Python.pdf Python for data science cheat sheet] - A cheat sheet for the different Scikit machine learning Python modules.<br />
 [http://scikit-learn.org/stable/tutorial/machine_learning_map/index.html Scikit Learn: Choosing the right estimator] - "The flowchart below is designed to give users a bit of a rough guide on how to approach problems with regard to which estimators to try on your data."<br />

@@ Line 26: / Line 26: @@
 [https://gallery.cortanaintelligence.com/Experiment/Binary-Classification-Network-intrusion-detection-2 Azure Binary Classification: Network Intrusion Detection] - This lab demonstrates intrusion detection security analysis using Azure's Machine Learning environment. The lab exercise can be performed using a free account. In addition, the Cortana Intelligence Gallery has other types of anomaly detection examples from Microsoft and other contributors available from this site.<br />
-=== Cheatsheets ===
+=== Cheat Sheets ===
-[http://download.microsoft.com/download/A/6/1/A613E11E-8F9C-424A-B99D-65344785C288/microsoft-machine-learning-algorithm-cheat-sheet-v6.pdf Microsoft Azure Machine Learning: Algorithm Cheat Sheet] - "This cheat sheet helps you choose the best Azure Machine Learning Studio algorithm for your predictive analytics solution. Your decision is driven by both the nature of your data and the question you’re trying to answer." <br/>
+[http://download.microsoft.com/download/A/6/1/A613E11E-8F9C-424A-B99D-65344785C288/microsoft-machine-learning-algorithm-cheat-sheet-v6.pdf Microsoft Azure Machine Learning: Algorithm Cheat Sheet] - "This cheat sheet helps you choose the best Azure Machine Learning Studio algorithm for your predictive analytics solution. Your decision is driven by both the nature of your data and the question you’re trying to answer." <br />
-[https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Scikit_Learn_Cheat_Sheet_Python.pdf Python for data science cheat sheet] - A cheat sheet for the different Scikit machine learning Python modules.<br/>
+[https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Scikit_Learn_Cheat_Sheet_Python.pdf Python for data science cheat sheet] - A cheat sheet for the different Scikit machine learning Python modules.<br />
-[http://scikit-learn.org/stable/tutorial/machine_learning_map/index.html Scikit Learn: Choosing the right estimator] - "The flowchart below is designed to give users a bit of a rough guide on how to approach problems with regard to which estimators to try on your data."<br/>
+[http://scikit-learn.org/stable/tutorial/machine_learning_map/index.html Scikit Learn: Choosing the right estimator] - "The flowchart below is designed to give users a bit of a rough guide on how to approach problems with regard to which estimators to try on your data."<br />
 <br />
 == ML Security Resources ==
-=== Consolidated resource sites ===
+=== Consolidated Resource Sites ===
 [https://www.mlsecproject.org/ MLSec Project] - "MLSec Project is a select community of like-minded individuals that want to work together on using machine learning and data science in information security. It features open source projects, blog posts and community content to help further the understanding of how this technology can help defenders handle the growing complexity and verbosity of their environments and tools."
 <br />