Summer Code Sprint2015 Progress Reports - Revision history

John Melton: sumanth damarla week 3 appsensor

2015-08-03T19:51:31Z

sumanth damarla week 3 appsensor

John Melton at 17:56, 28 July 2015

2015-07-28T17:56:31Z

John Melton: adding appsensor week 2 report

2015-07-28T17:55:45Z

adding appsensor week 2 report

Nikola Milosevic: /* Seraphimdroid - Kartik Kohli */

2015-07-25T09:25:25Z

‎Seraphimdroid - Kartik Kohli

John Melton at 17:19, 20 July 2015

2015-07-20T17:19:07Z

John Melton: fixing typo

2015-07-20T17:18:32Z

fixing typo

John Melton: creating new summer code sprint 2015 progress reports page

2015-07-20T16:41:40Z

creating new summer code sprint 2015 progress reports page

New page

=Main=

== OWASP Summer Code Sprint 2015 Progress Reports ==

This page will track the progress reports of the different selected projects for the OWASP Summer Code Sprint 2015 projects.

= OWTF Viyat Bhalodia =

= OWTF Arun Sori =

= OWTF Alexandra Sandulescu =

= Hackademic Anirudh Anand =

= Hackademic Minhaz AV =

= Hackademic Tapasweni Pathak =

= AppSensor Sumanth Damaria =

== Week 1 (2015.07.13 - 2015.07.17) ==

Operating System:
Ubuntu 15.04 Desktop
Processor: Intel Core i3 -3227U CPU @ 1.90GHz x 4
OS Type : 64-bit
Disk: 17.5GB
(Server or Desktop?) + (What hardware?)

Technologies researched on :
Overview of Time Series Databases.
Influxdb documentation.
Learnt InfluxQL (Query language for Influxdb).
Overview Elasticsearch and related tutorials.
Overview Logstash and related tutorials.
Overview Kibana and related tutorials.

Environement Setup:
Installed ElasticSearch 1.4.4, Logstash and Kibana 4.0 and Nginx environment.(To set up a reverse proxy to allow external access) individually. What versions of each component?
Working on linking all the three[how the logstash output be given to elasticsearch (the parameters to be mentioned) and how to generate metrics using Kibana from Elasticsearch inputs] but I need to configure it with private IP.(Linking all three what? )
Installed Appsensor environment.

Links I referenced (Teaching Resources):

InfluxDB Documentation: https://influxdb.com/docs/v0.9/introduction/overview.html

Logstash Documentation: https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html

Elastic Search Documentation: https://www.elastic.co/guide/en/elasticsearch/guide/current/getting-started.html

Kibana: https://www.elastic.co/webinars/whats-new-in-kibana-4

Visualizing Logs Using ElasticSearch, Logstash and Kibana : https://www.youtube.com/watch?v=Kqs7UcCJquM
Logs & Metrics: Use the Force, Gain the Insight (Need to have pro access which I have xD )
https://teamtreehouse.com/library/logs-metrics-use-the-force-gain-the-insight

Step by step installation on ELK Stack on Ubuntu: https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-ubuntu-14-04

== Week 2 ==

== Week 3 ==

== Week 4 ==

== Week 5 ==

== Week 6 ==

== Week 7 ==

= Seraphimdroid Kartik Kohli =

__NOTOC__ <headertabs />

← Older revision		Revision as of 19:51, 3 August 2015
Line 135:		Line 135:


−	== Week 3 ==	+	== Week 3 (2015.07.27 - 2015.07.31) ==
		+
		+	Parsed syslog with the help of “Grok” and “syslog-pri” filters.
		+	Built sample dashboards for Appsensor logs.
		+	Working on kv{} filter
		+
		+	Building Custom Dashboards using Kibana:
		+	Follow this guide: http://blog.trifork.com/2014/05/20/advanced-kibana-dashboard/
		+
		+	Successfully running “simple-websocket-dashboard” but working on executing “DemoDataPopulator.java”.
		+
		+	Reference Links:
		+	About Grok Filter: https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
		+	About Syslog_pri Filter: https://www.elastic.co/guide/en/logstash/current/plugins-filters-syslog_pri.html
		+	About KV{} Filter: https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html

	== Week 4 ==		== Week 4 ==

@@ Line 57: / Line 57: @@
 Step by step installation on ELK Stack on Ubuntu: https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-ubuntu-14-04
-== Week 2 ==
+== Week 2 (2015.07.20 - 2015.07.24) ==
 Configured Simple Websocket Dashboard from Appsensor and deployed in Tomcat 7:

← Older revision		Revision as of 17:55, 28 July 2015
Line 58:		Line 58:

	== Week 2 ==		== Week 2 ==
		+
		+	Configured Simple Websocket Dashboard from Appsensor and deployed in Tomcat 7:
		+
		+	Steps to Install Tomcat7 in Ubuntu 15.04:
		+	https://www.digitalocean.com/community/tutorials/how-to-install-apache-tomcat-7-on-ubuntu-14-04-via-apt-get
		+
		+	Building AppSensor Environment:
		+	AppSensor is a multi-module maven project. The project requires Java version 7 or higher. Building is generally handled by the following steps
		+
		+	clone the repo (or your fork)
		+	git clone https://github.com/jtmelton/appsensor.git
		+
		+	get into appsensor directory
		+	cd appsensor
		+
		+	install multi-module parent - one time requirement per version
		+	mvn -N install
		+	or
		+	sudo apt-get update
		+	sudo apt-get install maven
		+
		+	run the tests - done every time you make changes
		+	mvn test
		+
		+	Deploying Sample App in Standalone Container:
		+	If you'd like to deploy one of these applications to a standalone application server or servlet container, follow these simple steps:
		+	Download the source code (either zip download or git clone)
		+	Go into the folder containing the application you want to deploy (e.g. 'simple-dashboard')
		+	Execute 'mvn package'
		+	Look in the 'target' folder that gets generated and find the '.war' file
		+	Deploy this WAR file into your application server / servlet container and start it up
		+	You should now be able to interact with the application locally
		+
		+	Deploying Sample App in IDE-managed Container:
		+	If you'd like to deploy one of these applications to a standalone application server or servlet container, follow these simple steps:
		+	Download the source code (either zip download or git clone)
		+	Import the application into your IDE (using 'import maven project' mechanism)
		+	Setup an IDE managed container if you don't have one already
		+	Add the application to your container
		+	Startup the container
		+	You should now be able to interact with the application locally
		+
		+	Providing syslog as input for Logstash:
		+	Syslog and its configuration options: https://www.elastic.co/guide/en/logstash/current/plugins-inputs-syslog.html
		+
		+	ELK STACK has been configured
		+
		+	Ports used for ELK Stack:
		+	Port 9200: Elasticsearch
		+	Port 5610: Kibana
		+
		+	Plugins Installed:
		+	Marvel: Used as GUI for Elasticsearch.
		+	Kopf: Used as GUI for Logstash.
		+
		+	Sample Logs used in ELK Stack:
		+	Manual logs are inserted from terminal.
		+
		+	Ingesting CSV data with Logstash and analysing in Kibana.
		+
		+	Current Tasks:
		+	Researching on syslog format.
		+	Working on importing syslog format logs into Logstash.
		+	Learning the filters supported for syslog in ELK stack technology.
		+
		+	Tasks ahead:
		+	Extracting logs (syslog format) from the applications configured with AppSensor.
		+
		+	Reference Links:
		+	Centralized logging with an
		+	ELK stack (Elasticsearch-Logstash-Kibana) on Ubuntu: https://deviantony.wordpress.com/2014/05/19/centralized-logging-with-an-elk-stack-elasticsearch-logback-kibana/
		+	About Marvel Plugin: https://www.elastic.co/guide/en/marvel/current/index.html
		+	About Kopf Plugin: https://github.com/lmenezes/elasticsearch-kopf
		+	Ingesting CSV Data with Logstash: http://www.rittmanmead.com/2015/04/using-the-elk-stack-to-analyse-donors-choose-data/
		+	Download free CSV datasheets: http://data.donorschoose.org/open-data/overview/
		+

	== Week 3 ==		== Week 3 ==

← Older revision		Revision as of 09:25, 25 July 2015
Line 73:		Line 73:

	__NOTOC__ <headertabs />		__NOTOC__ <headertabs />
		+
		+	==Week one==
		+
		+	The student fixed bug regarding locking of application on Android 5+.
		+
		+	==Week two==
		+
		+	No activity. Student also said that he was home, so thats the reason of his slow progress, but that he will catch up from monday.

@@ Line 5: / Line 5: @@
 This page will track the progress reports of the different selected projects for the OWASP Summer Code Sprint 2015 projects.
-= OWTF Viyat Bhalodia =
+= OWTF - Viyat Bhalodia =
-= OWTF Arun Sori =
+= OWTF - Arun Sori =
-= OWTF Alexandra Sandulescu =
+= OWTF - Alexandra Sandulescu =
-= Hackademic Anirudh Anand =
+= Hackademic - Anirudh Anand =
-= Hackademic Minhaz AV =
+= Hackademic - Minhaz AV =
-= Hackademic Tapasweni Pathak =
+= Hackademic - Tapasweni Pathak =
-= AppSensor Sumanth Damarla =
+= AppSensor - Sumanth Damarla =
 == Week 1 (2015.07.13 - 2015.07.17) ==
@@ Line 69: / Line 69: @@
 == Week 7 ==
-= Seraphimdroid Kartik Kohli =
+= Seraphimdroid - Kartik Kohli =
 __NOTOC__ <headertabs />