Revision as of 19:37, 23 September 2013

WELCOME TO THE GRID

What is it?

The HIVE project started out as an idea for a learning platform. Using TAHOE-lafs as a storage solution we are able to interlock anything we do into the GRID. What’s the GRID? The GRID is a group of Tahoe enabled capable computers (The BeagleBone Black) could be any computer. The idea is to share data across a GRID in a secure anonymous fashion. Using Tahoe-LAFS we can share and distribute project files lots and lots of data all gathered from the GRID. This uses “provider-independent security” that means I don’t know what your data is and neither does anyone else. Only you know what you uploaded so unless someone beats you with sticks until you hand over you unique URI. It’s all secret! URI looks something like

URI%3ACHK%3Am6wj5avuhim3flsd2w54lhjcke:j2njlm3cm4sjzqdiwbadmts35lxa656kpvo66kx7wtlfyr6cab6a:3:10:3

Currently I am using a 3 to 10 ratio that means you will need at least 3 storage nodes to retrieve files from in order to download your erasure encoded stuff. There are two kinds of files: immutable and mutable. When you upload a file to the storage grid you can choose which kind of file it will be in the grid. Immutable files can't be modified once they have been uploaded. A mutable file can be modified by someone with read-write access to it. A user can have read-write access to a mutable file or read-only access to it, or no access to it at all. If you want more details the link will take you to a site interactive illustration by Drew Perttula http://bigasterisk.com/tahoe-playground/

Some ways I can see this being useful for OWASP is a HIVE cache that is currently what this project is becoming 10-1000 nodes strong but monitoring is controlled and nodes do what they please. The end goal is to provide a global grid for OWASP and any amount of users can use the GRID; a large, diverse ecosystem of people and organizations who want a storage grid with extremely high reliability and availability.

Example of project based use. Let’s say BOB has a tomcat server and he wants to share its contents to JOE. BOB has Tahoe up and sets up a dropfolder and this folder he sets up just happens to be his entire tomcat folder. As files change they are sent to the grid and encrypted, stored and shared out to people that JOE selected to give his folder URI to. JOE now has a working copy of BOB’s stuff and can help or just create his own environment. All the while we always use SFTP or SSL with our frontends. If a node gets hacked and stolen because JACK forgot to lock up then big deal there are X number of nodes left and the files on JACKS node are encrypted erasure style.

The milestones that I anticipate are:

Sweet user interface Find a LCD for the BeagleBone Black for IP or stats scrolling. The HIVE setup currently shows the nodes in real time. (expand more on this) Make a proposal to OWASP for a more stable introducer and private GRID. Tahoe-lafs homepage