OWASP Podcast/Transcripts/031

Jim Manico

We have with us today Mark Curphey. Mark is the original founder of the OWASP foundation. He is currently the Director of Information Security Tools at Microsoft and recently contributed to the collaborative security book for O’Reilly called Beautiful Security. Mark, you are the founder of OWASP. Can you give us some back story as to why you started the Open Web Application Security Project and who were some of the other players who started the foundation with you?

Mark Curphey

Sure, so in reality, I was just one of a bunch of people. I guess technically, I came up with the idea first, so the way it happened, I used to moderate a…security focus called WebAppSec. I think it is still running. It is not as popular as it used to be. I had just left a job. I actually was running a consulting team for ISS, Internet Security Systems, where we were predominately focused on network security. We had built an Internet scanner and real secure and other sort of network systems. I was running a consulting team. The majority of the guys that were working for me were breaking into clients through penetration testing, through the application there. It became intriguing to me how this was happening. It just so happened that two of the guys who were working for me actually were Caleb Seymour and Brian Christian who went off and founded Spikonomics, but kind of a side story, I guess, so I kind of come from this arena. Cut a long story short, my wife got pregnant. We were living in Atlanta and decided we didn't want to bring up children in the south, so I took a job at Charles Schwab, running network security. What I quickly discovered was that back in 2000, there were very few people talking about software security. There were a bunch of startups, a bunch of people really talking about technologies that they were already trying to sell, but it was heavily focused on marketing, real sort of… being pushed, so I punted out an idea amongst a couple of people that I knew, a guy from another big bank on the east coast and a guy called David Endler down at iDefense, Steve Taylor over there at…I had met who was in Germany. I said hey, we should all get together and write a guide that really kind of captures the things the really matter and go figure out what it does. No preconceived notion, no kind of grand plans. We bashed out a very quick guide, a very quick document collaboratively. All of a sudden everything kind of snowballed. People started joining and volunteering. The rest is history. Just one of these fantastic collaborative, open source internet projects. A whole bunch of people that I have met over the time have become very good friends. I hook up with folks like David Endler whenever I am in Austin and all sorts of fantastic people. It was a wonderful experience.

Jim Manico

So, Mark what do you think has changed in this industry since the early days of OWASP?

Mark Curphey

So, if anything, this sort of awareness. If you look back through history, I guess everything was originally network security based. Then, everything moved up the stack around operating systems. Then, everyone became aware of applications. If anything, we have seen that sort of natural progression. I think that things like OWASP have helped people understand where the real threats are and where the real issues are, and how to deal with them, so there is a huge credibility to that. I think that there has been this sort of natural evolution and I guess people always focus on the low hanging fruit and the lowest common denominator. Certainly, the security industry as I see it has evolved and is continuing to evolve. You know, it used to be all the network security guys and now there is a whole software security industry in there as well, so as that stuff changes, people start to understand and change and figure out the stuff that is really important.

Jim Manico

So, Mark what were the catalytic moments in the early days of the OWASP project?

Mark Curphey

You know, I look back and my wife would kind of laugh if she was here... back in the early days, we lived in California, and she would always come in at four o’clock in the morning and there would be me on my cell phone and IM. I think the real catalytic stuff in the real early days were a small tightly knit bunch of guys with very altruistic values enjoying working together. I think that was absolutely fantastic. There was no kind of grand plan. There was no politics. It was like hey, here is a cool thing to do. Let’s go ahead and do it. You are getting great feedback and great success immediately from the incident, so I think it was a new experience for all of us. Certainly it was extremely enjoyable. If you fast forward a little bit, the real catalytic moment from what I can see was actually what Jeff and Dave put in place. I have got to be honest, at that time I thought it was the absolute opposite thing to do, so OWASP was going through a real challenging phase. It had grown very quickly and produced a lot of content. Some of that content was of questionable quality. It had been open to all, and kind of naturally if you open things to everyone, you get absolutely fantastic quality content and you get some which is less so, so I really had this bee in my bonnet about what was in order for OWASP to grow. The most important thing is going to be figuring out a way to get a much better published provocation process, a much kind of tighter process with rules and processes around it. A lot of other people said what you actually want is a wiki. Wikis were new at the time, not really well understood. I certainly did not understand them at all. I understood this concept that you just allow everyone to edit and write. Now that problem is going to get ten times worse. When you look back now, that was absolutely a catalytic moment. It allowed everyone to contribute without going through the process. All of a sudden you got this swarm effect. The natural community stuff kicked in of validating and improving content, so I think looking back from the whole time that I have been actively involved or been watching, I think the real thing was that wiki. I think Jeff or Dave, one of those guys has absolute credit for that because that was thing that really made it take off.

Jim Manico

So what do you think of the leaders of OWASP who have taken the responsibility of stewardship of the organization as you have moved on to other endeavors?

Mark Curphey

You know, absolutely fantastic, so I had the pleasure of spending time with Dave and Jeff quite a lot and they came to the party…They were not there in the immediate phase, but it was obvious to me that when Jeff and Dave started getting involved that they were doing it for the right reasons. For various reasons, I was not involved in the way I was hoping. I started looking at doing other things. It was clear that we needed to find people who would take it in the right direction. There were a number of different options. There were certainly a lot of vendors who were interested in engaging and taking over. That certainly would have been for the wrong reasons. What I saw in Jeff and Dave, particularly, were a bunch of guys who were doing things for the right reasons. They were prepared to make commitments without making expectations of things in return. That was absolutely huge. Dennis is just a fantastic guy. I don't spend enough time with him. I have not done so since I have been in England. One of my favorite uber smart people and always engaging and always entertaining. Sebastian Schneider…I will butcher his name on your behalf, Jim, is also dedicated and fantastic. Tom Brennan, I don't know and have not had the pleasure of meeting, but I have heard nothing but fantastic things. When I look at it, I think great bunch of people. The results speak for themselves, like anything in life. You just look at the results, and that has got to speak volumes for the leaders.

Jim Manico

Mark, when I was researching your history I saw that you used to refer to yourself as a Java bigot, and when I read that, I was like I like this guy. Then there is this dark period in your life where we can't find much about you. Then, next thing we see is a pipe sticking out of your head, and you are a full time member for life of the Microsoft Corporation. Can you tell us about that transition from Java to dot net?

Mark Curphey

Yeah, so at Charles Schwab I was responsible for software security. Schwab was one of the biggest software producers on the west coast. We had at one point nearly a trillion dollars in assets, vast infrastructures, and we were a Java shop. That is really where I cut my teeth learning about software security in the real world. I kind of plummeted into building a software security program and going ahead and implementing it. We had 350,000 developers. We had QA in Russia and dev all over the world, stuff going on in India. That is really where I kind of cut my teeth. I really learned a lot about large scale enterprise Java implementation. I think that is where the Java bigot stuff came from. I am at Microsoft now and a huge proponent of dot net, still a big fan of Java, and absolutely a huge fan of the open source model, both in terms of the open source business model and social model, so I guess if anything, I would kind of just remove the bigot part from my name now, if that's at all possible.

Jim Manico

Would you care to tell us more about what you are doing at Microsoft today?

Mark Curphey

Sure. I run the information security calls team. I have around 25 developers, full-time developers around 15, vendors, contractors working in the team, and we are split between Redmond, Hyderabad in India, and the sustained engineering team is in Beijing and China, so we own a couple of different functions, one of which is to build software security calls, so we have a manage code scanning tool, a static analysis tool called CAT.NET. We have a protection library called MTXFS. Then we have one of the threat monitoring tools. There are two threat modeling tools at Microsoft, so we have one of those called the Threat Modeling and Analysis tool, so that is in sort of the software security cell. We have another cell that does identify and access engineering, so engineering around the implementation of the Microsoft identity management stuff, or our internal deployment, so we obviously have a large number of employees and a large number of systems. We have a product called Forefront Identity Manager, so we own the engineering of the implementation side of that. In the past, we have built all the tools around group management for active directory and all sorts of clever things that I have. You know, [email protected], even though my alias is [email protected], things like that…Then we have another cell of folks who are building operations tools, so have some tools that go out and scan the network and find the vulnerabilities and unpatch machines, and all those sort of things. Also, then looking at data classification tools…so tools that can go out and pass through a SharePoint or through a FileStore or through an exchange mailbox and find where people may be accidentally, nudge nudge, storing sensitive data and things like that, so a bunch of projects going on in that space, software security, identity management, operations. The fourth area is security management, so we have a cell…This is one of the reasons that I went to Microsoft. We have a cell that is building security management tools, so think about things like business continuity management tools to track business continuity plans, high level risk tracking plans, information score cards, all of those sort of things. We are fundamentally underneath that, building a development framework, so the notion is that every large scale organization has a combination of off the shelf tools and they have custom tools. Normally, the majority of those custom tools are architecturally balls of mud. They have kind of grown up from something that was originally built out of proof of concept. Some guy takes that and puts it into production. The next minute you know the business is relying on it. Then there are five or six of these things trying to work together and all of a sudden none of it scales, so what we are building is a development framework that allows you to build scalable custom web application including the ability to integrate your office shelf stuff, so we have to pull in information from the likes of cat UPC code scanner or maybe even other code scanning tools, network scanning tools, hook in to detect tracking systems, notification systems, all that sort of stuff so you can build a proper architecture to support the information security system. The fifth cell is sustained engineering. That stuff happens out in Beijing and doing sort of CRs and change request things.

Jim Manico

This is a question from Jeff Williams. Mark, why at first did you think that XSS was a nonissue and in general, how did your opinion on cross-site scripting change over time?

Mark Curphey

So, I went back to Jeff, I guess who sort of had this conversation…so I still believe there are huge amounts of hype around it. Let me fully qualify that statement. I fully understand the potential of it and I fully understand that things have certainly moved on. If you go back a number of years, there are a number of people essentially saying that the world is falling down. When I first became responsible for software security at Charles Schwab, we were on the front page of the Wall Street Journal for that cross-site scripting issue. Notice that you had all sorts of monitoring and all sorts of stuff in place…The reality was that there were very few customers complaining about the issue, but there were an awful lot of media people, so what we were seeing was that it was very easy for security people to say the potential for this is, wow, this is going to happen to you and everything else is going to happen. The reality was that those things were not happening, so fundamentally, it is my belief that there are these things called risk management and you have to stand by that. It was really a stance around absolutely understanding that the potential impact of this is pretty large, but we are not seeing those things get exploited. Now, I think that what we have seen clearly over time as things become more and more connected, that those things are becoming exploited. We are starting to see worms replicated and those sort of things. One of the things I have always believed is that cross-site scripting has diverted a lot of people's attention away from other issues, which are absolutely significant and in many cases much bigger issues. For example, the majority of application security people you talk to will absolutely understand how cross-site scripting works. They will be able to tell you all of the issues, all of the attack vectors. Then, you talk to a lot of them about how a MQ would work or some sort of thing like that, which is maybe pushing billions of dollars a day from a back end system and has a potential for massive amounts of damage. They probably will not be able to tell you how those things work at all. I think that another part of cross-site scripting is that I think it diverted and continues to divert a lot of attention away from really big really important issues.

Jim Manico

Mark, developers' tools in general have gotten less expensive, and in fact, there are a lot of very solid developers' tools in the open source space, especially in the Java world, so because of that, it is very often difficult to justify budget with developer tools, so in general, what do you think are the best ways that an organization can justify spending real money on software security initiatives?

Mark Curphey

So we agree that CAT.NET that my team produces is free and the intention is for it to always be that way. I think that there are a lot of great tools. I think if you look at software security in general, an awful lot of people think that they are going to be able to buy a shiny red button, press it, and the problem is going to go away. The reality is that it is not going to work like that, so it's a people process and technology problem. You require the right people with the right education and motivation, the right skill set. You require the right process to ensure that all the right things are being done. Then, you require the right technology, either through technology choices using frameworks or those sort of things or technology to help secure implementations. If you could go back and could spend any money on anything, I am still a firm believer…I know it is a real old cliché, teaching a man to fish or whatever that phrase is. I really think that the best money spent is on educating motivated, skilled people into how to solve the problems. I think that becomes a scalable thing. You know, if you go back to the days when I was running software security at Charles Schwab, one of the real key things that we did was find champions out in the business and developers. Do not try to scale up by trying to clone security people. Try to find development champions who understand security and go and implement that stuff in their development cells. I really think that is incredibly wise money to spend it around on training. That is not to say that the other choices do not have significant benefits. Clearly they do. Like you say, there are a lot of great software, the threat modeling tools that we produce give that a plug, CAT.NET, and there are a bunch of other great tools that can get people a long way. If you are not focused on that, people in process …which, in general requires a different sort of approach, I think it is often money well spent.

Jim Manico

I have heard a lot of folks from the network security, and I dare say the WAF community, who say well, fixing the code is a real tough proposition. We have tens of millions if not tens of billions of vulnerabilities out in the web and due to the scale problem, we will never be able to fix all these vulnerabilities. Do you think there is any truth to that? Do you have any thoughts in general on the scale of the problem?

Mark Curphey

People say that you can't tackle it by that. You have to create some shiny red button. My answer, I will give you the Microsoft/ Steve Ballmer answer, is hogwash, right? That is just absolute utter nonsense. What we have seen over the space of the past decade is that people follow the money. Network security people are trying to follow into the space of software security. It does not transcend well because they do not understand software. What they are trying to do is to apply old network security processes to software. That is really what has happened with these web application files. People are trying to apply protocol level protection to a software. It is just never going to solve the problem. Sure, it can make inroads, depth in defense, all those things, absolutely. Ultimately, you have to be able to produce software which can defend itself and protect itself. You cannot stick something else in front of it, so I think that the argument around can you train developers is fundamentally mute. If you look at what Microsoft has done with the SDL and how we have changed, it is an absolutely fantastic story. I would not have come to Microsoft five or seven years ago. It would not have been a great place to work. Now, they have the SDL, which is kind of held up as a place where we still have problems and challenges. We're certainly not perfect, but we are a very different company than the company that was several years ago. That has been through the SDL process, which Mike Howard and Steve Lipner, and those guys created and a superb bunch of talented people there. Educating all the developers, putting them through that training and making them go through that process. I think that there is really good tangible evidence to say that, that is hogwash.

Jim Manico

Mark, do you think that compliance is helping or hurting actual web security?

Mark Curphey

I guess, I always kind of smile…You cannot spell compliance without alliance, whatever that Dilbert cartoon is, so I have sat on both sides of the fence, right? I have run consulting teams. I have been in startups as a vendor. I have spent an equal amount of time running corporate security programs, so I guess I kind of see it from both sides. I think though that the reality is when you are running a corporate information security program, compliance is just something that you have to do. It is a tax. There are implications if you do not do it, and get caught I guess. There are issues. Compliance itself is not a driver around building scalable and sustainable security programs. I think that there is a vast amount of media hype, a vast amount of horrible marketing in the security industry and technology industry, in general. They say things like you have to go do XY and Z, and you have to do all sorts of things. I can tell you that I have had conversations with regulators, agreed things, got off the phone with them and some vendor will call me up and say, if you have…you are going to be fined 100 million dollars. I had just come off the phone conversation with a regulator and agreed that here is how we are going to approach something, and this is how the implications are, so I think that there is a real gap about what is being portrayed in the media and the reality. I think that you need to always ask yourself, why is someone telling me this and what is their motivation behind it. It is kind of what Al Gore said in that film, whatever that climate film he did…If someone’s salary is dependent on something, then it is pretty hard to convince them of the truth. I think that there is a vast amount of fear, uncertainty and doubt being portrayed around compliance. That said, the balanced view, of course, is people setting sensible standards, sensible ways to do things and building programs around it out of either enforcement or auditing. It clearly makes sense, right? It is the right thing to do. It is raising the bar and driving the right behaviors. I do not think there is a clear no it is bad or yes it is good. I think that there are certainly implementation issues and problems with the way that stuff happens. I think that is the nature of anything where there is money to be made.

Jim Manico

What do you think is necessary to interact with an offshore or some sort of outsource team and still build secure software?

Mark Curphey

I have been involved in that model since the early Schwab days, and we had QA and Russia, offshore development going on in India. I have a team in Beijing now and team in Hyderabad. My view is that software is a social process, building it. It is a people process. If you wind up with disjointed teams, then you are going to wind up with problems that are not just security problems anymore. What you need to do is to ensure that you understand the process and understand the workflow of how software gets created, ensuring that each of the stages in the life cycle or however has roles and responsibilities, that they clearly understand what is expected of them. I think that it is just naive of people to think that they can just ship someone a specification and expect it to turn back and become secure. No one can ever write down a document or build a checklist of all of the different issues. It is a social process that involves the right people, the process, and the right technology, so I think that it absolutely can be done because I have watched it being done. I have also watched it completely and utterly fail. I have found a lot of customers that do offshoring have completely and utterly failed. I do not think that there is any one specific answer. The things that I have seen that have been successful are where customers have partnered with the vendors. They made sure that they understand what those expectations are. They do not just make the expectation that someone is going to deliver those things. A lot of people think that I am going to write some contract here, write some specifications, hand it over. We are going to validate it against the specification. If it is not correct, I am going to whip out my appendix and my contract. I think that it is just a more complex thing to do, and failing software is about people. You have to get all the right people aligned, understanding all the right expectations, roles, and responsibilities. That stuff generally just creates results.

Jim Manico

So, Mark, your blog is entitled The Security Buddha. What are some of the common assumptions that you see in application security as just illusions?

Mark Curphey

So I still believe that there are huge amounts of fear, uncertainty, and doubt being peddled around the frequency of exploitation and the reality of the impact. The potential impact of many of these issues, absolutely, certainly, without doubt…To give you an example, I have not spoken to Ingo Strunk for years. Ingo would probably laugh at this. Back in the early days, we were building an XML Java portal with Gay Perfel. Ben and Ingo were building this Java portal to host this XML system using SourceForge as a source code management system. At the time, there was a whole bunch of attention on OWASP. I think googles, gobbles, whatever they called themselves, used to do through vulnerability alerts or posting things about OWASP cross-site scripting and all sorts of issues, a lot of people focusing attention, trying to put egg on the face. What we discovered one evening was that Ingo had accidentally committed an XML file into the source code suppository that actually had the password for the database that was being hosted on the Internet. What was interesting about that was that it was an open source project with open source code, and that stuff had been out there for about three months, and no one had noticed it. The natural reaction was oh shit. Christ, what have we done? Maybe we have been owned for a couple of months. The reality was that there are all of those issues out there. It is an absolute bug farm everywhere and we are arguing about things being exploited all the time. That is not the case that everything that is out there and everything that is a problem is being exploited. We have got to get back to this whole thing of risk management. A lot of people talk about risk management, but very few people act on it. It is one of these things that tends to be a buzz word. People kind of do not necessarily take it seriously. There is definitely risk involved. You basically have to make a bet. Making a bet involves putting money on the table and being prepared to living and dealing with the consequences. I think that there are an awful lot of people…I hear people at conferences talking about if you find this vulnerability, it has to be fixed immediately. There is this very blanket, very matter of fact kind of attitude in the security industry, in general. They are saying if you get this type of issue it has to be remediated, end of story. In some circumstances, absolutely, those things make sense. There are a lot of people who just make these blanket statements about things without really understanding what the impact could be, the frequency of these things, and all of those things that surround making informed decisions.

Jim Manico

So Mark, I see that you are not so much active in OWASP today. May I ask why?

Mark Curphey

Sure. In the early days, what I always hoped OWASP would be was a forum of software developers who had a secondary interest in security. What has happened is that I think it has become a forum for security people who have an interest in software. That is absolutely fine, fantastic people involved. Obviously it is a wonderful community and all those sort of things. It is just not what I had envisioned. What really interested me was design patterns and architectures and real software security rather than vulnerabilities and enumeration of those issues. So, Dennis and I know I have tried to steer things…I think it is absolutely fair…Jeff, Dave, and I have had this conversation…If you want things to change, it is an open source project. You just suggest the change and make it happen. That is absolutely true. It is just that I think it has gone in one direction naturally with the mass of people behind it. That is just not the natural direction that made sense for me. That certainly does not take anything away from the project, you know, absolutely fantastic. I had the pleasure of meeting a guy, Ward Cunningham, who created the Wiki. Ward used to work for Microsoft. We sat in a meeting with Ward about how the Wiki evolved and all about the design patterns he did. He said what happens with a lot of projects is that they change. They grow organically, and if they grow organically some people come in and other people leave. Some people stay the course. It is just the way that it happens. When I spoke through that with Ward, I realized that the project has changed. It is just different. Go away and find other things that keep that interest. Allow that thing to blossom the way that it is doing, so that is why today…Still, I am on speaking terms with Sebastian out at the OWASP Bureau picnic last year. I actively talk with Dennis and a whole lot of other people. I am certainly involved and a big supporter behind the scenes. In fact, I actually sponsored OWASP, got Microsoft to join OWASP earlier on in the year…not that I am not a supporter or not engaged in that sense. I am just not actively participating and contributing.

Jim Manico

Well Mark, I am really grateful that you took the time to interview with us. Do you have any final thoughts before we finish up today?

Mark Curphey

Gosh, no, I mean apart from saying thanks, and I am honored to interview with you. It is a pleasure. I think that the most important thing is to spend some time reflecting back on what happened when we started this thing. You look at where it is today with being promoted by NIST and recognized by all these people all around the world…I think it is absolutely phenomenal. I think that has taught me, if you have a great idea, talk to a bunch of people and decide to do it. Then you can do all sorts of fantastic stuff. I keep seeing all these other interesting ideas coming around. It is great inspiration for people to be able to say if you have an idea, get a collective bunch of people. Use the power of the Internet and the power of collaboration to go make a difference. It is certainly something I look at with pride.