Software Quality Assurance - Revision history

Adam.muntner: /* How to protect yourself */

2010-03-11T00:24:52Z

‎How to protect yourself

Adam.muntner: /* How to protect yourself */

2010-03-11T00:24:15Z

‎How to protect yourself

KirstenS: /* How to identify if you are vulnerable */

2009-03-04T15:58:59Z

‎How to identify if you are vulnerable

KirstenS: /* Links */

2008-08-28T12:07:27Z

‎Links

KirstenS: /* Best practices */

2008-08-28T12:05:22Z

‎Best practices

KirstenS at 12:03, 28 August 2008

2008-08-28T12:03:11Z

Weilin Zhong at 19:04, 22 May 2006

2006-05-22T19:04:29Z

Weilin Zhong: /* Best practices */

2006-05-22T17:18:24Z

‎Best practices

Weilin Zhong at 17:17, 22 May 2006

2006-05-22T17:17:51Z

Weilin Zhong at 18:35, 18 May 2006

2006-05-18T18:35:14Z

New page

[[Guide Table of Contents]]
[[Category:OWASP_Guide_Project]]

@@ Line 122: / Line 122: @@
 '''Test Data'''
-PCI DSS Section 6.3.4, ISO 27002:2005 section 12.4.2, COBIT 4.0 AI3.3 - in a nutshell, all require that production data containing personally identifiable information not be used for testing purposes.
+PCI DSS Section 6.3.4, ISO 27002:2005 section 12.4.2, COBIT 4.0 AI3.3 all require that production data containing personally identifiable information not be used for testing purposes.
 * Require data and application owner authorization for requests of production data.

← Older revision		Revision as of 00:24, 11 March 2010
Line 118:		Line 118:

	* Verify the application’s behavior under load. For example, what happens if 1,000 users login simultaneously? Or if a flood of TCP/IP connections are established, but no SYNs are received?		* Verify the application’s behavior under load. For example, what happens if 1,000 users login simultaneously? Or if a flood of TCP/IP connections are established, but no SYNs are received?
		+
		+
		+	'''Test Data'''
		+
		+	PCI DSS Section 6.3.4, ISO 27002:2005 section 12.4.2, COBIT 4.0 AI3.3 - in a nutshell, all require that production data containing personally identifiable information not be used for testing purposes.
		+
		+	* Require data and application owner authorization for requests of production data.
		+	* Replace email addresses, phone numbers, IP addresses, mailing address, etc. anatomizing the data as much as possible.
		+	o Example: replace the name portion of the email address with a one-way hash value, leaving the domain.
		+	o Other techniques are character scrambling, numerical variance (example: a random number +/- 10% of the original value), nulling, truncating, encoding, aggregating.
		+	o If possible, have your Operations team obfuscate the data prior to restoration.
		+	o Ensure this is performed for any data that can uniquely identify an individual.
		+	o Consider the process whereby data is extracted, stored, transmitted, and sanitized, ensuring that live data remains protected.
		+	* Consider artificial data generation using regular expressions and pre-defined ranges of values to generate realistic but fake test data.
		+	o For new new functions with no historical data, this may be the only option.
		+	o This is the safest option.
		+	* The full dataset may be too large to use efficiently on slower test hardware. A smaller artificial dataset or a subset of obfuscated data may be necessary.
		+	* Enlist the assistance of a Database Administrator, the SQL language contains commands like RAND, REPLICATE, and REPLACE.
		+	* There are open source data generation tools available, such as
		+	o dbMonster - http://dbmonster.kernelpanic.pl/
		+	o Generate Data - http://www.generatedata.com

	==Links==		==Links==

@@ Line 107: / Line 107: @@
 * Cookie or credentials manipulation testing has been performed.
-* Denials of Service scenarios have been checked. It is understood how the application will perform in the presence of connection, login, or transaction flooding.
+* Denial of Service scenarios have been checked. It is understood how the application will perform in the presence of connection, login, or transaction flooding.
 ===How to protect yourself ===

@@ Line 120: / Line 120: @@
 ==Links==
-[[Guide Table of Contents]]
+[[Guide Table of Contents| Development Guide Table of Contents]]
 [[Category:OWASP_Guide_Project]]
 [[Category:Activity]]
 [[Category:Test]]
 [[Category:Maintenance]]

@@ Line 16: / Line 16: @@
 * Plan to test and quantify application security behavior during the QA process, just like any other system functionality.
 * Include the following considerations in your test plans:
-*# The policy compliance framework requirements
+** The policy compliance framework requirements
-*# Overviews of security testing methods, tools, training, and resource allocations
+** Overviews of security testing methods, tools, training, and resource allocations
-*# The operating budget and schedule considerations
+** The operating budget and schedule considerations
-*# Select a preferred vulnerability scoring system (CVSS, OVAL, etc.) and a management/tracking system (Bugzilla, a third-party vulnerability management package or service, etc.)
+** Select a preferred vulnerability scoring system (CVSS, OVAL, etc.) and a management/tracking system (Bugzilla, a third-party vulnerability management package or service, etc.)
-*# Establish and collect useful metrics that will facilitate decision making (for example, the count of open defects by severity and category, the arrival count over time, the close rate, total testing coverage, etc.)
+** Establish and collect useful metrics that will facilitate decision making (for example, the count of open defects by severity and category, the arrival count over time, the close rate, total testing coverage, etc.)
-*# Identify the testing activities which will be automation candidates and discuss how it will be done.
+** Identify the testing activities which will be automation candidates and discuss how it will be done.
 * Have a set of QA entry criteria, which identifies the items necessary to begin testing:
-*# Policy compliance validation requirements
+** Policy compliance validation requirements
-*# The applicable threat modeling scenarios
+** The applicable threat modeling scenarios
-*# The testing schedule, resource list, and budget
+** The testing schedule, resource list, and budget
-*# The metric and vulnerability scoring system selections
+** The metric and vulnerability scoring system selections
-*# An organizationally meaningful certification, which shows the QA team participated in design reviews and was satisfied with the security parameters of the system.
+** An organizationally meaningful certification, which shows the QA team participated in design reviews and was satisfied with the security parameters of the system.
-*# The completed test plans
+** The completed test plans
 * The QA exit criteria should include proof of application security integrity and readiness including:
-*# A summary report with charts, which summarize the collected metrics.
+** A summary report with charts, which summarize the collected metrics.
-*# A security testing report, which describes how well the application performed, compared to the policy compliance requirements and threat modeling scenarios, and its readiness compared to the established security baselines.
+** A security testing report, which describes how well the application performed, compared to the policy compliance requirements and threat modeling scenarios, and its readiness compared to the established security baselines.
-*# No outstanding high-severity security defects (for example, a simple list showing that all severity 1 security bugs have been resolved and verified).
+** No outstanding high-severity security defects (for example, a simple list showing that all severity 1 security bugs have been resolved and verified).
-*# An assessment which uses metrics to show that application security meets or exceeds established baselines, and that all security-related design goals have been met (that is, proof that the job is well done).
+** An assessment which uses metrics to show that application security meets or exceeds established baselines, and that all security-related design goals have been met (that is, proof that the job is well done).
 '''Note:''' Ideally, the reports should use visual presentation techniques whenever possible, via charts, graphs, and other methods for displaying the information visually, so the numbers are easy to comprehend and will facilitate the decision making process.

← Older revision		Revision as of 12:03, 28 August 2008
Line 1:		Line 1:
−	[[Guide Table of Contents]]__TOC__	+	[[Guide Table of Contents\|Development Guide Table of Contents]]__TOC__

	==Objective ==		==Objective ==

← Older revision		Revision as of 17:17, 22 May 2006
Line 1:		Line 1:
		+	[[Guide Table of Contents]]__TOC__
		+
		+	==Objective ==
		+
		+	The software quality assurance goal is to confirm the confidentiality and integrity of private user data is protected as the data is handled, stored, and transmitted. The QA testing should also confirm the application cannot be hacked, broken, commandeered, overloaded, or blocked by denial of service attacks, within acceptable risk levels. This implies that the acceptable risk levels and threat modeling scenarios are established up front, so the developers and QA engineers know what to expect and what to work towards.
		+
		+	==Platforms Affected ==
		+
		+	All
		+
		+	==Best practices ==
		+
		+	* Leverage the available resources like the OWASP Top Ten list, CLASP, or the policy compliance frameworks described in Chapter 5 and the threat modeling processes described in Chapter 7. These processes will help identify design parameters, establish measurable goals, and ensure that security testing proceeds in a systematic, thorough, and quantified fashion.
		+
		+	* Effective software quality assurance involves three complementary factors: Process, Metrics, and Automation.
		+
		+	* Plan to test and quantify application security behavior during the QA process, just like any other system functionality.
		+
		+	* Include the following considerations in your test plans:
		+
		+	*# The policy compliance framework requirements
		+
		+	*# Overviews of security testing methods, tools, training, and resource allocations
		+
		+	*# The operating budget and schedule considerations
		+
		+	*# Select a preferred vulnerability scoring system (CVSS, OVAL, etc.) and a management/tracking system (Bugzilla, a third-party vulnerability management package or service, etc.)
		+
		+	*# Establish and collect useful metrics that will facilitate decision making (for example, the count of open defects by severity and category, the arrival count over time, the close rate, total testing coverage, etc.)
		+
		+	*# Identify the testing activities which will be automation candidates and discuss how it will be done.
		+
		+	* Have a set of QA entry criteria, which identifies the items necessary to begin testing:
		+
		+	*# Policy compliance validation requirements
		+
		+	*# The applicable threat modeling scenarios
		+
		+	*# The testing schedule, resource list, and budget
		+
		+	*# The metric and vulnerability scoring system selections
		+
		+	*# An organizationally meaningful certification, which shows the QA team participated in design reviews and was satisfied with the security parameters of the system.
		+
		+	*# The completed test plans
		+
		+	* The QA exit criteria should include proof of application security integrity and readiness including:
		+
		+	*# A summary report with charts, which summarize the collected metrics.
		+
		+	*# A security testing report, which describes how well the application performed, compared to the policy compliance requirements and threat modeling scenarios, and its readiness compared to the established security baselines.
		+
		+	*# No outstanding high-severity security defects (for example, a simple list showing that all severity 1 security bugs have been resolved and verified).
		+
		+	*# An assessment which uses metrics to show that application security meets or exceeds established baselines, and that all security-related design goals have been met (that is, proof that the job is well done).
		+
		+	'''Note:''' Ideally, the reports should use visual presentation techniques whenever possible, via charts, graphs, and other methods for displaying the information visually, so the numbers are easy to comprehend and will facilitate the decision making process.
		+
		+	==Process ==
		+
		+	===Description ===
		+
		+	Utilize the test planning, test results, and metrics data to quantify the application security meets or exceeds the policy compliance and risk assessment goals.
		+
		+	===How to identify if you are vulnerable ===
		+
		+	The presence of a working process results in an operating culture having certain distinguishing characteristics. Make sure you see some or all of these operating in yours.
		+
		+	For example,
		+
		+	* The development team members are getting routinely updated on secure coding practices.
		+
		+	* Design reviews incorporate and encourage security considerations.
		+
		+	* The QA process includes planning and testing time for security assessments, instead of covering them as an afterthought or in an ad-hoc fashion.
		+
		+	* Security-related bugs are specifically tracked and have an established escalation policy.
		+
		+	===How to protect yourself ===
		+
		+	* Make “Security” an operating word in the engineering team’s vocabulary. Encourage training opportunities, discussions, coding examples, and on-going interest.
		+
		+	* Select and employ a vulnerability scoring system, such as CVSS, OVAL, or the like. Or at least make sure that security related defects have some special tracking method or tag.
		+
		+	* Make sure the question of “Got security?” comes up during design reviews.
		+
		+	* Establish a working escalation procedure for security-related defects.
		+
		+	==Metrics ==
		+
		+	===Description ===
		+
		+	The QA group will identify, select, and employ the meaningful metrics to provide the baseline measurement of application security. This baseline will serve as a comparison point for future assessments, too.
		+
		+	===How to identify if you are vulnerable ===
		+
		+	A good system of metrics provides a basis for the following:
		+
		+	* Summary charts, showing the security-related bug counts over time, their open and closure rates, and the progress towards policy compliance and risk assessment goals.
		+
		+	* The numbers necessary to answer management’s questions about “How secure is the application?” or “Is risk increasing or decreasing over time?”
		+
		+	* A known security defect density (that is, the average number of security bugs per unit of code is being monitored and the rate is going in the right direction: Down!)
		+
		+	===How to protect yourself ===
		+
		+	* Establish a working set of metrics. For example, count the number of high, medium, and low severity security bugs as a start. Follow with rate assessments, which will answer questions like, “How fast are security-related bugs being discovered in QA testing?”, “How severe are the bugs that are being detected?”, and “How complete is the testing coverage for the areas prioritized by our policy compliance or risk assessment goals?”
		+
		+	* Track that all security related tests have been checked (a simple spreadsheet will do).
		+
		+	* Automate the calculation and charting of the metrics as possible, so accurate information is available on-demand, even in a dashboard summary fashion.
		+
		+	* Make sure all high-priority security bugs are fixed and regression-checked, prior to software release.
		+
		+	==Testing Activities ==
		+
		+	===Description ===
		+
		+	===How to identify if you are vulnerable ===
		+
		+	Not every QA team will employ all of the following testing activities, but the more you employ strategically, the better your security assurance will be:
		+
		+	* Cross-site scripting and SQL injection tests have been run.
		+
		+	* An assessment of how well the application handles user input, including special or multibyte characters, excessively long strings, null inputs, or invalid values has been done.
		+
		+	* Cookie or credentials manipulation testing has been performed.
		+
		+	* Denials of Service scenarios have been checked. It is understood how the application will perform in the presence of connection, login, or transaction flooding.
		+
		+	===How to protect yourself ===
		+
		+	* Run user agent injection tests (cross-site scripting, SQL query injections, data manipulation checks).
		+
		+	* Check how the application handles user input that is ill-formed, too short or too long, or that contains special or multibyte characters.
		+
		+	* Check how sensitive the application is to cookie manipulation or session tampering.
		+
		+	* Verify the application’s behavior under load. For example, what happens if 1,000 users login simultaneously? Or if a flood of TCP/IP connections are established, but no SYNs are received?


	[[Guide Table of Contents]]		[[Guide Table of Contents]]
	[[Category:OWASP_Guide_Project]]		[[Category:OWASP_Guide_Project]]