This site is the archived OWASP Foundation Wiki and is no longer accepting Account Requests.
To view the new OWASP Foundation website, please visit https://owasp.org

Difference between revisions of "Talk:Benchmark"

From OWASP
Jump to: navigation, search
(The diagonal's meaning: new section)
 
Line 1: Line 1:
== The diagonal's meaning ==
+
== The meaning of the diagonal ==
  
I don't think it's fair to call the diagonal line in the FPR/TPR chart a "random guess" line.  The FPR == TPR equation translates to FP/(FP+TN) == TP/(TP+FN), meaning FP*FN == TN*TP, or FP/TP == TN/FN.  The FPR > TPR area below the line does not put the tool into a "worse than guessing" shame list.  The last equation suggests a different interpretation of that area, "the error rate in reporting exceeds the correctness rate in staying silent".  --[[User:Eelgheez|Eelgheez]] ([[User talk:Eelgheez|talk]]) 20:01, 13 July 2016 (CDT)
+
I don't think it's fair to call the diagonal line in the FPR/TPR chart a "random guess" line.  The FPR == TPR equation translates to FP/(FP+TN) == TP/(TP+FN), meaning FP*FN == TN*TP, or FP/TP == TN/FN.  The FPR > TPR area below the line does not put the tool into a "worse than guessing" shame list.  The last equation suggests a different interpretation of that area, "the noise rate in reporting suspects exceeds the silence rate about non-issues". 
 +
 
 +
The "worse than guessing" interpretation seems to come from the following scenario.  We have 5 real and 5 fake vulnerabilitiesLet the tool (or a monkey) decide which vulnerability is real.  I guess this scenario ignores that the tool does not get the list of these vulnerabilities as its input. --[[User:Eelgheez|Eelgheez]] ([[User talk:Eelgheez|talk]]) 20:22, 13 July 2016 (CDT)

Revision as of 01:22, 14 July 2016

The meaning of the diagonal

I don't think it's fair to call the diagonal line in the FPR/TPR chart a "random guess" line. The FPR == TPR equation translates to FP/(FP+TN) == TP/(TP+FN), meaning FP*FN == TN*TP, or FP/TP == TN/FN. The FPR > TPR area below the line does not put the tool into a "worse than guessing" shame list. The last equation suggests a different interpretation of that area, "the noise rate in reporting suspects exceeds the silence rate about non-issues".

The "worse than guessing" interpretation seems to come from the following scenario. We have 5 real and 5 fake vulnerabilities. Let the tool (or a monkey) decide which vulnerability is real. I guess this scenario ignores that the tool does not get the list of these vulnerabilities as its input. --Eelgheez (talk) 20:22, 13 July 2016 (CDT)