Episode 10.11 - The problem with surveys

As I've said before, I get a lot of "studies" and "surveys" from cybersecurity firms with breathless and urgent warnings about a coming cyber-pocalypse of one sort or another. Funny thing, it's always about something that they supposedly defend against. As I started writing this note, I got another one.

I did one podcast about a survey from Huntress about phishing in February, which was actually pretty good. Then I did one a couple of weeks ago about a less-than-good survey from iProov. Well, my partner in Germany, Patrick Boch, wanted to get into the fun and we decided to talk about two more of these that were also less-than-good from HiddenLayer and Ontinue. No, we didn't interview representatives from either company on this one. We were just having some fun at, unfortunately, their expense.

Here are some of the highlights of our discussion.

• Many cybersecurity surveys lack scientific rigor, often using small, potentially biased samples (e.g., 250 IT decision-makers)
• Reports frequently make vague assumptions or present data in ways that may exaggerate threats or market demand
• Deep fake attacks, while concerning, are currently not as prevalent or successful in cybercrime as often portrayed
• The Verizon Data Breach Investigation Report (DBIR) is considered a gold standard for its concrete terms and unbiased approach