Learn more about this class!
"Jeremy and Thomas pull no punches, and bring years of in- side knowledge to bear on the murky world of security product testing" Stuart McClure, SVP Risk Management, McAfee Foundstone
"Thomas and Jeremy have both built and broken more commercial security products than almost anyone else in the industry, and a course sharing their tools and techniques for black-box product testing is long overdue"
Dug Song, Security Architect, Arbor Networks
This class offers a behind-the-scenes tour of the product evaluation process. Renowned security experts Jeremy Rauch and Thomas Ptacek offer a crash course on the most important aspects of validating - or debunking - security product claims. We'll show how to run a black-box test of a network security product, and provide an insiders view on how security products are designed - and marketed - to survive product bakeoffs.
What you will learn:
- Threat modeling applied to security products
- Verifying IPS/firewall performance claims
- Demystifying network security marketing jargon
- Evading detection and protection mechanisms
- This is a 2-day, lab/lecture lecture class.
Day one sets the stage for running a hard-core, black box test of a security product. We explain the concepts: how to reconcile a product's claims to your threat model and deployment environment. Then we introduce tools that will allow you to replicate highly advanced evasion attacks at the push of a button, and how to interpret the results.
Why bake off? Can I trust magazine reviews? What about NSS or ICSA? How vendors see it. How hackers see it.
Product Threat Modeling
What product criteria? What's a threat model? How do I turn them into test cases?
The Product Proving Ground
Our "mock" environment. Our "mock" products: "redwall" and "bluewall". Basic tools and techniques: traffic generation, inspection, attack simulation.
Critically Evaluating Marketing Pitches
Jargon. Buzzwords. "Red flag" features. What they hope you won't ask. The RedWall "pitch". The Green- Wall "pitch".
Understanding Network Security Hardware
ASICs. FPGAs. Signatures. Anomaly detection. Backplanes. TCAMs.
Black-Box Verification Concepts
Overview of testing steps. New attack simulation tools and lab.
Day one shows how to pop open the hood of a security product and test-drive it to see if it works at all. Day two dives into the product engine; can it really inspect 4 gigabits of traffic per second? Can a smart attacker walk right past it? How do I find out if I'm being asked to pay $50,000 per box for Snort? How do I know if deploying this product will actually make me less secure?
Real word traffic. Traffic patterns. Using your network to test. The vendor’s benchmark assumptions. Why they aren't your assumptions. Performance lab.
Network Security Features
Layer 3 evasion. Why TCP is so hard. App-layer evasion. Fingerprint evasion. Edge cases. Short sessions. Pipelining. Evasion lab.
Product Engine Deja-Vu
"This looks suspiciously like Snort". "Red-flag" features that are probably just free tools. Where to find them. How to evaluate them.
Putting It All Together
What it all means. Comparing different vendors. What features can I trust. Quantifying.
What have we learned? Unmasking our “mock” products.
Who should take this class:
Security administrators, architects, auditors, and consultants who are responsible for product selection or rely on commercial security products should take this class. Advanced networking and security competencies are required to gain the full benefit of the class, but no programming ability is necessary.
Attendees will need a to bring a laptop running Unix (or Mac OS X) or Windows and VMWare.
Students will conduct supervised lab evaluations of two security products and collaborate with other students and instructors on test planning.
Attendees will receive a course handbook, CD of tools and instructions.