Black Hat USA Training 2005
Caesars Palace Las Vegas • July 23-24 & July 25-26

Learn more about this class!

"Jeremy and Thomas pull no punches, and bring years of in- side knowledge to bear on the murky world of security product testing" —Stuart McClure, SVP Risk Management, McAfee Foundstone

"Thomas and Jeremy have both built and broken more commercial security products than almost anyone else in the industry, and a course sharing their tools and techniques for black-box product testing is long overdue"
—Dug Song, Security Architect, Arbor Networks

Overview:
This class offers a behind-the-scenes tour of the product evaluation process. Renowned security experts Jeremy Rauch and Thomas Ptacek offer a crash course on the most important aspects of validating - or debunking - security product claims. We'll show how to run a black-box test of a network security product, and provide an insiders view on how security products are designed - and marketed - to survive product bakeoffs.

What you will learn:

• Threat modeling applied to security products
• Verifying IPS/firewall performance claims
• Demystifying network security marketing jargon
• Evading detection and protection mechanisms
• This is a 2-day, lab/lecture lecture class.

DAY ONE
Day one sets the stage for running a hard-core, black box test of a security product. We explain the concepts: how to reconcile a product's claims to your threat model and deployment environment. Then we introduce tools that will allow you to replicate highly advanced evasion attacks at the push of a button, and how to interpret the results.

Introduction

Why bake off? Can I trust magazine reviews? What about NSS or ICSA? How vendors see it. How hackers see it.

Product Threat Modeling
What product criteria? What's a threat model? How do I turn them into test cases?

The Product Proving Ground
Our "mock" environment. Our "mock" products: "redwall" and "bluewall". Basic tools and techniques: traffic generation, inspection, attack simulation.

Critically Evaluating Marketing Pitches
Jargon. Buzzwords. "Red flag" features. What they hope you won't ask. The RedWall "pitch". The Green- Wall "pitch".

Understanding Network Security Hardware
ASICs. FPGAs. Signatures. Anomaly detection. Backplanes. TCAMs.

Black-Box Verification Concepts
Overview of testing steps. New attack simulation tools and lab.

DAY TWO
Day one shows how to pop open the hood of a security product and test-drive it to see if it works at all. Day two dives into the product engine; can it really inspect 4 gigabits of traffic per second? Can a smart attacker walk right past it? How do I find out if I'm being asked to pay $50,000 per box for Snort? How do I know if deploying this product will actually make me less secure?

Performance Qualification
Real word traffic. Traffic patterns. Using your network to test. The vendor’s benchmark assumptions. Why they aren't your assumptions. Performance lab.

Network Security Features
Layer 3 evasion. Why TCP is so hard. App-layer evasion. Fingerprint evasion. Edge cases. Short sessions. Pipelining. Evasion lab.

Product Engine Deja-Vu
"This looks suspiciously like Snort". "Red-flag" features that are probably just free tools. Where to find them. How to evaluate them.

Putting It All Together
What it all means. Comparing different vendors. What features can I trust. Quantifying.

Wrap-Up
What have we learned? Unmasking our “mock” products.

Who should take this class:
Security administrators, architects, auditors, and consultants who are responsible for product selection or rely on commercial security products should take this class. Advanced networking and security competencies are required to gain the full benefit of the class, but no programming ability is necessary.

Prerequisites:
Attendees will need a to bring a laptop running Unix (or Mac OS X) or Windows and VMWare.

Student Expectations:
Students will conduct supervised lab evaluations of two security products and collaborate with other students and instructors on test planning.

CLASS TAKE-AWAY
Attendees will receive a course handbook, CD of tools and instructions.

For over 10 years Jeremy Rauch has worked at the forefront of information security. An original member of the ISS X-Force and a co-founder of SecurityFocus, Jeremy is the discoverer of innumerable security vulnerabilities in widely used commercial products.

Thomas Ptacek is an internationally recognized authority, responsible for the discovery of many of the Internet's most serious security vulnerabilities. Thomas' groundbreaking paper, "Insertion, Evasion, and Denial of Service", is taught in university courses, cited widely in industry and academia, and guides the design of modern intrusion defense tools.