Free kit

Free AI Tabletop Exercise Kit for SMBs

Sign up for the Cyber Defense & AI Brief and get it free.

Get the free kit

Purple team methodology: the technical counterpart to the Cyber Defense Tactics AI tabletop exercise

Cyber Defense Tactics Detection Evidence

Tabletop exercises test your decisions. Detection Evidence exercises test your detections.

Purple teaming for the agentic era, measured by evidence.

Detection Evidence is a facilitated, defensive purple team methodology, delivered by Cyber Defense Tactics as a managed service. Your own team runs benign, pre-approved validation activity in your environment. We review the detection evidence (logs, alerts, and telemetry), map coverage and gaps against agreed threat scenarios mapped to MITRE ATT&CK and MITRE ATLAS, with OWASP Top 10 for LLM Applications and OWASP API Security Top 10 risk categories for AI agent and API paths, and record whether your detections fired, and how fast.

Based on the Detection Evidence Framework, open sourced by CarbeneAI and Cyber Defense Tactics.

Who it is for

Teams that need proof their detections fire

  • Security engineering, SOC, detection, and incident response teams that need proof their detections fire.
  • Organizations in healthcare, financial services, and public safety. The framework has a section for each.
  • Teams running AI agents or exposing APIs that want detection coverage on those paths measured, not only bugs reported.

The methodology

What the Detection Evidence methodology produces

Every exercise produces the same six outputs.

01

Scope and safety sign-off

Systems in scope, agreed threat scenarios, who approved them, and the stop conditions. Signed by you and Cyber Defense Tactics before any work begins.

02

Validation plan

A short list of benign validation steps for your team to run, chosen for your threat scenarios and environment, with the reason for each.

03

Scoring

Every validation step scored as Prevented, Detected-actionable, Detected-weak, Logged only, Missed, or Not testable.

04

Backlog with acceptance tests and retests

Each miss and weak hit becomes a backlog item with an owner, an acceptance test, and a retest.

05

Executive summary

One page: counts by result, named misses, and what is being fixed.

06

Coverage map and detection recommendations

A coverage map against the agreed scenarios, plus drafted detection and runbook improvements for your team to implement.

A downloadable kit of the methodology's templates is planned for later. Today the methodology is delivered by Cyber Defense Tactics.

Facilitated service

Detection Evidence Exercise

Cyber Defense Tactics facilitates the methodology with your team: one scoped exercise, a reviewed written report, and a backlog with owners and retests.

  1. 01

    Scope and authorize

    Written scope and authorization: systems in scope, agreed threat scenarios, stop conditions, and sign-off by both sides before any work begins.

  2. 02

    Analyze

    We review the detection rules, log samples, and alert data you provide, map coverage against scenarios mapped to MITRE ATT&CK and MITRE ATLAS, with OWASP risk categories for API and AI agent paths, and draft validation steps for your team.

  3. 03

    Validate and observe

    Your own team runs the agreed benign validation steps in your environment, on a shared schedule. We observe and record which detections fired and what evidence was captured.

  4. 04

    Score

    Record what your telemetry and detections actually did, using the six result levels.

  5. 05

    Backlog and report

    Turn gaps into backlog items with retests, then deliver a reviewed written report and walk you through it.

Every exercise is authorized in writing: systems, scenarios, and stop conditions are signed by you and Cyber Defense Tactics before any work begins. Anything not listed is out of scope, and either side can pause a validation step at once.

It is not a penetration test or a red team engagement. Cyber Defense Tactics does not run attacks or exploits, and by default does not run commands or tools in your environment. Your team runs benign, pre-approved validation activity, and Detection Evidence measures whether your detections fire, and how fast.

Analysis is AI-assisted and done by Cyber Defense Tactics staff, and every finding is reviewed by our staff before release. You receive written reports only: no access to the AI model, and no raw AI output, transcripts, or session logs.

Side by side

Tabletop vs Detection Evidence

Tabletop exercises test your decisions. Detection Evidence exercises test your detections.

AI Tabletop ExerciseDetection Evidence Exercise
What it testsDecisions, roles, communicationDetections, alerts, response runbooks
How it runsFacilitated discussion around a scenarioHands-on: your team runs benign, pre-approved validation steps in a scoped, safety-gated way, while Cyber Defense Tactics observes and reviews the evidence
Who takes partExecutives, IR, legal, comms, ITSecurity engineers, SOC, detection and IR teams
What you getAfter-action report and improvement listA reviewed written report: scored evidence, a coverage map, a backlog with acceptance tests and retests, and an executive summary
Best usedQuarterly, rotating scopesAfter tabletop gaps, or on its own, per scope

They work as a pair. A tabletop surfaces gaps in decisions. A Detection Evidence exercise proves whether the detections behind those decisions fire. Each one's findings feed the other's backlog. See the tabletop exercises.

The practice

Our team

Before Cyber Defense Tactics, our team built SOCs at several companies across 6 countries, led tabletops and war games while employed at Booz Allen Hamilton, EY and AT&T DFIR, and led response to high-profile incidents and breaches in those roles. We know this work from the inside.

Our lead practitioner holds the CISSP, CISM, CRISC, GCFA, and GSTRT certifications, has more than 10 years in incident response, and has hands-on experience with how AI, web and API attacks work, from bug bounty work, which informs the detections we review.

We facilitate the exercise, and we draft and audit the detections.

Cyber Defense Tactics is a practice of CarbeneAI.

Request a scoping call

Education and training

AI and API security training

Attacker-perspective training the team has completed in AI and API security. It informs how we review detections:

  • Arcanum: Bug Bounty
  • Arcanum: Attacking AI
  • Arcanum: Red Blue Purple Team AI
  • Evolve Security: OSCP bootcamp

FAQ

Questions

How is this different from a tabletop exercise?
A tabletop tests how people decide and communicate during an incident. Detection Evidence tests whether your telemetry and detections fire when your team runs benign, pre-approved validation activity, and records the result.
Is this a penetration test?
It is not a penetration test or a red team engagement. Cyber Defense Tactics does not run attacks or exploits, and by default does not run commands or tools in your environment. Your team runs benign, pre-approved validation activity, and Detection Evidence measures whether your detections fire, and how fast.
Does it cover APIs and AI agents?
Yes. Threat scenarios can include API and AI agent paths, mapped to MITRE ATT&CK and MITRE ATLAS, with risk categories from the OWASP API Security Top 10 and the OWASP Top 10 for LLM Applications (from the OWASP GenAI Security Project). We map your detection coverage on those paths; we do not attack them. Every exercise is authorized in writing: systems, scenarios, and stop conditions are signed by you and Cyber Defense Tactics before any work begins. Anything not listed is out of scope, and either side can pause a validation step at once.
Do we get access to the AI?
Analysis is AI-assisted and done by Cyber Defense Tactics staff, and every finding is reviewed by our staff before release. You receive written reports only: no access to the AI model, and no raw AI output, transcripts, or session logs.
Can we run it ourselves?
The Detection Evidence Framework is free under CC BY 4.0. The Detection Evidence Exercise is Cyber Defense Tactics facilitating it with your team. A downloadable kit of the methodology's templates is planned for later. Today the methodology is delivered by Cyber Defense Tactics.
Who delivers it?
Detection Evidence is a facilitated, defensive service, delivered directly by Cyber Defense Tactics as a managed service. Your team runs the validation steps; we review the evidence and write the report.
What does it cost?
Pricing is on request. Write to contact@carbene.ai or request a scoping call.

Pricing for the Detection Evidence Exercise is on request. Write to contact@carbene.ai or request a scoping call. Cyber Defense Tactics is a practice of CarbeneAI, LLC.