Seven months ago I posted about one letter: the “P” in TTPs. A few of us were going to turn adversary procedures into a formal STIX 2.1 object, and I said I would write up how. Between then and now, work and life got in the way, but we built it. This is the long version.
The short version: the procedure extraction pipeline reads a source (threat report, a tweet, or an incident case), pulls out how the adversary operated as first-class x-procedure objects, orders them into an attack flow, and walks the result past four human review gates before anything ships. The output is a self-contained STIX 2.1 bundle that any STIX consumer can read and, optionally, a graph. It is open source under Apache 2.0 at github.com/netandneedle/procedure-extraction-pipeline.
Fig 1 — the Explorer's flow view. One report's procedures in kill-chain order, an OR operator where the source describes a branch, and the selected procedure's techniques, observables and command line.
A quick disclaimer: I consider myself a subpar coder, so I leaned on LLM to help with the coding and wiring. We ran many rounds of unit, integration, and end-to-end tests, and worked real reports through the gates by hand. I am sure there are bugs here and there. The project is meant to be hosted locally for a small team, single user, no authentication, every port bound to loopback. If you want to scale it up for large-team production use, fork it and modify it. Some credit back to me would be nice =).
Procedures as a first-class object
The industry has been talking about TTPs for years, and two “Ts” are quite solid with robust definitions and taxonomy. Tactics tell you the adversary’s goal, techniques tell you the method, and both have ATT&CK identifiers, a catalog, and a place in every tool.
The “P” is where the actual work happens, and it lives in PDFs, Sigma rules, analyst notebooks, and institutional memory. ATT&CK’s own “procedure examples” are one-sentence prose attached to a group or software page. While they confirm that a technique was used, they don’t tell you exactly how to emulate or detect that procedure.
Fig 2 — an extracted procedure, with self-contained fields and inherent relationships.
That is a knowledge-transfer problem. No universally adopted way exists to write a procedure down or consume one, so procedure knowledge doesn’t move between teams, let alone between organizations. CTI, adversary emulation, and detection engineering all touch procedures, and without a common artifact the handoffs between them stall.
So the first job was to define the thing. This is the definition the whole pipeline anchors on, and it took more revisions than any of the code. A white paper co-authored with John McFarland on the taxonomy of what a procedure is will come out shortly. But the summarized definition is:
A procedure is a discrete, repeatable technical implementation that integrates one or more techniques, often spanning multiple tactics, to fulfill a specific adversarial objective as an atomic event within an attack sequence.
To unpack that statement:
Atomic means irreducible at the level of the objective. If a step splits into two parts that serve different objectives, it is two procedures.
Repeatable means it’s a recipe, not a sighting; the same recipe run against five hosts is one procedure seen five times.
Technical implementation means structured with commands, observables, a sequence of operations.
Fig 3 — one procedure and the three sets it has to carry. The name never contains the actor; attribution is a graph edge.
A procedure is well-formed when it carries three things. The techniques it implements, resolved against the ATT&CK catalog and never invented. Where a defender would see it, derived from ATT&CK’s own detection chain (technique → detection strategy → analytic → data component), so the detection side is on the object from the start. And its components: one process object per command line the source actually printed, with the binary it ran as a file. Names follow [Verb] [Object] via [Tool/Method], so Download web shell via certutil, never “APT-something downloads a web shell.” The actor sits on a relationship where it belongs.
Why extend STIX 2.1 instead of inventing a format? Because STIX is the interchange standard for threat intelligence, and it has a sanctioned mechanism for extension definition. A bundle carrying x-procedure objects stays spec-compliant, and any STIX consumer can at least read it. A procedure that lives only inside one platform with unique properties solves the owner’s problem, and nobody else’s.
We added elements of attack flow because a list of procedures is not an intrusion. The order and the branches are the intelligence, and that is the part a flat technique list throws away. Sequencing inside the bundle uses precedes relationships plus the attack-operator and attack-condition objects from MITRE CTID’s Attack Flow, an open specification, so a bundle reads as an attack chain rather than a list.
Credit where it’s due: the original procedure methodology draws on Tidal Cyber’s work on procedures, which John McFarland and I expanded into a mathematical approach. The sequencing vocabulary is MITRE CTID’s.
Use cases
The whole pipeline was built with threat-informed defense in mind. The output should be useful for teams within the cycle, such as CTI, adversary emulation/red team, and detection engineering.
CTI analysis
Reconstruct an intrusion as the source describes it into a structured attack flow, so you can analyze the sequence of procedures step by step rather than as a flat list of techniques.
Compare procedures across intrusions and threat groups such as TTP trend analysis, shared choke points, and which behaviors recur regardless of who is behind them.
Ask the graph “which procedures implement T1059.001, and what did they touch?” or “which procedures from different reports share two or more techniques?”
Each vendor account is maintained independently. When two distinct reports describe identical behavior, they generate separate procedure objects that share a name and fingerprint while remaining linked to their respective sources. Matching fingerprints indicate that two entries are worth comparing, not identical: the hash incorporates tactics, platforms, and techniques, explicitly excluding the specific CVE or tool.
Adversary emulation
A bundle can serve as a blueprint for adversarial emulation. Each procedure contains actions and, where applicable, components such as
processobjects carrying the command line and the binary it ran.Follow the order and the branches:
precedesrelationships give the sequence, and the AND/OR operators mark where paths split or join.Hand the result to Attack Flow tooling using the mapping in the docs. We are working on an attack flow converter script. But hey, we’re not stopping you from taking a crack at it =)
The bundle and emulation planning is based on the source itself. Since the pipeline does not fabricate command lines or suppose an action if the source does not contain it, the quality and depth of the adversarial attack thread is as good as the source itself. So a low confidence reflects a sparse source document, not model hallucination.
Detection engineering
Trace each behavior to ATT&CK’s detection chain through the log-source references on the object, which project data components and analytics onto the procedure.
Since procedures are stored systematically into the Neo4j database, we can cluster similar Ps together and evaluate detection/defensive strategies for groupings of adversarial behaviors.
We decided that it’s better if we don’t ship out with detection rules. The pipeline does not emit Sigma or any other rule format. This is in recognition that detection engineering processes differ for each entity. Therefore the output focuses exclusively on capturing the target behavior rather than generating the rule itself. That being said, each procedure object should contain all the elements one would need to generate detections.
The pipeline
Defining the object was only half the battle. Doing manual extraction (reading threat intel, breaking it down into procedures, mapping to techniques, extracting command lines, connecting the workflow, and formatting everything into a valid STIX bundle) is tedious and simply doesn’t scale. A good day brings dozens of reports, while a very busy day brings hundreds. With modern models now reliable enough to assist, this pipeline accelerates the entire process, which keeps the analyst in the loop, or running fully automated if you prefer (we call it the YEET mode).
Fig 4 — sixteen stages in one state machine. Seven of them call on an LLM model, the rest are ordinary Python, and four pause for a human to check.
The pipeline consists of sixteen stages, each checkpointed to a database after execution so that gate pauses stay independent of the initiating process. While seven stages use model calls, the remaining stages run deterministically, a deliberate boundary reflecting the principle that verifiable data should be retrieved rather than inferred.
The clearest example is technique mapping. LLM models have a training cutoff where a model doesn’t know about a technique MITRE added or renamed after it was trained, and it will happily produce a plausible-looking ID for one it half-remembers. So the model is never the source of truth for ATT&CK. The pipeline keeps an up-to-date catalog of official techniques saved directly on your computer.
When identifying a technique, the model suggests options from memory and a search tool automatically scans the local catalog for the best matches. The system then checks every suggested ID to remove made-up techniques and/or update any outdated ones. A second model call then picks from the candidate pool, and each pick must include a verbatim quote from the source as evidence.
For semantic retrieval, we run SecureBERT 2.0 locally. SecureBERT is a lightweight, 570 MB sentence-embedding model trained on security domain text. Threat intel reports and ATT&CK technique descriptions often describe identical activity with different phrasing. Relying only on term-matching mechanisms may miss connections like “impersonating IT support” versus “Impersonation,” while specialized embedding models such as SecureBERT compare underlying intent. Having the model locally hosted helps deterministic scoring across runs, eliminates API costs, and decouples retrieval performance from your chosen LLM backend. We also included a standard keyword search fallback if you prefer to bypass the model download.
The four gates
These review gates represent the core value of our approach: a sequence of clear decisions where human judgment ultimately provides the final outcome.
Gate 0, entities. Are these the right named things, with the right roles? Actors, malware, tools, vulnerabilities, indicators, and which organization is the victim versus the author of the report.
Gate 1, procedures. Are these the right procedures and in the right order? The chunks are drawn as a graph over the source text. You can edit, drop, add, rewire the edges, and mark whether a fork is AND, OR, or a condition the source describes.
Gate 2, techniques. Are these the right techniques for each procedure? Picks the model was less sure about sit in a review lane where you can promote them into the bundle.
Gate 3, bundle. Is this the right bundle? The relationships, reviewed as a graph, before anything is serialized.
A reject sends the run back to the stage that produced the mistake, with your reason attached to its prompt. Every gate can be switched off per source if you trust that stage for that kind of report.
Fig 5 — the source queue. A CISA advisory paused at Gate 0 with 125 entities waiting for review.
Offload, not outsource
We deliberately split model execution into two tiers:
Heavy-lifting extraction runs on a cheaper, mid-tier LLM that makes about nine calls per report, keeping processing costs down.
A separate, higher-capability model acts as our reviewer, reading the document once to generate recommendations and reasoning for each decision gate. Running distinct models avoids the rubber-stamp effect: if the evaluator shares the exact same architecture as the extractor, it just nods along to its own work and inflates accuracy metrics without giving you real oversight.
The gates work in three modes:
Interactive mode keeps human judgment front and center while the framework suggests recommendations.
Assist mode puts the AI reviewer’s notes right next to each piece of data.
Then there’s the mode which I call “Skynet”: fully automated execution without intervention. While automated processing works fine most of the time, it tends to miss subtle details a human analyst catches immediately, so enable it at your own risk. Checking quoted evidence against source documents helps, but it isn’t foolproof.
The point is to offload to the model, not to outsource to it.
The goal is to offload execution to the model, not to delegate judgment. I still strongly advocate for having an analyst/human examine the source document before evaluating each review stage, as this workflow directly preserves and strengthens your team’s subject matter expertise. That said, the pipeline offers flexibility through the configuration toggle if you prefer a different approach. (Again, YOLO I guess.)
The feedback flywheel
Inconsistency will always exist in threat report processing and analysis. When two analysts break down a single document, they produce two distinct outcomes. Ask the same analyst again on Tuesday, and they yield a third. Introducing an LLM does not resolve this friction because it merely adds another opinion to the mix. For the pipeline to succeed, human corrections must continuously refine the pipeline’s operation, steering both the model and analysts toward a unified baseline.
The simple (but not the smartest) fix is to edit the base prompt every time something goes wrong. But this can be tedious and impossible to maintain. The prompt grows, the rules contradict each other, and a rule written for a ransomware incident report gets applied to an AI-phishing catalog because both were filed under “wrong technique.” What you want is a store of corrections, a retriever that picks the relevant few, and a scoreboard that says which ones are earning their place.
Fig 6 — one correction becomes a rule, is retrieved only when it is relevant, and is graded on whether it helped.
Here is how it works in order:
Capture. Every decision at a gate that is not a plain approve is recorded: what the pipeline produced, what you changed it to, and your comment if you left one.
Synthesize. Once you finalize the bundle, the system runs a single model call to convert your edits into reusable rules. Each rule includes clear examples of what works and what doesn’t, tagged with tags for specific techniques, entities, and tactics so it’s easy to reuse later.
Retrieve. When running a new report, the pipeline pulls only the rules that apply to that specific document instead of searching the entire database. It ranks matching guidelines by checking details like ATT&CK IDs, software, and entity types, then adds the top fifteen directly to the model’s context window. This keeps things fast and simple without needing a complex vector database.
Settle. We create a “receipt” for every rule the model sees. The rule scores a hit if a human reviews that gate and the issue doesn’t happen again. It scores a miss if the mistake recurs. Unreviewed gates stay unscored because the model agreeing with itself isn’t real proof.
We decided to skip using a vector database to store the rules for now. However, we’ll likely need a more complex solution as the ruleset grows. RAG and optimizing LLM memory is definitely outside of our expertise, so let us know if you have a better solution for the feedback flywheel!
How salience works
The salience metric determines which directives get injected into the prompt context. It combines three distinct factors: past performance (giving new entries the benefit of the doubt), a 45-day fade-out period for inactive rules, and a boost for edits made repeatedly over one-off changes. If the model ignores a rule, it usually means there was a formatting issue, which you can fix in the Feedback tab.
Fig 7 — examples of recorded rules from a source run.
That tab is where the analyst controls live. A rule can be edited, dismissed, pinned so it is always injected regardless of ranking, or promoted to a denylist. The denylist is the only rule that strictly forces the system’s behavior: it uses fixed code to block specific terms after the model finishes, so a blocked term can never be re-added automatically. A human must manually choose to add something to this list.
As we mentioned, we’re not memory wizards or prompt-whispering geniuses here. In fact, our first attempt was so clumsy that about one in seven generated rules was pretty weak, so we had to throw them out. Our current workaround is basically keeping a copy of your exact edits, literal before-and-after notes, and trusting those over the LLM’s overconfident generalizations whenever they disagree. If you actually know what you’re doing in this space and have a non-dumb way to fix this, please reach out; our contact info is on the About page.
The output
The output is exported as a self-contained .json STIX 2.1 bundle. It packages everything necessary for standalone parsing: individual procedures, associated ATT&CK entities, referenced threat actors, software, and tools, the required pair of extension definitions, an overarching report object, and the underlying graph relationships. Observables are framed exclusively as STIX cyber observables; we did not include indicators, since rule generation belongs to a separate lifecycle. Validation operates on a strict, all-or-nothing basis. Schema correctness, reference integrity, and Attack Flow structure must validate fully before serialization.
Abridged from real serializer output on a synthetic fixture:
The component it points to is a plain STIX process object carrying the command line as the report printed it:
You can also load bundles directly into Neo4j alongside the base ATT&CK catalog. The catalog stays untouched: each pipeline-generated node carries a distinct tag, letting you roll back or strip out a single report’s graph contributions independently. One graph-side detail matters for analysis: the procedure-to-technique relationship is stored as IMPLEMENTS_TECHNIQUE rather than USES, because the catalog already holds eighteen thousand USES edges and “both used PsExec” should not read as technique overlap. The bundle keeps standard STIX; only the graph edge is renamed.
That is what makes the comparisons possible. A technique, and every procedure that implements it, with what each one touched:
A few lines longer gets you procedures from different reports that share two or more techniques, grouped into clusters, which is where you start looking at how one behavior varies across the groups that use it. Both queries are in the object’s documentation.
What’s next?
We’ve been processing a lot of open source reports and have been continuously tuning the pipeline, so there will likely be some updates to the project here and there. However, we make no promises to maintain it, answer issues, or review pull requests on any schedule. Fork it, change it, keep the license and the notice, and a link back is appreciated.
That said, our goal is to compile enough procedures to begin deeper analysis and cluster adversarial behaviors and intrusions. Some of the core questions that I personally want to start getting answers to include:
Which technique or a combination of techniques carries a high amount of procedure variations?
Do intrusion sets rotate through variations of a similar procedure? Or do they keep to doing the same exact behavior over a period of time?
With enough data on attacker sequences, are we able to quantitatively measure and deploy high-impact defensive chokepoints against intrusions?
There are obviously a lot of other interesting insights that we might be able to derive once we have a large enough corpus of data. So more to come!
Meanwhile, if you process threat reports and break down adversarial behavior for work or interest, feel free to try out the project and share your insights with us!