AI for SOC Analysts: Alert Triage With a 9B Model

Yes, in the one published test that checks it. On Alert-Bench, published in October 2026, the open-weight Qwen3.5 9B scored an F1 of 0.952 on alert triage, the same score GPT-4.1 reached on the same 300 alerts under the same investigation protocol. The protocol did as much work as the model: four other open models scored between 0.098 and 0.777.

For a SOC analyst, that makes the first pass over the alert queue a job a small model can do inside your own environment, with the hard cases escalated to you. This post covers what the benchmark tested, what AI alert triage costs per alert on both cost dimensions, and where analysts still make the call. Every figure is dated and was read at its source on 30 October 2026.

What is Alert-Bench?

Alert-Bench is a benchmark for LLM alert triage that replays enterprise telemetry through a live SIEM, so each system has to retrieve its own evidence before it closes or escalates an alert. It was built by Saimon Amanuel Tsegai, Alex Kantchelian, Danfeng (Daphne) Yao and Peng Gao, and posted to arXiv on 7 October 2026.

The authors ran 1,247 alerts from a multi-stage attack scenario through five common approaches, from a single prompt with tool access to self-review and explicit verification. Every one of them missed at least 40.4% of the attack-related alerts. Attack alerts were more likely to be dismissed when a search returned no records.

They then designed AIDA, a multi-agent protocol: one agent proposes a decision, a second challenges it in a separate context, and a judge rules on the evidence, with every step written to an append-only investigation ledger. Dismissing an alert needs stronger evidence than escalating it. With GPT-4.1, AIDA reached an F1 of 0.958 on all 1,247 alerts, cut the false-negative rate from 40.4% to 3.1%, and escalated 18.4% of alerts to analysts.

Can a 9B model triage alerts like GPT-4.1?

On Alert-Bench, yes. The authors ran the full AIDA protocol with five open-weight models on a 300-alert subset holding 54 attack alerts, the same subset they used for GPT-4.1's ablations.

Qwen3.5 9B and GPT-4.1 recorded the same confusion matrix on this subset while disagreeing on two individual alerts. In the paper's words, "model size alone does not explain the ordering": the 9B model beat 70B and 120B models under the same protocol.

The result has limits worth stating. It comes from one attack scenario and one SIEM, the subset holds 54 attack alerts, the models ran with no task-specific training, and GPT-4.1 is the only closed model tested. It shows that a small model can do this job inside a good harness. Your own alerts are the test that decides whether it does the job for you.

What does AI alert triage cost per alert?

AI alert triage costs from about two US cents to about three US dollars per alert on frontier models, depending on how deep the investigation goes. The published figures, as of October 2026:

Under AIDA, GPT-4.1 used 33,942 tokens and 4.3 tool calls per alert, with 12.7 seconds of latency. RunReveal (28 May 2026), across more than 50,000 investigations, put full agentic investigations at "between $1 and $3 per AI SOC investigation for Sonnet", about US$2 per alert. Cyril Simonnet of Sekoia, writing on 18 October 2026, noted that Dropzone AI lists about US$36,000 a year for up to 4,000 investigations.

Volume turns those unit costs into a budget. Vectra puts the average organisation at 2,992 security alerts a day in 2026, with 63% unaddressed. That is about 1.09 million alerts a year. At AIDA's US$0.109 per alert, triaging all of them on GPT-4.1 would cost about US$119,000 a year; at US$2 per full investigation, about US$2.2 million. That arithmetic is ours, from the sources above, and the bill grows with alert volume, which the attacker and a noisy scanner influence more than the SOC does.

How does a trained model lower the cost of alert triage?

Costs come down on two fronts: a lower price per token than the frontier model, and fewer tokens per alert, because the instructions and protocol you repeat in every API call get trained into the model itself.

A lower price per token. As of 30 October 2026, OpenAI lists GPT-4.1 at US$2 per million input tokens and US$8 per million output tokens. OpenRouter lists Qwen3.5 9B at US$0.17 and US$0.25. On the 300-alert subset, AIDA with GPT-4.1 used 36,988 tokens per alert. If a 9B model used the same count (the paper does not report its token use) and every token were billed at the 9B model's output rate, triage would cost under one US cent per alert, about US$10,000 a year at Vectra's average volume.

Fewer tokens per alert. Much of AIDA's token count is protocol: role instructions, the ledger format and tool definitions sent again with every API call, for every alert. A model trained on the protocol and on your team's past dispositions carries that knowledge in the model, so each API call sends the alert and its evidence. How much that saves depends on how much of each prompt is repeated boilerplate, and on whether you already use prompt caching, which bills repeated context at a fraction of the input rate on frontier APIs. Measure the repeated share on your own traffic before you put a number on it.

Australia's ASD makes the same point from the buyer's side: frontier AI "carries cost and sustainability considerations, including token consumption", and organisations should pick capability "commensurate with the requirements and value of the intended use case" (Opportunities for AI in cyber defence, 8 September 2026). To work out your own numbers, use the cost-per-task calculator, and see AI cost per task for the method.

Where do SOC analysts still make the call?

Analysts still decide escalated alerts, high-impact response actions and the tasks where AI evidence is thin. Even AIDA sent 18.4% of alerts to a human, and that escalation path is part of why its false-negative rate fell.

• Containment and response. ASD's guidance has AI automate triage and prepare "pre-approved and reversible response actions", with "human approval required for high-impact containment, disruption, recovery or communication decisions."
• Detection engineering. On CTI-REALM, which tests writing detection rules from threat intelligence, the best published reward is 0.637 for Claude Opus 4.6. All 16 models tested came from Anthropic or OpenAI, so no open-model result exists yet.
• Malware analysis and threat-intel reasoning. CyberSOCEval, part of Meta's CyberSecEval 4, found 2025 models from OpenAI, Google, Anthropic, Meta and DeepSeek bunched between about 23% and 34% on malware analysis. We found no published 2026 results, open or closed, as of October 2026.

The workable shape today is a model on the first pass and an analyst on the escalations. Hugging Face's July 2026 incident was "initially surfaced through AI-assisted detection", where "LLM-based triage over security telemetry" separated real signals from the daily noise, and its responders then ran the forensic analysis themselves on an open model.

What should a SOC check before adopting AI triage?

Five checks separate a useful triage model from a demo:

1. Recall on attack alerts and the false-negative rate, measured on your own labelled alerts. Headline accuracy hides missed attacks, because most alerts are benign.
2. The escalation rate, since it sets your analysts' workload.
3. Tokens and cost per alert, at your volume, on both cost dimensions.
4. Where alert data goes. ASD's vendor question is "What data is retained, reused, or used to retrain models and under what conditions?"
5. The harness as well as the model. Alert-Bench shows the same model scoring very differently under different protocols.

For which models fit your own hardware, see the best local LLM for cybersecurity. For the larger open models, see GLM-5.3 and Kimi K3 cybersecurity benchmarks, and for picks across every security task, Best LLM for Cybersecurity (October 2026).

Run these models on your own security data

Run Qwen3.5 9B or a larger open model for your SOC on infrastructure we manage, in your own cloud tenancy, or on-premise. Your alert data stays private, usage costs become more predictable than a per-alert bill that grows with alert volume, and the model is in place before an incident, ready for forensic work on real attack data. We host models for defensive work: triage, threat intelligence, malware analysis and detection.

A model trained on your own alerts and analyst dispositions goes further than an untrained one, and that is what our endpoint does: new versions train on your traffic in the background, are scored against your evaluations, and go live only with your sign-off. The endpoint is currently invitation only, and we are developing a more streamlined version for wider access.

Book a call and we will work out the right model, size and deployment for your team.

Frequently Asked Questions

Will AI replace SOC analysts?

The evidence points to AI taking the first pass. In Alert-Bench, the best protocol still escalated 18.4% of alerts to analysts, and ASD's guidance keeps a human approval step for high-impact containment and response decisions.

What is AI alert triage?

AI alert triage uses a language model, usually with tool access to the SIEM, to gather evidence on each security alert and decide whether to close it or escalate it to an analyst. The model reads the alert, queries logs, and records its reasoning for review.

How much does AI alert triage cost?

As of October 2026, published figures run from about US$0.02 to US$0.11 per alert for structured triage on GPT-4.1, and US$1 to US$3 per full agentic investigation on Claude Sonnet. A self-hosted or trained small model lowers both the price per token and the tokens per alert.

Can I run AI alert triage on my own infrastructure?

Yes. Qwen3.5 9B, the open model that matched GPT-4.1 on Alert-Bench, can run on your own servers, in your cloud tenancy, or in an air-gapped network, so alert data stays in your environment.

---

Sources, read 30 October 2026: Tsegai, Kantchelian, Yao and Gao, "From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage" (Alert-Bench, 7 October 2026); RunReveal, "AI SOC investigation cost" (28 May 2026); Cyril Simonnet, "220 AI SOCs, sixteen prices" (18 October 2026); Vectra AI, "Alert fatigue"; OpenAI API pricing; OpenRouter, Qwen3.5 9B; Australian Signals Directorate, "Opportunities for AI in cyber defence" (8 September 2026); Chakraborty et al., "CTI-REALM" (March 2026); Deason et al., "CyberSOCEval" (2025); Hugging Face, "Security incident, July 2026" (16 July 2026).