GitHub has published a new set of best practices for teams evaluating large language models before pushing them into production. The guidance zeroes in on precision, recall, and testing against real-world workflows to keep security tools reliable.
Why precision and recall matter
The company's advice is straightforward: don't just measure how often a model is right. Look at precision and recall separately. Precision tells you how many of the model's flagged items are actually true. Recall tells you how many true items the model actually caught. For security tools, a low precision means teams drown in false alarms. Low recall means real threats slip through.
GitHub's guidance walks through how to set up these measurements in a way that fits actual use cases. A generic score isn't enough. The context of the task changes what acceptable trade-offs look like. A model that's fine for code suggestions might be terrible for vulnerability detection.
Testing against real workflows
Benchmarks and test sets only get you so far, according to the guidance. The company suggests evaluating models against the specific workflows where they'll be deployed. That means feeding in examples that mirror real queries, real code, real logs, and real user input patterns. It's not about making a model look good on a leaderboard. It's about knowing how it behaves when a security analyst actually uses it.
GitHub also cautions against relying on static test cases. Workflows change. Threats change. An evaluation process that isn't updated regularly can give teams false confidence. The company recommends building evaluation loops that refresh with new data from production and incident responses.
What this means for security tools
Security tools that lean on LLMs are getting more common. But a model that fails quietly can be worse than no model at all. If it doesn't recognize a known attack pattern because the workflow was never tested, that's a gap nobody notices until it's too late.
The guidance pushes teams to think about failure modes from the start. Instead of asking only whether the model works, ask how it fails. Does it miss certain types of attacks? Does it hallucinate when context is thin? Those answers need to be part of the evaluation before the model goes anywhere near production.
GitHub's advice isn't a one-time checklist. It's an ongoing practice. The company's own tools have evolved alongside these models, and the guidance reflects that experience.
For teams just starting to evaluate LLMs, the immediate next step is to define what precision and recall should look like for their specific security workflow. That's the base. Without it, everything else is guesswork.
