Harvey, the legal AI company, and Baseten, an AI infrastructure provider, have completed post-training research on open-weight legal AI agents. The work targets three trouble spots that have slowed AI adoption in law: cost, governance, and domain expertise. Their findings suggest that open-weight models — rather than closed, proprietary ones — could deliver the performance law firms need without the usual trade-offs.
The three barriers the research tackled
Legal AI has been held back by high costs tied to running large models on expensive infrastructure. Governance is another hurdle: law firms need to know exactly how an AI handles confidential client data and whether it stays within ethical boundaries. The third barrier is domain expertise — off-the-shelf models often lack the nuance needed for tasks like contract analysis or case law retrieval.
Harvey and Baseten focused their post-training research on open-weight models, which allow users to inspect, modify, and run the AI on their own servers. The companies say the approach keeps control in the hands of legal professionals rather than cloud providers. They also note that open-weight systems can be fine-tuned more precisely for specific legal domains, cutting down on errors and hallucinations.
Why open-weight models are gaining attention
Most legal AI tools today rely on closed, API-based models. Open-weight alternatives let firms run the software on premises or in private clouds, which addresses data sovereignty concerns. The research indicates that after targeted post-training, open-weight legal AI agents can match — and in some tasks exceed — the accuracy of their closed counterparts.
Cost is another factor. Running an open-weight model on dedicated hardware can be cheaper per query than paying per-token fees on a commercial API. For firms processing thousands of documents, that difference adds up quickly. The research also found that governance improves because every inference stays within the firm's own infrastructure, making audits and compliance simpler.
Domain expertise, the third challenge, was tackled by training the models on a curated set of legal documents and judgment patterns. Harvey's existing work in legal AI provided the domain data, while Baseten contributed the infrastructure and optimization expertise. The result is an agent that can handle tasks like summarization, clause extraction, and due diligence with fewer mistakes.
The open-weight approach is not without friction. Deploying and maintaining these models still requires technical skills that many law firms lack. The research doesn't propose a turnkey product — it's a proof of concept that the pieces work together. Whether firms will invest in the necessary infrastructure or rely on managed services remains an open question. Harvey and Baseten have not announced a commercial product or a timeline for releasing the models.
