AvalonThe customer this was delivered for.Achieved 100% Precision and 83% Recall in Prior Authorization Question Generation
Key results
- 60 policy documents managed
- 100% precision in clinically accurate follow-up questions
- 83% recall across policy documents tested
Avalon managed over 60 complex, frequently updated policy documents used to determine patient eligibility for diagnostic tests. The prior authorization process was manual and time-consuming, requiring staff and providers to sift through extensive documentation. This created operational inefficiencies and increased the risk of delays or errors in patient care. Avalon partnered with Tribe to build a customized proof of concept using large language models to extract key information from policy documents and generate medically accurate prior authorization questions. The team used Claude 3 Opus to parse policy language, identify relevant coverage sections, and produce structured clinical questions for human reviewers. The PoC also supported PDF uploads, model selection, auto-generated test lists with manual adjustment, and a feedback loop to refine outputs with human-in-the-loop oversight. The PoC pilot achieved 100% precision in producing clinically accurate follow-up questions for human reviewers and delivered 83% recall across the policy documents tested, exceeding internal benchmarks. These results supported fewer eligibility assessment errors and faster review times. Avalon also shifted its policy documentation review cadence from annual to monthly cycles and planned to expand beyond the initial four test policies.
Tribe AI