Today's submission queue (synthetic)
| Case | Drug · Payer | Denial risk | Top driver | Recommended pre-flight action |
|---|
Risk scores are calibrated probabilities, not rankings: a 0.72 means 72 of 100 such cases historically bounced. Calibration is what lets you allocate finite reviewer time rationally instead of chasing a leaderboard.
Calibration (reliability curve)
Predicted denial probability (deciles) vs. observed denial rate. The senior framing: a sharp-but-miscalibrated model misallocates human review; a calibrated one converts directly into staffing math.
Model card (what production must handle)
- Decision first: the model exists to trigger an action: fix documentation pre-submission, or route to human review. Output is a work queue, not a score.
- Features: payer × plan × drug approval history, step-therapy documentation status (NLP-extracted from clinical notes), required-lab presence, prescriber historical approval rate, diagnosis-criteria match.
- Label hygiene: overturned-on-appeal denials are re-labeled; payer criteria are versioned because rules shift quarterly and stale labels poison the model.
- The feedback loop: if the model fixes submissions, observed denials fall and the training signal erodes. Maintained holdout traffic and importance weighting are in the design, not bolted on.
- Clinical NLP core: the highest-value features come from unstructured notes (symptoms, prior medications, failed therapies). Extraction quality is the model's ceiling.
One-pager: the first-pass approval machine
Every denial costs an appeal cycle (days to weeks), reviewer labor, and patient risk. A large share of denials are administrative, not medical-necessity: a missing lab, undocumented step therapy, a criteria mismatch the note actually satisfies but the form doesn't show. Prevention is strictly cheaper than appeal, and it moves the headline time-to-therapy metric directly.
- A triage queue ranked by calibrated denial probability, each case carrying its top risk driver and a concrete pre-flight action.
- The ops translation: flags cleared → projected first-pass lift → appeal-hours avoided → patient-days saved. Model quality expressed in the units the CEO runs the company on.
- A model card that takes the hard parts seriously: label hygiene under appeals, quarterly payer-rule drift, and the self-eroding feedback loop.
The score runs at submission-assembly time inside the existing workflow; flagged cases route to a fix-it queue with the missing artifact named. Holdout traffic preserves the training signal. The same NLP extraction layer that powers the features also powers the documentation-completeness pre-check: one investment, two products.
My ML research career started on exactly this substrate: extracting medications, symptoms, and side effects from scanned psychiatric notes at CAMH (Canada's largest mental-health hospital), then an MSc capstone on clinical NLP at the University of Toronto. Since then: privacy-safe ML data infrastructure for LLM training at Meta, and Head of AI at a digital-health startup building therapeutic conversational AI. Unstructured clinical text → reliable structured decisions is my home turf. Jeff Pinto · jeff@jeffpinto.com · jeffpinto.com
Sources & method
Why prevention beats appeal (the core claim): Medicare-Advantage plans denied 7.4% of PA requests in 2022; only 9.9% of denials were ever appealed, yet 83.2% of those appeals were overturned. Most denials are reversible but go un-appealed, which is exactly the waste a first-pass fix removes. KFF, 2024 (2022 data). kff.org/medicare/…
That many denials are administrative, not clinical: HHS OIG found 13% of MA prior-auth denials and 18% of payment denials actually met Medicare coverage rules, driven by clinical criteria not in Medicare rules and by "insufficient documentation" claims where records were in fact sufficient. The exact administrative-vs-medical-necessity split is described qualitatively, not as a published percentage, so we do not assert one. OIG OEI-09-18-00260, 2022. oig.hhs.gov/…/OEI-09-18-00260
The cost of an avoided cycle / appeal-hours: a manual PA costs the provider ~$12.88 and ~24 minutes of effort (~$5.38 / 16 min via portal); industry-wide there is a ~$7.50-per-PA and ~14-minute-per-PA savings opportunity from automation. CAQH Index Report 2024. caqh.org/…/CAQH_IndexReport_2024_FINAL.pdf
Patient-days at stake per prevented denial: a required PA adds ~4 days to biologic time-to-therapy at median and ~23 days when first denied; total biologic time-to-therapy averages ~42 days. The "~4.7 patient-days saved" headline sits inside this PA-attributable range. Burton et al., J Allergy Clin Immunol 2019/2021 (PMID 33404389); Arthritis Care & Research 2020 (PMC7062557).
Why this work is worth doing at all: 93 to 95% of physicians report PA-associated care delays and ~26% report a PA led to a serious adverse event. AMA Prior Authorization Physician Survey, 2024-2025. ama-assn.org/…/prior-authorization-survey.pdf
Labeled assumptions (no public anchor): the "~+14pp first-pass lift," the "~4.7 patient-days saved," the specific per-case risk probabilities, and the calibration deciles are illustrative, chosen to sit inside the reversible-denial headroom above, not measured. In production these are estimated from Forus's own labeled outcomes with a maintained holdout. CMS's 2026 Interoperability & Prior Authorization rule (72h expedited / 7-day standard decisions; CMS-0057-F) makes a pre-submission fix more valuable, not less.