EHR Risk Leakage-aware by design
Index truncation + audit job. See temporal diagram.
Features in detail
Everything the framework ships for the research workbench loop — from ingest to explainable prediction.
Free demo server — it may be slow. Check it with a small amount of data. For larger workloads or freer experimentation, run locally or on your own server. Open live demo
Calibrate + explain
Brier/ECE + SHAP in UI/API. See fine-tuning.
Data ingest
| Path | What it does |
|---|---|
| Bundled teaching CSVs | data/demo/ehr_data.csv (tiny; no index_time) and data/demo/sample_ehr.csv; horizon/diabetes tasks need data/raw/paper_synthetic_cohort.csv (has index_time) |
| File upload | POST /v1/datasets/upload — CSV into data/uploads/ |
| Form / JSON rows | POST /v1/datasets/from-form for small teaching examples |
| SQL import | POST /v1/datasets/from-sql (research DB; keyword denylist — use a read-only role) |
| Column mapping | API map-preview / map-import for BYO column aliases |
| Thin adapters | Optional OMOP / FHIR subset converters (lossy — see limitations) |
Dataset health gate
Before training, the API can report readiness blockers (too few patients, missing columns, etc.). The clinical_research persona can refuse forced training past blockers.
UI: Datasets → select → Run health → Continue to train when ready.
Feature engineering
- Longitudinal multi-window features — prefixes
w7d_,w30d_,w180d_(configurable). - Index time strategies — last event or named column (task YAML).
- Horizon labels — outcomes defined after the index, not mixed into past features.
- Patient vs temporal splits — choose group-aware or time-aware evaluation.
Train & compare
Models
Logistic regression, random forest, XGBoost (optional LightGBM). Task presets under tasks/ (diabetes, heart_failure, readmission_30d, custom).
Compare job
Multi-model ranking by hold-out ROC-AUC with optional promote-best to the active artifact.
Named runs
Artifacts under reports/runs/ with manifests (data hash, params). Browse and promote from Results.
Light HPO
Optional small hyperparameter grid on Train (POST /v1/jobs/hpo). Results presents the best trial and trial table, with unavailable metrics shown as n/a. Research-scoped — not full AutoML.
CLI
ehr-ai / python -m training.train for headless pipelines.
Trust surfaces
| Surface | How to run | Artifacts |
|---|---|---|
| Leakage audit | UI Train → Leakage / POST /v1/jobs/leakage-audit | reports/leakage_audit.json |
| Calibration | Train with calibrate flag | Brier, ECE, calibration plot |
| SHAP | Results → Generate SHAP | SHAP summary + explain on predict |
| Fairness helpers | Results → Run fairness / POST /v1/jobs/fairness | Group metrics (age bands / aligned CSV) |
| Analytics | Workbench Analytics | Cohort charts + ROC/PR/calibration curves; per-chart PNG + print/PDF |
| Trust pack | Auto after train; bind jobs with run_id | Per-run file reports/runs/<id>/trust_pack.json (not a dedicated /v1/reports/trust-pack route; flags also appear on GET /v1/runs/{id}) |
| External validation | Results / Research wizard / POST /v1/jobs/external-validate | external_validation_report.json |
| Analysis pack | Analytics KPIs + download / GET /v1/reports/analysis-pack | analysis_pack.json |
| Paper curves | Analytics after retrain / GET /v1/reports/curves | evaluation_report.json → curves, bootstrap_cis |
| Research wizard | Nav Research / route /research | Guided health → train → trust → leakage → external → export |
| Methods + ZIP | Results / wizard download (optional run_id) | methods.md + download.zip |
| Operating points | Results → Threshold table / GET /v1/reports/thresholds | Computed from evaluation scores; optional file reports/threshold_operating_points.json when written |
Angular workbench
Pages: Home, Research (wizard), Datasets, Train, Results, Analytics, Config, Predict, Docs. Use header Take tour for a per-page spotlight with short docs (Next / Back / Skip; optional “don’t auto-start”). Datasets can hide bundled demos, browse tabular rows, and delete selected allowed files; themes and density live in Config. End-to-end study loop: Research workflow. See the annotated screenshot tour.
Serve & explain
GET /v1/model/schema— feature columns + optional input stats (medians)POST /v1/predict— risk probability / level + optional SHAP explanation- Predict page — schema-driven form; Export session JSON for lab notebooks
- Results ZIP — metrics, audits, figures, trust pack (demo data only in public repos)
Extensibility
Task YAML
Encode horizon, windows, index strategy once; reuse from UI or CLI.
Workspace config
config/workspace.yaml — persona, compare models, UI prefs.
Tests
Pytest API contracts + Angular unit tests for workbench pages.
Honest non-goals (no LSTM production path, no full CDM, not a device): see Limits and repository LIMITATIONS.md.