Study Lifecycle: From Draft to Final Archive

A study is a governed object: it is created in one state, moved only by specific roles through specific states, and closed only by archiving. The state machine is the workflow — every queue item, every button on a study, and every audit entry follows from it. This page is the complete map: the states, who acts in each, what is persisted at each transition, and the two recovery paths for a study that did not go right.

The states

The enum has ten members, and those ten names are exactly what the UI prints on the badges, the tabs and the stepper (app/constants/workflow.py). Eight form the forward path; Rejected and Failed are the returns.

State	Label	Who acts	What happens there
DRAFT	Draft	Analyst (creator)	The study is configured — identity, tested party profile, parameters, data file — and can be edited freely
STUDY_INITIALIZED	Study Initialized	The system picks it up	The submission landed: the data file is attached and the analysis run is dispatched
PROCESSING	Processing	The system (analysis pipeline)	The run executes step by step; no user action advances it, and an in-flight run can be cancelled back to Draft
IN_REVIEW	In Review	Manager (the responsible role)	The comparables grid and the statistical results are worked: approve, send back with reasons, override dispositions
REPORT_GENERATED	Report Generated	No user transition — the report task sets it (the document task sets it too, when a master report is requested before approval)	The Excel workbook exists and its file id is on the study
DOCUMENT_GENERATED	Document Generated	No user transition — the document task sets it	The Word master report (and, best-effort, the branded PDF) exists on the record
SIGNED_OFF	Signed Off	Partner (or Firm Admin)	The firm’s final approval is recorded; the study is ready to archive
ARCHIVED	Archived	Partner (or Superadmin)	The study is closed out and immutable — the only terminal state
REJECTED	Rejected	Analyst (responsible role)	A sent-back study waits with its recorded reasons; re-submitted, it goes straight back to In Review
FAILED	Failed	Analyst (responsible role)	A run that errored waits to be re-run; re-submission sends it back into Processing

The state the study is in decides everything else: which buttons render, which roles can act, and what the next transition is. The study page answers “where is this study” with its state stepper; the roles and permissions page has the full permission matrix. Each state also carries a responsible role — the role the queue is grouped by — which is None for the two system legs (PROCESSING, REPORT_GENERATED) and for ARCHIVED.

The forward path, and what each transition persists

Each move is a single audited action; what is written to the record at each step:

Transition	Triggered by	Persisted at the transition
Draft → Study Initialized	The analyst attaches the data file and saves the submission	The study row with its parameters block and file pointer; a job row is created (Pending, Queued substate) and the run is enqueued
Study Initialized → Processing → In Review	The pipeline — system-only, no user involved	The full analysis output on the study: statistics, the tested party position, the conclusion, the per-comparable record with reasons; the raw dump with its column mapping and detected years; and in_review_at, the review-entry timestamp that starts the dump-retention clock
In Review → Report Generated	The reviewer approves — deferred, see below	The approval writes a REPORT_REQUESTED audit entry and dispatches the Excel report job; the status itself, the workbook file id and the real STATUS_TRANSITION entry land only when the report task succeeds
Report Generated → Document Generated	The final-document task, on the partner’s request	The Word master report file id (report_docx_file_id), and the PDF id when the best-effort PDF leg succeeds
In Review → Report Generated → Document Generated	The same final-document task, when the report is requested before approval	The approval leg is not skipped: the workbook is built in that run (or reused if the study already has one) and each transition gets its own audit entry, so the study arrives at Document Generated rather than sitting at In Review holding a client document
Document Generated → Signed Off	The partner or firm admin	The sign-off transition, actor, role, timestamp and notes
Signed Off → Archived	The partner (or a superadmin on their own study)	The lock: locked_at and locked_by. From this point the study accepts no edits and no further transitions

Three properties of the forward path:

• Two legs are system-only. PROCESSING and REPORT_GENERATED have an empty role list — no user can push a study out of them. Processing advances when the run finishes; Report Generated is set by the report task.
• Approval is deferred on the report. Approving at In Review records the request and starts the workbook build; the study stays In Review until the workbook is stored, then the task moves it. The state always reflects a real deliverable, and a failed build leaves the study where it was — there is nothing to revert. A duplicate approval is refused while a report job is active.
• Sign-off waits for the document. The sign-off button renders only at Document Generated, so the seven-chapter report is a step on the path, not a side action.

The working states: rejected and failed

These are not dead ends; they are the workflow’s repair paths.

Rejected — sent back with reasons

The reviewer sends a study back from In Review to Rejected. The justification notes are required: an empty send-back is refused with a 422, because a blank reason would land blank in the audit log and in the analyst’s notification. The reasons ride on the transition’s audit entry, and the audit taxonomy reserves a distinct REJECTION_REASON_CHANGE action for amending them.

There is also an email leg: the rejection sends a best-effort mail to the assigned analyst — the creator if nobody is assigned — and is skipped when the rejecter is the recipient. A failed send never fails the transition.

Re-submission goes straight back to In Review — no re-run (REJECTED → IN_REVIEW is the canonical forward step from that state), because the analysis already exists. A rejection the analyst disagrees with is answered by addressing the stated reasons, not by resubmitting the same record.

Failed — re-run the pipeline

A pipeline error — a file that fails validation at run time, a storage failure, a step that could not recover — lands the study in Failed with the error recorded on the study and the job, and sends a best-effort email to the assigned analyst (the creator if nobody is assigned) naming the step and the error. FAILED → PROCESSING re-runs the pipeline from the start; the failed run’s partial state is replaced by the new run’s output.

Cancel from Processing

While a run is in flight, an analyst (or a firm admin, or a superadmin) can send the study back to Draft and pull it out of the queue — the only user-invokable move out of Processing, and it requires justification notes. Any other manual attempt from Processing is refused with a 403: studies in Processing transition automatically.

Who can act, by role

The lists below are the backend’s. The study’s state endpoint computes this user’s rights server-side and the action buttons follow that flag; the role lists mirrored in the frontend are an offline fallback, not the authority.

Role	Acts
Analyst	Draft and Study Initialized (submit), cancel Processing back to Draft, re-submit from Rejected, re-run from Failed
Manager	Owns In Review: approve, send back, enter the review workspace, override comparables; may also advance a Draft. Not the re-submit from Rejected or Failed
Partner	Approve or send back at In Review; generate the final document; sign off at Document Generated; archive at Signed Off. Not the comparable-override right — the review workspace is manager/admin — and not the re-submit either
Firm Admin	Everything the Manager and the Partner do except the archive close-out, plus the re-submit from Rejected and Failed
Superadmin	Partner-level rights plus the re-submit, scoped to the studies they created

Re-submitting a study out of a repair state is the originator’s job: the backend allows REJECTED → IN_REVIEW and FAILED → PROCESSING for Analyst, Firm Admin and Superadmin only. Archival is the narrowest gate in the machine: only PARTNER and SUPERADMIN may move a study to Archived, and the check is hard-coded ahead of the role lists. Every other transition that the state machine or the role does not allow is rejected with an error — 400 for an illegal pair, 403 for an unauthorised role — and nothing is mutated locally.

Audit logging: every transition is a record

Every state change writes an audit log entry in the same operation as the move:

• The move itself — the from-state, the to-state, the actor with name and role, the timestamp, and the justification notes where there are any.
• The deferred case — an approval at In Review is logged as REPORT_REQUESTED, not as a transition to Report Generated, because at that moment no report exists. The STATUS_TRANSITION row is written by the report task when the workbook is actually stored.
• The surrounding record — creation (STUDY_CREATION), every field edit (STUDY_UPDATE, previous and new value), every assignment change (ASSIGNMENT_CHANGE), and every comparable override (REVIEWER_OVERRIDE) all land in the same study-scoped log.

The log is append-only and is the study’s chronology: the Audit Trails page aggregates it across the firm, and the study page renders its own slice. Transitions also publish events onto the activity feed, which is the same history in time order.

Idempotent runs and re-runs

The analysis run is a queued background job, and the re-run paths above lean on that:

• The job is idempotent on state. The pipeline advances a study into review only from the states a run legitimately starts in; a duplicate or late-arriving run cannot double-advance a study that has already moved on.
• Transient failures retry. A run that hits a transient error retries before it is marked Failed; a data-validation failure fails immediately with the specific reason named.
• The queue is bounded. Ten analysis runs queue globally and three per firm at a time; a submission over the cap is rejected with a retry message and the study stays where it was — nothing is queued silently. The report queue is separate and caps at five.
• Progress is polled, not pushed. The study page polls GET /jobs/{id} for the substate — Queued, Preparing Data, Quantitative Analysis, Basic Enrichment, Qualitative Screening, AI Screening, Final Analysis — and the stepper renders it; a polling error surfaces once and does not lose progress. The optional Advanced AI Screening step appears only when the tenant holds the advanced_ai_screening plan feature.

Where the state shows up

• The study page — the state stepper and the action footer, which render only the transitions the current state and the viewer’s role allow.
• The study list — tabbed by state, which is how a practice sees its workload by stage.
• The queue — My Tasks and Approvals is the per-role view of the same states.
• The activity feed — the firm-wide record of the same transitions.

The review step, in detail, is in Manager Review; the document, sign-off and archive are in Partner Sign-Off and Final Archiving.

FAQ

Can a study skip In Review? No. The pipeline is the only thing that moves a study into review, and the forward path is the only path — Draft, Study Initialized, Processing, In Review, Report Generated, Document Generated, Signed Off, Archived, with Rejected and Failed as the working returns.

Does approval put the study in Report Generated straight away? No — and this is deliberate. Approval records the request and dispatches the workbook build; the status changes when the file exists. If the build fails, the study is still In Review and the approver is told by the job and by email.

What happens to a study’s data if it fails mid-run? The partial run’s intermediate state is replaced by the re-run’s output when the study is re-submitted. The study row, its configuration and its file are untouched; the failed run’s error is the recorded reason.

Is there any state a study can get stuck in? Every working state has an exit owned by someone — approve or send back at In Review, re-submit from Rejected, re-run from Failed, sign off at Document Generated. The state with no exit is Archived; that is the point of it.