Stage 11: FreeSurfer reconstruction quality control

Author: Hadi Zaatiti <hadi.zaatiti@nyu.edu>

What & why

Stage 11 of the KIT general pipeline puts every FreeSurfer reconstruction of stage 10 in front of a reviewer before stage 12 may coregister the MEG data to it, and prepares the MNE files that the coregistration needs. A reconstruction that looks complete can still be wrong in ways that ruin the coregistration and the forward model: a skull strip that removed cortex, a pial surface that leaked into the dura, a scalp surface clipped by the field of view (a cropped nose or vertex biases the head-shape fit), or a Talairach registration that failed. recon-all does not fail on any of these.

The stage is built on the same rule as the bad channel and ICA stages: nothing is accepted by an algorithm. It computes metrics, flags candidates for a closer look, draws the evidence, and asks a person for a verdict. Flags never decide; the verdict does. A verdict is tied to the files it was given on and is asked again when they change.

It runs on the Windows workstation in the neurowaves environment: FreeSurfer is not needed, the scalp tessellations are made with MNE’s surface tools from the lh.seghead that stage 10 built (the steps of mne.bem.make_scalp_surfaces after its mkheadsurf call; make_scalp_surfaces itself refuses to run without FREESURFER_HOME even when it reuses an existing lh.seghead).

What you need to provide

  • The stage-10 SUBJECTS_DIR (10-freesurfer_reconstruction/subjects/) with, per subject, a complete recon-all and surf/lh.seghead. The completion evidence is the same as in stage 10 (recon-all.done present, recon-all.error absent, outputs non-empty).

  • A desktop session for the review window (PySide6, matplotlib, pyvista in the environment). For the Open in freeview button on Windows, FreeSurfer in WSL with WSLg (graphical WSL applications).

  • Subjects recorded as template_fsaverage by stage 10 are listed with status: template_fsaverage and need no review.

Configuration file

freesurfer_qc:
  enabled: true
  save_plots: true
  outlier_z: 3.0            # cohort robust z (median/MAD, this dataset only) that flags a metric
  min_cohort: 5             # no cohort z flags below this many reconstructions
  talairach_z_min: -9.0     # FreeSurfer's own Talairach QA z against buckner40
  fov_clip_fraction: 0.002  # flag when more scalp vertices than this lie on a non-inferior volume face
  fov_clip_tolerance_mm: 3.0
  min_scalp_brain_mm: 3.0   # flag when the scalp comes closer than this to the pial surface
  mosaic_slices: 12
  freeview: true            # visual check in freeview (step 5 of the review); false removes the step
  freeview_auto_open: true  # open freeview when step 5 is reached; false waits for the button
  freeview_executable: null # a freeview binary to run as is (no FreeSurfer setup), any OS
  wsl_distro: null          # Windows only: null -> FREESURFER_WSL_DISTRO or Ubuntu-22.04
  override: {}              # verdicts given in the YAML, e.g. {0989: {verdict: pass_with_note, note: "..."}}

Cohort z-scores are computed within this dataset only, with the median and the MAD, because adult norms do not apply to children and a cohort of five is the minimum for a robust estimate. override records a verdict without the GUI (for a subject reviewed elsewhere, or on a headless machine); it counts as a human decision and needs a note like a GUI verdict.

How to run

conda activate neurowaves
python -m pipeline.mne_pipelines.kit_general_pipelines.11-freesurfer_qc \
    -c pipeline/mne_pipelines/kit_general_pipelines/pipeline_config_files/<project>_config.yml

Options:

  • --no-gui: compute the scalp surfaces, the metrics, the flags and the figures only; verdicts stay pending. Use it on a compute node or to prepare the figures before a review session.

  • --review-all: reopen the review window for subjects that already have a verdict.

  • --reset-review: discard the verdicts given in the GUI (YAML overrides stay).

  • --overwrite: recompute metrics and figures that are up to date.

  • --no-plots: skip the figures (the review window then shows metrics and flags only).

The default run reviews every subject without a current verdict. The VS Code launch entries are <project> 11 FreeSurfer QC (review GUI) and ... metrics only (no GUI).

What the script does

Per subject with an individual reconstruction:

  1. Completion and scalp surface: check recon-all completion and the presence of surf/lh.seghead; a subject failing either is reported (recon_incomplete, no_scalp_surface) and not reviewed.

  2. MNE scalp tessellations: write bem/<subject>-head-{dense,medium,sparse}.fif from lh.seghead into the subject’s FreeSurfer folder: the dense surface is the lh.seghead mesh in metres, the medium and sparse ones its decimation to 30000 and 2500 triangles (mne.surface.decimate_surface). The decimated meshes usually have a few dozen vertices with fewer than three triangles; they are written anyway, as MNE does, and the coregistration GUI of stage 12 and mne.dig_mri_distances read them with on_defects="warn".

  3. Metrics from the FreeSurfer outputs: surface holes before topology fixing and the Euler number (aseg.stats), Talairach QA and FreeSurfer’s z against buckner40 (talairach_avi_QA.log), eTIV, brain segmentation, cortex, cerebral white matter and subcortical grey volumes, mean cortical thickness and white surface area per hemisphere (?h.aparc.stats), grey/white contrast (?h.w-g.pct.stats), white-matter SNR (norm.mgz inside the eroded white matter), the field of view of the source T1w and whether -cw256 was used, the fraction of scalp vertices on each face of the conformed volume (FOV clipping), the minimum and 1st percentile scalp-to-pial distance, and the scalp mesh topology (Euler characteristic, open edges).

  4. Flags: robust cohort z-scores beyond outlier_z for the volumes, thickness, area, contrast and holes (once the cohort has min_cohort reconstructions), plus the absolute rules: Talairach z below talairach_z_min, scalp vertices on a non-inferior volume face beyond fov_clip_fraction, scalp closer than min_scalp_brain_mm to the pial surface, and a scalp mesh with open edges or handles.

  5. Figures: full field-of-view head coverage, white/pial/scalp contours on the T1 in coronal, axial and sagittal mosaics, the brain mask (skull strip), the aseg overlay, and a 3-D rendering of the scalp and the pial surface.

  6. Review window (default): one guided window per subject, see The review window below. The reviewer confirms each check in order, answers the visual check in freeview, and records one of pass, pass_with_note, fail_needs_edit, use_fsaverage with a note (required for everything but pass).

  7. Provenance: the verdict is saved with the identities of recon-all.done, T1.mgz, lh.seghead and the dense head surface. When any of them changes (a --force rerun of stage 10, a manual edit followed by recon-all), the verdict becomes stale and the subject is reviewed again.

The review window

The window walks through six steps with Checked, next / Back; every step shows its figures on the left and, on the right, written instructions on what to look at and what counts as a problem, plus the automatic flags where they matter. The header line shows which steps have been confirmed.

  1. Head coverage and brain mask (coverage and skull-strip figures): the whole head inside the volume, no clipped vertex or nose; the mask covers all cortex and cerebellum and no skull, eyes or dura.

  2. Surfaces on the T1 (coronal, axial and sagittal mosaics): the white surface (blue) on the grey/white boundary, the pial surface (red) on the outer cortex without dura, vessels or missing gyri, the scalp (yellow) on the skin and closed.

  3. Subcortical segmentation and 3-D surfaces (aseg and 3-D figures, Open 3-D view for the interactive scene): structures labelled and not merged; no holes, spikes or folds in the scalp and pial meshes.

  4. Metrics and automatic flags: the metric table with the cohort z and the flagged rows highlighted; flags tell where to look, they decide nothing.

  5. Visual check in freeview: freeview opens by itself (WSL on Windows) on mri/T1.mgz with lh/rh.white (blue), lh/rh.pial (red), lh.seghead (yellow) and the brain mask and aseg as hidden layers; the instructions say how to scroll, switch views and toggle layers. After closing freeview the reviewer answers All good or Not good with a comment (required for Not good). This answer is stored as the visual_check of the review (result: pass | fail | not_performed, comment, checked_at, reviewer, and the error text when freeview could not start). The step cannot be passed without an answer; if freeview cannot be used, the reason goes in the comment and the verdict step then requires a note.

  6. Verdict: a summary of the steps and of the visual check, the verdict buttons and the note. All good preselects pass (pass_with_note when a comment was given); Not good locks the verdict to fail_needs_edit and copies the comment into the note. The time each step was confirmed is saved with the review (checks).

How freeview is started. On Linux and macOS the stage runs it in the local bash; on Windows in the WSL distribution named by wsl_distro through WSLg (the subject folder is addressed as /mnt/<drive>/...). In both cases freeview is found in this order: freesurfer_qc.freeview_executable when set (run as is); freeview already on PATH (a shell whose profile sources SetUpFreeSurfer.sh); freesurfer.freesurfer_home; FREESURFER_HOME; then the usual install folders (/usr/local/freesurfer[/<version>], /opt/freesurfer[/<version>], /Applications/freesurfer[/<version>], ~/freesurfer[/<version>]), whose SetUpFreeSurfer.sh is sourced first. freeview’s output goes to 11-freesurfer_qc/sub-<id>/freeview_log.txt; a start failure is shown in the window with the last lines of that log. On Windows a WSL session that the service refuses for a moment (Wsl/Service/E_UNEXPECTED, typically right after another WSL process ended) is retried up to three times.

What the verdicts mean downstream

  • pass and pass_with_note: stage 12 opens the coregistration GUI with this reconstruction.

  • fail_needs_edit: stage 12 reports the subject as blocked. Fix the reconstruction (FreeSurfer edits, recon-all -autorecon2 -autorecon3, or a new T1w), rerun stage 10, review again.

  • use_fsaverage: the individual MRI is not used; stage 12 coregisters the subject to the fsaverage template like a subject without a T1w.

  • pending or stale: stage 12 skips the subject with the reason until a verdict is recorded.

Outputs

Under <bids_root>/derivatives/11-freesurfer_qc/ (and in the stage-10 subject folders for the scalp surfaces):

freesurfer_qc_summary.csv            # one row per subject, merged across runs
freesurfer_qc_metrics.tsv            # cohort metrics with robust z-scores
configs/config_<timestamp>.yml
sub-<id>/
  sub-<id>_desc-reconqc.json         # metrics, flags, verdict, note, visual check, provenance
  freeview_log.txt                   # output of the last freeview launch
  figures/
    sub-<id>_desc-reconqc_coverage.png
    sub-<id>_desc-reconqc_surfaces_{coronal,axial,sagittal}.png
    sub-<id>_desc-reconqc_skullstrip.png
    sub-<id>_desc-reconqc_aseg.png
    sub-<id>_desc-reconqc_scalp_3d.png
../10-freesurfer_reconstruction/subjects/sub-<id>/bem/sub-<id>-head-{dense,medium,sparse}.fif

freesurfer_qc_summary.csv columns: subject, mri_source, status (ok | recon_incomplete | no_scalp_surface | template_fsaverage | error), review_status (reviewed | pending | not_applicable), verdict, verdict_note, visual_check (pass | fail | not_performed), visual_check_comment, reviewed_at, review_source (gui | yaml_override), n_flags, flags, the headline metrics (used_cw256, surface_holes, lh_euler, rh_euler, talairach_qa, talairach_qa_z_fs, etiv_cm3, thickness, contrast, wm_snr, scalp_brain_min_mm, fov_clip_anterior, fov_clip_superior, fov_clip_max_flagged_face), scalp_files, record_json, status_message, error and mne_version.

Report

The report’s stage-11 section explains the step, shows the review status of the cohort (verdicts, notes, visual checks, flags), the per-subject QC table with the headline metrics, cohort statistics of the metrics, and the evidence per subject: the figures with the verdict, its note, the visual check result and the times the review steps were confirmed. The pipeline console shows the verdict per subject in its stage-11 column (pass in the done colour, other verdicts in the warning colour, pending verdict with the number of flags while no verdict exists).

Edge cases & safeguards

  • Small cohorts: below min_cohort reconstructions no cohort z flag is raised; the absolute rules and the figures still apply. Do not read the absence of flags as a pass.

  • Cropped field of view (used_cw256): look at the coverage figure; a clipped vertex or nose shows as scalp vertices on the superior or anterior face. The head-shape points digitised there will not find a surface in stage 12; pass_with_note is appropriate when the clipped region is small, use_fsaverage or a new T1w when it is not.

  • Scalp touching the brain (scalp_brain_min_mm low): the mkheadsurf threshold let the scalp surface fall inside the skull, often with a dark T1; raise freesurfer.scalp_threshold in stage 10 and rebuild the scalp surface.

  • Verdict given on old files: stale; the subject reappears in the review. Give the verdict again (or --review-all to revisit others).

Troubleshooting

  • freeview does not open: the window shows the error and sub-<id>/freeview_log.txt has freeview’s output. FreeSurfer not found: set freesurfer.freesurfer_home (on Windows the WSL path, for example /usr/local/freesurfer/7.4.1), FREESURFER_HOME, or freesurfer_qc.freeview_executable; There is no distribution with the supplied name: set freesurfer_qc.wsl_distro or FREESURFER_WSL_DISTRO; Wsl/Service/E_UNEXPECTED after the retries: run wsl --shutdown in a terminal and press Open freeview again; no window although the log is clean: WSLg is not available (wsl --update, then wsl --shutdown). On macOS freeview needs XQuartz only for old FreeSurfer versions; 7.x runs natively.

  • Scalp surfaces fail (no_scalp_surface): lh.seghead is missing or empty; rerun stage 10 (it rebuilds the scalp surface for complete subjects).

  • No figures in the window: the stage ran with --no-plots; rerun with figures.

  • A subject is missing from the table: it is not in subjects.include or stage 10 did not list it; check freesurfer_summary.csv.