Working with the results in Python (the results module and the report’s code blocks)
Every figure and every table of the Quarto report carries a folded block,
Reproduce this figure in Python (or Load this table in Python), with
the code that loads the same files and draws the same thing with MNE-Python
and matplotlib, ending with a savefig to a vector file. The idea: a figure
of the report is a starting point for a publication figure, not an end
product. Copy the block with the button in its corner, change colours, sizes,
time windows or channels, and run it in any Python that has MNE installed with
the repository on PYTHONPATH (the environment that ran the pipeline does).
The code is written by pipeline/mne_pipelines/kit_general_pipelines/results.py,
which is also the module to use directly.
Opening a dataset
import os
from pathlib import Path
from pipeline.mne_pipelines.kit_general_pipelines.results import Results
res = Results(bids_root=Path(os.environ["MEG_DATA"]) / "epilepsy-project") # the dataset under $MEG_DATA
# or: Results("pipeline/mne_pipelines/kit_general_pipelines/pipeline_config_files/epilepsy_project_config.yml")
res.stages() # which stages have outputs: number, name, folder, rows, last run
res.table(15) # any stage's summary table as a pandas frame (text columns)
res.rows(15, subject="0989", task="flippedsquares") # filtered by BIDS entities
res.subjects(); res.conditions()
Results(bids_root=...) opens a dataset directly and reads the settings it
needs (the template subjects directory, the coregistration policy) from the
newest configuration snapshot the stages saved under derivatives.
Results(config) resolves the dataset exactly as the stages do
(project.root_override or $MEG_DATA, joined with project.name).
Everything is located through the stages’ own summary tables (the CSV every
stage writes), never by guessing file names, so the module follows any
dataset, any subject and any renaming of the derivative folders the stages
themselves understand. Tables with no entity columns (stages 1, 3, 8) are
filtered through the entities of the file names they list. The module never
writes to the dataset.
No user-specific paths. The code blocks of the report reach the dataset
through the data-root variable (project.root_env, MEG_DATA by
default), never through the path of the machine that rendered the report,
so a block copied from a report runs unchanged on a colleague’s machine. The
summaries themselves hold the absolute paths of the machine that ran each
stage; res.local(path) (used by every loader) re-anchors such a path on
the dataset as found here, from its derivatives part on, when it does not
exist as written. When the dataset is not under $MEG_DATA
(project.root_override), the blocks open it through the configuration
file, by its path inside the repository (run them from the repository root).
Recordings
rec = res.recording("0989", task="flippedsquares", run="01") # session may be left out when unambiguous
raw = rec.raw("filtered") # mne.io.Raw of stage 7 (bad channels marked, ICA applied, filtered)
raw = rec.raw("converted") # the stage-2 FIF
events, event_id = rec.events() # stage-8 events (sample, 0, code) and the condition names
rec.raw("filtered").plot(events=events, event_id=event_id, block=True) # MNE's browser (full block: report, stage 7)
rec.events_table() # the per-event detail table of stage 8
epochs = rec.epochs("224") # mne.Epochs of one code
evoked = rec.evoked("224") # mne.Evoked of a code or of a condition name
rec.evokeds() # every code, keyed by code
rec.bad_channels(); rec.bad_channel_metrics() # stage 5: decision and per-sensor metrics
ica, record = rec.ica(); rec.ica_scores() # stage 6: solution with ica.exclude set, scores per component
rec.forward(); rec.noise_cov(); rec.inverse() # stages 14 and 15
stc = rec.stc("flippedsquares_reversal_to_a") # mne.SourceEstimate on fsaverage (stage 15)
rec.labels("flippedsquares_reversal_to_a") # ROI time courses (pandas)
rec.psd(); rec.psd_sensors(); rec.band_powers(); rec.band_powers_sensors(); rec.spectrogram() # stage 21
rec.tfr("flippedsquares_reversal_to_a"); rec.tfr_bands("flippedsquares_reversal_to_a") # stage 22
Subjects and group results
sub = res.subject("0989")
sub.coreg() # trans_fif, mri_subject, subjects_dir, source (manual / auto), qa: the
# transform the source stages use (source.coregistration policy)
sub.trans() # mne Transform
dip, record = sub.dipole("visual_reversal_bilateral", "left") # mne.Dipole over the window + the fit's JSON
sub.dipole_timecourse("visual_reversal_bilateral", "left")
sub.dipole_evoked("visual_reversal_bilateral") # the pooled evoked the fit was made from
sub.recon_qc(); sub.head_movement()
res.group_stc("flippedsquares_reversal_to_a", kind="condition") # stage 16 grand average (kind="contrast" for contrasts)
res.group_labels(...) # group ROI time courses
res.cluster_test("a_vs_b", level="source") # stage 17: row, T_obs, times, locations, clusters
res.rsa(); res.rsa("0989"); res.decoding("a_vs_b", "0989") # stage 18
res.template_subjects_dir # the SUBJECTS_DIR holding fsaverage
res.local(res.table(15).loc[0, "stc"]) # a summary path as a path on this machine
Every loader returns the standard MNE object (mne.io.Raw, mne.Epochs,
mne.Evoked, mne.SourceEstimate, mne.Dipole,
mne.preprocessing.ICA, AverageTFR) or a pandas frame, so the whole
MNE-Python API applies to it.
The report’s code blocks
results.snippet_for_file(path, res) returns the code behind a figure of
the pipeline from its path (stage and entities are read from the file name),
and results.snippet_for_table(csv, res) the code that loads a table.
report.qmd calls them for every figure and table it shows. A snippet lists
the data files it reads (Files: under the code). The off-screen brain
views use brain_render.BrainFrames (the renderer of stages 15 and 19);
MNE’s interactive stc.plot is given next to it for a window with a mouse.