Stage 8: build MNE events from the triggers

Author: Hadi Zaatiti <hadi.zaatiti@nyu.edu>

What & why

Stage 8 turns the trigger information of each recording into an MNE events array (N x 3: sample, previous value, event id) saved as a .eve text file, plus an audit TSV with one row per detected event. Stage 9 reads these files to build epochs and evoked responses.

Three variants exist; pick the one that matches how the triggers were coded during acquisition:

Script

Input

Use when

8-kit_compute_events_single_channel.py

raw .con

Single-channel mode: each event type on its own analog line (KIT 224 to 231). The event id is the KIT channel number.

8-kit_compute_events_multi_channel.py

raw .con

Binary-coded mode: several lines rise together to encode one code. Channels rising within events.multi_channel.max_onset_skew_ms are combined into one event.

8-kit_compute_events_STI014.py

stage-2 .fif

The digital STI 014 stim channel already carries the event codes (the FIF was created with the kit2fiff stim settings). Uses mne.find_events. This is the KidLang variant.

All three write into their own derivatives/<script-name>/ folder, name the outputs with the recording’s BIDS entities plus desc-<label>, and register what they produced in an index CSV. Stage 9 reads events.desc and events.source from the YAML to find the right files.

What you need to provide

  • Analog variants (single and multi channel): the raw BIDS recordings sub-<id>/[ses-<id>/]meg/*_meg.con and a paired events JSON sidecar declaring the trigger mode ("TriggerMode": "single_channel" or a multi-channel/binary-coded label). The JSON is located with the same fallback hierarchy as the stage-1 sanity check (exact scope, then subject + task, subject, dataset root).

  • STI 014 variant: the FIFs written by stage 2 and listed in its kit2fiff_summary.csv; no events table or JSON is needed.

Configuration file

Shared YAML, given with -c / --config:

project:
  name: "kidlang"
  root_env: "MEG_DATA"

subjects:
  include: []
  exclude: []

bids_selection:
  tasks: []
  runs: []
  processings: []

events:
  desc: sti014              # label written by stage 8; defaults: sti014 (STI 014 variant), autopulses (analog variants)
  source: sti014            # branch read by stage 9: single_channel | multi_channel | sti014 | legacy | null
  find_events:              # STI 014 variant only: keyword arguments of mne.find_events
    stim_channel: "STI 014"
    shortest_event: 1
    min_duration: 0.002
  multi_channel:            # multi-channel variant (and stage 1 multi) only
    max_onset_skew_ms: 2.0

plots:                      # analog variants: stim-channel plots
  save_stim_plots: true
  format: pdf               # pdf | png | svg
  duration_s: 10.0
  misc_scaling: 1.0
  channels_mne: ["MISC 001", "MISC 002"]   # default: all eight trigger lines
  subdir: figures

How to run

From the repository root with neurowaves activated:

MODULE=pipeline.mne_pipelines.kit_general_pipelines
CONFIG=pipeline/mne_pipelines/kit_general_pipelines/pipeline_config_files/kidlang_config.yml

# KidLang: STI 014 codes from the stage-2 FIFs
python -m $MODULE.8-kit_compute_events_STI014 -c $CONFIG

# single-channel analog triggers
python -m $MODULE.8-kit_compute_events_single_channel -c <config.yml>

# binary-coded analog triggers
python -m $MODULE.8-kit_compute_events_multi_channel -c <config.yml>

Options (all variants):

  • --desc <label>: override events.desc for the output names. If you use it, set the same label in the YAML before running stage 9.

  • --overwrite: replace existing .eve / _detail.tsv pairs. Without it an existing pair is reported as skipped_existing_output (or incomplete_existing_output when one of the two files is missing or empty).

The VS Code entries <project> 8 Compute events (...) run the variant of each project (STI 014 for KidLang, single channel for the epilepsy project).

What the scripts do

Single-channel variant

  1. Resolve the BIDS root, subjects and recordings (bids_selection).

  2. Locate the events JSON; skip the run if it is missing or does not declare single_channel.

  3. Open the .con with mne.io.read_raw_kit (no preload).

  4. For each trigger line MISC 001 to MISC 008, detect pulses with detect_pulses_on_channel (same thresholds as stage 1) and map the line to its KIT channel number, which becomes the event id (224 to 231).

  5. Optionally save a stim-channel plot (plots block) under sub-<id>/figures/ with desc-stimplot.

  6. Write <base>.eve and <base>_detail.tsv; append to auto_events_index.csv.

Multi-channel (binary-coded) variant

Same discovery as the single-channel variant (events JSON with a multi-channel TriggerMode, accepted spellings as on the stage-1 page), then:

  1. Detect the high intervals of every trigger line MISC 001 to MISC 008 with the same robust threshold as the stage-1 multi-channel check (baseline from the first 10 s, median + MAD, hysteresis, minimum width 3 ms). A recording missing a trigger line is skipped (skipped_missing_trigger_channels): with one bit absent no code can be trusted. Non-finite samples are interpolated for the detection and reported.

  2. Build the code at every sample from the lines that are high (bit weights: MISC 001 = KIT 224 = 1, MISC 002 = 2, … MISC 008 = 128) and split the occupied stretches into events with the rule shared with stage 1 (multi_channel_events.combine_binary_channel_intervals): rising edges within events.multi_channel.max_onset_skew_ms of the first rising edge belong to the same event; a later rising edge starts a new event even if other lines are still high; a falling edge never starts one; a gap of up to 2 ms where every line is low is tolerated inside an event. The code of an event is built from the lines that rose within its onset group.

  3. Give each event the event id = its binary code (1 to 255) and write the events array with the sample of the first rising edge (including the recording’s first_samp, like MNE’s own events).

  4. Flag what needs a look, per event in the detail TSV and summed in the index: an earlier line still high at the onset (held_channels_at_onset), grouped lines that were never high together (grouped_bits_never_coactive), and a code change without a new rising edge (late_falling_code_transitions). Any of these makes the index status created_review_required instead of created; the .eve is still written and stage 9 can read it, but the flagged events should be checked against the raw lines (stim-channel plot) first. A recording whose events all have a single active line is warned as a possible single-channel dataset labelled as multi-channel.

  5. Print the event_id mapping (binary_00000011 -> 3, …), save the optional stim-channel plot, write <base>.eve and <base>_detail.tsv atomically, and append to auto_events_index.csv.

Changing max_onset_skew_ms after a run does not rebuild existing outputs: rerun with --overwrite (the index records both the configured and the applied value).

STI 014 variant

  1. Resolve the stage-2 folder (2-kit_con_to_fif, or its pre-renumbering name when only that exists) and keep only the FIFs listed as current in kit2fiff_summary.csv.

  2. For each FIF, call mne.find_events(raw, **events.find_events).

  3. Replace the desc-rawkit label and _raw suffix of the FIF name with desc-<label>_events; write the .eve and a detail TSV with sample, prev, event_id and onset.

  4. Append to events_summary.csv.

File naming

sub-<id>[_ses-<id>][_task-<task>][_acq-<l>][_run-<n>][_split-<n>][_rec-<l>][_space-<l>][_proc-<l>]_desc-<label>_events.eve
sub-<id>..._desc-<label>_events_detail.tsv

Examples:

  • sub-test1_task-400events_desc-autopulses_events.eve

  • sub-Y0312_task-dialogue_desc-sti014_events.eve

Trigger delay correction (single channel)

A trigger pulse is written on a video frame, which is not exactly the moment of the event it marks: a sound starts a few ms before or after its trigger frame, and a button press is only read on the next frame. The lab’s stimulus scripts measure that difference per event and store it in the run’s BIDS *_events.tsv (column trigger_delay, seconds; the events.json OnsetRule: onset = pulse time - trigger_delay). When the column exists (events.apply_trigger_delay: true, default), the pulses of each trigger channel are matched in order with the events.tsv rows of that channel and moved to the event (n/a counts as 0). A channel whose pulse count differs from its row count is left uncorrected and named in the index comments; the detail table keeps the original pulse sample (sample_pulse) and the delay applied (trigger_delay_ms). A derived variant of a recording (stage 0’s proc-CALMnoisereduction CON) uses the events table of the original (proc, rec, space and desc are dropped from the name, and the dataset’s own meg folder is searched), so both variants get the same correction; a recording without an events table is noted (no events.tsv for this recording; trigger_delay not applied). In the epilepsy project the button presses move 7 to 15 ms earlier and the tones about 3 ms later. Existing event files are rebuilt with --overwrite. The index (auto_events_index.csv) and the stage-1 summary keep the rows of the recordings not processed in a run.

Outputs

<bids_root>/derivatives/8-kit_compute_events_single_channel/     # or ..._multi_channel/
  auto_events_index.csv
  configs/config_<timestamp>.yml
  sub-<id>/[ses-<id>/]
    sub-<id>..._desc-autopulses_events.eve
    sub-<id>..._desc-autopulses_events_detail.tsv
    figures/sub-<id>..._desc-stimplot.pdf

<bids_root>/derivatives/8-kit_compute_events_STI014/
  events_summary.csv
  configs/config_<timestamp>.yml
  sub-<id>/[ses-<id>/]
    sub-<id>..._desc-sti014_events.eve
    sub-<id>..._desc-sti014_events_detail.tsv

Index columns:

  • auto_events_index.csv (single channel): subject, file, status, events_eve, detail_tsv, n_events, stim_plot, comments, mne_version.

  • auto_events_index.csv (multi channel): the same plus applied_max_onset_skew_ms, configured_max_onset_skew_ms, held_onset_events, noncoactive_event_codes, late_falling_code_transitions, event_id_mapping (JSON) and warnings. Statuses: created, created_review_required, skipped_existing_output, incomplete_existing_output, skipped_missing_events_json, skipped_wrong_trigger_mode, skipped_missing_trigger_channels, skipped_invalid_trigger_data, no_triggers_detected, failed.

  • events_summary.csv (STI 014): subject, input_file, events_found, output_eve, status, mne_version.

Detail TSV columns:

  • single channel: sample, onset_s, channel_mne, channel_kit, event_id, width_ms, amp_max, amp_mean;

  • multi channel: max_onset_skew_ms, sfreq, sample, end_sample, onset_s, width_ms, code, event_id, trigger_channels (the lines that formed the code), trigger_code_binary (eight bits, MISC 001 least significant), active_channels, n_codes_during_event, held_channels_at_onset, observed_code_at_last_onset, grouped_bits_never_coactive, late_falling_code_transitions, amp_max, amp_mean;

  • STI 014: sample, prev, event_id, onset.

Edge cases & safeguards

  • Missing or wrong-mode events JSON (analog variants): run skipped and recorded in the index (skipped_missing_events_json, skipped_wrong_trigger_mode).

  • No events found: nothing is written for that recording; the index says no_events.

  • MNE split files (..._raw-1.fif): the STI 014 variant processes the base file only.

  • Existing outputs: preserved unless --overwrite.

  • Stage 9 ambiguity: with events.source: null stage 9 searches every stage-8 branch and skips a recording if more than one event file matches. Pin source when more than one variant has been run.

Troubleshooting

  • No outputs (analog): check that the events JSON exists at a valid scope and that TriggerMode matches the script you chose.

  • No outputs (STI 014): stage 2 must have run for these recordings; the FIFs must appear in kit2fiff_summary.csv. Check that kit2fiff.stim / slope / stimthresh produced a non-empty STI 014 channel.

  • Unexpected counts: compare with the stage-1 sanity check and open the _detail.tsv; for find_events tune shortest_event / min_duration in events.find_events.

  • Stage 9 finds no events: events.desc in the YAML must equal the label used when stage 8 ran (including any --desc override).

  • ``created_review_required`` (multi channel): open the detail TSV, filter the flagged events and compare with the stim-channel plot. Pulses of one code that fall at different times produce late_falling_code_transitions; a stimulus presented while the previous code was still high produces held_channels_at_onset. Fix the stimulus script’s pulse timing for the next session; for the existing recording, decide per event whether the code is right and document it.

  • ``skipped_missing_trigger_channels`` (multi channel): a MISC line is absent from the .con; the recording cannot be decoded.

Summary

Stage 8 converts the trigger stream into canonical MNE event files with entity-rich names and an audit trail. Choose the variant by trigger coding (single channel, binary coded, or STI 014), keep events.desc and events.source in the YAML consistent with it, and stage 9 will epoch directly from these files.