Experiment example (Psychtoolbox): MSWM

Description

This experiment compares neural activity during cue-driven and self-driven selection of an item held in working memory (WM).

On each trial, three triangles appear briefly, each inside a circle. After a mask, a cue says which item will be tested: the experimenter’s choice (retro-cue) or the participant’s choice (self-selected). After a delay, the participant first selects the item with the Current Designs dial. They then turn the dial to report where that triangle’s tip pointed. Feedback shows the error and the points earned, and the points earn a bonus.

Code: experiments/psychtoolbox/MSWM (start with MEG_metacog_main.m).

Presentation sequence

All screens show a grey disc on a black background, with a fixation dot in the centre. The three items sit at polar angles of 45, 165 and 285 deg (counter-clockwise from the right) at 3.98 deg from fixation. Each item is a circle of 3.98 deg diameter. Sizes are in degrees of visual angle for a viewing distance of 88 cm (viewDist, to be checked in the MEG room).

Once per run, before the first trial, a white fixation dot is shown for 5000 ms (exp_start trigger). Each trial then runs:

Epoch

Duration

Screen

Fixation

500 ms

Black fixation dot.

Stimuli

300 ms

Three black triangles, one in each circle. Each orientation is drawn without replacement from 20, 60, …, 340 deg, plus a uniform 0-5 deg jitter.

Mask

50 ms

Each circle is filled black, with two white quarter-sectors.

ISI

250 ms

Empty circles.

Cue

3000 ms

Large red fixation dot. Retro-cue: the cued circle turns blue. Self-selected: all three circles turn blue.

Delay

3000 ms

Empty circles.

Select

self-paced

Large green fixation dot. Self-selected: the participant turns the dial, the circle it points at turns blue, and a press is accepted only on a circle. The dial starts at a random angle that points at no item. Retro-cue: the cued circle stays blue and the participant only presses.

Report

self-paced

A white dot on the rim of the selected circle, at a random start angle. The participant turns the dial until the dot sits where the triangle’s tip was, then presses.

Feedback

750 ms

The response (white dot) and the correct tip position (green dot), with the points for the trial and the running total.

ITI

2250, 3000 or 3750 ms, plus padding

White fixation dot. The ITI is padded so that the time from select onset to the end of the ITI is a whole number of 750 ms TRs.

Minimum response time for simulation. On the select and report screens, presses are ignored for 500 ms plus a uniform 0-500 ms jitter. A new jitter is drawn for each screen, and the participant can still turn the dial during this time. Presses are also ignored while the onset trigger is on. After an accepted press the screen stays (frozen) for the trigger duration, so the next trigger cannot start too early.

Timing. At start-up the script measures the frame duration and prints it (Frame timing in the command window). Each fixed-duration screen ends by the clock, not by a fixed number of frames. Frames are shown until the next flip is the one closest to onset + duration. On a display locked to the refresh, this gives exactly round(duration / frame) frames. The command window prints the trial number when a trial starts, and its duration from fixation onset to the end of the ITI when it ends.

Scoring

The error is the response minus the tip orientation, wrapped to (-180, 180] deg.

  • Points per trial: max(0, floor(10 - |error| * 10 / 50)). That is 10 points for a perfect report, and 0 points for an error of 50 deg or more.

  • Bonus at the end of the run, printed in the command window: min(6, round(totalPoints / (10 * nTrials * 0.9) * 6)) AED.

Conditions and randomization

  • Cue condition 1 = retro-cue (one item is cued).

  • Cue condition 2 = self-selected (the participant turns the dial onto an item).

A run has nTrials = 24 trials, which must be a multiple of 12. Half are retro-cue and half self-selected. In retro-cue trials, each of the three locations is cued equally often. Within each run the trial order is shuffled (saved as AllCond in the taskmap file, see Saved data). The three ITI values are used equally often and are also shuffled. The orientations are drawn at random on every trial.

Triggers

Triggers use DataPixx pixel mode. The RGB colour of the top-left pixel sets the KIT trigger channels 224-231. That pixel is 1 px in the MEG room and a 50 x 50 px patch on the own computer. Each event has its own code, and the same epoch has a different code in each cue condition. The codes are defined in SetupTriggers.m.

A trigger starts on the first frame of each epoch and lasts 30 ms (p.meg_trigger_duration), rounded to whole frames. It is shortened if needed, so that every fixed-duration epoch (the shortest is the 50 ms mask) ends with at least one trigger-free frame. Two triggers therefore never touch.

During the trial triggers, a 100 x 100 px photodiode patch in the top-right corner turns white; it stays black the rest of the time. exp_start and run_end do not flash it.

Event

KIT channels (retro-cue)

RGB (retro-cue)

KIT channels (self-selected)

RGB (self-selected)

exp_start

224+225+226

[84 0 0]

same

same

fixation

224+227+228

[4 5 0]

224+227+229

[4 17 0]

stimuli

224+227+230

[4 65 0]

224+227+231

[4 1 1]

mask

224+228+229

[4 20 0]

224+228+230

[4 68 0]

ISI

224+228+231

[4 4 1]

224+229+230

[4 80 0]

cue

224+229+231

[4 16 1]

224+230+231

[4 64 1]

delay

225+226+227

[80 1 0]

225+226+228

[80 4 0]

select

225+226+229

[80 16 0]

225+226+230

[80 64 0]

report

225+226+231

[80 0 1]

225+227+228

[16 5 0]

feedback

225+227+229

[16 17 0]

225+227+230

[16 65 0]

ITI

225+227+231

[16 1 1]

225+228+229

[16 20 0]

run_end

224+225+227

[20 1 0]

same

same

Channel bits: 224 = R 4, 225 = R 16, 226 = R 64, 227 = G 1, 228 = G 4, 229 = G 16, 230 = G 64, 231 = B 1. All eight channels 224-231 must be wired on the KIT.

Every trigger that is sent is written to the trigger log (see Saved data), with its channels, onset, offset, duration and the gap to the next trigger. At the end of the run the log is also written as the events file that the lab’s pipeline compares with the MEG recording (see Events file and trigger check).

Running the experiment

Before the session, set up the dial (Current Designs HHSC-1x1-DIAL through the FIU-905):

  1. Connect the dial and the fibre.

  2. Keep turning the dial while you plug in the 905 USB, and keep turning for a few seconds after.

  3. Check the dial with dial_dot_demo.

The FIU-905 appears as a USB keyboard: a clockwise step is a t tap, and a counter-clockwise step is a b tap. The dial has no button, so the participant confirms with the response box (DataPixx digital input).

cd experiments/psychtoolbox/MSWM
MEG_metacog_main      % opens the start-up window

The start-up window asks for:

  • Subject code, Session, Run, Age, Gender. Run and Age must be numbers, and Run a whole number (1, 2, 3, …). If that run is already saved for this subject and session, a message appears and the Run field changes to the next free number: check it and press Start again.

  • Testing on my own computer. Uses screen 0 and a visible 50 x 50 px trigger patch. Unchecked, it uses screen 2 and a 1 px trigger (MEG room).

  • Use DataPixx. Pixel-mode triggers and response-box buttons.

  • Eye tracker (EyeLink). Runs the 9-point calibration, then records to M<subject><session><run>.edf on the EyeLink host, which accepts names of 8 characters at most (subject and session as typed, run on 2 digits: M101101.edf). Keep subject and session short enough: the EyeLink host does not open a file with a longer name, and the eye-tracking data of the run is then not saved. At the end of the run the file is copied to the stimulus computer and renamed (see Saved data).

  • Dial + response box. Unchecked, the keyboard is used instead: hold t / b to turn, and press SPACE to confirm. With the dial and no DataPixx, SPACE confirms.

  • Simulate responses. Generates the responses automatically, so no participant, dial or response box is needed. The run first waits on a SIMULATION screen, with no trigger: start the MEG recording there, then press SPACE or ENTER (ESC quits). The instruction and end screens then continue by themselves after 2 s, and the run is saved in data_test/ instead of Data/, so simulated runs never mix with real ones. The simulated settings are in the sim.* block at the top of the script.

The participant starts the run from the instruction screen with a button press, and ends it from the end screen. ESC stops the run at any time.

Saved data

The files follow the lab’s BIDS naming. They are written inside the experiment folder: in Data/ for real runs and in data_test/ for simulated runs. Both folders are Git-ignored. Every file of a run starts with the same name:

sub-<subject>_ses-<session>_task-mswm_run-<run>

A numeric subject is padded to 3 digits, a numeric session to 2 digits and the run to 2 digits. Subject 101, session 1, run 1 gives:

Data/
  sub-101/ses-01/
    beh/sub-101_ses-01_task-mswm_run-01_beh.tsv                  one row per trial
    meg/sub-101_ses-01_task-mswm_run-01_events.tsv               one row per trigger (DataPixx runs)
    meg/sub-101_ses-01_task-mswm_run-01_events.json              trigger mode and duration, start time, timing constraints
  sourcedata/
    task-mswm_desc-runsummary_beh.mat                            points and bonus of every completed run
    sub-101/ses-01/
      beh/sub-101_ses-01_task-mswm_run-01_desc-triggers_beh.csv  trigger log, as written during the run
      beh/sub-101_ses-01_task-mswm_run-01_desc-console_log.txt   MATLAB Command Window output of the run
      beh/sub-101_ses-01_task-mswm_run-01_desc-taskmap_beh.mat   trial order (AllCond)
      beh/sub-101_ses-01_task-mswm_run-01_desc-error_beh.mat     only after a crash
      eyetrack/sub-101_ses-01_task-mswm_run-01_eyetrack.edf      if the eye tracker is on

The behaviour file and the trigger log are written as the run goes, one row per trial and one row per trigger, so a run that is stopped or crashes keeps what was recorded until then.

The console log holds everything MATLAB printed in the Command Window during the run: the frame timing measured at start-up, any warning, the completion message with the duration and the bonus, and the crash report if the run crashed. It starts after the run number check, so a launch that is refused writes nothing into the log of the existing run.

One run number, one launch. The script does not start a run whose behaviour file already exists. The start-up window checks this when Start is pressed, and the script checks it again before the screen opens. A run that was stopped or crashed keeps its files: start again with the next run number.

task-mswm_desc-runsummary_beh.mat holds the table subjectMat, with one row per completed run: experiment name (MetaCog_MEG, or MetaCog_MEG_SIM for a simulated run), subject, session, run, gender, age, total points and bonus.

After the session, copy the content of Data/ into the dataset folder on NYU Box (MEG/Data/MSWM) and save each KIT recording next to its events files, under the same name: sub-101/ses-01/meg/sub-101_ses-01_task-mswm_run-01_meg.con. A simulated run recorded as a trigger test is copied the same way, from data_test/.

One launch of the script is one run and one events file, so make one KIT recording per launch: start it before the first screen is passed, and stop it only after the end screen, which sends the last trigger (run_end).

Behaviour file

beh/<name>_beh.tsv, tab-separated, one row per completed trial. All times are in seconds relative to expStart, which is the end of the 5 s start screen. Durations and RTs are in ms.

Columns

Content

Subject, Session, Run, Trial, CueCondition

Trial identifiers (CueCondition 1 = retro-cue, 2 = self-selected).

Stim1, Stim2, Stim3

Tip orientation of each item (deg).

probeLoc, SelectedPosAngle

Selected item (1-3), and the dial angle when it was selected.

Target, Estimate, Error

True and reported orientation, and their wrapped difference (deg).

SelRT, RT

Select and report RTs (ms, screen onset to the press frame).

Points, totalPoints

Points for the trial and the running total.

tFix … tFeedback, tITI, avgFrame

Planned epoch durations (ms), and the measured frame duration (ms).

expStart, fixStart … ITIEnd

Onsets of every epoch (s relative to expStart). For the select, report, feedback and ITI screens, the end is also given.

variableTiming

Select onset to ITI end (ms).

selMinRT, repMinRT

Minimum RT drawn for the select and report screens (ms).

Trigger log

sourcedata/.../beh/<name>_desc-triggers_beh.csv, comma-separated. One row per trigger: Subject, Session, Run, Trial, CueCondition, Event, TrigR/TrigG/TrigB (8-bit pixel value), MEGChannels, OnsetAbs (GetSecs), OnsetRel and OffsetRel (s relative to expStart), DurationMs and GapToNextMs. GapToNextMs is the time with no trigger between this trigger’s offset and the next trigger’s onset; it is NaN for the last trigger.

Events file and trigger check

When DataPixx is on, the trigger log is written again at the end of the run, and also after a crash, as meg/<name>_events.tsv: tab-separated, with missing values as n/a. It holds every column of the trigger log, with six columns in front and trigger_duration at the end, in the form read by the lab’s KIT pipeline:

Column

Content

onset

n/a: the time of the trigger in the MEG recording, filled during preprocessing.

duration

Duration of the event (s): the time from this trigger to the next one. For the fixed-duration screens this is the time the screen was shown; for select and report it is the time until the response was confirmed. For the last ITI it also includes the saving done before the end screen. n/a for the last row.

trial_type

Name of the event (same as Event).

value

Binary trigger code, 1 to 255: the sum of the weights of the channels that are high (224 = 1, 225 = 2, 226 = 4, 227 = 8, 228 = 16, 229 = 32, 230 = 64, 231 = 128). fixation in the retro-cue condition (224+227+228) is 25.

stim_time

Flip time of the first frame with the trigger (same as OnsetAbs).

trial_index

Trial number (same as Trial; 0 for exp_start and run_end).

trigger_duration

Time the trigger code stayed on (s), from the flip times (DurationMs / 1000). n/a when the screen ended before the trigger did.

meg/<name>_events.json is written at start-up, on every launch. It declares:

  • TriggerMode: binary_coded.

  • TriggerDuration: the number of frames the trigger stays on and the measured frame duration, for example {"Frames": 4, "FrameMs": 8.3357} (33.3 ms).

  • RunStarted: date and time of the launch.

  • TimingConstraints: the time expected between the onsets of two consecutive fixed-duration screens (fixation to stimuli, stimuli to mask, mask to ISI, ISI to cue, cue to delay, delay to select, feedback to ITI). Each is the number of frames shown times the measured frame duration, with a tolerance of half a frame. A last rule asks that every interval between two consecutive triggers in the recording equals the interval logged in OnsetAbs.

With the recording saved next to these two files, stage 1 of the pipeline (trigger count sanity check, script 1-kit_count_sanity_check_multi_channel.py) compares the recording with the events file: the number of triggers, the count of every code, the order of the codes, the timing constraints and the trigger duration.

The pipeline configuration of the dataset is pipeline_config_files/mswm_config.yml (a copy of config_template.yml whose project.name is MSWM):

python -m pipeline.mne_pipelines.kit_general_pipelines.1-kit_count_sanity_check_multi_channel \
    -c pipeline/mne_pipelines/kit_general_pipelines/pipeline_config_files/mswm_config.yml

The result is in MSWM/derivatives/1-kit_count_sanity_check_multi_channel/sanity_check_overview.csv. A complete run has 242 triggers (10 per trial, exp_start and run_end). The count, the codes and their order decide PASS or FAIL. The timing and the trigger duration are reported next to it (timing_status, trigger_duration_status) and do not change the PASS. A run stopped with ESC is checked on the triggers it sent; its unfinished trial appears as one timing violation (“no To event”).

Files

File

Role

MEG_metacog_main.m

The experiment: start-up window, trial loop, drawing, dial and response-box input, response simulation, trigger log, DataPixx, EyeLink and cleanup.

SetupTriggers.m

Trigger codes per event and cue condition, and the positions of the trigger pixel and the photodiode patch.

MEG_metacog.m, MEG_metacog_dial.m

Earlier versions of the task, kept for reference.