@brainpilot/skills 0.0.6

package/skills/04_Psycholinguistics/self-paced-reading-designer/references/region-segmentation.md

@@ -0,0 +1,266 @@
+# Region Segmentation for Self-Paced Reading
+
+This reference provides detailed guidance on how to segment sentences into reading regions for self-paced reading (SPR) experiments. Region boundaries are the most consequential design decision in SPR because they determine what can be measured and what will be confounded.
+
+---
+
+## Fundamental Principles
+
+### 1. Regions Must Respect Syntactic Constituency
+
+Never split a region in the middle of a syntactic constituent. A region boundary within a noun phrase (e.g., splitting "the tall" from "man") forces participants to process an incomplete constituent, introducing processing difficulty that is an artifact of segmentation rather than a reflection of the intended manipulation (Jegerski, 2014).
+
+**Good segmentation** (respects constituency):
+
+```
+The maid | of the actress | who was | on the balcony | shouted | to the crowd.
+```
+
+**Bad segmentation** (splits constituents):
+
+```
+The maid of | the actress who | was on | the balcony shouted | to the | crowd.
+```
+
+### 2. Critical Regions Must Be Controlled
+
+The critical region -- where you expect to observe a reading time difference between conditions -- must be matched across conditions on:
+
+| Property | How to Match | Source for Norms |
+|---|---|---|
+| **Word length (characters)** | Within +/- 1 character across conditions | Count characters |
+| **Lexical frequency** | Within 0.5 log10 units (SUBTLEX-US) | Brysbaert & New (2009); SUBTLEX-US database |
+| **Number of syllables** | Same syllable count preferred | Standard dictionaries |
+| **Morphological complexity** | Same number of morphemes preferred | Linguistic analysis |
+| **Part of speech** | Identical across conditions | Linguistic analysis |
+| **Orthographic neighborhood size** | Match if feasible | English Lexicon Project (Balota et al., 2007) |
+
+If the same word appears in the critical region across all conditions (e.g., an ambiguity resolves on a word that is identical but disambiguates due to prior context), length and frequency matching is automatic. This is the ideal design.
+
+### 3. Pre-Critical and Post-Critical (Spillover) Regions Must Be Identical
+
+The words immediately before and after the critical region should be character-for-character identical across conditions. This is essential because:
+
+- **Pre-critical region**: Any difference here creates a baseline shift that propagates into the critical region via spillover (Mitchell, 2004)
+- **Post-critical (spillover) region**: Effects from the critical word reliably appear 1-3 words downstream in SPR (Just et al., 1982; Rayner, 1998). If spillover regions differ across conditions, you cannot attribute reading time differences to the critical manipulation
+
+### 4. Avoid Clause and Sentence Boundaries in Critical Regions
+
+Wrap-up effects inflate reading times at clause-final and sentence-final words by **50-100+ ms** (Just & Carpenter, 1980; Warren, White, & Reichle, 2009). This inflation is independent of experimental condition and adds noise that reduces statistical power. Additionally, clause-final words often co-occur with punctuation (commas, periods), which introduces visual complexity unrelated to your manipulation.
+
+**Rule**: Place the critical region at least 2 words before any clause or sentence boundary.
+
+---
+
+## Segmentation Strategies by Construction Type
+
+### Garden-Path Sentences
+
+Garden-path sentences temporarily mislead the parser. The critical region is typically the disambiguation point.
+
+**Example**: Main verb / reduced relative ambiguity
+
+Condition A (ambiguous):
+```
+The horse | raced | past the barn | fell | down | the hill.
+ pre-crit CRIT spill1 spill2 spill3
+```
+
+Condition B (unambiguous control):
+```
+The horse | that was raced | past the barn | fell | down | the hill.
+ pre-crit CRIT spill1 spill2
+```
+
+**Key considerations**:
+- The critical word "fell" is identical across conditions -- this is ideal
+- Spillover regions ("down the hill") are identical
+- In condition B, the relative clause marker "that was raced" makes the structure unambiguous before the critical region
+- The garden-path effect (longer RT in condition A at "fell") may spill over to "down" and even "the hill"
+
+### Relative Clause Attachment
+
+Relative clause attachment ambiguity (high vs. low attachment) typically manipulates the plausibility or gender match of the relative clause with one of two potential hosts.
+
+**Example**: High vs. low attachment
+
+```
+The maid | of the actress | who was | sitting | on the | balcony | spoke loudly.
+ R1 R2 R3 CRIT spill1 spill2 wrap-up
+```
+
+**Key considerations**:
+- The critical region is the verb inside the relative clause ("sitting") where semantic fit disambiguates attachment
+- "on the balcony" serves as spillover
+- "spoke loudly" is the wrap-up region -- do NOT analyze this as spillover because it includes sentence-final wrap-up effects
+- The two host nouns ("maid" and "actress") must be matched for length and frequency
+- If the disambiguation depends on semantic fit (e.g., "who was knitting" fits better with "actress"), the critical verbs across conditions must be matched for length and frequency
+
+### Filler-Gap Dependencies
+
+In filler-gap constructions, a moved element (the filler) is interpreted at a later position (the gap). The critical region is the gap site.
+
+**Example**: Object relative with gap
+
+```
+The book | that the author | wrote _ | in the | library | was | a bestseller.
+ filler intervener GAP spill1 spill2 spill3 wrap-up
+```
+
+Control: Subject relative (no object gap)
+
+```
+The author | that _ wrote | the book | in the | library | was | quite famous.
+ GAP post-gap spill1 spill2 spill3 wrap-up
+```
+
+**Key considerations**:
+- Compare reading times at the verb ("wrote") and spillover regions across the two structures
+- The gap site is implicit (no overt marker), so the parser must posit a gap based on syntactic expectations
+- Spillover regions must be identical or closely matched because gap-filling effects propagate downstream (Stowe, 1986)
+
+### Verb Argument Structure
+
+When manipulating subcategorization or argument structure expectations, the critical region is typically the post-verbal material where the argument structure becomes clear.
+
+**Example**: Transitive vs. intransitive bias
+
+```
+The doctor | hoped | the patient | would recover | quickly | from | the illness.
+ verb CRIT spill1 spill2 spill3 wrap-up
+```
+
+```
+The doctor | examined | the patient | would recover | quickly | from | the illness.
+ verb CRIT spill1 spill2 spill3 wrap-up
+```
+
+**Key considerations**:
+- "hoped" is intransitive-biased: "the patient" is unexpected as a direct object
+- "examined" is transitive-biased: "the patient" is expected as a direct object
+- The critical region ("the patient") is identical across conditions
+- The verbs differ across conditions -- they must be matched for length and frequency
+- The pre-critical verb is the only word that differs, so any RT difference at "the patient" or spillover must be due to the verb's argument structure bias
+
+### Agreement and Morphosyntactic Violations
+
+For studies of grammatical agreement or morphosyntactic processing, the critical region is the word where the violation becomes apparent.
+
+**Example**: Subject-verb number agreement
+
+Grammatical:
+```
+The key | to the cabinets | was | rusty | and | old.
+ subj intervener CRIT spill1 spill2 spill3
+```
+
+Ungrammatical:
+```
+The key | to the cabinets | were | rusty | and | old.
+ subj intervener CRIT spill1 spill2 spill3
+```
+
+**Key considerations**:
+- "was" vs. "were" differ by one character -- this is an acceptable minimal difference
+- The intervening noun ("cabinets") creates an attraction environment that is part of the manipulation
+- Spillover regions must be identical across conditions
+- With agreement violations, effects are often largest in the spillover region (Pearlmutter, Garnsey, & Bock, 1999), not the critical word itself
+
+---
+
+## Word-by-Word vs. Phrase-by-Phrase: When to Choose Each
+
+### Word-by-Word
+
+Use word-by-word presentation (the default and most common method) when:
+
+- You need maximal temporal resolution
+- The critical manipulation is localized to a single word
+- You want to track spillover word-by-word
+- You need comparability with the existing SPR literature
+
+### Phrase-by-Phrase
+
+Use phrase-by-phrase presentation when:
+
+- The critical manipulation spans a multi-word constituent (e.g., an entire PP or relative clause)
+- Word-by-word presentation would break up a tightly bound phrase unnaturally
+- You are studying phrase-level prosodic or structural effects
+
+When using phrase-by-phrase, follow these rules:
+
+1. **Segment at constituent boundaries.** Every region must be a complete syntactic constituent (NP, VP, PP, clause)
+2. **Match region length (in words and characters) across conditions.** If condition A has a 4-word critical region and condition B has a 3-word critical region, the length difference confounds the comparison
+3. **Report your segmentation criteria explicitly.** Phrase-by-phrase segmentation is more subjective than word-by-word. Readers of your paper need to evaluate whether your boundaries are defensible
+
+---
+
+## Controlling for Confounds Within Regions
+
+### Word Length
+
+Reading time in SPR increases approximately **30-40 ms per additional character** (Ferreira & Clifton, 1986; Trueswell, Tanenhaus, & Garnsey, 1994). This means a 2-character difference between conditions at the critical word creates a **60-80 ms confound** -- often larger than the effect of interest.
+
+**Solutions** (in order of preference):
+1. Use identical words in the critical region across conditions (best)
+2. Match word length exactly (within +/- 1 character)
+3. Include word length as a covariate in the statistical model
+4. Use residual reading times (Ferreira & Clifton, 1986)
+
+### Word Frequency
+
+Low-frequency words take longer to read than high-frequency words. The frequency effect in SPR is approximately **20-40 ms per log unit** of frequency difference (Rayner, 1998; Inhoff & Rayner, 1986).
+
+**Solutions**:
+1. Use identical words in the critical region across conditions (best)
+2. Match on log-transformed word frequency (SUBTLEX-US; Brysbaert & New, 2009)
+3. Include log frequency as a covariate in the statistical model
+
+### Word Position in Sentence
+
+Reading times are typically slower for the first 2-3 words of a sentence (start-up cost) and inflated at the final 1-2 words (wrap-up; Just & Carpenter, 1980). Avoid placing critical regions in these positions.
+
+**Safe zone**: Words 3 through (sentence_length - 3) are least affected by positional artifacts.
+
+---
+
+## Example: Complete Segmentation with Region Labels
+
+**Study**: Effect of verb bias on complement clause processing
+
+Condition A (SC-biased verb):
+```
+Position: R1 R2 R3-CRIT R4-spill1 R5-spill2 R6-wrap
+Sentence: The coach | knew | the team | would travel | to the game | on Friday.
+```
+
+Condition B (DO-biased verb):
+```
+Position: R1 R2 R3-CRIT R4-spill1 R5-spill2 R6-wrap
+Sentence: The coach | told | the team | would travel | to the game | on Friday.
+```
+
+- **R1** (pre-verb context): Identical across conditions
+- **R2** (verb): Differs across conditions -- "knew" (4 chars) vs. "told" (4 chars) -- matched for length. Must also match frequency.
+- **R3** (critical region): "the team" is identical -- this is where the sentence-complement interpretation becomes clear for DO-biased "told" but not SC-biased "knew"
+- **R4-R5** (spillover): Identical across conditions
+- **R6** (wrap-up): Do not analyze as spillover due to sentence-final wrap-up effects
+
+---
+
+## References
+
+- Balota, D. A., et al. (2007). The English Lexicon Project. *Behavior Research Methods, 39*, 445-459.
+- Brysbaert, M., & New, B. (2009). Moving beyond Kucera and Francis. *Behavior Research Methods, 41*, 977-990.
+- Ferreira, F., & Clifton, C. (1986). The independence of syntactic processing. *Journal of Memory and Language, 25*, 348-368.
+- Inhoff, A. W., & Rayner, K. (1986). Parafoveal word processing during eye fixations in reading. *Perception & Psychophysics, 40*, 431-439.
+- Jegerski, J. (2014). Self-paced reading. In J. Jegerski & B. VanPatten (Eds.), *Research methods in second language psycholinguistics*. Routledge.
+- Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. *Psychological Review, 87*, 329-354.
+- Just, M. A., Carpenter, P. A., & Woolley, J. D. (1982). Paradigms and processes in reading comprehension. *Journal of Experimental Psychology: General, 111*, 228-238.
+- Keating, G. D., & Jegerski, J. (2015). Experimental designs in sentence processing research. *Studies in Second Language Acquisition, 37*, 1-32.
+- Mitchell, D. C. (2004). On-line methods in language processing. In M. Carreiras & C. Clifton (Eds.), *The on-line study of sentence comprehension*. Psychology Press.
+- Pearlmutter, N. J., Garnsey, S. M., & Bock, K. (1999). Agreement processes in sentence comprehension. *Journal of Memory and Language, 41*, 427-456.
+- Rayner, K. (1998). Eye movements in reading and information processing. *Psychological Bulletin, 124*, 372-422.
+- Stowe, L. A. (1986). Parsing WH-constructions: Evidence for on-line gap location. *Language and Cognitive Processes, 1*, 227-245.
+- Trueswell, J. C., Tanenhaus, M. K., & Garnsey, S. M. (1994). Semantic influences on parsing. *Journal of Memory and Language, 33*, 285-318.
+- Warren, T., White, S. J., & Reichle, E. D. (2009). Investigating the causes of wrap-up effects. *Cognition, 111*, 132-137.

package/skills/04_Psycholinguistics/sentence-stimulus-norming/SKILL.md

@@ -0,0 +1,346 @@
+---
+name: "sentence-stimulus-norming"
+description: "Specifies norming procedures for linguistic stimuli including cloze probability, plausibility ratings, acceptability judgments, and lexical controls"
+domain: "psycholinguistics"
+version: "1.0.0"
+authors:
+ - "Claude (AI-assisted)"
+papers:
+ - "Taylor, 1953"
+ - "Sprouse & Almeida, 2012"
+ - "Brysbaert & New, 2009"
+ - "Schütze & Sprouse, 2014"
+ - "Baayen et al., 2008"
+dependencies:
+ required:
+ - research-literacy
+review_status: "ai-generated"
+---
+
+# Sentence Stimulus Norming
+
+## Purpose
+
+This skill encodes expert methodological knowledge for norming linguistic stimuli before running psycholinguistic experiments. A competent programmer without linguistics training would likely construct stimuli based on intuition, failing to control for critical lexical variables (word frequency, length, neighborhood density), skipping cloze norming, using inappropriate rating scales, or under-powering the norming study. Poor stimulus norming is the single most common methodological weakness in psycholinguistic research, because confounds in the materials propagate to every analysis.
+
+## When to Use
+
+Use this skill when:
+
+- Creating sentence stimuli for reading experiments (self-paced reading, eye-tracking, ERP)
+- Norming the predictability (cloze probability) of critical words in sentence contexts
+- Collecting plausibility, naturalness, or acceptability ratings for sentence materials
+- Controlling lexical properties of critical words across experimental conditions
+- Designing Latin square counterbalancing for within-item designs
+- Planning filler items and practice trials
+
+Do **not** use this skill when:
+
+- Working with single-word stimuli without sentence context (use lexical database tools directly)
+- Designing non-linguistic stimuli (visual search arrays, tones)
+- Analyzing existing normed materials without creating new ones
+
+## Research Planning Protocol
+
+Before executing the domain-specific steps below, you MUST:
+
+1. **State the research question** -- What specific question is this analysis/paradigm addressing?
+2. **Justify the method choice** -- Why is this approach appropriate? What alternatives were considered?
+3. **Declare expected outcomes** -- What results would support vs. refute the hypothesis?
+4. **Note assumptions and limitations** -- What does this method assume? Where could it mislead?
+5. **Present the plan to the user and WAIT for confirmation** before proceeding.
+
+For detailed methodology guidance, see the `research-literacy` skill.
+
+
+## ⚠️ Verification Notice
+
+This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please [open an issue](https://github.com/HaoxuanLiTHUAI/awesome_cognitive_and_neuroscience_skills/issues).
+
+## Cloze Probability Norming
+
+### What Is Cloze Probability?
+
+Cloze probability is the proportion of people who complete a sentence fragment with a particular word (Taylor, 1953). It is the standard measure of a word's predictability in context and is a critical control variable in nearly all sentence processing research.
+
+### Procedure
+
+1. **Create sentence fragments**: Truncate each sentence immediately before the critical word
+2. **Present fragments one at a time** to participants
+3. **Instruct**: "Please complete each sentence with the first word that comes to mind. Write only one word."
+4. **Score**: For each item, cloze probability = (number of completions matching the target word) / (total number of respondents)
+
+### Design Parameters
+
+| Parameter | Recommended Value | Citation / Rationale |
+|-----------|------------------|---------------------|
+| **N per item** | Minimum **30 raters** | Taylor, 1953; Bloom & Fischler, 1980; standard minimum for stable estimates |
+| **Preferred N** | **40-50 raters** | More stable estimates, especially for medium-cloze items |
+| **Items per participant** | **50-100 fragments** per norming session | Avoid fatigue; pilot to calibrate |
+| **Time limit** | **~10-15 seconds per item** or untimed | Untimed is standard; brief limit prevents overthinking |
+| **Population** | Same as experimental population (e.g., native English speakers, same age range) | Ensures cloze values generalize |
+
+### Scoring Conventions
+
+- **Exact match**: Only the target word counts (standard)
+- **Morphological variants**: Decide a priori whether "run" and "running" count as the same completion. Standard practice: count only the **exact form** (Staub et al., 2015)
+- **Spelling errors**: Accept obvious misspellings of the target
+- **Blank/nonsense responses**: Exclude from the denominator (participant did not engage)
+
+### Cloze Probability Benchmarks
+
+| Cloze Range | Label | Use Case |
+|-------------|-------|----------|
+| **> 0.80** | High cloze / highly predictable | N400 amplitude studies; predictability effects (Kutas & Hillyard, 1984) |
+| **0.30 - 0.70** | Medium cloze | Moderate predictability manipulations |
+| **< 0.10** | Low cloze / unpredictable | Baseline; unexpected completions |
+| **0.00** | Zero cloze | Anomalous or implausible continuations |
+
+### Online vs. Lab Norming
+
+| Aspect | Lab | Online (e.g., Prolific, MTurk) |
+|--------|-----|-------------------------------|
+| **Quality control** | Direct observation | Must include catch trials and attention checks |
+| **Sample size** | Limited by lab capacity | Easy to reach N = 40-50 per item |
+| **Population** | Typically university students | More diverse; specify inclusion criteria |
+| **Validity** | Gold standard | Comparable for cloze (Schütze & Sprouse, 2014) |
+| **Cost** | Lab time | Participant payment (~$10-15/hour; Prolific standards) |
+
+**Recommendation for online norming**: Include **10-15% catch trials** (sentences with obvious completions, e.g., "The dog chased the ___") and exclude participants who fail > **20%** of catch trials.
+
+## Plausibility and Naturalness Ratings
+
+### When to Collect
+
+- When cloze probability alone is insufficient (e.g., both conditions have low cloze but differ in plausibility)
+- When manipulating semantic fit or thematic role plausibility
+- When verifying that "anomalous" conditions are genuinely perceived as odd
+
+### Rating Scale Design
+
+| Parameter | Recommended | Citation / Rationale |
+|-----------|------------|---------------------|
+| **Scale type** | Likert scale | Standard for sentence ratings (Schütze & Sprouse, 2014) |
+| **Number of points** | **7-point scale** | Balances sensitivity and reliability; standard in psycholinguistics (Schütze & Sprouse, 2014) |
+| **Anchors** | 1 = "very unnatural/implausible" to 7 = "very natural/plausible" | Labeled endpoints with unlabeled intermediate points |
+| **N per item** | Minimum **20 raters**; preferred **30+** | Sufficient for stable means per item (Sprouse & Almeida, 2012) |
+| **Items per rater** | **40-80** items per session | Avoid fatigue effects |
+| **Practice items** | **3-5 items** spanning the full range before data collection | Calibrate scale use |
+
+### Instructions Template
+
+> "You will read a series of sentences. For each sentence, please rate how natural or plausible it sounds on a scale from 1 to 7, where 1 means 'very unnatural / makes no sense' and 7 means 'perfectly natural / makes complete sense.' There are no right or wrong answers; we are interested in your intuition."
+
+### Critical Design Considerations
+
+- **Within-list design**: Each rater sees only one version of each item (Latin square). Raters should **never** see multiple conditions of the same item, or they will rate contrastively rather than absolutely.
+- **Filler items**: Include filler sentences spanning the full rating range. This prevents range restriction.
+- **Order effects**: Randomize item order per participant.
+
+## Acceptability Judgments
+
+### When to Collect
+
+- When manipulating syntactic structure (grammaticality, island constraints, movement dependencies)
+- When testing formal linguistic predictions about sentence well-formedness
+- For factorial designs crossing syntactic factors (e.g., 2x2 designs testing island effects; Sprouse et al., 2012)
+
+### Rating Methods
+
+| Method | Description | Pros | Cons | Citation |
+|--------|-------------|------|------|----------|
+| **Likert scale (7-point)** | Rate acceptability 1-7 | Simple; familiar; sufficient for most purposes | Ceiling/floor possible; ordinal data | Schütze & Sprouse, 2014 |
+| **Magnitude estimation (ME)** | Assign a number proportional to perceived acceptability relative to a reference sentence | Unbounded scale; ratio-level data (in theory) | More complex; participants need training; debated whether it outperforms Likert | Bard et al., 1996; Sprouse, 2011 |
+| **Forced choice** | Choose the more acceptable of two sentences | Binary; easy; avoids scale-use differences | Low sensitivity; many trials needed | Sprouse & Almeida, 2012 |
+| **Yes/No judgment** | "Is this sentence acceptable?" | Simple; binary | Very low sensitivity; cannot distinguish degrees of unacceptability | -- |
+
+**Recommendation**: Use **7-point Likert** as the default. It provides sufficient sensitivity for most research questions and has been shown to replicate formal linguistic judgments as reliably as magnitude estimation (Sprouse & Almeida, 2012; Sprouse, 2011).
+
+### Sample Size for Acceptability
+
+| Design | Minimum N | Rationale | Citation |
+|--------|-----------|-----------|----------|
+| **Simple grammatical/ungrammatical** | **20 participants** | Large effect sizes (d > 1.0 typical) | Sprouse & Almeida, 2012 |
+| **Factorial (2x2) with interaction** | **30-40 participants** | Interaction effects are smaller | Sprouse et al., 2012 |
+| **Subtle contrasts** | **50+ participants** | Small effect sizes require more power | Power analysis recommended |
+
+## Lexical Controls
+
+### Variables That Must Be Controlled Across Conditions
+
+Every critical word manipulation must control for confounding lexical variables. The target word and its condition-matched alternatives should be equated on the following:
+
+| Variable | Database / Source | Why It Matters | Citation |
+|----------|------------------|----------------|----------|
+| **Word frequency** | SUBTLEX-US (log10 word frequency per million) | Most powerful predictor of reading time; ~30-60 ms effect for high vs. low (Brysbaert & New, 2009) | Brysbaert & New, 2009 |
+| **Word length** | Character count | Longer words = longer reading times; ~20-30 ms per character (Rayner, 2009) | Rayner, 1998 |
+| **Orthographic neighborhood density (N)** | N-Watch; CLEARPOND | Number of words differing by one letter; affects lexical access (Coltheart et al., 1977) | Andrews, 1997 |
+| **Concreteness** | Brysbaert et al. (2014) ratings | Concrete words processed faster than abstract words | Brysbaert et al., 2014 |
+| **Age of acquisition (AoA)** | Kuperman et al. (2012) ratings | Earlier-acquired words processed faster | Kuperman et al., 2012 |
+| **Number of syllables** | Any pronunciation dictionary | Affects phonological processing time | Rayner, 1998 |
+| **Morphological complexity** | Manual coding | Derived words (e.g., un-happi-ness) processed differently than monomorphemic words | Taft, 2004 |
+
+### Frequency Database Selection
+
+| Database | Language | Measure | Recommended? | Citation |
+|----------|----------|---------|--------------|----------|
+| **SUBTLEX-US** | English (US) | Subtitle-based frequency per million | **Yes** -- best predictor of processing times | Brysbaert & New, 2009 |
+| **SUBTLEX-UK** | English (UK) | Subtitle-based frequency | Yes, for British English materials | van Heuven et al., 2014 |
+| **HAL** | English | Usenet corpus frequency | Outdated; SUBTLEX preferred | Lund & Burgess, 1996 |
+| **CELEX** | English, Dutch, German | Mixed corpus frequency | Acceptable but less predictive than SUBTLEX | Baayen et al., 1995 |
+
+**Key recommendation**: Use **SUBTLEX** log frequency values. They explain more variance in lexical decision and naming times than older norms (Brysbaert & New, 2009).
+
+### How to Match Across Conditions
+
+1. **Select critical words** for each condition
+2. **Retrieve lexical metrics** from SUBTLEX-US and norming databases
+3. **Compute condition means** for each metric
+4. **Test for differences**: Run t-tests or ANOVAs across conditions on each lexical variable
+5. **Criterion**: No significant differences (p > 0.20 is a reasonable threshold; some use p > 0.30) on any controlled variable
+6. If matching fails: **replace items** or add the unmatched variable as a **covariate** in the analysis
+
+## Latin Square Counterbalancing
+
+### Purpose
+
+In a within-item design, each item appears in all conditions, but each participant sees each item in only one condition. A Latin square assigns items to conditions across participant lists.
+
+### Construction
+
+For a design with **k conditions** and **n items** (where n is divisible by k):
+
+1. Divide items into **k groups** of n/k items each
+2. Create **k lists**; in each list, assign each item group to a different condition
+3. Each participant receives one list
+4. Result: every item appears in every condition across participants; each participant sees an equal number of items per condition
+
+### Example: 2-Condition Design
+
+With 40 items and 2 conditions (A, B):
+
+| List | Items 1-20 | Items 21-40 |
+|------|-----------|-------------|
+| **List 1** | Condition A | Condition B |
+| **List 2** | Condition B | Condition A |
+
+### Requirements
+
+| Parameter | Value | Rationale |
+|-----------|-------|-----------|
+| **Minimum items per condition per list** | **16-24** | Standard for psycholinguistic experiments; fewer items = lower power (Brysbaert & Stevens, 2018) |
+| **Recommended items** | **24-40** per condition | More stable estimates, especially for eye-tracking |
+| **Participants per list** | Equal across lists; minimum **4-6 per list** | Ensures balanced representation |
+| **Total participants** | Divisible by number of lists | Critical for balanced design |
+
+## Filler Items
+
+### Purpose
+
+Fillers prevent participants from noticing the experimental manipulation and adopting strategies.
+
+### Design Parameters
+
+| Parameter | Recommended Value | Rationale |
+|-----------|------------------|-----------|
+| **Filler-to-target ratio** | **2:1 or 3:1** (fillers:targets) | Standard in psycholinguistics; prevents pattern detection (Schütze & Sprouse, 2014) |
+| **Filler diversity** | Fillers should span the full range of sentence types, lengths, and structures | Prevents target sentences from standing out |
+| **Filler acceptability range** | Include some clearly good and some mildly awkward fillers | Prevents raters from using only part of the scale |
+| **Filler length** | Match the average length of target sentences | Controls for sentence length expectations |
+
+### Filler Construction Tips
+
+- Use fillers from different syntactic constructions than your targets
+- Include some fillers with comprehension questions (for reading studies) to maintain attentive reading
+- If targets are semantically anomalous, include some fillers that are also slightly odd (but in different ways) so anomaly is not a cue
+
+## Practice and Warm-Up Items
+
+| Parameter | Recommended Value | Rationale |
+|-----------|------------------|-----------|
+| **Number of practice items** | **4-6 items** (minimum 3) | Familiarize participants with the task and interface |
+| **Practice item composition** | Span the range of difficulty/acceptability | Calibrate participant expectations |
+| **Practice data** | **Always exclude** from analysis | Practice responses are contaminated by learning effects |
+| **Warm-up items at start of main experiment** | **2-3 additional filler items** | Allow settling into the task; exclude from analysis |
+
+## Online Norming Considerations
+
+### Platform Recommendations
+
+| Platform | Pros | Cons | Typical Pay Rate |
+|----------|------|------|-----------------|
+| **Prolific** | Diverse participants; pre-screening; good data quality | Smaller pool than MTurk | ~$10-15/hour (Prolific minimum: $8/hour) |
+| **Amazon MTurk** | Large pool; fast recruitment | Lower data quality; less diverse; requires careful screening | ~$10-15/hour recommended |
+| **PCIbex / Ibex Farm** | Free hosting; designed for linguistics | Requires programming; no built-in recruitment | (hosting only) |
+| **Gorilla** | GUI-based; good for complex designs | Subscription cost | (hosting only) |
+
+### Quality Control for Online Studies
+
+| Measure | Implementation | Threshold |
+|---------|---------------|-----------|
+| **Catch trials** | Include **10-15%** filler items with obvious answers | Exclude participants failing > **20%** |
+| **Completion time** | Record total time | Exclude participants completing in < **50%** of median time |
+| **Straight-lining** | Check for same response on all items | Exclude participants with zero variance in ratings |
+| **Bot detection** | Include reCAPTCHA or similar | Exclude flagged responses |
+| **Native speaker check** | Self-report + brief language background questionnaire | Exclude non-native speakers (unless studying L2) |
+
+## Common Pitfalls
+
+1. **Not norming cloze probability**: Claiming words are "predictable" or "unpredictable" based on experimenter intuition rather than empirical cloze norms. Always collect cloze data (Taylor, 1953).
+
+2. **Too few raters per item**: With N < 20 raters for cloze, individual item estimates are unstable. A word with true cloze of 0.50 could yield observed cloze of 0.20-0.80 with only 10 raters. Use minimum **30 raters** (Bloom & Fischler, 1980).
+
+3. **Not controlling word frequency**: Frequency is the strongest single predictor of reading time. A 1 log-unit difference in SUBTLEX frequency corresponds to ~**30-40 ms** in gaze duration (Brysbaert & New, 2009; Rayner, 1998). Always match or control.
+
+4. **Using the wrong frequency database**: HAL and Kucera-Francis norms are outdated. SUBTLEX-US explains significantly more variance in behavioral data (Brysbaert & New, 2009).
+
+5. **Showing raters multiple conditions of the same item**: This introduces contrastive evaluation. Raters must see each item in only one condition (Latin square for norming too).
+
+6. **Insufficient filler items**: A 1:1 target-to-filler ratio makes the manipulation transparent. Use at least **2:1** fillers to targets (Schütze & Sprouse, 2014).
+
+7. **Not piloting the norming study**: Always pilot with **5-10 participants** to catch unclear instructions, ambiguous items, and timing issues before running the full norming sample.
+
+8. **Ignoring age of acquisition**: AoA effects are independent of frequency (Kuperman et al., 2012). Failing to control AoA can introduce confounds, especially for studies comparing concrete vs. abstract words.
+
+## Minimum Reporting Checklist
+
+Based on Schütze & Sprouse (2014) and current psycholinguistic standards:
+
+- [ ] Number of items per condition
+- [ ] Cloze probability values: mean, SD, and range per condition (if collected)
+- [ ] Cloze norming details: N raters, population, procedure, scoring criteria
+- [ ] Plausibility/acceptability ratings: scale type, N raters, mean and SD per condition
+- [ ] Lexical control variables: list each controlled variable, database source, and condition means
+- [ ] Statistical test confirming conditions do not differ on controlled variables
+- [ ] Latin square design: number of lists, items per list per condition, participants per list
+- [ ] Filler-to-target ratio and description of filler types
+- [ ] Number of practice/warm-up items
+- [ ] For online norming: platform, pay rate, attention check procedure, exclusion criteria and N excluded
+- [ ] Full item list (in supplementary materials or online repository)
+
+## References
+
+- Andrews, S. (1997). The effect of orthographic similarity on lexical retrieval: Resolving neighborhood conflicts. *Psychonomic Bulletin & Review*, 4, 439-461.
+- Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. *Journal of Memory and Language*, 59, 390-412.
+- Baayen, R. H., Piepenbrock, R., & Gulikers, L. (1995). *The CELEX lexical database* (CD-ROM). Linguistic Data Consortium, University of Pennsylvania.
+- Bard, E. G., Robertson, D., & Sorace, A. (1996). Magnitude estimation of linguistic acceptability. *Language*, 72, 32-68.
+- Bloom, P. A., & Fischler, I. (1980). Completion norms for 329 sentence contexts. *Memory & Cognition*, 8, 631-642.
+- Brysbaert, M., & New, B. (2009). Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. *Behavior Research Methods*, 41, 977-990.
+- Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models: A tutorial. *Journal of Cognition*, 1, 9.
+- Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. *Behavior Research Methods*, 46, 904-911.
+- Coltheart, M., Davelaar, E., Jonasson, J. T., & Besner, D. (1977). Access to the internal lexicon. In S. Dornic (Ed.), *Attention and performance VI*. Hillsdale, NJ: Erlbaum.
+- Kuperman, V., Stadthagen-Gonzalez, H., & Brysbaert, M. (2012). Age-of-acquisition ratings for 30,000 English words. *Behavior Research Methods*, 44, 978-990.
+- Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. *Nature*, 307, 161-163.
+- Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. *Behavior Research Methods, Instruments, & Computers*, 28, 203-208.
+- Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. *Psychological Bulletin*, 124, 372-422.
+- Rayner, K. (2009). Eye movements and attention in reading, scene perception, and visual search. *Quarterly Journal of Experimental Psychology*, 62, 1457-1506.
+- Schütze, C. T., & Sprouse, J. (2014). Judgment data. In R. J. Podesva & D. Sharma (Eds.), *Research methods in linguistics*. Cambridge University Press.
+- Sprouse, J. (2011). A test of the cognitive assumptions of magnitude estimation: Commutativity does not hold for acceptability judgments. *Language*, 87, 274-288.
+- Sprouse, J., & Almeida, D. (2012). Assessing the reliability of textbook data in syntax: Adger's Core Syntax. *Journal of Linguistics*, 48, 609-652.
+- Sprouse, J., Schütze, C. T., & Almeida, D. (2012). A comparison of informal and formal acceptability judgments using a random sample from Linguistic Inquiry 2001-2010. *Lingua*, 134, 219-248.
+- Staub, A., Grant, M., Astheimer, L., & Cohen, A. (2015). The influence of cloze probability and item constraint on cloze task response time. *Journal of Memory and Language*, 82, 1-17.
+- Taft, M. (2004). Morphological decomposition and the reverse base frequency effect. *Quarterly Journal of Experimental Psychology*, 57A, 745-765.
+- Taylor, W. L. (1953). "Cloze procedure": A new tool for measuring readability. *Journalism Quarterly*, 30, 415-433.
+- van Heuven, W. J. B., Mandera, P., Keuleers, E., & Brysbaert, M. (2014). SUBTLEX-UK: A new and improved word frequency database for British English. *Quarterly Journal of Experimental Psychology*, 67, 1176-1190.
+
+See `references/lexical-databases-guide.md` for detailed instructions on accessing and querying lexical control databases.