Quality ledger
Review response patterns without pretending to read minds.
Short answer: keep every validated response visible, describe observable patterns, separate incomplete from completed files, and require a researcher reason for excluding completed evidence. Never treat a speed or shape flag as proof of low effort.
What the field lab flags
| Flag | Observable condition | What it does not prove |
|---|---|---|
| incomplete | Response state is saved, not completed | Why the participant stopped |
| very-fast | Duration is below the researcher-set threshold | Carelessness, expertise, or invalidity |
| unsorted | One or more cards remain in the unsorted tray | Whether the card was confusing or intentionally deferred |
| all-in-one | Every card is in one non-unsorted group | That the participant ignored the task |
| all-singletons | Each used group contains one card | That the mental model is unreasonable |
| unnamed/repeated label | Created groups lack or repeat names | Language ability, motivation, or comprehension |
Recommended review order
- Verify identity and completion. Confirm edition fingerprint, response revision, state, timestamps, and that every card appears exactly once.
- Inspect the whole grouping. Do not decide from duration alone. Read group contents, order, raw labels, unsorted cards, and the optional final comment.
- Apply a pre-agreed rule where possible. If the study protocol defines exclusions, use it consistently and record the rule. Avoid inventing a rule after seeing a desired result.
- Write a bounded reason. Describe evidence such as “excluded under protocol: completion below X seconds and debrief confirmed accidental submission,” not a judgment about the person.
- Compare sensitivity. Re-run descriptive analysis with and without borderline completed responses. Report material changes rather than hiding them.
- Retain the record. Exclusion changes analysis membership but does not erase the imported file. Deletion is a separate explicit action.
Incomplete files
The app preserves them but prevents inclusion in analysis. A resumed higher-revision completed response replaces the earlier saved revision under the same response ID. This avoids double-counting one participant's progress and preserves a deterministic amendment rule.
Duplicates, amendments, and collisions
- Exact duplicate: same response ID and digest; no project change.
- Amendment: same response ID and a higher revision; validated replacement.
- Collision: same response ID with a different digest at the same or lower revision; batch stops.
- Mismatched edition: response edition/fingerprint is unavailable; batch stops.
Human interpretation remains accountable. Do not infer protected traits, cognition, motivation, mental health, competence, or suitability from a card-sort pattern. This tool is not for clinical, educational, or employment decisions about individuals.