Method record
Exact descriptive formulas, visible assumptions.
Evidence identity
A frozen edition contains the participant-facing study settings, active cards, and predefined categories. Canonical JSON serialization is hashed with SHA-256. Every response repeats the edition ID and fingerprint, so a response cannot be silently analyzed against a different card set or wording.
Inclusion
Completed responses are included by default. Saved/incomplete responses are excluded and cannot be included. A researcher may exclude or re-include a completed response with a bounded review note. Imported evidence is not deleted by exclusion. Analysis is recomputed from the current included set.
Pair similarity
pair similarity(A, B) = included responses grouping A and B together / included responses containing both A and B
The table exposes numerator, denominator, and percentage. A response's group labels do not need to match another response's labels for its card pairs to count as together. When the denominator is zero, the pair is unknown and displays as 0 with a zero denominator, not as observed disagreement.
Predefined-category placement
placement(A, category C) = included responses placing A in C / included responses
This applies to closed and hybrid editions. The strongest placement share can describe agreement for a card, but it does not establish that the category is correct or usable.
Raw and standardized labels
Participant-created labels are grouped only after Unicode normalization, trim, whitespace folding, and locale-insensitive case folding. Display variants remain visible. Semantic merging is never automatic. A researcher can attach or remove one standard label while preserving the raw labels and counts.
Response-pair agreement
agreement(R1, R2) = card pairs treated the same way in both responses / all card pairs
For each card pair, the responses agree when both grouped the pair together or both kept it apart. This is an exact descriptive comparison, not an inferential reliability statistic.
Agglomerative clustering
Every card starts as one cluster. Pair distance is 1 - pair similarity. The closest active clusters merge until one root remains. Ties break by stable cluster signatures, making the same evidence and settings deterministic.
| Linkage | Distance between two clusters | Common sensitivity |
|---|---|---|
| Single | Smallest cross-cluster card-pair distance | Can chain clusters through one strong bridge. |
| Complete | Largest cross-cluster card-pair distance | Requires all cross-cluster pairs to remain comparatively close. |
| Unweighted average | Mean cross-cluster card-pair distance | Balances all cross-cluster pair distances without weighting by participant identity. |
A threshold cut joins merges whose similarity is at or above the chosen basis-point threshold. The app compares all three cuts and lists card pairs whose together/apart status changes. A cut is never labeled correct or recommended.
Quality flags
The ledger can flag incomplete, very-fast, unsorted, all-in-one, all-singleton, unnamed-created-group, or repeated-created-label patterns. The fast threshold is researcher-configured. Flags do not infer motivation, ability, protected traits, or research validity, and they never exclude automatically.
Method sources and context
- Nielsen Norman Group: Card Sorting, Definition and Overview, including qualitative/quantitative distinctions and published participant-count guidance.
- Optimal Workshop: choose open, closed, or hybrid, for common method distinctions.
- Optimal Workshop: participant completion options, noting completeness/bias tradeoffs.
- Capra (2005): Factor Analysis of Card Sort Data, for limits of hierarchical clustering when overlapping groups matter.
- A comparison of card-sorting analysis methods, illustrating that different methods expose different structure.