On this page
Define what one click row proves
The source years are the publication years shown by the papers; both source pages were checked on 22 October 2026. Begin with a single impression opportunity: query or request, ranked item, rank position, interface layout, timestamp, eligibility state, and click event. A row with click=true proves only that the logging system associated an interaction with that exposure under its event definition. It does not prove the item was read, preferred, relevant, or the best available result.
| Field | Record | Hold if |
|---|---|---|
| Exposure id | Unique display or request id | Click has no display context |
| Rank | Position and layout coordinates | Grid and list are combined |
| Eligibility | Items eligible and filtered | Unshown items are mixed with shown |
| Query context | Synthetic query or request family | Query is redacted to an opaque total |
| Interface | Device, viewport, or component version | Layout changed within sample |
| Event | Click definition and timestamp | Duplicate or replay rule absent |
Keep the candidate set and the shown set separate. A zero click can mean no interest, no exposure, a poor position, an early exit, or a missing event. The review should preserve those competing explanations.
Audit exposure before comparing click rates
For each comparison, ask whether items had the same opportunity to be seen. Position can mean a one-based list rank, a grid cell, a carousel slot, or a promoted component. Record the interface and any pinned, filtered, or missing positions. If a ranking policy changed, segment the evidence instead of pooling rows under one position number.
| Comparison | Required context | Unsafe shortcut |
|---|---|---|
| Item A vs B | Same request family and exposure rule | Compare raw click totals |
| Rank 1 vs Rank 5 | Same layout and candidate policy | Call rank effect relevance |
| Before vs after | Policy and interface version | Treat time as randomization |
| Shown vs not shown | Exposure eligibility and logging | Use unshown as zero-click |
| Mobile vs desktop | Viewport and component geometry | Assume same examination |
The Google Research, (opens in a new tab) paper discusses click models that account for examination behavior beyond a simple position assumption. The practical implication for a review is modest: retain enough context to state which assumptions were made, and mark unobserved exposure as unknown.
Separate descriptive counts from adjustment claims
A descriptive table can report impressions and clicks by rank or interface. An adjusted estimate requires a stated model, assumptions, training or holdout boundary, and validation evidence. Do not call a click rate debiased because a rank column exists. If a policy uses randomized or naturally varying rank evidence, record how the variation arose and which rows were excluded.
- Freeze the event and exposure definitions.
- Split by ranking policy and interface version.
- Check candidate availability and unshown items.
- Report counts by position with missing-context counts.
- Describe any adjustment model and its assumptions.
- Keep adjusted outputs separate from raw click evidence.
Use a synthetic click log worksheet
Synthetic impressions and clicks
exposure | request | layout | rank | item | shown | clicked | note
fixture-1 | qA | list-v1 | 1 | item-X | yes | yes | click after exposure
fixture-2 | qA | list-v1 | 2 | item-Y | yes | no | visible opportunity only
fixture-3 | qA | list-v1 | 3 | item-Z | no | blank | not shown
fixture-4 | qB | grid-v1 | cell-1 | item-X | yes | no | geometry differs
report: keep qA and qB separate; do not score not-shown as a zero click| Test | Expected result | Hold condition |
|---|---|---|
| Not shown | No click denominator | Counted as a non-click |
| Layout change | Separate interface cohort | Rank treated as equivalent |
| Duplicate event | Replay rule requested | Two clicks assumed two users |
| Policy change | Segmented report | Before and after pooled |
| Unknown query | Context gap flagged | Relevance inferred from item total |
All identifiers and queries in this fixture are invented. Add a row with a click but no exposure id, a pinned item, and a missing rank. Each should produce a stable finding and a request for the missing context.
Keep click evidence separate from ranking conclusions
Click-log review can document exposure and confounding. It cannot establish user intent, relevance, fairness, quality, or the correctness of a ranking policy. Preserve raw events, candidate context, interface versions, and any adjustment method so a qualified analyst can interpret them.
Pair this guide with impression and click denominator review for event counting and event ordering and replay review for duplicate handling. After a supported non-health scope, authority to describe the collection, and a controlled click-log packet are established, the qualified appraisal request path may be a preparation step. It does not rank items, infer relevance, create a quote, match a buyer, or grant a license.
Takeaways
- Define a click as an event attached to an exposure, not as a relevance label.
- Keep rank, layout, candidate eligibility, policy, and query context.
- Treat unshown items and missing exposure ids as unknown, not zero.
- Separate descriptive click counts from modeled adjustment claims.
- Use synthetic logs to test missing context and duplicate events.