On the public citation graph
The Impact of Judgment Variability on the Consistency of Offline Effectiveness Measures
0 signed · 47 inferred · view as graph
Inferred edges come from two machine sources: sweeps of open repository metadata (mostly SSRN and Zenodo), and a bulk OpenAlex citation snapshot. Every row below is labelled with the one it came from. Neither is a complete citation index — absence here is not evidence of absence.
Is this your work? Sign in with ORCID and put your name on the
claims that are yours.
Reference health. See whether every reference in this work resolves to a
known work and none has been retracted — and get a dated CiteStamped
badge you can embed.
What this work says
Outgoing claims: the papers this work cites, builds on, or argues with.
inferredcites →https://doi.org/10.1002/(sici)1097-4571(199104)42:3<166::aid-asi2>3.0.co;2-aopenalex-snapshot
A study of probabilistic information retrieval systems in the case of inconsistent expert judgments
inferredcites →https://doi.org/10.1002/asi.20583openalex-snapshot
TREC: Experiment and evaluation in information retrieval
inferredcites →https://doi.org/10.1007/s10791-012-9205-0openalex-snapshot
An analysis of human factors and label accuracy in crowdsourcing relevance judgments
inferredcites →https://doi.org/10.1007/s10791-015-9274-yopenalex-snapshot
Test collection reliability: a study of bias and robustness to statistical assumptions via stochastic simulation
inferredcites →https://doi.org/10.1007/s10791-022-09411-0openalex-snapshot
Shallow pooling for sparse labels
inferredcites →https://doi.org/10.1016/0020-0271(68)90029-6openalex-snapshot
Relevance assessments and retrieval system evaluation
inferredcites →https://doi.org/10.1016/0306-4573(92)90031-topenalex-snapshot
Variations in relevance judgments and the evaluation of retrieval performance
inferredcites →https://doi.org/10.1016/s0306-4573(00)00010-8openalex-snapshot
Variations in relevance judgments and the measurement of retrieval effectiveness
inferredcites →https://doi.org/10.1109/mic.2012.71openalex-snapshot
Obtaining High-Quality Relevance Judgments Using Crowdsourcing
inferredcites →https://doi.org/10.1109/tkde.2019.2947049openalex-snapshot
Fixed-Cost Pooling Strategies
inferredcites →https://doi.org/10.1145/1416950.1416952openalex-snapshot
Rank-biased precision for measurement of retrieval effectiveness
inferredcites →https://doi.org/10.1145/1835449.1835540openalex-snapshot
The effect of assessor error on IR system evaluation
inferredcites →https://doi.org/10.1145/1852102.1852106openalex-snapshot
A similarity measure for indefinite rankings
inferredcites →https://doi.org/10.1145/2094072.2094076openalex-snapshot
Multiple testing in statistical analysis of systems-based information retrieval experiments
inferredcites →https://doi.org/10.1145/3015022.3015025openalex-snapshot
Judgment Pool Effects Caused by Query Variations
inferredcites →https://doi.org/10.1145/3052768openalex-snapshot
Incorporating User Expectations and Behavior into the Measurement of Search Effectiveness
inferredcites →https://doi.org/10.1145/3086701openalex-snapshot
Using Replicates in Information Retrieval Evaluation
inferredcites →https://doi.org/10.1145/3239572openalex-snapshot
Estimating Measurement Uncertainty for Information Retrieval Effectiveness Metrics
inferredcites →https://doi.org/10.1145/3310364openalex-snapshot
Using Collection Shards to Study Retrieval Performance Effect Sizes
inferredcites →https://doi.org/10.1145/3336191.3371857openalex-snapshot
Crowd Worker Strategies in Relevance Judgment Tasks
inferredcites →https://doi.org/10.1145/582415.582418openalex-snapshot
Cumulated gain-based evaluation of IR techniques
inferredcites →https://doi.org/10.1353/lib.0.0000openalex-snapshot
Effects of Inconsistent Relevance Judgments on Information Retrieval Test Results: A Historical Perspective
inferredcites →https://doi.org/10.1561/1500000009openalex-snapshot
Test Collection Based Evaluation of Information Retrieval Systems
What points at this work
Incoming claims: where this work appears in other papers’ reference lists and arguments.
inferred← citeshttps://doi.org/10.1145/3664190.3672519openalex-snapshot
Query Variability and Experimental Consistency: A Concerning Case Study
inferred← citeshttps://doi.org/10.1145/3674127.3674133openalex-snapshot
Query-driven Search Functionality
inferred← citeshttps://doi.org/10.1145/3674127.3674137openalex-snapshot
Temporal Extraction and Retrieval
inferred← citeshttps://doi.org/10.1145/3674127.3674139openalex-snapshot
Privacy in Information Retrieval
inferred← citeshttps://doi.org/10.1145/3674127.3674140openalex-snapshot
eCommerce Search and Discovery
inferred← citeshttps://doi.org/10.1145/3674127.3674142openalex-snapshot
Searching Personal Collections
inferred← citeshttps://doi.org/10.1145/3788872openalex-snapshot
On the Use of LLMs for Relevance Labelling
Editorial review for independent work. ICSAC — the Institute for Complexity
Science and Advanced Computing — runs a structured editorial panel for researchers
publishing outside journals, and publishes on CiteStamp rails. Submission is free
and goes through ORCID: icsacinstitute.org/submit.