On the public citation graph
Obtaining High-Quality Relevance Judgments Using Crowdsourcing
0 signed · 69 inferred · view as graph
Inferred edges come from two machine sources: sweeps of open repository metadata (mostly SSRN and Zenodo), and a bulk OpenAlex citation snapshot. Every row below is labelled with the one it came from. Neither is a complete citation index — absence here is not evidence of absence.
Is this your work? Sign in with ORCID and put your name on the
claims that are yours.
Reference health. See whether every reference in this work resolves to a
known work and none has been retracted — and get a dated CiteStamped
badge you can embed.
What this work says
Outgoing claims: the papers this work cites, builds on, or argues with.
inferredcites →https://doi.org/10.1007/978-3-642-20161-5_17openalex-snapshot
In Search of Quality in Crowdsourcing for Search Engine Evaluation
inferredcites →https://doi.org/10.1007/s10791-011-9181-9openalex-snapshot
Increasing cheat robustness of crowdsourcing tasks
inferredcites →https://doi.org/10.1016/s0306-4573(00)00010-8openalex-snapshot
Variations in relevance judgments and the measurement of retrieval effectiveness
inferredcites →https://doi.org/10.1145/1357054.1357127openalex-snapshot
Crowdsourcing user studies with Mechanical Turk
inferredcites →https://doi.org/10.1145/1401890.1401965openalex-snapshot
Get another label? improving data quality and data mining using multiple, noisy labelers
inferredcites →https://doi.org/10.1145/1480506.1480508openalex-snapshot
Crowdsourcing for relevance evaluation
inferredcites →https://doi.org/10.1145/1753326.1753688openalex-snapshot
Are your participants gaming the system?
inferredcites →https://doi.org/10.1145/1837885.1837906openalex-snapshot
Quality management on Amazon Mechanical Turk
inferredcites →https://doi.org/10.1145/2009916.2010039openalex-snapshot
Repeatable and reliable search system evaluation using crowdsourcing
inferredcites →https://doi.org/10.1145/2009916.2010057openalex-snapshot
Quantifying test collection quality based on the consistency of relevance judgements
inferredcites →https://doi.org/10.1145/2063576.2063860openalex-snapshot
Worker types and personality traits in crowdsourcing relevance labels
inferredcites →https://doi.org/10.1214/074921708000000336openalex-snapshot
On Detecting Fake Coin Flip Sequences
What points at this work
Incoming claims: where this work appears in other papers’ reference lists and arguments.
inferred← citeshttps://doi.org/10.1002/cae.21868openalex-snapshot
Correctness assessment of a crowdcoding project in a computer programming introductory course
inferred← citeshttps://doi.org/10.1007/978-3-030-12334-5_6openalex-snapshot
What You Sow, So Shall You Reap! Toward Preselection Mechanisms for Macrotask Crowdsourcing
inferred← citeshttps://doi.org/10.1007/978-3-030-26142-9_8openalex-snapshot
Adversarial Active Learning in the Presence of Weak and Malicious Oracles
inferred← citeshttps://doi.org/10.1007/978-3-030-33702-5_10openalex-snapshot
N2TM: A New Node to Trust Matrix Method for Spam Worker Defense in Crowdsourcing Environments
inferred← citeshttps://doi.org/10.1007/978-3-319-07692-8_38openalex-snapshot
Agreement between Crowdsourced Workers and Expert Assessors in Making Relevance Judgment for System Based IR Evaluation
inferred← citeshttps://doi.org/10.1007/978-3-319-16354-3_17openalex-snapshot
A Discriminative Approach to Predicting Assessor Accuracy
inferred← citeshttps://doi.org/10.1007/s10606-018-9336-yopenalex-snapshot
Crowd Anatomy Beyond the Good and Bad: Behavioral Traces for Crowd Worker Modeling and Pre-selection
inferred← citeshttps://doi.org/10.1007/s10791-013-9222-7openalex-snapshot
Crowdsourcing for information retrieval: introduction to the special issue
inferred← citeshttps://doi.org/10.1016/j.artint.2014.04.005openalex-snapshot
Efficient crowdsourcing of unknown experts using bounded multi-armed bandits
inferred← citeshttps://doi.org/10.1016/j.asoc.2019.02.019openalex-snapshot
On the evaluation and selection of classifier learning algorithms with crowdsourced data
inferred← citeshttps://doi.org/10.1016/j.future.2015.07.012openalex-snapshot
Cybermatics: Cyber–physical–social–thinking hyperspace based science and technology
inferred← citeshttps://doi.org/10.1016/j.respol.2019.103825openalex-snapshot
When does crowdsourcing benefit firm stock market performance?
inferred← citeshttps://doi.org/10.1080/17538947.2018.1480670openalex-snapshot
Assessing relevance of tweets for risk communication
inferred← citeshttps://doi.org/10.1080/19331681.2023.2231436openalex-snapshot
Categorizing political campaign messages on social media using supervised machine learning
inferred← citeshttps://doi.org/10.1109/access.2020.3022773openalex-snapshot
A Robust Consistency Model of Crowd Workers in Text Labeling Tasks
inferred← citeshttps://doi.org/10.1109/cvprw.2013.112openalex-snapshot
Leveraging Crowdsourced Data for Creating Temporal Segmentation Ground Truths of Subjective Tasks
inferred← citeshttps://doi.org/10.1109/hicss.2015.549openalex-snapshot
A Systematic Literature Review of Crowdsourcing Research from a Human Resource Management Perspective
inferred← citeshttps://doi.org/10.1109/icb.2015.7139094openalex-snapshot
Annotating Unconstrained Face Imagery: A scalable approach
inferred← citeshttps://doi.org/10.1109/iccv.2019.00437openalex-snapshot
Why Does a Visual Question Have Different Answers?
inferred← citeshttps://doi.org/10.1109/tkde.2019.2896926openalex-snapshot
Quality Control in Crowdsourcing Using Sequential Zero-Determinant Strategies
inferred← citeshttps://doi.org/10.1109/tmc.2014.2316517openalex-snapshot
Utilize Signal Traces from Others? A Crowdsourcing Perspective of Energy Saving in Cellular Data Communication
inferred← citeshttps://doi.org/10.1109/tpds.2013.215openalex-snapshot
An Iterative Method for Calculating Robust Rating Scores
inferred← citeshttps://doi.org/10.1111/ijmr.12135openalex-snapshot
Crowdsourcing: A Review and Suggestions for Future Research
inferred← citeshttps://doi.org/10.1145/2872427.2883070openalex-snapshot
Using Hierarchical Skills for Optimized Task Assignment in Knowledge-Intensive Crowdsourcing
inferred← citeshttps://doi.org/10.1145/3122865.3122867openalex-snapshot
Deep learning for video classification and captioning
inferred← citeshttps://doi.org/10.1145/3122865.3122868openalex-snapshot
Audition for multimedia computing
inferred← citeshttps://doi.org/10.1145/3122865.3122869openalex-snapshot
Multimodal analysis of free-standing conversational groups
inferred← citeshttps://doi.org/10.1145/3122865.3122870openalex-snapshot
Encrypted domain multimedia content analysis
inferred← citeshttps://doi.org/10.1145/3122865.3122872openalex-snapshot
Social-sensed multimedia computing
inferred← citeshttps://doi.org/10.1145/3122865.3122873openalex-snapshot
Situation recognition using multimodal data
inferred← citeshttps://doi.org/10.1145/3122865.3122874openalex-snapshot
Hawkes processes for events in social media
inferred← citeshttps://doi.org/10.1145/3122865.3122875openalex-snapshot
Utilizing implicit user cues for multimedia analytics
inferred← citeshttps://doi.org/10.1145/3479586openalex-snapshot
Quality Control in Crowdsourcing based on Fine-Grained Behavioral Features
inferred← citeshttps://doi.org/10.1145/3596511openalex-snapshot
The Impact of Judgment Variability on the Consistency of Offline Effectiveness Measures
inferred← citeshttps://doi.org/10.1155/2014/135641openalex-snapshot
Creation of Reliable Relevance Judgments in Information Retrieval Systems Evaluation Experimentation through Crowdsourcing: A Review
inferred← citeshttps://doi.org/10.14257/astl.2013.31.31openalex-snapshot
Crowdsourcing Fraud Detection Algorithm Based on Psychological Behavior Analysis
inferred← citeshttps://doi.org/10.14257/ijsia.2014.8.1.26openalex-snapshot
Crowdsourcing Fraud Detection Algorithm Based on Ebbinghaus Forgetting Curve
inferred← citeshttps://doi.org/10.18297/etd/3054openalex-snapshot
Spam elimination and bias correction : ensuring label quality in crowdsourced tasks.
inferred← citeshttps://doi.org/10.22517/23447214.13541openalex-snapshot
Control de calidad en sistemas crowdsourcing: un mapeo sistemático
inferred← citeshttps://doi.org/10.25300/misq/2024/16610openalex-snapshot
Design Principles for Information Categorization Quality in Crowdsourced Crisis Mapping Platforms
inferred← citeshttps://doi.org/10.26686/wgtn.17019560openalex-snapshot
Establish Crowdsourcing as an Organisational Business Process: A Design Science Approach
inferred← citeshttps://doi.org/10.2991/iset-15.2015.47openalex-snapshot
Application Research of Iterative Detection Strategy in the Crowdsourcing Quality Evaluation
inferred← citeshttps://doi.org/10.3390/s21155007openalex-snapshot
A Time-Series-Based New Behavior Trace Model for Crowd Workers That Ensures Quality Annotation
inferred← citeshttps://doi.org/10.3390/su11102834openalex-snapshot
Monetary and Social Rewards for Crowdsourcing
inferred← citeshttps://doi.org/10.4018/978-1-5225-0466-5.ch006openalex-snapshot
Massive Open Program Evaluation
inferred← citeshttps://doi.org/10.4018/978-1-5225-8362-2.ch004openalex-snapshot
Massive Open Program Evaluation
inferred← citeshttps://doi.org/10.48550/arxiv.1211.0390openalex-snapshot
Rating through Voting: An Iterative Method for Robust Rating
inferred← citeshttps://doi.org/10.70675/f0985d6fz1e21z4016z9b2fz62a948318fc7openalex-snapshot
Using hierarchical skills for optimized task selection in crowdsourcing
Editorial review for independent work. ICSAC — the Institute for Complexity
Science and Advanced Computing — runs a structured editorial panel for researchers
publishing outside journals, and publishes on CiteStamp rails. Submission is free
and goes through ORCID: icsacinstitute.org/submit.