On the public citation graph
On the State of Reporting in Crowdsourcing Experiments and a Checklist to Aid Current Practices
0 signed · 100 inferred · view as graph
Inferred edges come from two machine sources: sweeps of open repository metadata (mostly SSRN and Zenodo), and a bulk OpenAlex citation snapshot. Every row below is labelled with the one it came from. Neither is a complete citation index — absence here is not evidence of absence.
Is this your work? Sign in with ORCID and put your name on the
claims that are yours.
Reference health. See whether every reference in this work resolves to a
known work and none has been retracted — and get a dated CiteStamped
badge you can embed.
What this work says
Outgoing claims: the papers this work cites, builds on, or argues with.
inferredcites →https://doi.org/10.1001/jama.291.20.2457openalex-snapshot
Empirical Evidence for Selective Reporting of Outcomes in Randomized Trials
inferredcites →https://doi.org/10.1002/jrsm.1252openalex-snapshot
An exploration of crowdsourcing citation screening for systematic reviews
inferredcites →https://doi.org/10.1007/978-1-4939-0378-8_9openalex-snapshot
Experimental Research in HCI
inferredcites →https://doi.org/10.1007/978-3-642-36257-6_13openalex-snapshot
Data Fusion: Resolving Conflicts from Multiple Sources
inferredcites →https://doi.org/10.1007/s11135-017-0574-8openalex-snapshot
Saturation in qualitative research: exploring its conceptualization and operationalization
inferredcites →https://doi.org/10.1016/j.cosrev.2015.05.001openalex-snapshot
Crowdsourcing: A taxonomy and systematic mapping study
inferredcites →https://doi.org/10.1016/j.jtbi.2011.03.004openalex-snapshot
The promise of Mechanical Turk: How online labor markets can help theorists run behavioral experiments
inferredcites →https://doi.org/10.1017/s1930297500002205openalex-snapshot
Running experiments on Amazon Mechanical Turk
inferredcites →https://doi.org/10.1097/prs.0b013e3181de24bcopenalex-snapshot
Identifying and Avoiding Bias in Research
inferredcites →https://doi.org/10.1109/cvprw.2008.4562953openalex-snapshot
Utility data annotation with Amazon Mechanical Turk
inferredcites →https://doi.org/10.1111/j.1552-3934.2011.02081.xopenalex-snapshot
Publication Manual of the American Psychological Association
inferredcites →https://doi.org/10.1126/science.49.1254.51openalex-snapshot
The American Psychological Association
inferredcites →https://doi.org/10.1136/bmj.g7647openalex-snapshot
Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation
inferredcites →https://doi.org/10.1136/medethics-2012-100798openalex-snapshot
Internet-based crowdsourcing and research ethics: the case for IRB review
inferredcites →https://doi.org/10.1145/1357054.1357127openalex-snapshot
Crowdsourcing user studies with Mechanical Turk
inferredcites →https://doi.org/10.1145/1600150.1600175openalex-snapshot
Financial incentives and the "performance of crowds"
inferredcites →https://doi.org/10.1145/1809400.1809422openalex-snapshot
Financial incentives and the "performance of crowds"
inferredcites →https://doi.org/10.1145/2145204.2145355openalex-snapshot
Shepherding the crowd yields better work
inferredcites →https://doi.org/10.1145/2441776.2441848openalex-snapshot
Quality control mechanisms for crowdsourcing
inferredcites →https://doi.org/10.1145/2464464.2464485openalex-snapshot
Experiences surveying the crowd
inferredcites →https://doi.org/10.1145/2556288.2557155openalex-snapshot
Cognitively inspired task design to improve user performance on crowdsourcing platforms
inferredcites →https://doi.org/10.1145/2631775.2631819openalex-snapshot
A taxonomy of microtasks on the web
inferredcites →https://doi.org/10.1145/2702123.2702541openalex-snapshot
Supporting the Modern Polyglot
inferredcites →https://doi.org/10.1145/2736277.2741102openalex-snapshot
Incentivizing High Quality Crowdwork
inferredcites →https://doi.org/10.1145/2858036.2858115openalex-snapshot
Embracing Error to Enable Rapid Crowdsourcing
inferredcites →https://doi.org/10.1145/2858036.2858330openalex-snapshot
Novices Who Focused or Experts Who Didn't?
inferredcites →https://doi.org/10.1145/2872427.2883035openalex-snapshot
Crowdsourcing Annotations for Websites' Privacy Policies
inferredcites →https://doi.org/10.1145/3025453openalex-snapshot
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
inferredcites →https://doi.org/10.1145/3025453.3025870openalex-snapshot
Narratives in Crowdsourced Evaluation of Visualizations
inferredcites →https://doi.org/10.1145/3027385.3027402openalex-snapshot
Improving learning through achievement priming in crowdsourced information finding microtasks
inferredcites →https://doi.org/10.1145/3078714.3078715openalex-snapshot
Clarity is a Worthwhile Quality
inferredcites →https://doi.org/10.1145/3159652.3159661openalex-snapshot
Demographics and Dynamics of Mechanical Turk Workers
inferredcites →https://doi.org/10.1145/3173574.3174023openalex-snapshot
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
inferredcites →https://doi.org/10.1145/3274423openalex-snapshot
Resolvable vs. Irresolvable Disagreement
inferredcites →https://doi.org/10.1145/3287560.3287596openalex-snapshot
Model Cards for Model Reporting
inferredcites →https://doi.org/10.1145/3290605.3300637openalex-snapshot
Understanding and Mitigating Worker Biases in the Crowdsourced Collection of Subjective Judgments
inferredcites →https://doi.org/10.1145/3313831.3376448openalex-snapshot
Transparency of CHI Research Artifacts: Results of a Self-Reported Survey
inferredcites →https://doi.org/10.1145/3331184.3331353openalex-snapshot
Revealing the Role of User Moods in Struggling Search Tasks
inferredcites →https://doi.org/10.1147/jrd.2019.2942288openalex-snapshot
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
inferredcites →https://doi.org/10.1162/tacl_a_00041openalex-snapshot
Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science
inferredcites →https://doi.org/10.1177/1745691617706516openalex-snapshot
An Evaluation of Amazon’s Mechanical Turk, Its Rapid Rise, and Its Effective Use
inferredcites →https://doi.org/10.1186/1741-7015-8-18openalex-snapshot
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
inferredcites →https://doi.org/10.1186/1745-6215-11-32openalex-snapshot
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
inferredcites →https://doi.org/10.1186/s13104-019-4858-zopenalex-snapshot
Crowdsourced dataset to study the generation and impact of text highlighting in classification tasks
inferredcites →https://doi.org/10.1371/journal.pone.0039116openalex-snapshot
The Effects of Demand Characteristics on Research Participant Behaviours in Non-Laboratory Settings: A Systematic Review
inferredcites →https://doi.org/10.1371/journal.pone.0057410openalex-snapshot
Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research
inferredcites →https://doi.org/10.1371/journal.pone.0233154openalex-snapshot
Enhancing big data in the social sciences with crowdsourcing: Data augmentation practices, techniques, and opportunities
inferredcites →https://doi.org/10.15779/z38z92xopenalex-snapshot
Working The Crowd: Employment And Labor Law In The Crowdsourcing Industry
inferredcites →https://doi.org/10.1609/hcomp.v2i1.13154openalex-snapshot
Scaling-Up the Crowd: Micro-Task Pricing Schemes for Worker Retention and Latency Improvement
inferredcites →https://doi.org/10.1609/hcomp.v4i1.13270openalex-snapshot
MicroTalk: Using Argumentation to Improve Crowdsourcing Accuracy
inferredcites →https://doi.org/10.1609/hcomp.v4i1.13284openalex-snapshot
Crowdsourcing Relevance Assessments: The Unexpected Benefits of Limiting the Time to Judge
inferredcites →https://doi.org/10.1609/hcomp.v5i1.13317openalex-snapshot
Confusing the Crowd: Task Instruction Quality on Amazon Mechanical Turk
inferredcites →https://doi.org/10.1609/hcomp.v7i1.5264openalex-snapshot
Platform-Related Factors in Repeatability and Reproducibility of Crowdsourcing Tasks
inferredcites →https://doi.org/10.1609/hcomp.v7i1.5268openalex-snapshot
Understanding the Impact of Text Highlighting in Crowdsourcing Tasks
inferredcites →https://doi.org/10.1609/hcomp.v7i1.5283openalex-snapshot
Fair Work: Crowd Work Minimum Wage with One Line of Code
inferredcites →https://doi.org/10.1609/icwsm.v5i1.14105openalex-snapshot
An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets
inferredcites →https://doi.org/10.18653/v1/n16-1104openalex-snapshot
Effective Crowd Annotation for Relation Extraction
inferredcites →https://doi.org/10.2307/2346806openalex-snapshot
Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm
inferredcites →https://doi.org/10.3115/1613715.1613751openalex-snapshot
Cheap and fast---but is it good?
inferredcites →https://doi.org/10.3389/fninf.2017.00076openalex-snapshot
Reproducibility vs. Replicability: A Brief History of a Confused Terminology
inferredcites →https://doi.org/10.3758/s13428-011-0124-6openalex-snapshot
Conducting behavioral research on Amazon’s Mechanical Turk
inferredcites →https://doi.org/10.3758/s13428-013-0365-7openalex-snapshot
Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers
inferredcites →https://doi.org/10.4324/9781410601155openalex-snapshot
Statistics As Principled Argument
inferredcites →https://doi.org/10.48550/arxiv.1810.10733openalex-snapshot
Cicero: Multi-Turn, Contextual Argumentation for Accurate Crowdsourcing
inferredcites →https://doi.org/10.48550/arxiv.1909.02800openalex-snapshot
CrowdHub: Extending crowdsourcing platforms for the controlled evaluation of tasks designs
inferredcites →https://doi.org/10.48550/arxiv.2003.12206openalex-snapshot
Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)
What points at this work
Incoming claims: where this work appears in other papers’ reference lists and arguments.
inferred← citeshttps://doi.org/10.1002/asi.24724openalex-snapshot
What scholars and IRBs talk when they talk about the Belmont principles in crowd work‐based research
inferred← citeshttps://doi.org/10.1007/978-3-031-19433-7_32openalex-snapshot
WDV: A Broad Data Verbalisation Dataset Built from Wikidata
inferred← citeshttps://doi.org/10.1007/978-3-031-20984-0_46openalex-snapshot
Quality Engineering in AI Services
inferred← citeshttps://doi.org/10.1007/s10606-023-09472-9openalex-snapshot
Undue Influence or Exploitation — A Qualitative Inquiry into an Ethical Dilemma Around Payment in Crowd Work-Based Research in the U.S.
inferred← citeshttps://doi.org/10.1007/s10676-023-09725-7openalex-snapshot
The landscape of data and AI documentation approaches in the European policy context
inferred← citeshttps://doi.org/10.1037/pha0000546openalex-snapshot
Careless responding in crowdsourced alcohol research: A systematic review and meta-analysis of practices and prevalence.
inferred← citeshttps://doi.org/10.1108/jices-01-2024-0011openalex-snapshot
Necessary but not sufficient – examining the Belmont principles’ application in social and behavioral research ethics from a Confucian perspective
inferred← citeshttps://doi.org/10.1109/thms.2023.3319290openalex-snapshot
NLP-Crowdsourcing Hybrid Framework for Inter-Researcher Similarity Detection
inferred← citeshttps://doi.org/10.1145/3491101.3516384openalex-snapshot
Towards fair and pro-social employment of digital pieceworkers for sourcing machine learning training data
inferred← citeshttps://doi.org/10.1145/3531146.3534647openalex-snapshot
CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation
inferred← citeshttps://doi.org/10.1145/3544548.3581253openalex-snapshot
Overcoming Algorithm Aversion: A Comparison between Process and Outcome Control
inferred← citeshttps://doi.org/10.1145/3592616openalex-snapshot
A Survey of Data Quality Requirements That Matter in ML Development Pipelines
inferred← citeshttps://doi.org/10.1145/3603555.3603567openalex-snapshot
Investigating Responsible Nudge Design for Informed Decision-Making Enabling Transparent and Reflective Decision-Making
inferred← citeshttps://doi.org/10.1145/3613904.3642834openalex-snapshot
If in a Crowdsourced Data Annotation Pipeline, a GPT-4
inferred← citeshttps://doi.org/10.1145/3641023openalex-snapshot
The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and Guidelines
inferred← citeshttps://doi.org/10.1145/3706599.3719962openalex-snapshot
Creating Conversational Datasets for Retrieval-Augmented Generation Applications is Hard: Challenges & Research Opportunities
inferred← citeshttps://doi.org/10.1145/3757466openalex-snapshot
Designing Value-Centered Consent Interfaces: A Mixed-Methods Approach to Support Patient Values in Data-Sharing Decisions
inferred← citeshttps://doi.org/10.1145/3805689.3806510openalex-snapshot
The Consensus Trap: Dissecting Subjectivity and the “Ground Truth” Illusion in Data Annotation
inferred← citeshttps://doi.org/10.1609/hcomp.v11i1.27547openalex-snapshot
Collect, Measure, Repeat: Reliability Factors for Responsible AI Data Collection
inferred← citeshttps://doi.org/10.1609/hcomp.v9i1.18939openalex-snapshot
A Checklist to Combat Cognitive Biases in Crowdsourcing
inferred← citeshttps://doi.org/10.18653/v1/2022.gem-1.12openalex-snapshot
Evaluation of Response Generation Models: Shouldn’t It Be Shareable and Replicable?
inferred← citeshttps://doi.org/10.3390/app13042198openalex-snapshot
Designing for Hybrid Intelligence: A Taxonomy and Survey of Crowd-Machine Interaction
Editorial review for independent work. ICSAC — the Institute for Complexity
Science and Advanced Computing — runs a structured editorial panel for researchers
publishing outside journals, and publishes on CiteStamp rails. Submission is free
and goes through ORCID: icsacinstitute.org/submit.