On the public citation graph
On Crowdsourcing Relevance Magnitudes for Information Retrieval Evaluation
0 signed · 161 inferred · view as graph
Inferred edges come from two machine sources: sweeps of open repository metadata (mostly SSRN and Zenodo), and a bulk OpenAlex citation snapshot. Every row below is labelled with the one it came from. Neither is a complete citation index — absence here is not evidence of absence.
Is this your work? Sign in with ORCID and put your name on the
claims that are yours.
Reference health. See whether every reference in this work resolves to a
known work and none has been retracted — and get a dated CiteStamped
badge you can embed.
What this work says
Outgoing claims: the papers this work cites, builds on, or argues with.
inferredcites →https://doi.org/10.1002/(sici)1097-4571(1999)50:3<254::aid-asi8>3.0.co;2-yopenalex-snapshot
Towards the identification of the optimal number of relevance categories
inferredcites →https://doi.org/10.1002/1097-4571(2000)9999:9999<::aid-asi1564>3.0.co;2-lopenalex-snapshot
Regions and levels: Measuring and mapping users' relevance judgments
inferredcites →https://doi.org/10.1002/asi.20583openalex-snapshot
TREC: Experiment and evaluation in information retrieval
inferredcites →https://doi.org/10.1002/asi.20682openalex-snapshot
Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance
inferredcites →https://doi.org/10.1007/978-3-319-16354-3_23openalex-snapshot
Judging Relevance Using Magnitude Estimation
inferredcites →https://doi.org/10.1007/978-3-642-28997-2_16openalex-snapshot
On Aggregating Labels from Multiple Crowd Workers to Infer Relevance of Documents
inferredcites →https://doi.org/10.1007/s10791-012-9205-0openalex-snapshot
An analysis of human factors and label accuracy in crowdsourcing relevance judgments
inferredcites →https://doi.org/10.1007/s10791-012-9206-zopenalex-snapshot
Crowdsourcing interactions: using crowdsourcing for evaluating interactive information retrieval systems
inferredcites →https://doi.org/10.1016/0306-4573(88)90042-8openalex-snapshot
Measuring relevance judgments
inferredcites →https://doi.org/10.1016/j.ipm.2006.07.020openalex-snapshot
On the reliability of information retrieval metrics based on graded relevance
inferredcites →https://doi.org/10.1016/j.ipm.2012.01.004openalex-snapshot
Using crowdsourcing for TREC relevance assessment
inferredcites →https://doi.org/10.1080/19312450709336664openalex-snapshot
Answering the Call for a Standard Reliability Measure for Coding Data
inferredcites →https://doi.org/10.1109/mic.2012.95openalex-snapshot
Examining the Limits of Crowdsourcing for Relevance Assessment
inferredcites →https://doi.org/10.1111/1467-9884.t01-9-00356openalex-snapshot
The Handbook of Parametric and Nonparametric Statistical Procedures
inferredcites →https://doi.org/10.1111/j.1745-4557.1977.tb00942.xopenalex-snapshot
MAGNITUDE ESTIMATION: NOTES ON WHAT, HOW, WHEN, AND WHY TO USE IT
inferredcites →https://doi.org/10.1126/science.151.3710.530openalex-snapshot
A Metric for the Social Consensus
inferredcites →https://doi.org/10.1145/1102351.1102363openalex-snapshot
Learning to rank using gradient descent
inferredcites →https://doi.org/10.1145/1645953.1646032openalex-snapshot
Empirical justification of the gain and discount function for nDCG
inferredcites →https://doi.org/10.1145/1645953.1646033openalex-snapshot
Expected reciprocal rank for graded relevance
inferredcites →https://doi.org/10.1145/1835449.1835540openalex-snapshot
The effect of assessor error on IR system evaluation
inferredcites →https://doi.org/10.1145/2009916.2010057openalex-snapshot
Quantifying test collection quality based on the consistency of relevance judgements
inferredcites →https://doi.org/10.1145/2484028.2484156openalex-snapshot
Assessor disagreement and text classifier accuracy
inferredcites →https://doi.org/10.1145/2505515.2507884openalex-snapshot
An analysis of crowd workers mistakes for specific and complex relevance assessment task
inferredcites →https://doi.org/10.1145/2600428.2609577openalex-snapshot
Multidimensional relevance modeling via psychometrics and crowdsourcing
inferredcites →https://doi.org/10.1145/2766462.2767760openalex-snapshot
The Benefits of Magnitude Estimation Relevance Assessments for Information Retrieval Evaluation
inferredcites →https://doi.org/10.1145/564376.564433openalex-snapshot
Liberal relevance criteria of TREC -
inferredcites →https://doi.org/10.1145/582415.582418openalex-snapshot
Cumulated gain-based evaluation of IR techniques
inferredcites →https://doi.org/10.1177/002224378001700401openalex-snapshot
The Optimal Number of Response Alternatives for a Scale: A Review
inferredcites →https://doi.org/10.1177/154193120304700406openalex-snapshot
Usability Magnitude Estimation
inferredcites →https://doi.org/10.1177/154193121005401229openalex-snapshot
How Much is too Much? Using Magnitude Estimation for User Experience Research
inferredcites →https://doi.org/10.1198/tech.2004.s209openalex-snapshot
Handbook of Parametric and Nonparametric Statistical Procedures
inferredcites →https://doi.org/10.1609/aimag.v36i1.2564openalex-snapshot
Truth Is a Lie: Crowd Truth and the Seven Myths of Human Annotation
inferredcites →https://doi.org/10.2307/1421606openalex-snapshot
Sensory Processes: The New Psychophysics
inferredcites →https://doi.org/10.2307/416793openalex-snapshot
Magnitude Estimation of Linguistic Acceptability
What points at this work
Incoming claims: where this work appears in other papers’ reference lists and arguments.
inferred← citeshttps://doi.org/10.1007/978-1-4614-7163-9_110177-1openalex-snapshot
Spatiotemporal Recommendation in Geo-Social Networks
inferred← citeshttps://doi.org/10.1007/978-3-030-02925-8_4openalex-snapshot
Geographical Proximity Boosted Recommendation Algorithms for Real Estate
inferred← citeshttps://doi.org/10.1007/978-3-030-12450-2_13openalex-snapshot
Exploring the Potential of Social Media Content for Detecting Transport-Related Activities
inferred← citeshttps://doi.org/10.1007/978-3-030-15712-8_25openalex-snapshot
Optimizing Ranking Models in an Online Setting
inferred← citeshttps://doi.org/10.1007/978-3-030-15712-8_29openalex-snapshot
On Cross-Domain Transfer in Venue Recommendation
inferred← citeshttps://doi.org/10.1007/978-3-030-25213-7_11openalex-snapshot
Analysis of Online Social Network after an Event
inferred← citeshttps://doi.org/10.1007/978-3-030-37309-2_16openalex-snapshot
Correlation Analysis of Applications’ Features: A Case Study on Google Play
inferred← citeshttps://doi.org/10.1007/978-3-030-45442-5_26openalex-snapshot
Crowdsourcing Truthfulness: The Impact of Judgment Scale and Assessor Bias
inferred← citeshttps://doi.org/10.1007/978-3-030-69143-1_43openalex-snapshot
Mobile Application Software Usability Evaluation: Issues, Methods and Future Research Directions
inferred← citeshttps://doi.org/10.1007/978-3-031-02318-7openalex-snapshot
The Practice of Crowdsourcing
inferred← citeshttps://doi.org/10.1007/978-3-319-75683-7_10openalex-snapshot
Multimedia Data Management for Disaster Situation Awareness
inferred← citeshttps://doi.org/10.1007/978-3-319-76941-7_15openalex-snapshot
Modelling Randomness in Relevance Judgments and Evaluation Measures
inferred← citeshttps://doi.org/10.1007/978-3-658-37512-6openalex-snapshot
Popularität und Relevanz in der Suche
inferred← citeshttps://doi.org/10.1007/978-981-10-5209-5_9openalex-snapshot
Deep Learning in Social Computing
inferred← citeshttps://doi.org/10.1007/978-981-15-7527-3_69openalex-snapshot
An Empirical Study on Usability and Security of E-Commerce Websites
inferred← citeshttps://doi.org/10.1007/s00500-020-05397-3openalex-snapshot
Recommendation system based on semantic scholar mining and topic modeling on conference publications
inferred← citeshttps://doi.org/10.1007/s00521-021-05705-4openalex-snapshot
AutoFM: an efficient factorization machine model via probabilistic auto-encoders
inferred← citeshttps://doi.org/10.1007/s00779-019-01271-8openalex-snapshot
Personalized recommendation of film and television culture based on an intelligent classification algorithm
inferred← citeshttps://doi.org/10.1007/s00779-021-01604-6openalex-snapshot
Can the crowd judge truthfulness? A longitudinal study on recent misinformation about COVID-19
inferred← citeshttps://doi.org/10.1007/s10115-020-01477-wopenalex-snapshot
TAILOR: time-aware facility location recommendation based on massive trajectories
inferred← citeshttps://doi.org/10.1007/s10115-020-01528-2openalex-snapshot
A hybrid neural network approach to combine textual information and rating information for item recommendation
inferred← citeshttps://doi.org/10.1007/s10462-020-09873-yopenalex-snapshot
Review text based rating prediction approaches: preference knowledge learning, representation and utilization
inferred← citeshttps://doi.org/10.1007/s10619-020-07311-0openalex-snapshot
Influential facilities placement over moving objects
inferred← citeshttps://doi.org/10.1007/s10791-020-09374-0openalex-snapshot
On the nature of information access evaluation metrics: a unifying framework
inferred← citeshttps://doi.org/10.1007/s10791-022-09411-0openalex-snapshot
Shallow pooling for sparse labels
inferred← citeshttps://doi.org/10.1007/s10844-020-00616-7openalex-snapshot
Empirical study of sentiment analysis tools and techniques on societal topics
inferred← citeshttps://doi.org/10.1007/s11042-018-7019-9openalex-snapshot
Collaborative tensor–topic factorization model for personalized activity recommendation
inferred← citeshttps://doi.org/10.1007/s11069-020-04429-3openalex-snapshot
Can we detect trends in natural disaster management with artificial intelligence? A review of modeling practices
inferred← citeshttps://doi.org/10.1007/s11280-017-0476-7openalex-snapshot
Joint user knowledge and matrix factorization for recommender systems
inferred← citeshttps://doi.org/10.1007/s11280-021-00865-8openalex-snapshot
An integrated model based on deep multimodal and rank learning for point-of-interest recommendation
inferred← citeshttps://doi.org/10.1007/s11432-017-9243-7openalex-snapshot
Integrating a weighted-average method into the random walk framework to generate individual friend recommendations
inferred← citeshttps://doi.org/10.1007/s13369-018-3456-zopenalex-snapshot
A Two-stage Iterative Approach to Improve Crowdsourcing-Based Relevance Assessment
inferred← citeshttps://doi.org/10.1007/s40685-020-00108-yopenalex-snapshot
Frontiers of business intelligence and analytics 3.0: a taxonomy-based literature review and research agenda
inferred← citeshttps://doi.org/10.1007/s41019-019-00115-yopenalex-snapshot
Deep Learning for User Interest and Response Prediction in Online Display Advertising
inferred← citeshttps://doi.org/10.1016/j.asoc.2018.04.042openalex-snapshot
Interactive object retrieval via cost-minimizing queries
inferred← citeshttps://doi.org/10.1016/j.comcom.2021.01.030openalex-snapshot
The prediction model for residence based on reliability in social network
inferred← citeshttps://doi.org/10.1016/j.cor.2022.105995openalex-snapshot
Task design in complex crowdsourcing experiments: Item assignment optimization
inferred← citeshttps://doi.org/10.1016/j.eswa.2020.113193openalex-snapshot
Intelligent personal assistants: A systematic literature review
inferred← citeshttps://doi.org/10.1016/j.eswa.2020.114065openalex-snapshot
A robust personalized location recommendation based on ensemble learning
inferred← citeshttps://doi.org/10.1016/j.ijhcs.2021.102650openalex-snapshot
Recognising Intrinsic Motivation using Smartphone Trajectories
inferred← citeshttps://doi.org/10.1016/j.im.2021.103557openalex-snapshot
Explore for a day? Generating personalized itineraries that fit spatial heterogeneity of tourist attractions
inferred← citeshttps://doi.org/10.1016/j.ipm.2021.102688openalex-snapshot
On the effect of relevance scales in crowdsourcing relevance assessments for Information Retrieval evaluation
inferred← citeshttps://doi.org/10.1016/j.jmapro.2020.12.067openalex-snapshot
A vision-based method for lap weld defects monitoring of galvanized steel sheets using convolutional neural network
inferred← citeshttps://doi.org/10.1016/j.jvcir.2019.102724openalex-snapshot
Quality-related English text classification based on recurrent neural network
inferred← citeshttps://doi.org/10.1016/j.knosys.2021.107623openalex-snapshot
Enabling the interpretability of pretrained venue representations using semantic categories
inferred← citeshttps://doi.org/10.1049/cit2.12056openalex-snapshot
A multi‐agent system for itinerary suggestion in smart environments
inferred← citeshttps://doi.org/10.1049/iet-its.2019.0017openalex-snapshot
Convolutional LSTM based transportation mode learning from raw GPS trajectories
inferred← citeshttps://doi.org/10.1080/1206212x.2021.1949117openalex-snapshot
A survey on non-factoid question answering systems
inferred← citeshttps://doi.org/10.1080/19312458.2021.1965973openalex-snapshot
Building the Bridge: Topic Modeling for Comparative Research
inferred← citeshttps://doi.org/10.1088/1742-6596/1642/1/012008openalex-snapshot
Deep Pyramid Convolutional Neural Network Integrated with Self-attention Mechanism and Highway Network for Text Classification
inferred← citeshttps://doi.org/10.1088/1742-6596/1757/1/012038openalex-snapshot
An Overview on Fine-grained Text Sentiment Analysis: Survey and Challenges
inferred← citeshttps://doi.org/10.1108/ajim-05-2022-0253openalex-snapshot
Reflection on future directions: a systematic review of reported limitations and solutions in interactive information retrieval user studies
inferred← citeshttps://doi.org/10.1109/access.2022.3211668openalex-snapshot
Batch Evaluation Metrics in Information Retrieval: Measures, Scales, and Meaning
inferred← citeshttps://doi.org/10.1109/access.2024.3377239openalex-snapshot
Reliable Information Retrieval Systems Performance Evaluation: A Review
inferred← citeshttps://doi.org/10.1109/iac.2018.8780481openalex-snapshot
A Proposed Crowdsourcing Engine for Indonesian Cultural Heritage
inferred← citeshttps://doi.org/10.1109/jiot.2020.3049024openalex-snapshot
Contact Tracing Incentive for COVID-19 and Other Pandemic Diseases From a Crowdsourcing Perspective
inferred← citeshttps://doi.org/10.1109/nmitcon58196.2023.10276252openalex-snapshot
Optimizing Task Distribution Systems: A Comparative Study of Micro-Task Job Replication, Accuracy, and Budget Constraints
inferred← citeshttps://doi.org/10.1109/tkde.2019.2948168openalex-snapshot
The Impact of Task Abandonment in Crowdsourcing
inferred← citeshttps://doi.org/10.1109/tmc.2021.3052873openalex-snapshot
A Misreport- and Collusion-Proof Crowdsourcing Mechanism Without Quality Verification
inferred← citeshttps://doi.org/10.1145/3209978.3210052openalex-snapshot
On Fine-Grained Relevance Scales
inferred← citeshttps://doi.org/10.1145/3269206.3271779openalex-snapshot
Studying Topical Relevance with Evidence-based Crowdsourcing
inferred← citeshttps://doi.org/10.1145/3309994openalex-snapshot
On Annotation Methodologies for Image Search Evaluation
inferred← citeshttps://doi.org/10.1145/3357384.3357988openalex-snapshot
On Transforming Relevance Scales
inferred← citeshttps://doi.org/10.1145/3391197openalex-snapshot
Serendipity-based Points-of-Interest Navigation
inferred← citeshttps://doi.org/10.1145/3397271.3401112openalex-snapshot
Can The Crowd Identify Misinformation Objectively?
inferred← citeshttps://doi.org/10.1145/3404835.3462916openalex-snapshot
Standing in Your Shoes: External Assessments for Personalized Recommender Systems
inferred← citeshttps://doi.org/10.1145/3404835.3462947openalex-snapshot
Evaluation Measures Based on Preference Graphs
inferred← citeshttps://doi.org/10.1145/3409256.3409816openalex-snapshot
Offline Evaluation without Gain
inferred← citeshttps://doi.org/10.1145/3411755openalex-snapshot
When to Stop Reviewing in Technology-Assisted Reviews
inferred← citeshttps://doi.org/10.1145/3466640openalex-snapshot
Interpretable Aspect-Aware Capsule Network for Peer Review Based Citation Count Prediction
inferred← citeshttps://doi.org/10.1145/3476063openalex-snapshot
Using Worker Avatars to Improve Microtask Crowdsourcing
inferred← citeshttps://doi.org/10.1145/3477495.3531991openalex-snapshot
Human Preferences as Dueling Bandits
inferred← citeshttps://doi.org/10.1145/3480965openalex-snapshot
A Game Theory Approach for Estimating Reliability of Crowdsourced Relevance Assessments
inferred← citeshttps://doi.org/10.1145/3485447.3511960openalex-snapshot
Preferences on a Budget: Prioritizing Document Pairs when Crowdsourcing Relevance Judgments
inferred← citeshttps://doi.org/10.1145/3503516.3503526openalex-snapshot
Crowdsourcing Backstories for Complex Task-Based Search
inferred← citeshttps://doi.org/10.1145/3539618.3591694openalex-snapshot
Hear Me Out: A Study on the Use of the Voice Modality for Crowdsourced Relevance Assessments
inferred← citeshttps://doi.org/10.1145/3539618.3591801openalex-snapshot
A Preference Judgment Tool for Authoritative Assessment
inferred← citeshttps://doi.org/10.1145/3597201openalex-snapshot
How Many Crowd Workers Do I Need? On Statistical Power when Crowdsourcing Relevance Judgments
inferred← citeshttps://doi.org/10.1145/3600211.3604655openalex-snapshot
How do you feel? Measuring User-Perceived Value for Rejecting Machine Decisions in Hate Speech Detection
inferred← citeshttps://doi.org/10.1145/3626772.3657712openalex-snapshot
Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs
inferred← citeshttps://doi.org/10.1145/3626772.3657849openalex-snapshot
Evaluating Generative Ad Hoc Information Retrieval
inferred← citeshttps://doi.org/10.1145/3674127.3674133openalex-snapshot
Query-driven Search Functionality
inferred← citeshttps://doi.org/10.1145/3674127.3674137openalex-snapshot
Temporal Extraction and Retrieval
inferred← citeshttps://doi.org/10.1145/3674127.3674139openalex-snapshot
Privacy in Information Retrieval
inferred← citeshttps://doi.org/10.1145/3674127.3674140openalex-snapshot
eCommerce Search and Discovery
inferred← citeshttps://doi.org/10.1145/3674127.3674142openalex-snapshot
Searching Personal Collections
inferred← citeshttps://doi.org/10.1145/3726302.3730091openalex-snapshot
The Magnitude of Truth: On Using Magnitude Estimation for Truthfulness Assessment
inferred← citeshttps://doi.org/10.1186/s40537-021-00425-xopenalex-snapshot
User profile correlation-based similarity (UPCSim) algorithm in movie recommendation system
inferred← citeshttps://doi.org/10.1371/journal.pone.0201531openalex-snapshot
Framework for fusing traffic information from social and physical transportation data
inferred← citeshttps://doi.org/10.2200/s00904ed1v01y201903icr066openalex-snapshot
The Practice of Crowdsourcing
inferred← citeshttps://doi.org/10.2200/s01063ed1v01y202012aim048openalex-snapshot
Network Embedding: Theories, Methods, and Applications
inferred← citeshttps://doi.org/10.3233/ida-194961openalex-snapshot
Hybrid recommendation model based on deep learning and Stacking integration strategy
inferred← citeshttps://doi.org/10.3233/jifs-189227openalex-snapshot
Student sentiment classification model based on GRU neural network and TF-IDF algorithm
inferred← citeshttps://doi.org/10.3233/jifs-189413openalex-snapshot
Students’ emotional analysis on ideological and political teaching classes based on artificial intelligence and data mining
inferred← citeshttps://doi.org/10.3233/jifs-202092openalex-snapshot
Pairwise deep learning to rank for top-N recommendation
inferred← citeshttps://doi.org/10.3233/jifs-219039openalex-snapshot
Research on the influence of music educational psychology on saxophone players’ mental state and stage performance
inferred← citeshttps://doi.org/10.3233/jifs-219040openalex-snapshot
Music psychology using fuzzy models to promote mental health
inferred← citeshttps://doi.org/10.3233/jifs-219047openalex-snapshot
Sentiment analysis of students in ideological and political teaching based on artificial intelligence and data mining
inferred← citeshttps://doi.org/10.3233/jifs-219048openalex-snapshot
Research on the data analysis system of student stress in English MOOC based on fuzzy C-means algorithm
inferred← citeshttps://doi.org/10.3233/jifs-219050openalex-snapshot
The system for relieving students’ boring emotion in English learning based on fuzzy neural network
inferred← citeshttps://doi.org/10.3233/jifs-219053openalex-snapshot
Evaluation of the effect of music education on improving students’ mental health based on intelligent fuzzy system
inferred← citeshttps://doi.org/10.3233/jifs-219057openalex-snapshot
The music psychology that uses fuzzy models to promote mental health
inferred← citeshttps://doi.org/10.3233/jifs-219161openalex-snapshot
Students’ emotion recognition and psychological stress during the exam
inferred← citeshttps://doi.org/10.3389/frvir.2021.681482openalex-snapshot
Conducting Unsupervised Virtual Reality User Studies Online
inferred← citeshttps://doi.org/10.48550/arxiv.2005.06915openalex-snapshot
Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor's Background
inferred← citeshttps://doi.org/10.48550/arxiv.2007.11682openalex-snapshot
Assessing top-$k$ preferences
inferred← citeshttps://doi.org/10.48550/arxiv.2011.00479openalex-snapshot
Cheap IR Evaluation: Fewer Topics, No Relevance Judgements, and Crowdsourced Assessments
Editorial review for independent work. ICSAC — the Institute for Complexity
Science and Advanced Computing — runs a structured editorial panel for researchers
publishing outside journals, and publishes on CiteStamp rails. Submission is free
and goes through ORCID: icsacinstitute.org/submit.