On the public citation graph

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application

https://doi.org/10.1145/3699518
0 signed · 127 inferred · view as graph

Inferred edges come from two machine sources: sweeps of open repository metadata (mostly SSRN and Zenodo), and a bulk OpenAlex citation snapshot. Every row below is labelled with the one it came from. Neither is a complete citation index — absence here is not evidence of absence.

Is this your work? Sign in with ORCID and put your name on the claims that are yours.
Claim with ORCID
Reference health. See whether every reference in this work resolves to a known work and none has been retracted — and get a dated CiteStamped badge you can embed.
See reference health

What this work says

Outgoing claims: the papers this work cites, builds on, or argues with.

inferredcites →https://doi.org/10.1007/s11263-021-01453-zopenalex-snapshot
Knowledge Distillation: A Survey
inferredcites →https://doi.org/10.1016/j.cviu.2023.103720openalex-snapshot
Self-knowledge distillation via dropout
inferredcites →https://doi.org/10.1016/s0898-1221(96)90463-0openalex-snapshot
Advances in neural information processing systems 7
inferredcites →https://doi.org/10.1038/s41586-022-04501-xopenalex-snapshot
FAIR data enabling new horizons for materials research
inferredcites →https://doi.org/10.1109/access.2018.2883637openalex-snapshot
Multi-Scale Attentive Interaction Networks for Chinese Medical Question Answer Selection
inferredcites →https://doi.org/10.1109/jproc.2020.2976475openalex-snapshot
Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
inferredcites →https://doi.org/10.1145/2507157.2507163openalex-snapshot
Hidden factors and hidden topics
inferredcites →https://doi.org/10.1145/3411763openalex-snapshot
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
inferredcites →https://doi.org/10.1145/3616855.3635772openalex-snapshot
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization
inferredcites →https://doi.org/10.1145/3641289openalex-snapshot
A Survey on Evaluation of Large Language Models
inferredcites →https://doi.org/10.1162/tacl_a_00290openalex-snapshot
Neural Network Acceptability Judgments
inferredcites →https://doi.org/10.1162/tacl_a_00305openalex-snapshot
Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension
inferredcites →https://doi.org/10.1162/tacl_a_00317openalex-snapshot
T y D i QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages
inferredcites →https://doi.org/10.1162/tacl_a_00370openalex-snapshot
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
inferredcites →https://doi.org/10.1609/aaai.v33i01.33018076openalex-snapshot
TallyQA: Answering Complex Counting Questions
inferredcites →https://doi.org/10.1609/aaai.v34i05.6319openalex-snapshot
QASC: A Dataset for Question Answering via Sentence Composition
inferredcites →https://doi.org/10.18653/v1/2020.acl-main.195openalex-snapshot
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
inferredcites →https://doi.org/10.18653/v1/2020.coling-mainopenalex-snapshot
Proceedings of the 28th International Conference on Computational Linguistics
inferredcites →https://doi.org/10.18653/v1/2020.findings-emnlp.372openalex-snapshot
TinyBERT: Distilling BERT for Natural Language Understanding
inferredcites →https://doi.org/10.18653/v1/2021.sigdial-1openalex-snapshot
Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue
inferredcites →https://doi.org/10.18653/v1/2022.emnlp-mainopenalex-snapshot
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
inferredcites →https://doi.org/10.18653/v1/2022.emnlp-main.759openalex-snapshot
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
inferredcites →https://doi.org/10.18653/v1/2022.naacl-mainopenalex-snapshot
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
inferredcites →https://doi.org/10.18653/v1/2023.acl-longopenalex-snapshot
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
inferredcites →https://doi.org/10.18653/v1/2023.acl-shortopenalex-snapshot
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
inferredcites →https://doi.org/10.18653/v1/2023.findings-acl.714openalex-snapshot
A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models
inferredcites →https://doi.org/10.18653/v1/2023.findings-emnlp.454openalex-snapshot
MCC-KD: Multi-CoT Consistent Knowledge Distillation
inferredcites →https://doi.org/10.18653/v1/2023.findings-emnlp.725openalex-snapshot
HuatuoGPT, Towards Taming Language Model to Be a Doctor
inferredcites →https://doi.org/10.18653/v1/2026.acl-longopenalex-snapshot
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
inferredcites →https://doi.org/10.18653/v1/d15-1202openalex-snapshot
Solving General Arithmetic Word Problems
inferredcites →https://doi.org/10.18653/v1/d17-1openalex-snapshot
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
inferredcites →https://doi.org/10.18653/v1/d18-1206openalex-snapshot
Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization
inferredcites →https://doi.org/10.18653/v1/d19-1openalex-snapshot
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
inferredcites →https://doi.org/10.18653/v1/d19-1259openalex-snapshot
PubMedQA: A Dataset for Biomedical Research Question Answering
inferredcites →https://doi.org/10.18653/v1/p18-2124openalex-snapshot
Know What You Don’t Know: Unanswerable Questions for SQuAD
inferredcites →https://doi.org/10.18653/v1/p19-1openalex-snapshot
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
inferredcites →https://doi.org/10.18653/v1/s17-2openalex-snapshot
Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)
inferredcites →https://doi.org/10.18653/v1/w17-44openalex-snapshot
Proceedings of the 3rd Workshop on Noisy User-generated Text
inferredcites →https://doi.org/10.18653/v1/w18-54openalex-snapshot
Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
inferredcites →https://doi.org/10.2139/ssrn.4335905openalex-snapshot
ChatGPT Goes to Law School
inferredcites →https://doi.org/10.3390/app11146421openalex-snapshot
What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams
inferredcites →https://doi.org/10.3929/ethz-b-000653476openalex-snapshot
Distilling Reasoning Capabilities into Smaller Language Models
inferredcites →https://doi.org/10.48550/arxiv.1606.00709openalex-snapshot
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
inferredcites →https://doi.org/10.48550/arxiv.1812.01193openalex-snapshot
e-SNLI: Natural Language Inference with Natural Language Explanations
inferredcites →https://doi.org/10.48550/arxiv.2009.13081openalex-snapshot
PRAG: Paninian Retrieval-Augmented Generation for Safety-Critical Medical Question Answering
inferredcites →https://doi.org/10.48550/arxiv.2203.02155openalex-snapshot
Discernment and Social Learning as a Companion Training Layer
inferredcites →https://doi.org/10.48550/arxiv.2307.15190openalex-snapshot
f-Divergence Minimization for Sequence-Level Knowledge Distillation
inferredcites →https://doi.org/10.52202/068431-0182openalex-snapshot
Learn to Explain: Multimodal Reasoning Via Thought Chains for Science Question Answering
inferredcites →https://doi.org/10.7717/peerj-cs.474openalex-snapshot
Knowledge distillation in deep learning and its applications

What points at this work

Incoming claims: where this work appears in other papers’ reference lists and arguments.

inferred← citeshttps://doi.org/10.1007/978-3-031-98414-3_18openalex-snapshot
Training LLM-Based Tutors to Improve Student Learning Outcomes in Dialogues
inferred← citeshttps://doi.org/10.1007/978-981-96-6929-5_35openalex-snapshot
Automatic Annotation of Clinicaltrials.gov Entities Using Large Language Models
inferred← citeshttps://doi.org/10.1007/s00521-025-11162-0openalex-snapshot
Mitigating exposure bias in large language model distillation: an imitation learning approach
inferred← citeshttps://doi.org/10.1007/s00530-026-02245-6openalex-snapshot
Cross-branch knowledge distillation via shallow layer guidance
inferred← citeshttps://doi.org/10.1007/s10822-026-00911-5openalex-snapshot
A survey of transformer and LLM-based architectures, workflows, and systems in drug discovery
inferred← citeshttps://doi.org/10.1007/s10845-026-02897-1openalex-snapshot
EC-Distill-ZeroDiag: a cloud-edge collaborative framework for zero-shot industrial fault diagnosis via large language model distillation
inferred← citeshttps://doi.org/10.1007/s11390-025-5684-5openalex-snapshot
MoFedNet: Semantic Link is All Models Need
inferred← citeshttps://doi.org/10.1007/s11704-025-50814-1openalex-snapshot
A study and formal framework of the composability of LLM compression techniques
inferred← citeshttps://doi.org/10.1007/s12243-025-01134-9openalex-snapshot
Large language models: applications, limitations, challenges, and recommendations in cybersecurity, digital forensics, and ethical hacking
inferred← citeshttps://doi.org/10.1007/s40747-025-02019-zopenalex-snapshot
A review of state-of-the-art techniques for large language model compression
inferred← citeshttps://doi.org/10.1007/s44254-025-00156-9openalex-snapshot
Applications and challenges of large language models in anesthesiology: narrative review and future perspectives
inferred← citeshttps://doi.org/10.1016/j.array.2026.101351openalex-snapshot
A Collaborative Framework for Rapid Model Adaptation and Deployment Based on Meta-Learning
inferred← citeshttps://doi.org/10.1016/j.engappai.2025.110602openalex-snapshot
Leveraging large language models to examine the interaction between investor sentiment and stock performance
inferred← citeshttps://doi.org/10.1016/j.eswa.2026.133034openalex-snapshot
A classification framework of large language model in the post-transformer era
inferred← citeshttps://doi.org/10.1016/j.future.2025.107829openalex-snapshot
The role of Large Language Models in addressing IoT challenges: A systematic literature review
inferred← citeshttps://doi.org/10.1016/j.ipm.2026.104815openalex-snapshot
Graph-driven knowledge stream method for knowledge management in vertical domain large language models: Case study on pit engineering application
inferred← citeshttps://doi.org/10.1016/j.knosys.2026.117186openalex-snapshot
Answer-oriented collaborative distillation for multi-stage evidence-augmented question answering
inferred← citeshttps://doi.org/10.1016/j.neucom.2025.130374openalex-snapshot
MSLoRA: Meta-learned scaling for adaptive fine-tuning of LoRA
inferred← citeshttps://doi.org/10.1016/j.neunet.2025.108061openalex-snapshot
Data-free knowledge distillation via text-noise fusion and dynamic adversarial temperature
inferred← citeshttps://doi.org/10.1016/j.trc.2026.105672openalex-snapshot
A dynamic prompting and scenario generation method for autonomous driving perception via large-model optimization
inferred← citeshttps://doi.org/10.1038/s42256-025-01137-0openalex-snapshot
Densing law of LLMs
inferred← citeshttps://doi.org/10.1039/d5dd00429bopenalex-snapshot
MOFReasoner: think like a scientist—a reasoning large language model via knowledge distillation
inferred← citeshttps://doi.org/10.1063/5.0337453openalex-snapshot
Amplified topic coupling and polarization by higher-order interactions in multidimensional opinion dynamics
inferred← citeshttps://doi.org/10.1080/20964471.2025.2603810openalex-snapshot
A dual-track machine learning framework for andesite tectonic environment identification: ensemble learning and few-shot learning
inferred← citeshttps://doi.org/10.1080/2833373x.2025.2546791openalex-snapshot
Large language models for data extraction in toxicology: implications and lessons learned from the clinical evidence domain
inferred← citeshttps://doi.org/10.1088/1361-6501/ae0e88openalex-snapshot
Learning while teaching for autonomous driving
inferred← citeshttps://doi.org/10.1088/2631-8695/ae7898openalex-snapshot
Robust and efficient bearing fault diagnosis: a parallel Swin-MSCA framework with knowledge distillation
inferred← citeshttps://doi.org/10.1108/er-12-2024-0786openalex-snapshot
Sustainable HRM in the age of analytics: capabilities, risks and a path forward
inferred← citeshttps://doi.org/10.1109/access.2025.3605110openalex-snapshot
Make Large Language Models Efficient: A Review
inferred← citeshttps://doi.org/10.1109/access.2026.3724635openalex-snapshot
DABO: Difficulty-Aware Binary Offloading for Collaborative Large-Small Model Inference
inferred← citeshttps://doi.org/10.1109/access.2026.3737802openalex-snapshot
SLIM-KT: Transferring Large Language Model Item Semantics into a Lightweight, Cold-Start-Robust Knowledge Tracing Model
inferred← citeshttps://doi.org/10.1109/caos69069.2026.11666192openalex-snapshot
Cross-modal Depth-privileged Knowledge Distillation for Underwater Salient Object Detection with Only RGB Images
inferred← citeshttps://doi.org/10.1109/cisat66811.2025.11181792openalex-snapshot
Collaborative Distillation Strategies for Parameter-Efficient Language Model Deployment
inferred← citeshttps://doi.org/10.1109/comst.2026.3682638openalex-snapshot
Model Compression for Sustainable AI in xG Wireless Networks: Recent Advances, Challenges, and Future Directions
inferred← citeshttps://doi.org/10.1109/dcoss-iot69657.2026.00094openalex-snapshot
Crop and Weed Segmentation by an Ensemble of Distilled Models
inferred← citeshttps://doi.org/10.1109/dlcv69906.2026.11635711openalex-snapshot
CSSD3D: Cross-Stage Self-Distillation for Accurate and Efficient LiDAR-based 3D Object Detection
inferred← citeshttps://doi.org/10.1109/icassp49660.2025.10888451openalex-snapshot
Enhancing Small Model Performance in Educational Classification Tasks through Knowledge Distillation
inferred← citeshttps://doi.org/10.1109/icassp55912.2026.11462247openalex-snapshot
What is the risk? Evaluating the impact of knowledge distillation on LLM vulnerabilities
inferred← citeshttps://doi.org/10.1109/icassp55912.2026.11464298openalex-snapshot
Dual-Space Knowledge Distillation with Key-Query Matching for Large Language Models with Vocabulary Mismatch
inferred← citeshttps://doi.org/10.1109/iccces62661.2026.11436497openalex-snapshot
Model Compression and Knowledge Distillation Techniques for Accelerating Inference in Large Generative AI Models
inferred← citeshttps://doi.org/10.1109/iccke65377.2024.10874673openalex-snapshot
Distilled BERT Model in Natural Language Processing
inferred← citeshttps://doi.org/10.1109/icicse66971.2025.11429888openalex-snapshot
Edge Intelligence in Civil Aviation: Paradigms, Techniques, and Applications
inferred← citeshttps://doi.org/10.1109/icinvents64613.2025.11402162openalex-snapshot
Interpretable Knowledge Distillation Framework for Detection of Ewing Sarcoma in Young Adults
inferred← citeshttps://doi.org/10.1109/icnp65844.2025.11192368openalex-snapshot
Mamba4Net: Distilled Hybrid Mamba Large Language Models For Networking
inferred← citeshttps://doi.org/10.1109/ictbai68361.2025.00011openalex-snapshot
A Lightweight Optimization of A Vehicle Networking Task Offloading Scheme Based on Knowledge Distillation
inferred← citeshttps://doi.org/10.1109/ies67184.2025.11161662openalex-snapshot
The Survey of Model Scale Impact on Knowledge Distillation Efficiency in LLM
inferred← citeshttps://doi.org/10.1109/ijcnn64981.2025.11227819openalex-snapshot
SAM-RAG: An Self-adaptive Framework for Multimodal Retrieval-Augmented Generation
inferred← citeshttps://doi.org/10.1109/ijcnn64981.2025.11229360openalex-snapshot
A White-Box Knowledge Distillation Approach for Task-Optimized Large Language Model Compression
inferred← citeshttps://doi.org/10.1109/jiot.2026.3685248openalex-snapshot
TinyML for Eddy Current Testing: A Review of Advances, Challenges, and Applications
inferred← citeshttps://doi.org/10.1109/mc.2026.3667296openalex-snapshot
Converging AI: Merging Data-Centric and Model-Centric AI Systems
inferred← citeshttps://doi.org/10.1109/ojcoms.2026.3680064openalex-snapshot
LLMs and IoT: A Comprehensive Survey on Large Language Models and the Internet of Things
inferred← citeshttps://doi.org/10.1109/secon68281.2026.11579156openalex-snapshot
Lightweight Dynamic Global Filter Network for Efficient Image Analysis on Edge Devices
inferred← citeshttps://doi.org/10.1109/taffc.2026.3684445openalex-snapshot
Bootstrap Wayfinding Questions to Elicit Emotion Shift Reasoning With Large Language Models
inferred← citeshttps://doi.org/10.1109/tai.2025.3562162openalex-snapshot
ARDIAL-BERT: Advancing Multidialectal Arabic Named Entity Recognition Through Continual Pretraining
inferred← citeshttps://doi.org/10.1145/3742891openalex-snapshot
Towards Message Brokers for Generative AI: Survey, Challenges, and Opportunities
inferred← citeshttps://doi.org/10.1145/3770762.3772560openalex-snapshot
You Don't Need a Data Center to Explain in Plain English! Comparing Open-Source and Proprietary LLMs for EiPE Grading
inferred← citeshttps://doi.org/10.1145/3785353.3815086openalex-snapshot
Robust Black-Box Fingerprinting for Large Language Models Against Composite Attacks via LLM Self-Evaluation and ArcFace
inferred← citeshttps://doi.org/10.1145/3806980.3807011openalex-snapshot
Seek-R1: A Lightweight Training Framework for Long-Chain Reasoning Models Based on Curriculum Learning
inferred← citeshttps://doi.org/10.1145/3837125openalex-snapshot
StaleFlow: Staleness-Aware Data Management for Mitigating Data Skewness in Fully Disaggregated RL Post-Training
inferred← citeshttps://doi.org/10.1631/fitee.2500282openalex-snapshot
Knowledge distillation for financial large language models: a systematic review of strategies, applications, and evaluation
inferred← citeshttps://doi.org/10.21275/sr25331222304openalex-snapshot
Vision LLMs: Bridging Language and Visual Understanding - Case study of IndoAI
inferred← citeshttps://doi.org/10.2139/ssrn.5222342openalex-snapshot
Handling Causal Tasks by Large Language Models: From Discovery to Reasoning
inferred← citeshttps://doi.org/10.2196/66905openalex-snapshot
Using Large Language Models for Chronic Disease Management Tasks: Scoping Review
inferred← citeshttps://doi.org/10.2196/94764openalex-snapshot
Local Deployment of Open-Weight Language Models in Dermatology: Viewpoint on Privacy, Equity, and Practical Implementation
inferred← citeshttps://doi.org/10.26599/tst.2025.9010130openalex-snapshot
KDLLM: Copyright-Preserving LLM Based on Knowledge Distillation
inferred← citeshttps://doi.org/10.2738/engtm.2026.0007openalex-snapshot
The Future of Mechanical Metastructures: There is Plenty of Room at the Top
inferred← citeshttps://doi.org/10.3389/fendo.2026.1747468openalex-snapshot
Real-world performance of open-source large language models in diabetes diagnosis
inferred← citeshttps://doi.org/10.3390/app15179605openalex-snapshot
Interpretable Knowledge Tracing via Transformer-Bayesian Hybrid Networks: Learning Temporal Dependencies and Causal Structures in Educational Data
inferred← citeshttps://doi.org/10.3390/computers14110490openalex-snapshot
Structured Prompting and Collaborative Multi-Agent Knowledge Distillation for Traffic Video Interpretation and Risk Inference
inferred← citeshttps://doi.org/10.3390/computers15070415openalex-snapshot
MTKD-RL: Multi-Teacher Knowledge Distillation Method for Reinforcement Learning Based on Few-Shot Node Classification
inferred← citeshttps://doi.org/10.3390/data11060126openalex-snapshot
Temporally Informed Distillation of Embedding Semantics: Beyond Continued Pretraining for Modeling Gender Ideology in Dated Texts
inferred← citeshttps://doi.org/10.3390/electronics14081538openalex-snapshot
Knowledge Distillation Based Recommendation Systems: A Comprehensive Survey
inferred← citeshttps://doi.org/10.3390/electronics14193860openalex-snapshot
Design and Evaluation of Knowledge-Distilled LLM for Improving the Efficiency of School Administrative Document Processing
inferred← citeshttps://doi.org/10.3390/info16080688openalex-snapshot
Ensemble Large Language Models: A Survey
inferred← citeshttps://doi.org/10.3390/info17090824openalex-snapshot
A Human Factors Framework for Operational Risk Management in Banking Using Deep Learning and Large Language Models
inferred← citeshttps://doi.org/10.3390/rs17121998openalex-snapshot
A Dual-Branch Network for Intra-Class Diversity Extraction in Panchromatic and Multispectral Classification
inferred← citeshttps://doi.org/10.51387/25-nejsds84openalex-snapshot
Knowledge Distillation Decision Tree for Unravelling Black-Box Machine Learning Models
inferred← citeshttps://doi.org/10.7717/peerj-cs.4000openalex-snapshot
Generative small language models in clinical NLP: applications, adaptation, and evaluation
Editorial review for independent work. ICSAC — the Institute for Complexity Science and Advanced Computing — runs a structured editorial panel for researchers publishing outside journals, and publishes on CiteStamp rails. Submission is free and goes through ORCID: icsacinstitute.org/submit.