Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on QMSum

TL;DR

QMSum ships without a scorer, so we rescored 15 systems under one implementation. A 406M specialist retrained on the same retrieved transcript spans as our 1.2B system scored 36.33 ROUGE-1 against 35.41, a difference QMSum cannot statistically resolve, while using about a third of the parameters and under half the peak inference memory.

Key findings

Figures and tables

Figure 1. ROUGE-1 for 15 summarization systems on the QMSum test split

Figure 2. ROUGE-1 differences between the promoted 1.2B system and seven others, with 95% intervals

Figure 3. What retrieval and fine-tuning each add within one 1.2B base model

Why it matters

The largest lever this study isolated was training the summarizer on the same retrieved-span inputs it reads at inference, worth 5.29 ROUGE-1 within one base model. A 406M specialist trained that way scored within QMSum's resolution of our 1.2B system while needing under half the peak inference memory.

For summarization run at volume, that footprint sets serving cost, so a smaller specialist is worth measuring before choosing a larger model.

Limitations

All conclusions are limited to QMSum and automatic metrics (ROUGE and BERTScore); no human or factuality evaluation was run. The 406M and 1.2B systems are not separated on any reported metric, and crossing intervals do not establish formal equivalence.

The strongest released specialist's full-input row is scored from its authors' released predictions, our port of that system is not a reproduction, and training cost was measured on one machine at the edge of its memory.

Abstract

QMSum provides no scorer, making query-focused meeting summarization results difficult to compare. We rescore or generate 15 systems under one implementation. Through a common inference port, a released 406M Fusion-in-Decoder specialist loses 6.30 ROUGE-1 when moved from capped long input to 2,000-word retrieved spans. Fine-tuning it on this span regime recovers the loss. On test it scores 36.33 ROUGE-1 versus 35.41 for our 1.2B system; the meeting-cluster 95% interval for the difference is [-0.27, +2.22], so QMSum does not statistically separate them. The smaller system uses about one-third as many total parameters and less than half the peak inference memory. Within the fixed 1.2B base, span-regime fine-tuning adds 5.29 [+4.02, +6.56], while replacing the first 4,500 transcript words with 2,000 retrieved words adds 1.55 on test and 0.29 on validation. Separately, under one concise prompt and reference-overlap scorer, a released 406M specialist exceeds five proprietary hosted models by at least 6.2 ROUGE-1, but output length and absent human or factuality evaluation limit this ordering. Conclusions are limited to QMSum and automatic metrics.

Abstract, arXiv:2609.25028v1

Paper, code and models

Common questions

About the authors

Cite this work

@misc{yang2026retrievedspan,
  title         = {Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on {QMSum}},
  author        = {Yang, Edward Xi},
  year          = {2026},
  eprint        = {2609.25028},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  doi           = {10.48550/arXiv.2609.25028},
  url           = {https://arxiv.org/abs/2609.25028}
}

References

• Zhong et al. (2021), QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization, NAACL
• Vig et al. (2022), Exploring Neural Models for Query-Focused Summarization, Findings of NAACL
• Pagnoni et al. (2023), Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization, ACL
• Sotudeh and Goharian (2024), Learning to Rank Salient Content for Query-focused Summarization, EMNLP

Related reading

• RAG Pipeline Architecture: Indexing vs Retrieval as Separate Concerns
• Glossary: Fine-Tuning
• Glossary: LoRA
• Glossary: Retrieval-Augmented Generation (RAG)