[JCD26x2] KPViz: A Framework for Keyphrase Prediction Experiments & A Few Good Instances: How Much Data Does Frugal Keyphrase Generation Actually Need?
My PhD student Saber Zahhar has published 2 papers at JCDL 2026 (Proceedings of the 26th ACM/IEEE Joint Conference on Digital Library), both on the same topic on Keyphrases recommendation and benchmarking.
This work has been developped jointly with Devoteam (many thanks to Benoit Borenstein & Mostafa Maazi), Nedra Mellouli and Christophe Rodrigues (at DVRC/ESILV).
A Few Good Instances: How Much Data Does Frugal Keyphrase Generation Actually Need? [JCDL26]
Fine-tuning sequence-to-sequence models such as BART is the state of the art for keyphrase generation, at tens to hundreds of GPU hours per corpus. We ask whether data selection makes it frugal: does choosing which documents to train on buy anything over simply taking fewer of them at random? We fine-tune BART-base under budgets of 1–70% of the training pool on three corpora from distinct domains (KP20k, KPBiomed, KPTimes), comparing random sampling against frequency-weighted keyphrase coverage, gradient diversity (BADGE) and influence-based selection (LESS): 75 runs, 108 paired comparisons. The exchangeable unit is the (dataset, budget) block: within one block every difference is measured against the same RANDOM run. Over the 18 blocks, an exact sign-flip permutation test puts the two off-the-shelf strategies 0.83 points behind random sampling (95% CI [ − 1.55, −0.11], p = 0.026); LESS loses by 1.07 despite its privileged access to the validation set. The claim that survives any multiplicity correction is the interval: the best strategy’s 95% CI reaches only + 0.15 points, so informed selection cannot repay its own overhead. What pays is the budget. At matched exposure, 70% of the data reproduces full-data quality within 0.4 points on both metrics and all three corpora at 64–77% of the cost; on a redundant editorial corpus, 30% beats full-data present F1@5 at 35% of the cost and under half the exposure, while small budgets disproportionately damage absent-keyphrase generation. We distil this into a training-free triage rule and release code, checkpoints and inferences.
And the second one is the corresponding demonstration:
KPViz: A Framework for Keyphrase Prediction Experiments [JCDL26]
Keyphrases are core metadata for digital libraries, yet progress in keyphrase extraction and generation is hard to judge from the literature: each project rebuilds its own evaluation (dataset readers, matching rules, cost bookkeeping, significance tests), so published numbers are hard to reproduce and to compare. We demonstrate KPViz, an open-source tool that turns a folder of datasets, model outputs and cost descriptions into one reproducible evaluation and an interactive dashboard. Users describe each dataset, model and machine once in a short JSON card; KPViz then aligns every prediction with its gold keyphrases and its cost through these cards, so adding a dataset, a model or a run requires no code. It computes corpus statistics, present/absent keyphrase classes, exact-match scores and significance tests, and recomputes only what changed after an edit. Every table and figure exports to camera-ready LaTeX with an editable caption, so the figure a researcher sees on screen is the one that lands in the paper. The demonstration follows one question end to end: does a model’s context window limit what it can extract? Demo data and a demo video are public, and the code is open source at https://github.com/saberzahhar/kpviz.
The corresponding video: