The Machine Learning Reproducibility Challenge has run as a community effort since 2018, mostly as a satellite workshop. For 2026 it becomes an official track at NeurIPS, with accepted papers presented in person in Sydney, December 6-13, 2026, alongside the Main Track and the Evaluations & Datasets Track.

NeurIPS states the intent directly: to signal that reproducibility has become a scientific question worthy of its own rigorous study. That is a shift in what the field treats as a contribution, and the mechanism chosen to implement it is at least as interesting as the announcement.

The route runs through TMLR

MLRC 2026 does not accept papers directly. A reproducibility paper must first be accepted as is or with minor revisions to TMLR (Transactions on Machine Learning Research), within an eligibility window running from June 20, 2025 through September 30, 2026, 23:59 AOE. Only then does it undergo a light compatibility review by the MLRC committee to confirm suitability for the track.

The call for papers sets out three submission paths — expression of interest before acceptance, self-nomination after TMLR acceptance, or Area Chair nomination — all requiring prior TMLR acceptance. The soft deadline for expressions of interest was June 4, 2026; the hard deadline for having a TMLR acceptance in the system is September 30, 2026, with author notifications October 7, 2026.

Routing through a journal rather than a conference review cycle is a deliberate design decision, and it addresses a real problem with reviewing reproduction work.

Conference review is calibrated for novelty under a fixed deadline. A reproduction study is a poor fit: its value depends on methodological care rather than on a new idea, and the most valuable outcome — a careful failure to reproduce — reads as a negative result to reviewers trained to look for contribution. TMLR’s criteria ask whether the claims are supported and whether the work is of interest to some audience, without a novelty bar and without a batch deadline. That is a substantially better match, and it means a reproduction is evaluated on whether it was done well rather than on whether it surprised anyone.

Failed reproductions count

MLRC explicitly welcomes the full spectrum of outcomes: positive confirmations, partial replications, and failures to reproduce, provided reproducibility is the central focus. Eligible scope includes reproduction studies, generalizability investigations, meta-reproducibility analyses, and methodological contributions to reproducibility research. Source papers may be any published in top ML conferences and journals from 2025 to present.

Admitting negative results is where the scientific value concentrates, and it is the hardest thing for a venue to sustain. A field that publishes only successful reproductions produces a literature in which reproduction appears easy — which is precisely the misleading signal the exercise is meant to correct.

The requirement that reproducibility be the central focus, rather than a section in a paper about something else, is what keeps the track from becoming a place to park incremental extensions.

What actually determines whether work gets reproduced

The submission mechanics say what MLRC will accept. The more useful question for anyone publishing is what makes their own work reproducible, and the recurring obstacles are consistent across years of these challenges.

Undocumented environment. Framework and library versions, CUDA versions, hardware. Results that depend on a specific version combination without saying so are not reproducible even with the code in hand. A pinned environment specification costs nothing at publication and is the single most common gap.

Hyperparameters that differ from the paper. The values in the config in the repository frequently do not match the values in the table in the paper. Neither is annotated as authoritative. This is rarely deception — it is drift between the last experiment run and the last version of the manuscript — and it consumes an enormous share of reproduction effort.

Undocumented preprocessing. Data cleaning, filtering, splitting, and normalization steps performed once in a notebook and never committed. The model code is public; the pipeline that produced its inputs is not.

Seeds and variance. A single reported number with no seed and no variance estimate cannot be checked. A reproduction that lands two points lower has no way to determine whether that is a discrepancy or noise. Reporting mean and spread over several seeds converts an unfalsifiable claim into a checkable one.

Compute that nobody else has. Not solvable by the authors, but honest reporting of total compute lets a would-be reproducer decide whether to attempt the work and lets readers interpret a partial reproduction correctly.

Evaluation code. Metric implementations differ in edge-case handling more often than the metric names suggest. Shipping the evaluation script matters as much as shipping the model.

MLRC’s guidance reflects this: sharing code is strongly recommended, and authors are asked to archive implementations in the Software Heritage Archive with a citable identifier in the paper. Archival rather than a repository link is a small detail with a large effect — a link to a repository that later moves, changes, or disappears documents nothing.

What the track changes

The credential argument is straightforward: a NeurIPS track publication is career-legible in a way a workshop paper is not, which changes the calculus for a graduate student deciding whether to spend three months reproducing someone else’s result. That was the binding constraint on this work getting done, and it has been loosened.

The second-order effect is on incentives for original authors. When reproduction studies are published at a venue people read, with failures included, the cost of publishing work that cannot be reproduced rises. That is a slower mechanism than any artifact-checklist requirement, and probably a more durable one.

It also complements evaluation-integrity work that has drawn attention over the past year — benchmark contamination and what a credible evaluation pipeline requires is the same underlying concern arriving from the measurement side rather than the replication side. Both are symptoms of a field that scaled its publication rate faster than its verification capacity.

Whether the track holds depends on something not yet observable: whether the accepted set includes a meaningful share of failed reproductions, or whether it drifts toward confirmations of results that were never in doubt. The 2026 cohort will be the first evidence either way.