ML Docking Rescoring: What Works and How to Trust It

ML docking rescoring means taking the poses a conventional docking engine has already generated and re-ranking them with a machine-learned scoring model, so that true binders rise above decoys. It can improve a virtual screen, but only when the poses are good, the model has seen chemistry like your target, and you measure the result prospectively rather than trusting a retrospective benchmark. This guide covers how to set it up, how to test it, and how to attach an honest confidence estimate to the scores.

Innovabiotech

Make Virtual Screening More Project-Specific

Innovabiotech provides tailored bioinformatics and computational biology solutions for virtual screening, designed around each project's needs.

Explore Innovabiotech

TL;DR

Rescoring cannot fix bad sampling. If the docking step never places the ligand near its native binding mode, no ML model recovers it.
Retrospective gains are often inflated. Enrichment improvements above 100% on benchmarks sit next to prospective hit rates near 2% in larger studies.
ML scorers generalize unevenly. Use ensembles or consensus, and compare against simple baselines such as interaction fingerprints.
Treat uncertainty as a deliverable. Conformal prediction and ensemble variance tell you when a score or pose deserves trust.
Run rescoring as one tier in a funnel: cheap filters first, expensive models on a short list.

What rescoring does, and what it cannot do
The retrospective trap
Classical docking plus an ML rescorer, or end-to-end AI?
Build the rescoring step as one tier in a funnel
Estimating uncertainty in docking and rescoring
Our perspective
A short checklist before you trust a rescored list
Where our team fits
FAQ
Sources

What rescoring does, and what it cannot do

A docking program does two jobs at once. It samples poses, and it scores them with a fast function built for speed. Rescoring separates these jobs. You keep the sampling, then pass the saved poses through a second, usually slower and more specialized, scoring model.

The second model can be physics-based (for example MM/GBSA), a classical machine-learning model such as a random forest on interaction features, or a deep network such as a 3D convolutional or graph neural network. A review of scoring functions from machine learning to deep learning traces how the field moved between these families. A 2024 overview of modern ML for binding affinity estimation covers the progress and the open challenges.

The limit is built in. In a critical benchmark on the LIT-PCBA library, the quality of the input poses was a major bottleneck, and AI rescorers were sensitive to how close those poses were to the native mode. A rescorer re-ranks what it is given. It does not invent a pose that was never sampled.

That has a practical consequence for how you spend your effort. Before you tune a rescoring model, check that your docking step reproduces known ligand poses for your target, ideally across several receptor conformations. Our note on pocket-focused ensemble docking explains how a small set of conformers improves sampling without a large compute bill.

What rescoring does, and what it cannot do — overview diagram

The retrospective trap

The most common mistake with ML rescoring is believing the benchmark.

Benchmarks such as LIT-PCBA are built from experimental screening data, which makes them harder than older decoy-based sets, but they are still retrospective. A Journal of Cheminformatics study on IRAK1 discusses the gap: complex models such as wide neural networks improved EF1% by more than 109% in a retrospective setting, while prospective studies testing at least 470 compounds report a median hit rate of about 2.16%.

The lesson is not that ML rescoring is useless. It is that a number from a benchmark is a ceiling on your expectations, not a forecast. A 2021 study titled "The Simpler the Better" examined fast scoring functions against high-throughput screening data from docking poses and found that simple methods, interaction fingerprints among them, often prove more robust than complex deep learning models. It also noted that many deep models predict affinities in a narrow band around the training-set mean, which blunts their ability to discriminate between compounds on a new target.

If you are evaluating a vendor, a tool or your own pipeline, ask for three things:

Performance on targets that share no close homologs with the training data.
A comparison with a simple baseline, such as the native docking score plus an interaction fingerprint model.
Prospective results, even a small experimental batch, not only retrospective enrichment.

Our guide to validating virtual screening results walks through this kind of check in more detail.

Classical docking plus an ML rescorer, or end-to-end AI?

End-to-end deep learning docking tools are fast and sometimes impressive, but physical realism has been a recurring concern. A Nature Machine Intelligence benchmark of AI-powered docking from the virtual screening perspective points out that many of these methods have neglected the physical plausibility of their predicted poses.

In the LIT-PCBA benchmarking study cited above, AutoDock-GNINA, a traditional engine paired with a CNN-based rescorer, reached the highest median EF1% at 2.14 and outperformed several DiffDock-based methods. Note the modest absolute figure. Even the best performer in that comparison delivered enrichment far from the dramatic numbers retrospective papers can show.

A fair summary of the evidence for a new target:

Approach	Strength	Main risk
Classical docking, native scoring only	Transparent, fast, physically constrained poses	Scoring noise, weak ranking of close analogs
Classical docking plus ML rescoring	Keeps physical poses, improves ranking where the model has seen similar chemistry	Poor generalization to dissimilar targets, dependence on pose quality
End-to-end AI docking	Very fast, can sample unusual poses	Physical plausibility, opaque failure modes
Physics-based rescoring (MM/GBSA)	Explicit energetics, interpretable terms	Cost, sensitivity to setup and protonation states

These are tendencies, not laws, and they shift as models improve. Treat the model's shelf life as short and re-benchmark on your own target when you change versions.

Build the rescoring step as one tier in a funnel

The teams that get value from ML rescoring rarely run it on everything. A WIREs review and practitioner experience point the same way: start with rapid, inexpensive filtering, then apply the heavier, more accurate models to a reduced, prioritized subset.

A workable layout:

Prepare the target and library. Curate protonation and tautomer states, and choose one or several receptor conformations.
Dock with a fast classical engine. Keep several poses per ligand, not only the top one, so the rescorer has something to choose between.
Filter cheaply. Remove compounds with clear strain, steric clashes or missing key interactions.
Rescore the survivors with an ML ensemble. Use at least two models built on different representations.
Apply physics-based refinement to the short list. This is where MM/GBSA rescoring done ensemble-first fits.
Check chemistry and diversity. Cluster, inspect binding modes by eye, and select a balanced set for purchase or synthesis.

Keeping multiple poses per ligand matters more than it appears. A model that scores only the top-ranked pose inherits every sampling error. Pose selection with structure-based deep learning is a topic of its own, and a review of docking pose selection describes the persistent difficulty of identifying the near-native pose.

For the wider workflow around these steps, see our virtual screening workflow guide.

Estimating uncertainty in docking and rescoring

A score is a point estimate. A decision needs to know how much to trust it. There are three practical ways to get there.

Ensemble disagreement

Train or run several models and look at the spread of their predictions. A 2024 paper on an ensemble-based approach to estimate the confidence of predicted binding affinities starts from a basic limit: a scoring function is trained on a small fraction of all possible protein-ligand complexes, so predictions for unfamiliar complexes deserve less trust. Large disagreement among ensemble members flags exactly those cases.

Conformal prediction

Conformal methods attach an interval to a prediction with a coverage guarantee that does not depend on assuming a particular data distribution. ConfDock, a preprint, applies this to atom-level pose uncertainty. It reports an average 57.2% reduction in mean interval width compared with standard split conformal prediction, while keeping the target coverage. The work is a preprint and has not been through peer review, so treat it as a promising direction rather than a settled standard.

Applicability domain checks

The simplest check is also useful: how similar is this ligand, and this pocket, to the training data? If the answer is "not very", discount the score regardless of how confident the model sounds. This addresses the generalization problem noted in the "Simpler the Better" study directly.

Whichever you use, report uncertainty next to the score in your hit list. A compound ranked 40th with a tight interval can be a better bet than one ranked 10th with a wide one.

Our perspective

We see the same pattern on client projects. Teams arrive asking which rescoring model is best, and the more useful question is whether their pose set is any good. When sampling is sound, a small ensemble of diverse scorers, checked against a simple baseline, usually beats a single sophisticated network, and it is much easier to explain to a medicinal chemist who has to decide what to order.

We also push for a short prospective test early. A few dozen compounds tested against the target tell you more about your rescoring protocol than another round of retrospective tuning. And we ask for uncertainty on every list we deliver, because a ranked list without confidence invites false precision.

Finally, we keep the protocol documented and the data confidential. Models age quickly in this field, so a clear record of what was run, with which version and which settings, is what lets a team repeat or revise the work six months later.

— Hooman

A short checklist before you trust a rescored list

Does the docking step recover the poses of known ligands for this target, within a reasonable RMSD?
Are several poses per ligand passed to the rescorer?
Is the ML model compared with a simple baseline on your own data?
Is there at least one model from a different family in the ensemble?
Have you looked at decoys and property bias, so the model is not just learning molecular weight?
Is each score accompanied by a confidence measure or applicability check?
Is a prospective experimental batch planned, and is its success criterion set in advance?

If several answers are "no", the rescoring step is not yet adding reliable signal, and the effort is better spent on sampling or on reducing false positives further upstream.

Where our team fits

If you are building or auditing a docking and rescoring pipeline and would rather have a specialist team run it, we offer custom virtual screening and hit-to-lead services for biotech and pharma R&D groups. That includes structure-based and ligand-based screening, project-specific protocol design and regular technical updates through delivery. You can also start from our main site to describe your target and goals.

FAQ

Is ML rescoring better than the native docking score?

Not automatically. It can improve ranking when the model has seen chemistry similar to your target, but ML scorers often generalize poorly to dissimilar targets. Compare it with the native score and a simple baseline on your own data before adopting it.

How many poses should I rescore per ligand?

More than one. Rescoring only the top pose carries forward any sampling error from the docking step. Passing a handful of diverse poses lets the rescorer, or a consensus of rescorers, choose between them.

How do I know if a rescored pose or score is reliable?

Use several signals together: disagreement across an ensemble of models, an applicability domain check against training data, and where available a conformal prediction interval. Methods such as ConfDock are still at the preprint stage, so confirm important picks experimentally.

Should I replace classical docking with an end-to-end AI docking tool?

For most new targets, pairing a well-understood docking engine with targeted ML re-ranking is the safer starting point. Benchmarks have flagged physical plausibility concerns with end-to-end tools, so test them side by side on your target before switching.