How we predict what a public tender will cost, and who will bid

Quebec publishes every public contract it puts out to tender: who is buying, what they want, who bid, who won, and for how much. It is a remarkably complete public record, it is open data, and almost nobody can use it.

The reason is not access. It is that the record is written for accountability, not for decisions. It answers what happened beautifully and what is about to happen not at all. A contractor looking at a notice that closes in eighteen days gets a title, a category code, a buyer name and a PDF. Everything that would actually inform the decision — what this kind of work goes for with this kind of buyer, how many firms will show up, whether the notice is a real competition or a formality — is present in the historical record and unavailable at the moment it matters.

This is what we built, and — more importantly — how we know it works.

Four models, and one thing that is deliberately not one

Given a tender on the day it is published, before a single envelope is opened, four separate models run.

What will it cost. Quantile regressors produce a low-to-high range on the eventual award value, which is then calibrated so that the stated interval means what it says.

How much competition. An expected number of bidders, plus a separately trained probability that the tender draws exactly one.

Who will bid. A two-stage system: a retrieval stage narrows tens of thousands of companies with a public bidding record down to a few hundred plausible candidates for this specific tender, and a ranking stage scores them. The output is a calibrated probability per company, not a leaderboard — more on why that distinction matters below.

Who will win. A calibrated win probability for each company in the race.

Then there is the fifth output, the go/no-go recommendation, and no model produces it. It is an explicit weighted rubric over five factors — sector fit, expected competition, your historical price position, your relationship with this buyer, and contract size relative to what you normally deliver — each one traceable to the section of the report that demonstrates it.

That is a deliberate choice, and the reasoning is worth stating plainly: *nobody publishes whether a company should have bid. There is no label. Any "should you bid" model would be fitting a proxy — usually "did they win" — and quietly answering a different question than the one asked. So the four things that have ground truth are learned, the one that does not is an arithmetic grid the reader can dispute factor by factor, and the report never blurs the line between them. It also has a fourth state, insufficient signal*, which is a refusal rather than a middle score. A weak recommendation and no recommendation are different products.

Why most prediction claims in this space are worthless

Before any results: the reason to distrust them by default.

The standard way to report a model is to hold out a random slice of history and score against it. On procurement data this is close to meaningless. A random split puts 2019 tenders in the test set and 2024 tenders in training, so the model gets to know how the market turned out before predicting a past it has already seen. Worse, the features leak: any statistic about a company's track record, computed over the whole dataset, encodes wins that had not yet been announced when the tender was published. The resulting numbers are excellent and do not survive contact with a live notice.

Three things prevent that here.

The split runs forward. Models learn from tenders published through the end of 2024, are tuned on the first half of 2025, and are measured — once — on everything published after. Nothing in this article is a score on data a model was fitted to.

Every historical fact is reconstructed as of the publication date. Not "as of today, filtered". Reconstructed. A company's win rate on the day a tender was published counts only the awards that had actually been made public by that date — participations dated at bidder-list release, wins at award release, because those are two different dates and using the earlier one for both leaks the outcome of a race into the features that predict it. Same-day facts are excluded outright: dates in this record are day-granular, so a fact published on the same day as the tender may well postdate it within that day.

A structural allowlist forbids outcome-time inputs. Award value, number of tenderers, winner count, award release date and their relatives are named in code as ineligible, asserted at every build. The rule extends past the obvious cases to anything describing how a company's story ended rather than where it stood on the day. A field that quietly encodes which firms were about to stop bidding is a leak, even when it looks like a static attribute — and we measured that this is not hypothetical before writing the rule.

Ranges, not point estimates — and the range is measured, not declared

A single predicted number is nearly useless for a bid decision. What a bidder needs is the shape of the distribution: how wide is the plausible range, and where inside it does my cost structure sit.

Quantile regression gives the shape. It does not, on its own, give an honest shape — a model trained to emit a 10th and a 90th percentile will not naturally produce an interval that contains the truth 80% of the time. Ours, raw, contains it 71% of the time.

So the interval is calibrated after the fact against held-out data, with the correction fitted separately for each band of predicted value — small contracts and large ones are wrong in different ways and by different amounts, and a single global correction over-widens one to fix the other. After calibration the stated 80% interval contains the real award price 79% of the time on tenders the model never saw, and holds between 74% and 82% across the ten value bands — with one band sitting below the 75% floor we set ourselves, which the accuracy page marks as failing rather than footnotes.

That property — the number on the page means the thing it says — is the one we would defend first if only one claim survived.

What it measures out at

All figures below: test split, open procurement only, published on or after 2025-07-01, models trained through 2024-12-31. Every figure is stated against the naive method a competitor would reach for first — usually "what does this category of contract normally go for", which is a genuinely reasonable baseline and the one most people are implicitly using today.

Price

Across 12,076 unseen tenders, model against category benchmark:

• Median error on the likely price: 34% for the model, 79% for the benchmark.
• The stated 80% range actually holds: 79% of the time, against 72%.
• Range width, low to high: about 4 times, against about 30 times.

The width is the row that matters commercially. Both methods produce an interval that is right about 80% of the time — that is the easy part, and a wide enough range is always right. Ours is roughly seven times narrower to get there, and wrong half as often. An interval you cannot act on is not a prediction.

Public contract values are extraordinarily dispersed — the same category routinely spans several orders of magnitude — so a 34% median error is a strong result rather than a weak one. It is also not uniform, and we publish where it is worst. Grouped by what contracts actually turned out to be worth rather than by what was predicted, the bottom decile reads 58%. That gap is largely a selection effect that any predictor, even a perfect one, would show — conditioning on the outcome picks out exactly the contracts that came in cheaper than anyone expected — and the accuracy page says so rather than hiding behind it.

Competition

Across 8,540 tenders:

• Average error on the number of bidders: about 1.9, against 2.6 for the benchmark.
• A separately trained probability that the tender draws exactly one bidder, reported as a probability rather than a yes-or-no label.

The single-bidder case is the one that changes a bidder's behaviour most. About one open tender in six draws exactly one bid, and knowing in advance which ones is worth more than any amount of extra precision on the price.

Who bids

Across 17,197 tenders:

• Share of actual bidders caught in a 10-name shortlist: 43%.
• Same, excluding companies with no public bidding history: 44%.
• Precision at ten names: 14%.

That last figure comes with a framing caveat we enforce in the product. Most names on any top-ten list are wrong by construction and cannot be otherwise. Read as "these are the bidders", such a list is false. Read as "recognise these names, and here is a calibrated probability for each", it is useful and honest. So the report ships probabilities and an expected bidder count, with the shortlist as a surface rather than the claim.

Who wins

Across 50,902 company-tender pairs, the calibration error on the per-company win probability is 0.006 — the number that decides whether a probability can be read as a probability at all.

Note what is not headlined. The overall ROC-AUC is 0.72, and we do not lead with it, because a diagnostic that predicts 1 / number of bidders and knows nothing else lands close behind on the same data. Most of that headline is bidder-count structure, not insight about firms. The honest read is calibration, and the contested strata discussed further down.

The controls

Reporting the score is the easy half. These are the checks that make it mean something.

Scramble the dates and everything should collapse. We rebuild the entire dataset with publication dates randomly permuted before the point-in-time joins, keeping labels and splits honest, and re-run. If hindsight were leaking through the history features, performance would survive. It does not: the candidate-retrieval ceiling collapses, and the share of bidders the system considers unreachable rises sharply. The collapse is the evidence.

And the honest limit of that control. It only bites on features built from history. A feature welded to the tender's own row — text from the notice, the buyer's own published estimate — stays attached to that row under any permutation of dates, so the shuffle cannot say anything about it. We say so, and use different checks for those: a direct scan for outcome values appearing in the input text, and a slice of the corpus where the feature has exactly 0% coverage, which must therefore show exactly zero effect. It does.

A second, independent training window. Adopted improvements are re-measured on an entirely different origin — trained through 2023 instead of 2024 — and must reproduce. Several did, at the same magnitude. Several candidate features looked promising on the first origin and were dropped when they did not.

A noise floor, stated before the run. Repeated seeds give a measured variance for each metric, and each experiment declares a minimum detectable effect before it is executed. Anything under the noise band is not an improvement, however pleasant the direction.

The failure nobody looks for: the data changed shape mid-backtest

This is the finding we would most want another team in this space to steal.

The source data's payload changed shape around June 2024 — new fields, renamed values, different spellings for the same thing. That date lands almost exactly on our train/test boundary: training straddles it, evaluation is entirely on the far side.

None of the standard guards can see this. Date-shuffling permutes dates, which leaves a train-versus-serve vocabulary mismatch completely intact. The forbidden-input allowlist targets a different failure mode entirely. The split-integrity check asserts that train and test do not overlap in time — which is exactly the condition that makes this hazard possible, not the one that detects it.

So we built a dedicated check that compares, feature by feature, what the model saw in training against what it will actually meet at serving time: does every category value still exist, does coverage hold, has the missingness pattern changed, has the distribution shifted, is the column degenerate.

It found, among other things:

• The single most important feature in the price model — carrying, at the time, over 40% of its signal — had a straight-apostrophe spelling of several category values that simply stops at the changeover. One value appears 3,639 times in 2023, four times in 2024 and never again, while its curly-quote twin continues. About 46,000 training rows sat under labels the model would never see again.
• A column that was 26% populated in training and 100% empty in both evaluation splits. Nobody found that by hand; the check found it.
• A field that was 44% populated in training and 0% at serve time.
• Two features that were not really features at all but proxies for "this record predates a classification change".

Seven failures on the live feature list at the start. Zero today.

Calibrating the check was itself instructive. Some features grow by construction — a running count of a company's history should be larger in 2026 than in 2021 — and a check that fires on correct-by-construction behaviour gets ignored, which is worse than no check at all. Those are exempted explicitly, with the reason recorded next to the exemption. Two later revisions were needed because the check's own arithmetic was wrong in ways that would have hidden real problems: once because it compared averages of quantities that legitimately average to zero, and once because it compared coverage rates rather than gap rates, so a feature going from 6.9% missing to 0.07% missing — a 94-fold change in how often the model meets a hole — read as a pass.

What we measured and threw away

Around three dozen experiments are on record, each with a pre-registered hypothesis, a threshold and a ruling. Roughly a third are documented rejections, with the numbers that killed them:

• Macro-economic indicators — consumer and industrial price indices, regional job-vacancy rates, construction cost indices, correctly lagged for publication delay. Flat, and flat on a second origin too. Publication year and month already absorb inflation drift.
• Dense multilingual text embeddings of tender descriptions. About 2% on price and nothing elsewhere. A lexical representation of the same text, tried much later, worked — French procurement writing is driven by exact tokens like déneigement and street addresses, and a sentence encoder smears precisely those. The lesson is that "text didn't help" was the wrong conclusion drawn from the right measurement.
• A language model's own labels for the price features. The gain came in under the noise floor, the median error moved the wrong way, and one of the two features contributed exactly zero. On tabular data, a tuned gradient-boosted model with proper calibration is simply the stronger instrument.
• A listwise ranking objective for the bidder shortlist. Lost outright to plain binary classification, by half a point of recall.
• Bidding-rhythm features — is this firm overdue relative to its own historical cadence. Dead flat. Recency and intensity counts already encode it.
• Competitor-context features for win probability. Zero measured contribution across all five.
• Harder negative sampling for the ranker: six points worse, and structurally so — selecting negatives by rank teaches the model that top-ranked implies non-bidder.

Two of the most useful records are not about features at all.

The first: a cheap pre-build screen, designed to decide whether an expensive feature was worth constructing, was built, run, and then discarded before it was allowed to decide anything. On inspection it was confounded, it was biased low, and above all it was underpowered — with roughly a hundred quasi-independent readings at the measured noise level, the expected maximum of pure noise was larger than the threshold the screen was going to use to greenlight the build. It would have said "build" on nothing. The generalised rule we now follow: kill an idea before building it only when the diagnostic answers a reach question with a wide margin; once the artifact has to exist anyway, screen nothing and measure.

The second: an adoption constant that named itself the single source of truth for a shipped configuration was, in fact, read only by the deployment path. Every ad-hoc experiment silently resolved to a different configuration — one that had been explicitly examined and rejected. The control run that caught it did so because it failed to reproduce a recorded baseline by more than the noise band. Anything that can diverge between backtest and deployment should be resolved in one place that both call.

That body of negative results is the part of this work with the longest shelf life. Knowing where the headroom is not is what stops the next quarter being spent re-running someone else's dead ends.

What the record cannot support

The limits are what make everything above credible, so they are stated in the product, not just here.

Close races have a ceiling, and we hit it. In the two-to-four bidder stratum — the case a bidder most wants an answer on — three separate attempts to add competitive context measured flat: incumbency signals, contract-expiry timing, and aggregate features describing the rival field. Not marginal, not noisy. Flat. The plainest reading is that what decides a close race is how aggressively each firm priced, and the public feed does not hand that over in a form anything can learn from at the firm-versus-firm grain.

About 3% of the firms that bid have no public history at all. No system built on track records can name a first-time bidder, and the shortlist's theoretical ceiling — everything the retrieval stage could possibly reach — is 0.839, not 1.0. The honest product answer is to predict how many newcomers a tender will attract rather than pretend to name them, which the system does, and to state the unreachable share on the page next to the list.

The weakest price band is published rather than buried, as the coverage figures above show.

One open tender in six draws a single bidder. Predicting the winner there is trivial. The genuinely valuable case is also the hard one, and conflating the two is how this kind of system gets oversold.

And one thing we could build and will not. This dataset supports statistical anomaly screens on bidding patterns, and we have run them. They will not become a product claim. A screen detects a statistical anomaly, which is not the same thing as detecting wrongdoing, and naming a company on the strength of one is a defamation exposure with no corresponding upside. If market-integrity analytics are ever offered, they will be aggregate, unnamed and advisory.

Where this goes

The engine is built and graded. What it ships as is a report on a single tender that has not yet been awarded — the price range, the expected level of competition, the firms most likely to show up, and a recommendation you can argue with — accessed through a private link and priced per report. The pricing page has the rest, refund included.

One property of that report is worth stating here because it follows directly from everything above. A sold prediction is never mutated. New information arrives as a new version; version one stays byte-identical for the life of the report and remains readable forever. Predictions are point-in-time by construction, so silently re-scoring one after the fact would fold in information that did not exist when the client made the decision — which would quietly undo the discipline this entire article is about.

A report is written for a specific notice and exists before it is sold, so the first thing we need is which one. Tell us the tender you are looking at, and twenty minutes is enough to see what comes out and whether your sector is one where the forecasts hold.

Figures: test split, open procurement only. Models trained through 2024-12-31, calibrated through 2025-06-30, and evaluated only on tenders published after 2025-07-01. Source data: Système électronique d'appel d'offres du gouvernement du Québec (SEAO), CC-BY 4.0.