A bank put 2,000 engineers on a self-hosted open-weight model and reported 60% better coding time and 70% better lead time. The model that produced those numbers is not the model it is running now.
Both figures come from David Abitbol’s talk at AI Engineer NYC on 13 October 2026, where Revolut’s ops and infrastructure lead walked through the economics of serving coding models on its own GPUs. The conference publishes the session as a page with two distinct layers: an editorial summary written up by the organisers, and the full machine transcript, 202 timestamped passages of what was actually said. Reading both, the figures that travelled into the trade press came from one layer, the operating constraints came from the other, and almost none of the caveats came from either.
The version mismatch
The improvement figures arrive attached to a date and a model. From the transcript: “around April 26 is when we started self-hosting open source model”, then “April 26th, GLM 5.1”, then “we see 60% improvement” / “in coding time”, and “70% improvement in lead time.”
The serving figures arrive attached to a different model. “to serve 2,000 engineers, we roughly need 56 B200”, and “And that’s to host GLM 5.3.”
So the measured gain belongs to a deployment of GLM 5.1 beginning in April, and the 56-GPU, $0.30-per-million-token configuration describes GLM 5.3, which the bank moved to later. The conference’s own summary states the same split in its own words, saying the “move to GLM 5.1 coincided with roughly 60% better coding time and 70% better lead time on Revolut’s chart”, and separately that “Abitbol reports using 56 B200 GPUs to serve GLM 5.3 to roughly 2,000 engineers.”
Nobody misstated anything. But every retelling I could read merges the two into one sentence about one model, and that merged sentence is not in either layer of the source. Whether GLM 5.3 reproduces GLM 5.1’s measured improvement is not addressed in the talk, and no figure in it answers the question.
The precise numbers exist in only one layer
The inference-tuning result is the part most widely quoted: time to first token fell dramatically on a constant GPU count. Here the two layers disagree on precision.
The spoken transcript gives round numbers. Starting in April, “time to first token over 10 seconds, which is very slow”, ending in August at “requests per second were over 7, and time to first occur, and we’re around 1.6.” The percentages are stated aloud: “88% improvement in performance for 76% increased demand.”
The exact figures appear only in the organisers’ summary, describing what was on the slide: “the slide reports 76% higher requests per second and an 88% reduction in time to first token, from 14.26 seconds during migration to 1.68 seconds in the stable setup.”
I tested this as an explicit negative. The strings 14.26 and 1.68 have zero occurrences in the transcript layer and appear only in the summary layer. A reader working from the transcript alone cannot source either number. One widely-shared write-up rendered them as “about 14 seconds to 1.7 seconds”, which is a reasonable rounding of a figure that is in the slide note rather than in anything the speaker said.
The caveats went the other way
The conference page carries three limiting statements. Only one survived into the coverage I checked.
The summary says plainly that “The internal comparison does not isolate the model change from other causes”, and that the cost figures “describe Revolut’s deployment and utilization, not a portable price for any self-hosted model.” The transcript page carries a third, about itself: “Machine transcription with reviewed terminology and names”, and “Some recognition errors may remain.”
Counting those three against two retellings: the Dealroom write-up carried one of three, reproducing the no-isolation point and adding two caveats of its own that are not in the talk at all, that the saving “excludes cloud, staff and monitoring costs” and that the gains may partly reflect the removal of token limits. A longer explainer carried zero of three. The “not a portable price” line, which is the one that most directly limits whether any of this transfers to another company, appears in neither.
The transcription caveat is not pedantry. The transcript has the speaker sizing GLM at “8.500 billion parameters”, which is a recognition artefact: the published model card gives GLM 5.3 as “753B total parameters (744B excluding the multi-token-prediction layer) and approximately 40B active per token” with a 1M-token context window. Anyone quoting the parameter count off the transcript would be wrong by three orders of magnitude, which is exactly the failure the caveat warns about.
What only the transcript has
The split runs in the other direction too, and this is the half worth reading the long version for. The operating rules that would actually let someone attempt this appear only in the spoken layer, and no retelling I read carried them.
“Long context is a scam” — the advice being that even with a million-token window available, “256 is really what you need”. On quantisation, for Blackwell-generation hardware, “you don’t want to run anything higher than FP4”, with the claim that moving between FP4 and FP8 “is a 40% saving in GPU footprint for zero loss in precision”. On parallelism, “even though GLM fits on TP4, you don’t have enough KV cache, and so maybe a TP8 works better”. On cache, “coding test needs to be above 90% cache hit”. And the thing the whole tuning exercise was for: a target of “sub-3 second time to first token”, because “as soon as you exceed 5 seconds, my Slack inbox starts to blow up”.
The compliance framing is also transcript-only, including the sandbox rule given as a concrete example rather than a principle: “you can allow grep but not gradle.” The argument is that coding agents under a sandbox that guarantees human approval “are not considered autonomous agents”, and that because the weights are downloaded rather than called, “you’re just getting the weights and putting them on a US-based service.” Those are his descriptions of one EU-regulated bank’s position, and the conference page says so, noting they “should not be read as a general determination” for another organisation.
What is not checkable here
This is one practitioner’s account of his own deployment, published by the conference that hosted him, and every number in it is Revolut’s own. There is no independent measurement of the bank’s coding time, lead time, token cost or GPU count, and the talk offers none. The speaker is clear about one of the limits himself, saying the early Claude Code rollout showed engineers spending “a thousand a month, 10,000 a month with not so much ROI” — a comparison baseline that is a budget-limited deployment, which is not the same experiment as an unlimited one.
The figure I would most want and cannot get is the one for GLM 5.3. The gains are measured on 5.1; the hardware bill is quoted for 5.3; nothing published connects them.