NS Toor’s initiative to facilitate financial literacy ·

Banking India Update

Points for effort, ranks for accuracy: how trainees read scoreboards

Effort and accuracy scoreboards shape trainee decisions differently, revealing how the metrics people attend to influence judgment under uncertainty

5 min read

Points for effort, ranks for accuracy: how trainees read scoreboards
Points for effort, ranks for accuracy: how trainees read scoreboards

Trainees in finance and banking spend a great deal of time looking at numbers that are supposed to tell them how they are doing. Course dashboards, simulated P&L, accuracy scores, leaderboards, peer rankings, pass rates on regulatory modules. The question worth asking is not whether these scoreboards motivate people — they plainly do — but which part of the scoreboard trainees actually attend to, and what that attention does to the way they make decisions under uncertainty. A scoreboard that rewards effort and a scoreboard that rewards accuracy are not the same instrument, even when they sit on the same screen.

Two signals, one dashboard

Most corporate training platforms in banking and finance present, at minimum, a completion metric and a correctness metric. The first is a count: modules finished, cases attempted, hours logged. The second is a ratio: share of decisions that matched the model answer, error rate on reconciliation exercises, deviation from the expected hedge in a risk case.

Behaviorally these are different reinforcers. Completion behaves like a fixed-interval schedule — you get credit for showing up and persisting, and the reward is predictable once you learn the rhythm. Accuracy is closer to a variable schedule, because you cannot reliably predict which case will go well. Work on variable-ratio reinforcement has long shown that unpredictable payoffs sustain attention well past the point where a predictable one would have lost it; our own note on variable-ratio payoffs holding attention past satiety makes the point that this persistence is not the same thing as learning.

The practical consequence: if the effort counter is large and animated and the accuracy figure is a small grey percentage, trainees will optimize for the counter. They will open modules, click through cases, and finish — while their decision quality stays flat.

What trainees actually read on a scoreboard

Eye-tracking work in educational dashboards, and the broader literature on scoreboard design in serious games, converges on a modest finding: learners look first at rank, then at their own recent change, and only then at absolute accuracy. Rank is social and immediate. Change is a story about the self. Accuracy requires a comparison against a standard that is often not visible on the same screen.

This matters because rank and accuracy can diverge sharply in small cohorts. In a batch of twenty analysts, being third on completion points tells you almost nothing about whether your credit-risk judgments are calibrated. Yet the rank is what gets screenshotted and discussed.

There is a parallel in how people read probabilistic fields under time pressure. Our analysis of when users actually read the RTP field found that the headline number is consulted early and then abandoned once a session is underway; the same pattern shows up in trainees who check their accuracy score at the start of a module and never return to it while working the cases. The scoreboard is read as a verdict, not as feedback.

Loss aversion and the accuracy number

Kahneman and Tversky's framing work is directly relevant here, and not in the abstract. An accuracy score can be presented as a gain ("you got 78% right") or as a loss ("you got 22% wrong"). The two framings are arithmetically identical and behaviorally distinct. Loss framing tends to produce more cautious, more conservative decisions — sometimes appropriately, sometimes not.

We tested something close to this in a training context and found that loss-framed risk warnings shifted 27% of trainee hedges at case 3. The shift was not toward better answers; it was toward safer ones. Trainees who saw the loss-framed warning hedged more, including on cases where the model answer called for exposure. That is a scoreboard teaching caution rather than judgment.

The design implication is uncomfortable but clear. If your accuracy metric is displayed as a shortfall, you will get conservative behavior, and conservative behavior will look like competence on a risk module and like timidity on a trading or credit-growth module. The scoreboard is not neutral between the two.

Reward loops that reward the wrong thing

The most common failure mode in finance training dashboards is a streak mechanic bolted onto accuracy. Seven correct in a row, a badge appears. This is a textbook variable-reinforcement structure layered on top of a competence measure, and it produces a recognizable pathology: trainees begin selecting easier cases to protect the streak.

There is a useful analogue in how session-level reward structures distort stopping behavior. Our work on how variable rewards predict session stop-loss timing shows that when payoffs are unpredictable, people manage the end of a session by feel rather than by rule. In a training context the equivalent is managing the difficulty of the next case by feel — quietly avoiding the cases that would break the streak. The scoreboard has become a competitor to the curriculum.

A partial fix that has decent evidence behind it is tiered unlocking tied to demonstrated skill rather than to time or streaks. We found that skill-tier unlocks lifted 60-day course completion by 17% by module 9, and the mechanism appears to be that the unlock is contingent on a threshold of accuracy, so effort and correctness point the same direction instead of competing.

Designing a scoreboard that reads honestly

Three design choices seem to carry most of the weight.

Separate the panels, and put accuracy first. If completion and accuracy share a visual field, completion wins. Give accuracy its own panel, with the standard visible — the model answer, the cohort distribution, or both. A number without a reference point is decoration.

Show calibration, not just correctness. A trainee who is right 70% of the time and confident on exactly those 70% is in a different position from one who is right 70% of the time with uniform confidence. Brier scores and simple confidence-accuracy plots are not hard to compute and they change the conversation from "how many did I get" to "do I know what I know."

Frame errors as information, not as shortfall. This is the hardest one to implement politically, because compliance and audit functions often want the shortfall framing for legitimate reasons. The compromise is to keep the loss-framed number for the record and present the learning-facing number as a diagnostic — where the errors clustered, which case types, which decision points.

The forward-looking question for training teams is not whether to keep scoreboards. It is whether the scoreboard is measuring the thing the desk needs. A trainee who finishes every module and cannot size a position under uncertainty has been trained by the dashboard, not by the curriculum. The next generation of these systems will likely be judged less on engagement metrics and more on whether the accuracy number on screen at module 20 predicts the quality of the decision made at the desk in month six. That is a testable claim, and it is worth building the instrumentation to test it.