Two Correct Commits, One Silent Regression: Why a Rails Auto-Link Feature Died Three Months Later

Some regressions stay invisible even if you read both PRs, because each PR is right. By the end of this you can hunt for the same drift in your own Rails app and write the one spec that stops it.

Short version: the scope you write with and the scope you read with need to match. Hand-write them separately and one will eventually narrow on its own. Code review won't catch that. A spec that writes and then reads back will.

"No candidates, for all of them"

A work project of mine (a contract management SaaS) links related contracts automatically. Upload a memorandum or an amendment PDF, AI reads which contract it refers to and when that was signed, and the system attaches it to the original as a child.

One day a trial customer wrote in. Dozens of documents uploaded that day had all come back with "no candidates." I opened the ones they mentioned. The signing dates matched. Document name similarity was comfortably above the auto-link threshold. The candidate list was still empty.

High similarity and zero candidates? That made no sense to me until I looked at the order of the checks.

1. Date        parent signed or start date matches   SQL
2. Company     partner looked up by ID, not name     SQL   no IDs: return nil
3. Similarity  document and file names compared      Ruby  expensive, so last

Cheapest filter first, the usual order. Stop at step 2 and step 3 never runs, however high the similarity would have been.

The two scopes

When a name isn't in the partner master, we create an "unconfirmed" partner from the OCR result and attach it to the contract. Boiled down, the code looked like this (names changed).

class Org < ApplicationRecord
  has_many :partners
  has_many :active_partners, -> { where(confirmed: true) }, class_name: 'Partner'
end

# Attach side: reuses unconfirmed rows to avoid duplicates
org.partners.find_or_create_by!(name: name)

# Match side: can't see unconfirmed rows
ids = org.active_partners.where(name: name).pluck(:id)
return nil if ids.empty?

An org that never registered a partner has only unconfirmed ones. The attach side sees the row and the match side doesn't, so ids is always empty and the chain ends at gate 2.

What threw me further was that the partner's name came through a different path and looked fine. Name present, ID missing. The logs read as healthy.

Why it took three months

It only made sense once I laid git log out in order.

mid-June        unconfirmed partners added, list screen filters on confirmed
26 min later    filter moved inside active_partners
                commit message: "contract matching unaffected"
~2 weeks later  unrelated refactor drops the name string fallback  <- regression complete
August          production release
early Sept      trial customer writes in

"Unaffected" was true when it was written, because the fallback that matched by name was still alive. The refactor two weeks later did nothing wrong either. The definition of active_partners isn't in that diff. It just looks like clearing away a path nobody needs.

I don't think any reviewer would have caught this in either PR. The first commit's safety rested on code that was deleted later, and that dependency was recorded in a commit message rather than in code.

Reproducing it

To check I had this right, I built a minimal model with ActiveRecord 8.0.4 and in-memory SQLite. Org A has a confirmed partner, org B has only an unconfirmed one. Attach one contract to each, then look it up.

                       A (confirmed)   B (unconfirmed only)
with the fallback       1               1
fallback removed        1               0   <- here
match scope fixed       1               1

Look at A alone and nothing broke. That's how it survived three months. Orgs with registered partners kept working, so the overall link count never dropped enough to notice. When we counted later, a bit over a third of orgs had zero confirmed partners. For those orgs, auto-linking had returned nothing since the release.

The fix was one line: make the match scope agree with the attach scope. One line to fix, three months to find.

Having two scopes wasn't the mistake

My first reaction was "why hide unconfirmed rows at all?" The numbers changed my mind. Close to 80% of the partner table was unconfirmed rows generated by OCR. Show those in autocomplete or the master screen and the tens of thousands people registered vanish under a hundred thousand or so the AI made. Hiding them from display was correct.

The mistake, I think, was calling a display scope active_partners. It reads as "the live partners," so of course you reach for it when querying. Had it been listable_partners, someone would have paused before using it in matching code.

The same shape was already piling up

Hundreds of groups in the table had the same partner name twice in one org, most of them mixing confirmed and unconfirmed. That's what a registration path leaves behind when it looks up through a relation that can't see unconfirmed rows and creates one if nothing turns up. This reproduces too.

org.partners.where(name: 'Acme').pluck(:id, :confirmed)
# => [[2, false]]

org.active_partners.find_or_create_by!(name: 'Acme')

org.partners.where(name: 'Acme').pluck(:id, :confirmed)
# => [[2, false], [3, true]]   now the name exists twice

Create through a scoped relation and the where condition becomes attribute values. So a new confirmed partner appears right next to the unconfirmed one it couldn't see. One name, a whole family of bugs.

Is there a word for this?

The closest is Martin Fowler's semantic conflict: changes that merge cleanly as text but make the program behave differently. His piece is about parallel branches, while ours landed in sequence two weeks apart. The answer is the same though. Tests catch it, not a reviewer's eyes.

What to use when

Option               Stops                         Cost          When
round-trip spec      this whole regression         one spec      start here
single match scope   the next hand-written copy    refactor      2+ places look it up
rename the scope     picking the wrong one         find/replace  the scope's meaning is narrow
zero-count per org   nothing, but you hear sooner  dashboard     any automatic feature
code review          not this kind of bug          -             don't rely on it

If you pick one, pick the round-trip spec. Write data with the real attach code, then read it with the real match code. Don't test the scope with a hand-built fixture. Fixtures tend to use confirmed partners, and then this exact bug passes.

it 'finds contracts attached through an unconfirmed partner' do
  org = create(:org)                       # zero confirmed partners
  ContractAttacher.call(org, 'Acme', date) # the real attach code

  expect(ParentFinder.call(org, 'Acme', date)).to be_present
end

Run this shape of check against the reproduction above and it fails on the version without the fallback.

What to do in your own code

# relations and scopes with conditions
grep -rnE "has_many .*-> *\{|scope :" app/models
# places that quietly bail on an empty result
grep -rnE "return( nil)? (if|unless) .*(empty\?|blank\?|none\?|any\?|present\?)" app

Where the two lists meet, look closer. If a display scope feeds matching, a duplicate check or find_or_create_by, give it a spec. And when you review a PR that deletes a fallback, ask first what relies on it.

Features that quietly go to zero, like this one, can't be watched through a grand total. Split it per org or you won't see it.

If you'd like a hand finding quiet regressions like this in a Rails app, or pinning them with specs, feel free to ask negabaro.