How to Test an AI-Assisted Code Migration: What Google’s Fortran-to-Rust Port Shows

AI agents can now port the core of a large legacy codebase to a modern language in about an hour, at least in vendor showcases like Google's recent Fortran-to-Rust run. That changes the question a business owner should ask. It is no longer whether the old code can be translated, but how anyone will know the new version still does exactly what the old one did. For a billing system, a booking engine or a customer portal, that second question decides whether a migration is safe to release, and the answer starts with characterization tests.

To test an AI-assisted code migration, record what the old system actually does before changing anything (characterization tests), make automated checks enforce the rules the new code must follow, move one slice at a time, run old and new side by side on the same real inputs and compare the results, and have a named person decide every mismatch. The agent writes the new code. People decide what "the same" means.

In the sections below, we look at a recent public example, a Fortran-to-Rust port reported by Google's Antigravity team, then at what characterization tests are, why business applications are harder to compare than scientific code, what to do when the old code turns out to be wrong, and a test plan you can use on your own system.

What the Fortran port shows

Google's Antigravity team reported migrating the core of FVCOM, a coastal ocean model with 174,774 lines of Fortran 90 and C across 264 files, to memory-safe Rust in a single 59-minute autonomous run with Gemini 4 Argon. The authors describe it as one of their favorite internal stress tests, and they explain the method in their write-up on modernizing 40-year-old scientific code.

The run had three stages:

• Baseline capture (11 minutes). The agent compiled the original program and built a harness to record golden outputs, runtime and memory use across 100 random seeds.
• Audit and blueprint (3 minutes). It traced dependencies across all 264 files, wrote an architectural blueprint before any Rust existed, and logged six bugs in the original Fortran and C code.
• Port and verify (45 minutes). It rewrote the solver as Rust modules, generated tests, fixed the one test that failed, and compared the result with the Fortran baseline.

A single prompt set four hard rules: no unsafe code, no heap allocations inside hot simulation loops, every Rust file under 400 lines with runnable doc-tests for every module, and at least 95% branch test coverage with zero difference from the Fortran output across 100 randomized seeds. All 100 runs matched to 12 decimal places (3.91 × 10⁻¹³). The authors also report a 1.88x speed-up in their benchmark. That is their result on one scientific workload, not a guide to what an AI code migration of a business application will deliver.

Two details are worth copying more than the headline. First, 14 of the 59 minutes went to understanding and recording the old system before any new code was written. Second, several rules were checked by tools rather than trusted: the compiler blocks unsafe code, the test runner measured coverage, and the parity harness compared every seed with the Fortran baseline. The recorded outputs worked as characterization tests, and the enforced rules did the rest. The translation sat between the two.

Characterization tests: record what the system actually does

The term comes from Michael Feathers' book Working Effectively with Legacy Code. As InfoQ's summary of characterization testing explains, characterization tests characterize the actual behavior of a piece of code: they check "what the code actually and currently does", not what it is supposed to do. Mark Seemann adds the practical reason they exist: test-driven development is sometimes impractical, and this often happens with legacy code, so tests are added afterwards while gathering evidence about what the code does (Empirical Characterization Testing).

The Antigravity team called its recorded results golden outputs. Recording the full output of a whole flow, such as a generated invoice or a monthly report, and comparing later runs against it is usually called golden master testing. For a business application, characterization tests start with real inputs: a month (or a full business cycle) of orders, invoices, bookings or payments. Run them through the old system, store the exact outputs, and treat them as the baseline the new code must match. A simple test harness then replays the same inputs against the new code and compares the results automatically.

That baseline includes the old system's bugs and quirks, on purpose. It records behavior; decisions about changing behavior come later, and they come from people.

Why business applications are harder to compare

A simulation takes numbers in and gives numbers out, so "the same" can be defined as a tolerance. Even there, the rule asked for zero difference and the reported match was to 12 decimal places, which shows that someone always has to define what counts as a match. Business systems make that definition harder, and their characterization tests have to cover more than outputs:

• Stored state. The new code must produce the same database writes as the old code, so characterization tests should capture the data the old code writes, not just the screen it returns.
• Money. Rounding and currency rules can make totals agree while individual lines differ.
• Dates and cycles. Month-end and year-end jobs run rarely, and time-zone and daylight-saving edge cases appear only on certain dates, so characterization tests built from one ordinary day miss them.
• Side effects. Charging a card, sending an email or posting to an ERP must never happen twice.

The answer is a parallel run. Thoughtworks' Technology Radar describes a parallel run with reconciliation as "executing the same production flow through both the old and new code, returning the response from the legacy code but comparing the results to gain confidence in the new code". We add four working rules. Users keep getting the old system's answer. Writes and outbound actions stay on the old system until the new one is trusted. Agree reconciliation rules before comparing (formatting, rounding, empty values) so the comparison flags real differences rather than noise. Keep the run going for at least one full business cycle, including a month-end, and replay last year's data for any quarter-end or year-end job the slice touches.

Move one slice at a time. Martin Fowler's Strangler Fig describes a gradual approach to legacy modernization: start with small additions built on top of, yet separate from, the legacy code base, and move bits of behavior from the old system into the new one. Small slices keep each set of characterization tests small enough for a person to understand every difference. It is the same release-by-release logic as upgrading a legacy PHP app one release at a time.

When the old code is wrong: fix it or match it?

The Fortran audit found six old bugs, including a buffer overflow, an integer overflow and reads of uninitialized memory. The write-up does not say whether the Rust port fixed each one or reproduced it. Business migrations hit the same question constantly. The Divert the Flow pattern by Ian Cartwright, Rob Horn and James Lewis, published on martinfowler.com, notes that many legacy reports contain undiscovered issues and bugs, so the new outputs rarely, if ever, match the existing ones, and without a full understanding of the legacy implementation it is often very hard to find the cause of a mismatch. Characterization tests show you the difference; they cannot tell you which side is right.

So keep a written mismatch log alongside the characterization tests. Every difference between old and new gets one of three labels:

1. A bug in the old system to fix. The new behavior is correct, and the change is deliberate.
2. A bug to keep for now. A customer, report or downstream system depends on the old behavior.
3. An error in the new code. The agent has to fix it.

A named business owner decides each label and records the reason. Not the agent, and not an engineer alone. Over time the log becomes the record of what the migration changed on purpose.

The Fortran run also shows where human review has to be added for business systems. When a test failed, the agent "didn't stop to ask a human what to do"; it read the error, added a guard and re-ran the tests. For a research stress test that is reasonable. In a business system, an agent trying to get a failing test to pass can do it the wrong way, by loosening an assertion or regenerating the baseline. Keep the characterization tests where the agent cannot edit them, and make every change to a test or to the baseline a human approval. As our agentic SDLC playbook explains, a written rule is only advice until an automated check enforces it.

A test plan for your own migration

Before you let an agent run a legacy code migration on any part of a business system, work through this list:

1. Choose one slice: a single flow or module with clear inputs and outputs.
2. Record the baseline from real inputs, covering at least one full business cycle, and store it where the agent cannot change it. These are your first characterization tests.
3. Write the rules the new code must follow, including what counts as a match, and make automated checks enforce them.
4. Let the agent port the slice.
5. Compare the new code against the characterization tests, then run them in parallel on live inputs with users still on the old system.
6. Log every mismatch and have the named owner decide each one.
7. Release the slice with a tested rollback plan.
8. Move to the next slice and extend the characterization tests as you go.

AI makes the translation cheap. Characterization tests, a parallel run and a named decision-maker make it safe. This is how we approach legacy application modernization: baseline first, one slice at a time, with a person accountable for every release. It is also how our AI-assisted development and modernization work runs day to day.

If you have a system nobody wants to touch and want a second opinion on how to prove a migration, Book a free Next-Step Review. It is a 30-minute conversation about the system you want to modernize, followed by a one-page brief within two working days.

Frequently Asked Questions

What are characterization tests?

Characterization tests record what an existing system actually does today, including its bugs and quirks, rather than what it is supposed to do. In a migration they become the baseline the new code must match before anyone changes behavior on purpose.

How do you test an AI code migration?

Record characterization tests from real inputs before changing anything, make automated checks enforce the rules the new code must follow, move one slice at a time, run old and new in parallel and compare the results, and have a named person decide every mismatch.

What is golden master testing?

Golden master testing records the complete output of a flow, such as an invoice, report or file, and compares every later run against that recorded master. It is a common way to build characterization tests for a whole business process.

What is a parallel run with reconciliation?

The same production flow goes through both the old and new code. Users keep getting the old system's answer while the two results are compared. Differences go to a mismatch log for a person to decide.

Should a migration fix old bugs or keep them?

It depends on who relies on the old behavior. Log every difference, label it as a bug to fix, a bug to keep for now or an error in the new code, and let a named business owner decide each one.

Can AI agents migrate legacy code without human review?

Agents can translate and test code quickly, but they should not change tests or the baseline, or decide what counts as a match, without a person approving it. In a business system, those decisions carry real money and real customers.