Chronos — AI-Powered Legacy Code Modernization with IBM Bob

The Loop

A closed measurement loop, not a one-shot rewrite.

Never “trust the AI” — prove the improvement with the same static analyzer, before and after.

STEP 1

Measure

tree-sitter baseline, whole codebase

STEP 2

Target

rank by Likelihood × Impact risk

STEP 3

Modernize

IBM Bob rewrites, scope-constrained

STEP 4

Re-measure

identical analyzer, no moved goalposts

STEP 5

Impact

a hard before/after delta, not a vibe

30 0

Unguarded allocation sites eliminated across 7 files, verified by the same analyzer before and after.

Full Before / After

Every metric, measured twice.

Same tree-sitter analyzer, same 7 files, run before IBM Bob touched anything and again after.

Bar chart comparing before and after values for unguarded allocations, risk scores, cyclomatic complexity, raw pointers, allocation calls, unpaired allocations, and lines of code across 7 files.

Swipe to see all columns →

Metric	Before	After	Change
LOC	3299	3489	+5.8%
Unguarded allocations	30	0	−100.0%
Unpaired allocations	1	0	−100.0%
Cyclomatic complexity	788	893	+13.3%
Raw pointer count	1229	1290	+5.0%
Risk score (v2)	26.53	27.99	+5.5%
Risk score (v3)	25.72	3.89	−84.9%

Risk score (v2) includes cyclomatic complexity, which can increase when defensive NULL-checks are added — so it can rise even as the code gets safer. Risk score (v3) is OWASP-style Likelihood × Impact and excludes complexity; read it together with the unguarded/unpaired allocation metrics above for the safety signal.

Bug Spotlights

Two bugs Bob found along the way.

Not synthetic examples — both are the actual diffs Bob produced while modernizing these files.

A guaranteed crash on any network failure

clib-package.c · clib_package_new_from_slug_with_package_name()

Before

res = http_get(json_url);json = res->data;_debug("status: %d", res->status);if (!res || !res->ok) {  goto download;}

After

res = http_get(json_url);if (!res || !res->ok) {  goto download;}json = res->data;_debug("status: %d", res->status);

A guaranteed crash on any network failure. http_get() returns NULL when the request fails, and the original code dereferenced it one line before the NULL check it already had. Now the check runs first.

A heap overflow, self-detected

kopen.c · http_open(), 64 KB header buffer Not in our prompt scope — Bob found this independently

Before

while (read(fd, buf + l, 1)) {  if (buf[l] == '\n' && l >= 3)    if (strncmp(...) == 0) break;  ++l;   /* no upper bound */}buf[l] = 0;  /* l reaches 0x10000 */

After

while (read(fd, buf + l, 1)) {  if (buf[l] == '\n' && l >= 3)    if (strncmp(...) == 0) break;  ++l;  if (l >= 0xffff) break;  /* bound */}buf[l] = 0;  /* always in range */

A header larger than 64 KB walks l to 0x10000, and buf[l] = 0 writes one byte past the heap allocation. The loop now stops one byte short.

Scoring Model

Why we didn’t just use one risk number.

Adding defensive NULL-checks raises cyclomatic complexity. A blended score would punish the fix.

Likelihood

unguarded & unpaired allocations

Impact

pointer density & code scope

Risk

OWASP style — complexity excluded

Shin & Williams (2010) found only a weak correlation (r ≈ 0.30) between cyclomatic complexity and real vulnerabilities — a poor proxy for safety, and a terrible one for measuring a safety fix.