ModMess and ModMess2 as evolutionary search benchmarks, and SuperFizzBuzz as a palimpsest of another

I have big piles of old notebooks and electronic notes from meetings and seminars and all kinds of just sitting around thinking up stupid shit. This is about a relatively simple problem, which might have been a benchmark idea, or might be something somebody else told me about, or might have been from a seminar?

Later: I am 55% convinced this came from a chat with Terry Jones about his tunable RS Landscapes. Explaining below.

It has no identifying information in that regard, except being from between 2001 and 2004, when other pages in the same notebook were dated. Tell me again about AGI and superintelligence.

Anyway, it looks from context like it was intended for comparing various genetic algorithms, simulated annealing, and other metaheuristics for population-based parametric optimization. There are two variants I can glean here: let’s call them ModMess and ModMess2.

The ModMesses

In both ModMess and Modmess2, a solution (aka “genome”) is an array of G integers, each in the range (1...R) inclusive. In my notes, I stuck to G = 16 and R = 100, but you do you.

A specific landscape for search is defined by picking a specific (uniform random) target genome, and the goal of search is to identify the target genome’s values of the genes in all G positions.

Slightly confusingly, and input vector (or training case) is also an array of G integers in (1...R). For any given input vector, a genome (including the target genome) is used to produce an output vector using the algorithm shown below. To evaluate an arbitrary individual at a specified input vector, we produce the output vector for that individual and return the summed absolute error compared to the output vector produced from the same input applied to the target genome.

How that works, in words:

1. We take the input vector of numbers, and construct a new vector of the same length, where each element is the running sum of the preceding values in the input.
2. We take these sums and collect a new vector of the remainders obtained modulo the genome’s value at each position.
3. If we’re doing ModMess2, we then take these remainders and collect a vector of them modulo the gene value at the position just to the right (wrapping back to the head as needed).
4. We compare this vector of integers to the target vector (summed absolute error) to determine the error score.

I asked Opus 5.5 to write me a little legible python version of this for explanatory purposes, and when it calmed down enough to hear me it produced this, which isn’t too bad, right?

"""
How a genome is evaluated in ModMess and ModMess2.

genome, target and each input are lists of G integers in 1...R
(G = 16, R = 100 in our experiments). The target scores 0 on every input.
"""
from itertools import accumulate


def modmess_error(genome, target, inputs):
    sums = list(accumulate(inputs))                         # running sums of the input
    mine = [s % g for s, g in zip(sums, genome)]            # each sum mod the genome's gene there
    goal = [s % t for s, t in zip(sums, target)]            # ... and mod the target's gene there
    return sum(abs(a - b) for a, b in zip(mine, goal))      # summed absolute difference


def modmess2_error(genome, target, inputs):
    sums = list(accumulate(inputs))                         # running sums of the input
    next_genes = genome[1:] + genome[:1]                    # gene p + 1, the last wrapping to the first
    next_targets = target[1:] + target[:1]
    mine = [(s % g) % h for s, g, h in zip(sums, genome, next_genes)]     # mod the gene, then mod the next gene
    goal = [(s % t) % u for s, t, u in zip(sums, target, next_targets)]   # (0 mod x is 0)
    return sum(abs(a - b) for a, b in zip(mine, goal))      # summed absolute difference


def case_errors(error, genome, target, training_inputs):
    # one error per training case (what lexicase uses); their sum is what
    # tournament and fitness-proportionate selection use
    return [error(genome, target, inputs) for inputs in training_inputs]


# Worked example, G = 5:
#   target [19, 67, 30, 48, 75], genome [14, 48, 84, 89, 60], input [33, 95, 11, 78, 37]
#   running sums                     [33, 128, 139, 217, 254]
#   ModMess   target remainders      [14, 61, 19, 25, 29]
#             genome remainders      [5, 32, 55, 39, 14]    error 9 + 29 + 36 + 14 + 15 = 103
#   ModMess2  target remainders      [14, 1, 19, 25, 10]
#             genome remainders      [5, 32, 55, 39, 0]     error 9 + 31 + 36 + 14 + 10 = 100
if __name__ == "__main__":
    target, genome, inputs = [19, 67, 30, 48, 75], [14, 48, 84, 89, 60], [33, 95, 11, 78, 37]
    print(modmess_error(genome, target, inputs), modmess2_error(genome, target, inputs))  # 103 100

And… that’s about it.

ModMess is relatively easy to search over, since each gene can be independently optimized. The structure of the landscape of each is still a bit up-and-downy, but it’s not complicated for any reasonable search method.

As far as I’m aware, there’s only one solution in ModMess.

ModMess2 however has a little frisson of nonlinearity thrown in. You take a remainder mod another value from a different gene. So the two genes affecting the outcome, that’s nonlinear epistasis in terms of gene effects. And the fact that if the number you mod by second is bigger than the one you used first, well nothing will happen. But if it’s lower, then something about how different they are will have a big effect.

And (having asked Opus 5.5 to check for me) it’s also true that ModMess2 can have multiple solutions that are not exactly the same as the target genome. As it explained:

… So if a target gene is divisible by the next target gene and is at least as large as the previous one, then any other multiple of the next gene that is also at least the previous one works too. For example, with targets …30, 40, 20… at three adjacent positions, the middle gene can be 40, 60, 80 or 100….

Why?

I was paging through some old idea books looking for GA problems where lexicase selection would be obviously better than fitness-proportionate or tournament selection. And, frankly, a quick hand-wave by Claude to check things seems to support that idea.

One try led me to do a bunch of runs on a lot of landscapes, and yup: ModMess2 is better this way.

One of the interesting things, to me, is the way ModMess2 makes things wobble around when you try to independently optimize one gene at a time (because of that second neighboring gene’s effects).

SuperFizzBuzz

I was working through these summaries and remembered where it all led, back in the day: A GA benchmark I called SuperFizzBuzz. Given the name, it was definitely another “find multiples of things” sort of modulo-based fitness landscape, and I thought earlier (in a previous version of this piece) I had remembered it.

But when I had Claude poke at it, it did one of the most useful things I’ve found LLM coding assistants can do for research coding: it said “I know you’re looking at fitness landscapes, but maybe this one isn’t what you had in mind, since there’s a trivial solution and lots of copies of it?”

Damned if Claude wasn’t right. I made some kind of memory error writing it out, and now I’m afraid SuperFizzBuzz is lost to time, at least in the manner of a metaheuristics benchmark problem. There might be a reason those old notebooks never went anywhere—a reason, that is, besides “it’s a lot harder to hand-code and run stuff on a c.2000 laptop than it is on a 2026 laptop with an LLM running the code and keeping track of the goals for you”.