Dean Byrne PRO

Really appreciate this, honestly. You took it somewhere I hadn't measured, first that the honest object is a channel mixer built from a growth process not the fungal story, then that fan-out against a matched dense MLP is the number that settles the efficiency side. That got me to actually go measure instead of speculate, which I wouldn't have done otherwise.

Thanks for taking it seriously enough to push. I'll get you the fixed-fan-out numbers from the clustering variant when my current run frees up, since I want to know the answer too.

Measured it: the readout weights all 96 tips basically uniformly and the tips spread across ~15/16 density regions, so there's no clustering to exploit yet, a top-k readback today would be a cliff and it'd need a learned gate. Makes sense in hindsight, negative autotropism is an anti-clustering force by design, so the thing that makes it a colony is what keeps it dense.

I'll build the second variant (flip in a clustering/positive-tropism term so the growth concentrates) and post the fixed-fan-out perplexity numbers. I've got another experiment running right now, so it'll be a bit, but I'll get to it when I can since I want to see this one too.

Appreciate it, but efficiency wasn't the point, and the post already framed it the way you did: a research LM whose channel-mixing block isn't an MLP, just a few differentiable steps of colony growth read back into the hidden state. The only thing I was testing is whether a weird dynamical system can carry the channel-mixing role and still learn language under a fixed pipeline. "Curiosity or cheaper mixer" is a great question, just a different experiment than the one I ran.

That said, I measured your number: active-channel fan-out (Jacobian participation ratio, /768) was ~469 for the growth block vs ~489 for a matched dense MLP, at 3.86M vs 3.54M params and ~1.07x MACs/token. So you're right, it densifies to roughly MLP cost. The growth math is cheap (O(N·F)), but it's bracketed by dense projections, so it's a hair more expensive, not less. Curiosity, plainly. The version that'd make your point true is a sparsified readback (each tip writing to a local slice of channels, or top-k tip-to-channel routing) so fan-out drops well under d_model. That one I'd like to run. Thanks for pushing on it.