NeuroGenesys.
Program 01 • Genesys Frontier Models

Genesys Frontier Models

Last updated • By

Genesys is our research path to continual-learning world models: models that keep learning without losing what they already know. It rests on two ideas. When each skill sits in its own frozen expert, forgetting becomes a problem of choosing the right expert. And an exploitable verifier, not an inaccurate one, limits how far extra samples help.

Neither idea is ours alone. The decomposition is a 2022 theorem, and the verifier limit is published by at least five groups. Our own evidence is from toy models, CIFAR-100 ResNets and small open language models (3B to 8B, plus one 30B mixture-of-experts), and in places it cuts against the program: with real judges, Best-of-N did not decline by N = 32. This page sets out each claim, where it holds and where it fails, and the pre-registered tests behind them. A test that fails is reported here.

What does the program claim, and how strong is the evidence?

Evidence levels follow the findings ledger: L1 is a toy model, L2 a small LLM, L3 published work we have not reproduced. Bracketed numbers point to the sources at the end of the page.

Genesys claims with evidence level, support, counter-evidence and the test that decides them
ClaimEvidenceWhere it holdsWhat cuts against itDeciding test
With one frozen expert per task, class-incremental accuracy = selection accuracy × within-task accuracy.Identity (prior art)Exactly, for hard routing to frozen experts. Kim et al. proved the general bound in 2022 [1].Not ours, and not informative alone: frozen experts cannot forget by construction. It breaks once a shared head can recover from wrong routing, as in HiDe-Prompt, where final accuracy beats task-ID accuracy [4].None needed. It is the measurement frame, not a result.
Given frozen experts, selection error is where class-incremental accuracy is lost.L1 toySplit-MNIST, one seed: selection error ε rose from 0.21 to 0.57 as tasks went from 2 to 10.Our original selector (0.635) scored below the GDumb replay baseline (0.859). A selector that never reads expert confidence (random-projection ridge over streaming-PCA features) reached 0.926 on the same Split-MNIST stream over 5 seeds, but its feature basis sees the whole unlabelled stream (ledger 9.1). In the literature, selectors that beat replay train on replay data or synthetic stand-ins for other tasks [1, 3]. A strong pretrained backbone with no task-ID step beats prompt-selection methods [5].Split-CIFAR-100, pre-registered, 10 seeds: frozen experts route to the right task 36% of the time but are 96% right within it (ledger 8.15), holds; ε rises with task count, 0.53 → 0.64 → 0.79 at 5, 10 and 20 tasks, while within-task accuracy rises (ledger 8.18)
As an agent's skill library grows, failures come from picking the wrong skill, not from losing skills.L2 small LLMIn-context library of 4 → 64 skills: ε rose with library size, and forced accuracy on the original skills did not collapse in any of four runs. ε at 64 skills depends steeply on model size: 0.170 for a 3B model, 0.030 for 8B, 0.003 for a 30B-A3B model.Mis-selection explained at most 6% of the routed-accuracy drop in every run; the rest was execution. A 30B-class model barely mis-selects at 64 skills, and richer tool descriptions did not lower ε. The 3B model mis-selected most but ignored the chosen skill and still answered 42 of 68 such requests right. At 52–202 real skills on frontier models, wrong-skill selection explains 38–67% of the drop [16], so the thesis may hold only for larger, real libraries.Phase 1b: size predictions held, documentation prediction failed, done; next, real skill libraries at 100+ skills
Verifier exploitability ε, not average accuracy, bounds Best-of-N test-time compute.L1 toyIn toy models, Best-of-N accuracy peaks at a computable N* when ε > 0. On real systems, weak reward models and log-probability selection do decline at large N [9, 11].The qualitative claim is published [7, 8, 9, 10, 14]. Our quantitative match to Gao et al. failed its pre-registered bar (R² 0.73 and −0.49). Strong verifiers plateau rather than decline [12, 13]. Benchmark accuracy predicts low-N gains (r = 0.87 at N = 16) [15]. The bound applies to argmax selection, not to test-time compute in general [9]. Our own pre-registered test found no decline by N = 32 for a reward model, an LLM judge or self log-probability, and measured ε did not predict the best N (ledger 8.13).Real judges on GSM8K, MATH-500 and HumanEval+, pre-registered: no decline by N = 32 and ε did not order N*, not supported at N ≤ 32; plurality over the top-scored half beat argmax in all 6 math cells (ledger 8.14); next, the N = 1024 measurement
Selector errors follow a single ε law across domains.Correctionε* = p/(1−p) and the measured constants stand.The law is Thurstone's 1927 Case V identity, not a discovery, and it is not universal: it breaks 723× under heavy tails. See the correction.Closed.
Frozen experts plus a selector extend to continual-learning world models.L1 toyPoint-mass simulator: replay prevents forgetting of changed dynamics only with a dynamics-context signal. Given task identity, isolation methods forget almost nothing (PackNet: 0.80 performance, 0.00 forgetting) [19].Mixture-of-dynamics models with an inferred regime date to 2018 [20]. In practice continual model-based RL runs on replay [18]. Frozen experts cannot revise knowledge that has gone out of date.Not yet designed. The open gap is an oracle-vs-inferred-selector ablation on unlabelled streams where regimes recur.

Where does the selection view hold, and where does it not?

  • Holds: hard routing to frozen experts, where tasks are easy to tell apart. Expert Gate picks the right expert 99.2% of the time across distinct datasets [2].
  • Does not hold:tasks that are class splits of one distribution. The same gate reaches 43.2% on 5-task split MNIST [1], and selection becomes the whole problem. Selection has to see other tasks' data to be any good, which is replay by another name [3, 6].
  • Changes shape: with a strong pretrained backbone. Methods that skip task-ID entirely come within 1.6 points of joint training on CIFAR-100 [5], so the remaining gap is representation, not selection.
  • For agents: at 64 in-context skills on an 8B model, the loss was mostly execution. At 100–200 skills, selection dominates [16], and models under about 12B route skills poorly [21]. Where the crossover sits is an open measurement.
  • For verifiers:the limit applies to argmax Best-of-N. LLM judges are easily fooled: content-free answers like ":" pass an 8B Llama judge 79% of the time on GSM8K [17]. Strong reward models plateau instead [12, 13], and pessimistic selection rules avoid the decline [9].

What has been tested, and what is next?

Each study fixes its hypotheses, analysis and kill criteria in writing before the main run, and every outcome is reported. Protocols and code are in the lab's research archive, at the paths shown.

Completed and planned Genesys studies
StudyQuestion and designWhat would count against usStatus
Forgetting as selection at scaleresearch_notes/Forgetting as selection at scale/Does the Split-MNIST result survive on Split-CIFAR-100, and does explicit selection still trail replay?5, 10 and 20 tasks; 10 seeds; Naive, Joint, GDumb, DER++, BiC and frozen experts with three selectors (Avalanche 0.6.0). H1–H4 with Holm correction.Oracle within-task accuracy below 0.60 makes H1 only weakly informative, and the result is reported as such. A positive H2 (selection trails GDumb) is reported as a result against the program, not hidden.Done 2026-11-11: H1, H3 and H4 supported after Holm (selection error exceeds within-task error by 0.60; ε rises 0.53 → 0.64 → 0.79 from 5 to 20 tasks; selection carries 91% of between-method variance); H2 not supported (experts beat GDumb, 0.34 vs 0.16, but trail BiC and Joint)
Judge exploitability on real Best-of-Nresearch_notes/Judge exploitability on real Best-of-N/Do a real reward model, an LLM judge and self log-probability make Best-of-N peak and then fall, and does measured ε predict where?Llama 3.1 8B, 32 candidates on 100 GSM8K, 100 MATH-500 and 100 HumanEval+ problems. Judges: Skywork-Reward-V2-Qwen3-0.6B, Qwen2.5-Coder-7B as judge, and mean log-probability. N* is cross-fitted so it is never scored on the problems it was chosen on.If no judge declines at N ≤ 32, we make no exploitability-saturation claim for these judges. If the ε–N* correlation interval includes 0, we make no claim that ε predicts the safe budget.Done 2026-11-10: H1 and H2 not supported (no judge declined at N ≤ 32; ε did not order N*), so both kill criteria apply; H3 supported (top-half plurality matched or beat argmax in all 6 math cells, +0.03 to +0.17)
Forced vs routed skill selection, phase 1bresearch_notes/Forced vs routed skill selection/Does mis-selection grow faster for smaller models, and do better tool descriptions cut it?Llama 3.2 3B, Qwen3-VL 30B-A3B, and Llama 3.1 8B with documented descriptions; K = 4 vs 64; 400 requests and 200 probes per size.Forced accuracy on the original skills falling more than 10 points (significant at 95%) means real forgetting, and the thesis narrows.Done 2026-11-10: P1, P2, P4 supported; P3 (richer descriptions) not supported; kill not triggered
Phase 2: skills in the weightsresearch_notes/Forced vs routed skill selection/The same forced-vs-routed probe with one LoRA adapter per skill, so forgetting means weight-level forgetting.Small open model, one adapter per skill or sequential fine-tuning, the same probes after each addition.Same as phase 1.Needs a GPU

What could the program add that nobody has measured?

A literature review across continual learning, Best-of-N verification, agent tool selection and continual world models found four measurements no one has published. They, rather than the mechanism, are what Genesys aims to own:

  1. Task-ID accuracy as a function of the number of tasks, for a fixed selector. No paper we read plots it, and Kim et al.'s bound weakens as tasks grow.
  2. A pre-registered, out-of-sample prediction of the best Best-of-N budget from verifier statistics measured beforehand. We found none.
  3. How well an open 8B model executes the correct skill as the library around it grows, with deliberately similar-sounding skills.
  4. For a continual world model, a given task label against an inferred one, on unlabelled streams where regimes recur.

Common questions

Is "forgetting is a selection problem" a new discovery?
No. Kim et al. proved that class-incremental learning decomposes into within-task prediction and task-identity prediction (NeurIPS 2022), and Expert Gate used frozen per-task experts with a learned gate in 2017. The program uses the decomposition as a measurement frame. What it adds is measurement: how selection error grows with tasks, library size and model size.
Does Best-of-N always get worse with more samples?
No. With weak or exploitable verifiers it can peak and decline (Cobbe et al. 2021; Huang et al. 2025). With strong verifiers it usually plateaus (Lightman et al. 2023; Brown et al. 2024). The program asked whether a verifier's measured exploitability predicts which regime you are in before you spend the compute; for three real judges at N ≤ 32 it did not.
Is the Genesys SDK or Copilot available?
No. The SDK and Copilot shown on the programs page are a concept preview, still in design. What ships today are the research tools: the selection-reliability package (selrel), the exploitability audit and the safe Best-of-N budget calculators.

Papers, posts and tools behind the program

Sources

  1. A Theoretical Study on Solving Continual Learning — Kim, Xiao, Konishi, Ke & Liu, NeurIPS 2022
  2. Expert Gate: Lifelong Learning with a Network of Experts — Aljundi, Chakravarty & Tuytelaars, CVPR 2017
  3. Class Incremental Learning via Likelihood Ratio Based Task Prediction (TPL) — Lin et al., ICLR 2024
  4. Hierarchical Decomposition of Prompt-Based Continual Learning (HiDe-Prompt) — Wang et al., NeurIPS 2023
  5. RanPAC: Random Projections and Pre-trained Models for Continual Learning — McDonnell et al., NeurIPS 2023
  6. Three scenarios for continual learning — van de Ven & Tolias, 2019
  7. Training Verifiers to Solve Math Word Problems — Cobbe et al., 2021
  8. Scaling Laws for Reward Model Overoptimization — Gao, Schulman & Hilton, ICML 2023
  9. Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment — Huang et al., 2025
  10. The Limits of Inference Scaling Through Resampling — Stroebl, Kapoor & Narayanan
  11. Scalable Best-of-N Selection for Large Language Models via Self-Certainty — Kang et al., 2025
  12. Let's Verify Step by Step — Lightman et al., 2023
  13. Large Language Monkeys: Scaling Inference Compute with Repeated Sampling — Brown et al., 2024
  14. Inference-Time Reward Hacking in Large Language Models — Khalaf et al., NeurIPS 2025
  15. RewardBench 2: Advancing Reward Model Evaluation — Malik et al., 2025
  16. More Skills, Worse Agents? — Song & Wei, 2026
  17. One Token to Fool LLM-as-a-Judge — 2025
  18. The Effectiveness of World Models for Continual Reinforcement Learning — Kessler et al., CoLLAs 2023
  19. Continual World: A Robotic Benchmark for Continual Reinforcement Learning — Wołczyk et al., NeurIPS 2021
  20. Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL (MOLe) — Nagabandi, Finn & Levine, 2018
  21. Agent Skill Framework for small language models — Xu et al., 2026

Leadership

A word from the founder — on why the brain is the blueprint, and why the next decade of intelligence will be built here.

"The brain is the universe's only proof that general intelligence is possible. We treat it as the blueprint."

From cortical interfaces to photonic silicon, every program at NeuroGenesys exists to answer one question: how does intelligence arise — and how do we build it safely.

Will Schulz

Will Schulz

Founder, NeuroGenesys • AW3 Technology