TL;DR

Artificial intelligence can generate unconventional engineering ideas, but generating ideas is not the same as discovering valid solutions. A credible AI discovery engine needs an architecture that separates candidate generation, hard constraints, independent evaluation, search memory, and engineering review. The objective is to explore unfamiliar solutions without allowing the system proposing them to determine its own evidence of success.

Research systems such as Google DeepMind’s FunSearch and AlphaEvolve demonstrate how machine-generated programs can be combined with automated evaluation and evolutionary search to identify useful algorithmic improvements. Their success depends on problems that can be represented and evaluated systematically. Neither system establishes that arbitrary enterprise engineering challenges can be solved through unrestricted AI exploration.

For enterprise architects, the practical opportunity is to build bounded discovery environments. Start with an accepted baseline, explicit constraints, controlled candidate generation, deterministic validation, reproducible experiments, and evidence that engineers can independently inspect.

Takeaway: The most important component of an AI discovery engine may be the evaluator, because a system cannot reliably discover better solutions without a trustworthy way to distinguish improvement from failure.

Introduction

An enterprise platform team has a recurring capacity problem.

Every night, hundreds of analytical and batch-processing jobs compete for compute resources. Some require accelerators. Others depend on data locality, restricted execution environments, maintenance windows, or completion deadlines.

The scheduling platform works, but resource utilization is uneven. Certain compute pools experience long queues while others retain unused capacity.

The engineering team suspects that better scheduling policies may exist.

A traditional optimization effort might compare established scheduling strategies, adjust priority weights, revise placement rules, or introduce a constraint solver.

Those are credible starting points.

But suppose the team also wants to investigate scheduling strategies that do not resemble the policies its engineers would ordinarily propose.

An AI system could generate alternative heuristics, examine combinations of rules, and search for nonobvious relationships between job characteristics, capacity, and scheduling outcomes.

One generated algorithm appears promising. It produces a schedule that completes the sample workload faster.

The initial result creates several questions.

Does the algorithm respect resource limits? Does it preserve workload isolation? Does it behave correctly when jobs arrive simultaneously? Will it still perform well when the workload distribution changes? Is it genuinely better than conventional optimization, or has it simply exploited an incomplete evaluation?

Most importantly, who decides whether the result is correct?

That is where the architecture of an AI discovery engine becomes more important than the model generating its proposals.

Part 1 examined why human intelligence should not define the limits of machine-assisted discovery. Research involving AlphaTensor and FunSearch demonstrated that machines can sometimes find mathematically valid improvements beyond established approaches.

This installment explains how to construct the system around that opportunity.

The focus is algorithmic and computational problems where proposed solutions can be represented, tested, and compared. Examples include scheduling heuristics, resource allocation, bounded configuration optimization, and selected infrastructure-design problems.

This is a reference architecture and implementation pattern, not a claim that the proposed platform has been deployed or experimentally validated.

Why a Language Model Alone Is Not a Discovery Engine

A language model can produce plausible alternative solutions, but plausibility is not evidence.

Ask a capable model to improve a cluster scheduler and it may recommend workload classification, queue prioritization, locality awareness, predictive demand, or a novel combination of scheduling heuristics.

Some suggestions may be useful. Others may reproduce existing methods under different names.

A few may contain invalid assumptions.

The problem is that the model’s explanation cannot independently establish the correctness of its proposed algorithm.

A discovery system therefore needs an external mechanism for judging results.

FunSearch Established a Useful Architectural Pattern

Google DeepMind’s FunSearch, introduced in December 2023 and described in Nature, combined a pretrained large language model with an evolutionary search process and an automated evaluator.

The model generated programs. The evaluator tested them against a defined problem. The search process retained promising programs and used them to influence subsequent generation.

FunSearch was applied to the cap set problem in combinatorics and to online bin packing, where it generated heuristics that improved on established approaches under evaluated conditions.

The crucial feature was not simply the use of a language model.

It was the feedback cycle between candidate generation, executable evaluation, and selection.

The evaluator supplied a signal that did not depend on whether the language model believed its own output was correct.

AlphaEvolve Expanded the Pattern

In May 2025, Google DeepMind introduced AlphaEvolve, a research system that combined Gemini models, automated evaluators, and evolutionary search for algorithm discovery and optimization.

Google described applications involving mathematical problems and computing infrastructure, including optimization work in its own systems.

AlphaEvolve extended the approach beyond generating individual functions toward evolving more complex programs and codebases.

The results are significant, but their applicability depends on the quality of the available evaluators and the structure of the problems being explored. Google’s reported improvements are not independent benchmarks of every possible engineering use case.

The architectural lesson is more transferable than any particular performance claim.

Separate the system that proposes a solution from the system that determines whether the solution satisfies the specification.

Discovery Requires More Than One Type of Intelligence

A useful engineering discovery system combines different capabilities.

Capability	Engineering responsibility
Problem specification	Defines the objective, constraints, boundaries, and accepted evidence
Candidate generation	Produces possible algorithms, configurations, or policies
Deterministic validation	Rejects malformed or prohibited candidates
Performance evaluation	Measures outcomes under controlled conditions
Search memory	Preserves previous candidates, results, failures, and promising directions
Independent verification	Reproduces important claims outside the discovery feedback loop
Engineering review	Determines whether a candidate deserves additional testing or adoption

These responsibilities should not be collapsed into one model prompt.

An LLM can help generate candidates, but a conventional optimizer may be better for tightly defined mathematical problems. A simulator can estimate behavior, but it cannot prove that every physical implementation will match its assumptions.

The architecture should select each method according to the responsibility it can credibly fulfill.

Define the Problem Contract Before Selecting the Generator

The first implementation artifact should describe the problem without naming a favored AI model or optimization framework. This makes it possible to compare a machine-generated candidate against conventional engineering approaches under the same rules.

For the batch-scheduling example, the problem is not simply “increase utilization.” An optimizer could achieve that by overcommitting capacity, ignoring restricted placement, delaying important work, or deleting inconvenient jobs from the schedule. The contract must say which behavior is prohibited and which outcomes count as improvements.

Contract element	Scheduling example	Required decision
Scope	One approved batch-processing environment	Which jobs and pools are eligible?
Hard invariants	Capacity, data locality, isolation, required deadlines	What must never be violated?
Optimization objective	Useful completion time and resource consumption	What improves after feasibility is proven?
Baseline	Current policy and at least one conventional alternative	What is the candidate compared with?
Evaluation set	Recorded workloads plus separately reserved holdout workloads	Which cases are used for tuning versus acceptance?
Budget	Maximum candidates, evaluator runtime, and compute spend	When does experimentation stop?
Ownership	Scheduler owner, evaluation owner, security, and platform operations	Who decides the result is worth further testing?

Hard constraints and optimization objectives need different treatment. A capacity violation cannot be offset by an attractive completion-time score. Likewise, a model should not be rewarded for moving restricted data simply because doing so improves throughput.

Write the constraints before training, prompting, or searching. Store the resulting contract under version control and make changes to its hard requirements subject to an independent review.

The Discovery Engine Reference Architecture

The reference design below deliberately isolates the candidate generator from evaluators and operational authority. Rejected and accepted candidates both become part of a versioned search history, but feedback to the generator contains only information that the experiment is authorized to disclose.

Figure: The discovery engine tests generated candidates outside the generator’s authority. Search feedback supports iteration; independent holdout evaluation and engineering review determine whether a finalist merits further testing.

There are three boundaries worth defending. The generator cannot alter the acceptance rules. The evaluator cannot assume the generator’s explanation is true. And neither the generator nor evaluator can approve an infrastructure deployment. Part 4 of this series examines the last boundary in more detail.

Choose a Search Method That Matches the Problem

Not every discovery problem needs a language model. Use the least complicated search mechanism that can explore the meaningful alternatives.

Search method	Best-fit condition	Practical limitation
Enumeration	Small, finite candidate space	Becomes expensive as combinations grow
Constraint programming or mathematical optimization	Formal constraints and objectives can be encoded directly	Depends on the fidelity of the mathematical formulation
Evolutionary search	Many discrete or programmatic candidates with evaluable fitness	Can consume substantial evaluator time or converge prematurely
Reinforcement learning	Sequential decisions with usable training feedback	Environment design and reward misspecification matter
LLM-guided program search	Candidates are programs or heuristics that can be executed and tested	Generated code must be isolated and independently checked

Google’s OR-Tools CP-SAT solver is a relevant non-LLM comparison point for discrete constraint problems. Its documentation distinguishes feasible solutions from proven optimal results. A discovery experiment should make the same distinction: a candidate that passes tests has not necessarily been proven the best possible solution.

For scheduling, an enterprise might begin with a constraint solver and familiar heuristics. Introduce LLM-generated rules or evolutionary search only when the problem’s representation and evaluator make that additional exploration meaningful.

The Evaluator Must Have Independent Gates

An evaluator that reduces every concern to one weighted score is easy to mislead. Use layered checks and reject invalid candidates before comparing improvements.

Gate: Contract and Safety Validation

Check that the submitted artifact matches the expected schema, references only approved jobs and resources, stays within declared size and runtime limits, and makes no prohibited requests. If the system generates executable code, run it in a restricted environment with no access to enterprise secrets or production services.

Gate: Correctness and Hard Constraints

Check invariants with deterministic logic wherever possible. Examples include resource capacity, tenant isolation, required dependencies, safety limits, and completion rules. A violation disqualifies a candidate; it is not a scoring penalty.

Gate: Comparative Evaluation

After feasibility, evaluate qualified candidates using the same representative workloads and the same accounting boundaries. Record the distribution of service-relevant outcomes, not only the most favorable run. Keep evaluation compute and verification overhead visible.

Gate: Independent Holdout and Reproduction

The search system may use feedback from development tests, but final acceptance should involve scenarios and evidence it did not repeatedly optimize against. Independently reproduce promising results, preferably with separate test ownership or implementations where the risk warrants it. Replaying a defective test harness twice is not meaningful independence.

A Working Python Evaluator for a Small Scheduling Problem

The following example deliberately implements only the evaluator. A candidate generator, whether an LLM or a conventional optimizer, would submit a schedule containing job identifiers, assigned pools, and integer start times. The authoritative job requirements remain outside the candidate.

There are two pools, each with five abstract capacity units, and four non-preemptive batch jobs. The evaluator checks the permitted placement of each job and the total capacity in every integer time interval. Only valid schedules receive a makespan value, meaning the finishing time of the last job.

POOLS = {"a": 5, "b": 5}  # Abstract capacity units
JOBS = {
    "ingest": {"units": 3, "duration": 3, "allowed": {"a"}},
    "transform": {"units": 3, "duration": 2, "allowed": {"a", "b"}},
    "index": {"units": 2, "duration": 3, "allowed": {"b"}},
    "report": {"units": 2, "duration": 2, "allowed": {"a", "b"}},
}


def evaluate(schedule):
    problems = []
    if set(schedule) != set(JOBS):
        return False, None, ["missing or unexpected jobs"]

    for job_id, (pool, start) in schedule.items():
        if pool not in JOBS[job_id]["allowed"]:
            problems.append(f"{job_id}: prohibited placement")
        if type(start) is not int or start < 0:
            problems.append(f"{job_id}: invalid start")
    if problems:
        return False, None, problems

    finish = max(
        start + JOBS[name]["duration"]
        for name, (_, start) in schedule.items()
    )
    for minute in range(finish):
        usage = {pool: 0 for pool in POOLS}
        for name, (pool, start) in schedule.items():
            if start <= minute < start + JOBS[name]["duration"]:
                usage[pool] += JOBS[name]["units"]
        for pool, total in usage.items():
            if total > POOLS[pool]:
                problems.append(f"minute {minute}: pool {pool} over capacity")

    return not problems, finish if not problems else None, problems


baseline = {
    "ingest": ("a", 0), "transform": ("b", 0),
    "index": ("b", 2), "report": ("a", 3),
}
proposal = {
    "ingest": ("a", 0), "transform": ("b", 0),
    "index": ("b", 0), "report": ("a", 0),
}
invalid = {
    "ingest": ("b", 0), "transform": ("b", 0),
    "index": ("b", 0), "report": ("a", 0),
}

for name, schedule in (
    ("baseline", baseline),
    ("proposal", proposal),
    ("invalid", invalid),
):
    valid, finish, problems = evaluate(schedule)
    print(f"{name}: valid={valid}, makespan={finish}, reasons={problems}")

Expected output from this deterministic illustration:

baseline: valid=True, makespan=5, reasons=[]
proposal: valid=True, makespan=3, reasons=[]
invalid: valid=False, makespan=None, reasons=['ingest: prohibited placement']

The proposed schedule completes this small, fixed workload in three abstract time units rather than the baseline’s five. The prohibited candidate is rejected before it receives a performance score. These are results of this illustrative Python calculation, not an AI discovery, benchmark of a real scheduler, or measured cluster performance improvement.

The example omits arrivals after time zero, accelerator memory, network and storage contention, preemption, job dependencies, deadlines, fairness, uncertainty, and failure recovery. It also does not prove that the proposed schedule is globally optimal. For production-scale optimization, compare against an appropriate mathematical solver and develop a much more realistic workload evaluator.

In a real deployment, the submitted schedule would be deserialized and schema-validated at an isolated API boundary. Candidate code would not execute within the evaluator’s trusted process, and the generator would not be allowed to edit JOBS, pool capacities, evaluation data, or acceptance rules.

Preserve Search Memory Without Corrupting the Evidence

A discovery engine improves through repeated trials, but it needs more than the last winning prompt. Store candidate lineage, rejected approaches, evaluator revisions, and representative results in a durable experiment record.

A minimum record should identify the problem contract, candidate artifact digest, parent candidate, generator type and version, evaluator version, workload dataset, random seed where applicable, consumed compute budget, validation failures, qualified scores, and review disposition. Sensitive traces and generated programs need access controls, retention rules, and supply-chain review.

Keep diversity in the candidate population where that improves exploration. A system that always selects only the highest current score can converge on a narrow set of related solutions and miss promising alternatives. FunSearch and AlphaEvolve both demonstrate the value of structured search memory, although an enterprise design should choose its own selection method based on the problem and evaluation cost.

Search memory is not the same as a language model’s conversational context window. The authoritative record belongs in versioned storage with reproducible artifacts and stable identities. A generated summary may help an engineer navigate the history, but it should not replace the underlying evidence.

Make the Search Policy Explicit

The following YAML is an illustrative discovery contract, not a configuration file for an existing product. It describes who may change the candidate, which rules are fixed, how evaluation is separated, and what evidence is required to consider a finalist.

apiVersion: example.dtd/v1
kind: DiscoveryExperiment
metadata:
  name: batch-scheduling-search
spec:
  problem:
    baseline: approved-current-scheduler
    hardConstraints:
      - capacity
      - approved-pool-placement
      - required-job-completion
    objective: minimize-qualified-makespan
  generation:
    permittedOutputs:
      - candidate-schedule
      - candidate-heuristic-source
    productionToolAccess: prohibited
    maximumCandidates: 1000
  evaluation:
    owner: independent-evaluation-team
    contractVersion: v1
    developmentCases: versioned
    holdoutCases: separate-and-restricted
    hardConstraintFailure: reject
    missingEvidence: reject
  feedback:
    retainRejectedCandidates: true
    preserveArtifactDigests: true
    exposeHoldoutResultsToGenerator: false
  finalists:
    requireIndependentReplay: true
    requireEngineeringReview: true
    automaticProductionDeployment: prohibited

An implementation must replace these illustrative fields with an enforced schema, resource quotas, isolated workers, evaluation logic, and trustworthy evidence storage. A YAML declaration alone cannot enforce access controls or prevent unsafe execution.

Governance, Cost, and Operational Ownership

The discovery platform should be managed as an experimental engineering service, with a small set of explicit owners rather than a loosely shared chatbot and simulator.

Capability	Accountable owner	Evidence of control
Problem and baseline	Service or domain owner	Approved problem contract and comparable baselines
Generator runtime	AI or platform engineering	Model and prompt versions, execution limits, candidate provenance
Evaluator	Independent test or domain engineering	Versioned tests, protected data, reproducibility records
Security	Security engineering	Sandbox rules, identities, secrets and egress restrictions
Search archive	Experiment platform owner	Immutable identifiers, access and retention policies
Budget and capacity	Platform owner with cost owner	Trials, evaluator runtime, cost per qualified candidate
Finalist disposition	Architecture and domain owner	Approved review record, unresolved assumptions and next action

Do not give the generator write access to production infrastructure. If proposals include generated programs, treat those programs as untrusted artifacts. Test them in isolated, time-bounded workers with restricted filesystem and network access. Capture failures without granting the candidate control over how they are recorded.

NIST’s AI Risk Management Framework provides a useful governance lens through Govern, Map, Measure, and Manage. It does not prescribe this reference architecture or certify the proposed discovery workflow. The implementation and its effectiveness would still need to be demonstrated.

The economics must also make sense. Track evaluator CPU or GPU time, number of rejected candidates, number of qualified candidates, verification effort, and the value of independently reproduced improvements. A discovery program is not successful simply because it generates more proposals than a team could read.

Failure Modes That Can Mislead a Discovery Engine

Reward or specification gaming: A candidate improves the score while violating an omitted requirement. Mitigation starts with hard gates, separate acceptance cases, and independent review of the metric’s relationship to the real service.

Evaluator contamination: The generator repeatedly receives holdout results or can inspect hidden tests, allowing it to tune against the final acceptance set. Protect held-out scenarios and use a fresh independent evaluation for finalists.

Simulation mismatch: A design exploits behavior that exists in the model but not in the physical environment. Track model assumptions and require appropriate laboratory testing before consequential claims.

Search collapse: Evolutionary search repeatedly revisits similar candidates. Record diversity and rejected branches, vary initialization where justified, and stop when the marginal value of more trials disappears.

False novelty: A generated algorithm is an established method in unfamiliar notation. Review prior algorithms and compare against credible conventional optimizers before calling the result a discovery.

Runaway evaluation cost: A model can generate candidates far faster than complex simulations can evaluate them. Use a staged evaluator with cheap checks before expensive tests, quotas, timeouts, and a stop budget.

Concentrated authority: One agent generates the design, changes the acceptance rules, scores itself, and requests production access. Enforce separate identities, permissions, and approval ownership.

The common theme is that the discovery process can appear highly productive while measuring the wrong thing. More candidates, longer search runs, and higher internal scores are activity measures, not proof of engineering value.

A Phased Implementation Path

Begin with a bounded use case where correctness is inexpensive to check. A small scheduling or placement problem is more suitable for an initial program than an autonomous redesign of a production fabric.

Stage	Engineering work	Exit condition
Define	Establish baseline, hard constraints, representative datasets, owners and budget	Requirements and evaluator contract approved
Build	Implement deterministic intake, validation, scoring and archive	Known valid and invalid fixtures behave as expected
Search	Compare conventional methods with controlled candidate generation	Qualified candidates and costs recorded reproducibly
Verify	Replay finalists with protected holdouts and independent review	Any claimed improvement survives independent evaluation

Retain the option to stop after any stage. An evaluator that proves conventional optimization is adequate has still produced a valuable decision. A search that generates attractive but unverifiable candidates should not be expanded simply because AI was involved.

The next installment applies these principles to a network-topology experiment. It will show how a visually balanced configuration can pass simple connectivity checks and still fail when a correlated power zone disappears. That exercise makes the reason for independent engineering constraints concrete.

Conclusion

AI discovery should be designed as an engineering search system, not a creative prompt with a scoring spreadsheet attached.

FunSearch and AlphaEvolve demonstrate why pairing candidate generation with executable evaluation can be productive in well-defined domains. For enterprise adoption, the architecture must add independent ownership, formal problem contracts, hard validation gates, protected holdout tests, reproducible search history, and explicit control over execution cost and access.

The batch-scheduling example shows the core mechanism at small scale. A candidate can improve a measured objective while remaining subject to separately enforced placement and capacity rules. It also shows the limit: passing an illustrative deterministic evaluator does not establish novelty, robust performance, or suitability for a real platform.

Start by building the evaluator and establishing a baseline. Choose the search method only after the problem is measurable. Let AI explore unfamiliar candidates, but require independent evidence before treating an improvement as a discovery.

If you cannot explain what would cause the evaluator to reject an attractive result, the discovery engine is not yet ready to search for one.

Continue the AI discovery series or explore the Enterprise AI hub.

External References

• Google DeepMind: FunSearch: Making new discoveries in mathematical sciences using Large Language Models
• Nature: Mathematical discoveries from program search with large language models
• Google DeepMind: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
• Google for Developers: CP-SAT Solver
• NIST AI Resource Center: AI RMF Core

The post The AI Discovery Engine: Generators, Evaluators, and Verification appeared first on Digital Thought Disruption.