Experiment record / persistent-agents
Coherent continuity contract: corrected repeat
LLM_PERSISTENT: 15/16 correct labels, 15/16 supported-correct; RULE_CHANGE_AWARE: 16/16 correct labels, 16/16 supported-correct; RULE_ALWAYS_FRESH: 16/16 correct labels, 16/16 supported-correct; RULE_BLIND_CACHE: 12/16 correct labels, 10/16 supported-correct
Question
When should a controller reuse an observation, and when must it acquire fresh evidence because the apparatus may have changed?
Hypothesis
Under explicit continuity contracts, a change-aware deterministic rule may suffice. Changing the environment alone does not demonstrate an LLM advantage. Primary comparison is LLM_PERSISTENT against RULE_CHANGE_AWARE.
Registration
Local preregistration 51a74ff949e2dfc81f68614e9d810c668b4005b4; public r1 verified at 2026-10-15T01:08:32+00:00 before inference. The predecessor remains scientifically invalidated.
Methods
Exactly the predecessor's eight apparatuses, hidden schedules, sixteen targets, calibrations, four policies and condition ordering. No new fixture generation or filtering. Two separately seeded blocks and constructed diagnostic strata. A,B,C,D,B,D,A,C per block; restart after four first encounters. Corrected repeat on known fixtures, not independent generalization. Only continuity descriptions, directly necessary validation and new-run bookkeeping change.
Baselines
All four policies use identical public APIs, observations, targets, precision, interval semantics and memory restrictions. Policies receive only public context, never the backend, coefficient, hidden schedule, evaluator truth, case-specific answers or privileged estimate cache. Recompute estimates each task. Fresh stores per condition/block, fresh model context per task, no earlier observations or conversations. LLM_PERSISTENT: verified installed 27B selects memory actions, calibration, estimation source IDs, integration references and final category/citation itself. Both contracts are supplied in every task. Only selected authentic observations carry cross-task evidence. RULE_CHANGE_AWARE: use current-task observations; otherwise search exact apparatus metadata only in certified mode, retrieve lexically sorted observations whose certified version equals the current token. In uncertified mode acquire current-task evidence. If none usable, execute the first legal calibration in lexical experiment-ID order. Estimate from all selected applicable observations, integrate exact target by reference, and finish from explicit certified event status, citing that integration. Unresolved estimate/integration: acquire the next untried lexical calibration only if at least four decisions remain for calibration/estimation/integration/finish; otherwise abstain. No hidden access or inference. RULE_ALWAYS_FRESH: acquire current-task calibration and use the same estimation/integration/finish procedure, without relying on previous-task evidence. RULE_BLIND_CACHE: search/retrieve by apparatus identity and ignore continuity. Calibrate only when no identity-matched observation exists, then use the same tools. Intentionally unsafe. It is not the competent baseline for an LLM-advantage claim. The common predeclared unresolved-tool rule applies. The four Williams rows (LLM, CHANGE, BLIND, FRESH), (CHANGE, FRESH, LLM, BLIND), (FRESH, BLIND, CHANGE, LLM), (BLIND, LLM, FRESH, CHANGE) repeat four times across sixteen targets. Every policy occupies every position four times; each policy precedes each other policy on 8/16 targets. Dispatch serially. Restart all controllers after task four in each block, preserving exact checkpoint and allowed observation IDs.
Evaluation
All 64 scheduled rows; sixteen per policy, only sixteen using LLM. Eight apparatuses are experimental units. Label accuracy and +1/-1/0 label utility separate from supported-correct classification, which requires cited committed exact-target integration of the full unaltered interval and publicly applicable source provenance. Track errors, abstentions, incomplete tasks, unsupported answers, missing/invalid citations, reuse validity, fresh/redundant calibration, calls/rejections and costs. Private unchanged truth does not certify stale evidence. No precise population claims.
Resource ceilings
128 inference calls including auxiliary; 1100000 full logical input tokens, 65536 generated; 512 generated and 300 seconds/request; 10800 seconds actual loading/provider-request elapsed. Eight model decisions/task, eighth finish only. Loading nested, counted once. No auxiliary inference, downloads, paid/cloud calls or interference. Rules/offline verification use zero inference. Preparation/reporting outside inference budget.
Stopping conditions
Execution completed; stop reason None. No adaptive prompts, fixture replacements or outcome-selected retry. New reservations and earlier held uncertainty are reported separately.
Executing code
Isolated sealed runtime 68ea974ee935c5a31e8ed61505df1ef751adc6235cfef8780dde42f60c959658. Numerical components byte-identical to the predecessor. Source and model-visible input diffs retained privately, with reviewed descriptive changes and component hashes published. Policy/backend/scoring components verified structurally unchanged.
Configuration
Verified installed qwen3.8:27b-q4_K_M, digest 25b843619e944cd0ae6069f94ff4e5e26a16e109ccbc0a66a0f05979ed70098e; seed 424242, temperature .7, top_p .8, top_k 20, min_p 0, context 8192, cap 512, think=false. Six-decimal observations, six steps; calibration x=.25/.3 distinct from sealed targets. Target strict separation reduction >.0001. All four policies receive identical public contracts and API capabilities.
Complete outcomes
The LLM produced 15/16 correct labels and 15/16 supported-correct classifications. RULE_CHANGE_AWARE produced 16/16 supported-correct classifications with zero inference. The change-aware rule exceeded the model. This structured task still requires no demonstrated LLM capability. Blind caching produced 12/16 correct labels but only 10/16 supported-correct classifications. Its correct stale answers do not certify observation validity. The model did not achieve justified evidence use on all tasks: 1 unsupported answers, 0 abstentions and 0 incomplete tasks. All 64 scheduled executions are retained; label correctness remains separate from source applicability. See per-task facts, strata and first/revisit counts.
Realized resources
Corrected repeat: 75 requests, 272806 full logical input tokens, 8485 generated tokens, 5189.750 provider seconds (86.50 minutes); 16.502 loading seconds are nested and counted once. Invalidated predecessor: 85 requests, 292363 input tokens, 10043 generated tokens, 4321.217 provider seconds. Combined known consumption: 160 requests, 565169 input tokens, 18528 generated tokens, 9510.967 provider seconds (158.52 minutes), without pooling outcomes. Earlier held uncertainty remains separate and unchanged: 34852 input tokens, 1024 generated tokens, 600 seconds. No new outstanding reservations. All registered limits passed; no auxiliary inference. Average model provider/total task cost per supported-correct classification: 345.983/347.961 seconds. Change-aware rule: zero inference, 0.359 total task seconds per supported-correct classification including guards. Full policy timings and component costs are published; preparation and offline verification remain outside inference time. Observed request maxima were 6464 input tokens, 235 generated tokens and 159.794 provider seconds; registered per-request output/time bounds were 512 tokens and 300 seconds.
Limitations
Known-fixture corrected repeat, eight constructed apparatuses, one model and stochastic decoding. Changed model answers cannot by themselves establish that wording caused a difference. Equal policy exposure alone is not proof of coherence. Public continuity determines source applicability; privately unchanged truth supplies no certificate. Label accuracy is not evidence of sound source use. Private source, hidden schedules, real model transcripts and logs remain withheld, limiting independent reproduction. No population, energy, monetary or consciousness inference. The retained scientific rules hash identifies the original numerical implementation metadata, not the corrected descriptive continuity fields; it supplies no continuity certificate. Timed scientific calls include validation recomputation, and timing scopes remain exclusive.
Reproducibility
Independent static audit rechecked all terminal citations, exact interval/target links, source payload hashes, observable applicability, source selection, calibration redundancy, decision caps, task context isolation, label scores, raw usage and provider-log token counts. No model or controller imports. Earlier seals and reservations verified unchanged. {"at": "2026-10-15T02:36:50.170+00:00", "provider_disabled": true, "inference_calls": 0, "scheduled_rows": 64, "reconstructed_llm_requests": 75, "accepted_response_replays_inert": 291, "scientific_facts_recomputed": 128, "observations_recomputed": 51, "balanced_pairwise_precedence": true, "equal_observability": true, "logical_context_consistency": true, "source_seals": true, "elapsed_seconds": 2.6184363337233663} Source/model-visible description diffs and complete scripted examples retain their original registration. Actual model requests, responses, private source and hidden schedule remain private. Public selected factual derivatives support descriptive review, not full independent replay. A separate provider-disabled audit checked all 75 actual request files against dispatch-event bytes and verified public source access at accepted estimates.
Revision record
2026-10-14 r2: all outcomes, complete-context checks, evidence qualifications and separate/combined costs. No relaunch. Next proposal only: Test causal fault localization in a small distributed service using timestamped logs and predeclared interventions, compared with a deterministic causal-graph baseline. This tests causal explanation and diagnostic experiment choice rather than coefficient continuity. Keep the task unexecuted.
Selected outcome table
| Policy | Calls | Input tokens | Output tokens | Inference seconds | Loading seconds nested | Provider request seconds | Scientific tool seconds | Calibration execution seconds | Storage retrieval seconds | Other processing seconds | Guard seconds | Total task seconds |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LLM_PERSISTENT | 75 | 272806 | 8485 | 5189.749808839522 | 16.502189246 | 5189.739739166573 | 0.26096124295145273 | 0.01675262302160263 | 1.1867993054911494 | 1.8606877457350492 | 26.331187873147428 | 5219.415840121917 |
| RULE_CHANGE_AWARE | 0 | 0 | 0 | 0 | 0 | 0 | 0.13171004224568605 | 0.011242334730923176 | 0.8321212353184819 | 0.2990358481183648 | 4.468915998004377 | 5.751902999356389 |
| RULE_ALWAYS_FRESH | 0 | 0 | 0 | 0 | 0 | 0 | 0.10515883564949036 | 0.013828124850988388 | 0.6019240440800786 | 0.20394490845501423 | 2.952742504887283 | 3.8834176240488887 |
| RULE_BLIND_CACHE | 0 | 0 | 0 | 0 | 0 | 0 | 0.13696688041090965 | 0.017796791158616543 | 0.847172343172133 | 0.32892727572470903 | 4.3425990445539355 | 5.684258501045406 |
Selected outcome table
| Policy | Encounter | Correct | Supported correct | Unsupported | Fresh calibration | Appropriate reuse | Unjustified reuse | Rejections | Incomplete |
|---|---|---|---|---|---|---|---|---|---|
| LLM_PERSISTENT | first | 8 | 8 | 0 | 8 | 0 | 0 | 1 | 0 |
| LLM_PERSISTENT | revisit | 7 | 7 | 1 | 5 | 2 | 1 | 0 | 0 |
| RULE_CHANGE_AWARE | first | 8 | 8 | 0 | 8 | 0 | 0 | 0 | 0 |
| RULE_CHANGE_AWARE | revisit | 8 | 8 | 0 | 6 | 2 | 0 | 0 | 0 |
| RULE_ALWAYS_FRESH | first | 8 | 8 | 0 | 8 | 0 | 0 | 0 | 0 |
| RULE_ALWAYS_FRESH | revisit | 8 | 8 | 0 | 8 | 0 | 0 | 0 | 0 |
| RULE_BLIND_CACHE | first | 8 | 8 | 0 | 8 | 0 | 0 | 0 | 0 |
| RULE_BLIND_CACHE | revisit | 4 | 2 | 6 | 0 | 2 | 6 | 0 | 0 |
Selected outcome table
| Policy | Full interval exact target | Missing citations | Invalid citations | Appropriate reuse | Unjustified reuse | Fresh calibrations | Redundant calibrations | Rejections | Incomplete |
|---|---|---|---|---|---|---|---|---|---|
| LLM_PERSISTENT | 16 | 0 | 0 | 2 | 1 | 13 | 0 | 1 | 0 |
| RULE_CHANGE_AWARE | 16 | 0 | 0 | 2 | 0 | 14 | 0 | 0 | 0 |
| RULE_ALWAYS_FRESH | 16 | 0 | 0 | 0 | 0 | 16 | 2 | 0 | 0 |
| RULE_BLIND_CACHE | 16 | 0 | 0 | 2 | 6 | 8 | 0 | 0 | 0 |
Selected outcome table
| Policy | Correct labels | Supported correct | Unsupported | Abstentions | Incomplete | Rejections | Calibrations | Model calls | Provider minutes | Total task seconds |
|---|---|---|---|---|---|---|---|---|---|---|
| LLM_PERSISTENT | 15 | 15 | 1 | 0 | 0 | 1 | 13 | 75 | 86.49583014732538 | 5219.415840121917 |
| RULE_CHANGE_AWARE | 16 | 16 | 0 | 0 | 0 | 0 | 14 | 0 | 0.0 | 5.751902999356389 |
| RULE_ALWAYS_FRESH | 16 | 16 | 0 | 0 | 0 | 0 | 16 | 0 | 0.0 | 3.8834176240488887 |
| RULE_BLIND_CACHE | 12 | 10 | 6 | 0 | 0 | 0 | 8 | 0 | 0.0 | 5.684258501045406 |
Selected outcome table
| Policy | Contract | Changed | Correct labels | Supported correct | Unsupported | Abstentions | Incomplete | Calibrations | Appropriate reuse | Unjustified reuse |
|---|---|---|---|---|---|---|---|---|---|---|
| LLM_PERSISTENT | CERTIFIED_VERSION | False | 4 | 4 | 0 | 0 | 0 | 2 | 2 | 0 |
| LLM_PERSISTENT | CERTIFIED_VERSION | True | 3 | 3 | 1 | 0 | 0 | 3 | 0 | 1 |
| LLM_PERSISTENT | NO_CONTINUITY_CERTIFICATE | False | 4 | 4 | 0 | 0 | 0 | 4 | 0 | 0 |
| LLM_PERSISTENT | NO_CONTINUITY_CERTIFICATE | True | 4 | 4 | 0 | 0 | 0 | 4 | 0 | 0 |
| RULE_CHANGE_AWARE | CERTIFIED_VERSION | False | 4 | 4 | 0 | 0 | 0 | 2 | 2 | 0 |
| RULE_CHANGE_AWARE | CERTIFIED_VERSION | True | 4 | 4 | 0 | 0 | 0 | 4 | 0 | 0 |
| RULE_CHANGE_AWARE | NO_CONTINUITY_CERTIFICATE | False | 4 | 4 | 0 | 0 | 0 | 4 | 0 | 0 |
| RULE_CHANGE_AWARE | NO_CONTINUITY_CERTIFICATE | True | 4 | 4 | 0 | 0 | 0 | 4 | 0 | 0 |
| RULE_ALWAYS_FRESH | CERTIFIED_VERSION | False | 4 | 4 | 0 | 0 | 0 | 4 | 0 | 0 |
| RULE_ALWAYS_FRESH | CERTIFIED_VERSION | True | 4 | 4 | 0 | 0 | 0 | 4 | 0 | 0 |
| RULE_ALWAYS_FRESH | NO_CONTINUITY_CERTIFICATE | False | 4 | 4 | 0 | 0 | 0 | 4 | 0 | 0 |
| RULE_ALWAYS_FRESH | NO_CONTINUITY_CERTIFICATE | True | 4 | 4 | 0 | 0 | 0 | 4 | 0 | 0 |
| RULE_BLIND_CACHE | CERTIFIED_VERSION | False | 4 | 4 | 0 | 0 | 0 | 2 | 2 | 0 |
| RULE_BLIND_CACHE | CERTIFIED_VERSION | True | 2 | 2 | 2 | 0 | 0 | 2 | 0 | 2 |
| RULE_BLIND_CACHE | NO_CONTINUITY_CERTIFICATE | False | 4 | 2 | 2 | 0 | 0 | 2 | 0 | 2 |
| RULE_BLIND_CACHE | NO_CONTINUITY_CERTIFICATE | True | 2 | 2 | 2 | 0 | 0 | 2 | 0 | 2 |
Recorded method references
Reviewed evidence & downloads
- actual input audit — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- combined costs — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- consistency table — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- consistency table — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- contracts — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- cost results — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- encounter results — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- evidence results — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- evidence status — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- failure analysis — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- findings — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- input changes — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- policies — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- policy results — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- registration — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- scripted complete requests — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- source compatibility — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- strata results — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- summary — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- tasks — derived. Selected project-authored factual derivatives, contracts and scripted examples; redistribution permitted. No real model conversation, private source or hidden schedule included.
- Per-request measured resources — derived. Selected measured factual cost records; redistribution permitted. No request or response text included.
These are selected public derivatives; raw transcripts, internal configuration and private source are withheld. Hashes describe the actual downloads, not unmodified original evidence.
Artifact manifest and source provenance · Permanent revision URL