config: profile system (baseline / low-token / security / max-build / all-on)#291
Merged
Conversation
A profile is a small JSON patch that gets merged into TConfig BEFORE
the operator's config.json is applied. Five built-ins ship; operators
can drop more at \$PASCLAW_HOME/profiles/<name>.json.
Five built-in profiles:
baseline Everything off. A/B reference -- "how does pasclaw behave
with no features enabled?"
low-token Condenser, output cap 8KB, MEMORY task-aware slicing,
prompt cache, progressive disclosure, auto-router.
security Workspace restriction + shell deny + private-network
block, promptware scan, no web_fetch / vault,
agent-authored skill writes staged for approval.
max-build _inherits: ["low-token"] plus web_fetch, vault, vector
search, checkpoints, stats, 16KB cap, 1h prompt cache,
all four self-improving skill switches.
all-on _inherits: ["max-build"]. Every boolean knob flipped on.
Selection precedence (highest wins):
1. pasclaw <cmd> --profile <name> per-invocation
2. PASCLAW_PROFILE env var process scope
3. "profile": "<name>" field in config.json persistent default
4. None today's behaviour
Layering: TConfig.Create defaults -> profile (with _inherits chain)
-> operator config.json. Each layer is applied via FromJSON, which is
merge-style -- every field defaults to the current TConfig value so
unset fields preserve the lower layer and set fields override. The
operator's explicit config.json always wins so an operator can pick
max-build and still pin one field their own way.
Inheritance: a profile may declare "_inherits": ["other"]. Ancestors
apply first in order, then the current profile. Cycles are rejected;
depth is capped at 4. The meta keys (_description / _inherits / _name)
are left in the body -- TConfig.FromJSON ignores unknown keys so
`pasclaw profile show` reuses the same blob unchanged.
User shadow: a file at $PASCLAW_HOME/profiles/<name>.json with the
same name as a built-in wins, mirroring the skills loader's convention.
New unit and command:
PasClaw.Config.Profile ListAvailableProfiles, LookupProfile,
ResolveProfileBodies (recursive _inherits
resolver + cycle guard), ExtractProfileField
(peeks "profile" out of a config blob).
PasClaw.Cmd.Profile `pasclaw profile {list,show,use}`. diff
and bench are Stages C-rest / D, follow-up.
Wired into LoadConfig as a new overload `LoadConfig(ProfileOverride)`;
the parameterless version delegates with '' so existing callers don't
change. CLI surfaces (agent/tui/gateway/serve) take a --profile flag,
each threading the resolved name down to LoadConfig.
Tests (test-config-profile):
- catalogue contains all five built-ins, each with a description
- LookupProfile + ResolveProfileBodies handle base + _inherits +
unknown name + the parent-first ordering
- user-shadow: $PASCLAW_HOME/profiles/<name>.json wins over the
built-in with the same name
- ExtractProfileField: present / absent / malformed / empty cases
- End-to-end LoadConfig: profile applies, operator config.json
wins over the profile, config.json "profile" field is honoured
- LoadConfig with max-build: tool_output_cap=16384 (override wins
over the parent's 8192), orient_task_aware inherited from
low-token, progressive_disclosure inherited, self_manage set
by max-build
The test uses PASCLAW_HOME via the Makefile target wrapper (mktemp
+ trap rm) so it touches no real state. fpSetenv isn't reliable across
FPC versions; the env-var-in-shell pattern is simpler.
Docs: docs/configuration.md gains a "Profiles" section covering the
five built-ins, precedence, layering rules, the CLI commands, and the
user-shadow pattern.
Out of scope (separate PRs):
- profile diff <a> <b>
- profile bench --task ... --profiles ... -- the A/B test harness
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f82ce30ce6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+1818
to
+1823
| ProfileName := ProfileOverride; | ||
| if ProfileName = '' then | ||
| ProfileName := GetEnvironmentVariable('PASCLAW_PROFILE'); | ||
| if ProfileName = '' then | ||
| ProfileName := ExtractProfileField(S); | ||
| if ProfileName <> '' then |
There was a problem hiding this comment.
When config.json is absent, this entire profile-selection block is skipped because it sits under if FileExists(Path), so pasclaw agent --profile low-token ... and PASCLAW_PROFILE=low-token silently use plain defaults instead of the requested profile. The config-file profile field needs the file body, but the CLI/env cases can be resolved before or outside this file-exists branch.
Useful? React with 👍 / 👎.
| end; | ||
| if Cfg = nil then Cfg := TJsonObject.Create; | ||
| try | ||
| Cfg.PutStr('profile', Argv[1]); |
There was a problem hiding this comment.
profile use writes a top-level profile key, but LoadConfig never stores that value in TConfig and SaveConfig rewrites the file from C.ToJSON, which has no corresponding profile emission. After selecting a persistent profile, any config-mutating command that calls SaveConfig (for example auth login, model set, or settings writes) will drop this key and the next startup will no longer apply the profile.
Useful? React with 👍 / 👎.
Comment on lines
+302
to
+304
| Path := UserProfilePath(HomeDir, Name); | ||
| if FileExists(Path) then | ||
| begin |
There was a problem hiding this comment.
With the documented fork pattern of $PASCLAW_HOME/profiles/low-token.json containing "_inherits":["low-token"], this lookup returns the user file for both the child and the parent name. The resolver then sees low-token twice and reports a cycle instead of loading the built-in as the base layer, so same-name user shadows cannot actually fork built-ins as advertised.
Useful? React with 👍 / 👎.
…self-shadow inherit Three Codex P2 findings, all legit. P2 #1 -- the profile-selection block sat under `if FileExists(Path)` so `pasclaw agent --profile low-token` and `PASCLAW_PROFILE=low-token` silently used plain defaults on a fresh deploy that hadn't created config.json yet. Moved profile resolution OUT of the FileExists guard so it runs unconditionally; config.json is then layered on top when it exists. The third-tier "profile field in config.json" fallback is still gated on HasConfigFile (only meaningful when the file exists). P2 #2 -- `pasclaw profile use <name>` wrote the "profile" key by hand-crafting JSON, so any subsequent SaveConfig() (auth login, model set, /v1/config PUT, the TUI's settings tab, ...) would drop the key because TConfig didn't know about it. Added Profile: string field to TConfig that round-trips through FromJSON / ToJSON. DoUse now goes through LoadConfig + SetField + SaveConfig like every other config-mutating command. P2 #3 -- the documented "fork a built-in" pattern -- $PASCLAW_HOME/profiles/low-token.json containing `{"_inherits":["low-token"], "tool_output_cap":4096}` -- false-cycled. LookupProfile returned the user file for both the child AND its inherited parent (because both lookups asked for "low-token"), and the resolver then saw the same name twice in Visited and reported a cycle. Added a LookupProfileInternal overload with a `BypassUserShadow` switch + threaded NameToBypass through ResolveInto. When a user-shadow's _inherits contains its own name, the recursive parent lookup bypasses the user file and reaches the built-in directly. Visited is briefly popped + re-added across the recursive call so the cycle guard doesn't false-fire. Tests: TestProfileWithoutConfigFile P2 #1 TestSaveConfigPreservesProfile P2 #2 (round-trip + SaveConfig preserves through a second mutation) TestSelfShadowInherit P2 #3 (verifies two-layer resolution: built-in first, user override second) All existing test-config-profile, test-config-secret-merge, test- config-env-subst, test-plan-build-mode, test-loop-shaping-defaults remain green. Live-verified each fix end-to-end via the CLI.
Two follow-ups to PR #291's profile system. 1) Adds a sixth built-in profile, `stock`, that mirrors TConfig.Create's exact defaults. Applying it is a no-op (every GetX call in FromJSON sees its declared value matching the current field), but it exists so the no-profile fresh-install state is explicit and inspectable: pasclaw profile show stock # see what `pasclaw agent` does # with no --profile / no # PASCLAW_PROFILE / no config.json # profile field set. The test-config-profile suite gains TestStockMatchesDefaults which loads `stock` and compares every relevant field against TConfig. Create. A future drift between the two -- e.g. a flag default getting flipped in TConfig.Create without the same change to the stock profile -- fails the test. 2) Adds PromptStarterProfile to the onboarding wizard, sitting at the top of the loop-shaping prompt block. The operator picks one of the six profiles (1=stock, 2=baseline, ..., 6=all-on) or option 7 to skip. When they pick a profile: - the profile body chain is applied to Cfg via FromJSON - Cfg.Profile := <name> so SaveConfig persists it - the per-feature loop-shaping prompts that follow (PromptWebFetch / PromptPromptware / PromptCondenseReversible / PromptToolOutputCap / PromptOrientTaskAware / PromptSelfImprovingSkills) are SKIPPED, because the profile already encapsulates those choices. Other prompts (VaultTools / Vector / KB / Stats / Checkpoints / ShellBackend / Heartbeat / AutoRouter / MCP installs) still run unconditionally -- those touch operator-environment concerns the profile doesn't cover (e.g. whether to fetch the MiniLM weights for vector search, which docker image to use, ...). Picking option 7 (default) preserves the pre-PR-#291 flow exactly: no profile written, every per-feature prompt fires. Docs (docs/configuration.md) updated with the `stock` row and a new "Onboarding" subsection describing the picker behaviour. Verified live: - `profile list` shows six profiles, stock first - `profile show stock` prints a single layer mirroring defaults - `profile use stock` writes profile=stock; status differs zero from a no-profile install - onboard pick=security writes profile=security AND skips the loop-shaping prompts - onboard pick=7 runs every loop-shaping prompt and leaves the profile field absent
FMXExpress
pushed a commit
that referenced
this pull request
Jun 16, 2026
Completes the profile-system plan: PR #291 shipped Stages A, B, list/ show/use and the onboarding picker (Stage E). This commit adds the two missing subcommands. `pasclaw profile diff <a> <b>` Applies each profile (with _inherits resolved) against a fresh TConfig in isolation, then walks a hand-curated list of comparable fields (loop-shaping booleans, tool_output_cap, sandbox, self- improving skills, prompt cache, auto-router) and prints rows where the two sides disagree. Suppresses rows where they agree. Output: $ pasclaw profile diff baseline max-build field baseline max-build vault_tools_enabled false true web_fetch_enabled false true condense_reversible false true tool_output_cap 0 16384 self_improving_skills.self_manage false true ... (16 differing field(s)) Why a hand-listed field set instead of a JSON walk of ToJSON output: ToJSON is "tidy" -- it suppresses fields matching their default, which would make the comparison ambiguous (no row could mean "same value" OR "both at default"). Walking live TConfig fields after the profile chain is applied is unambiguous. `pasclaw profile bench --task "<prompt>" --profiles <a,b,c> [--runs N]` For each (profile, run) pair, spawns this same binary as pasclaw agent --profile <p> --quiet \ --session bench-<p>-<n>-<ts> -m "<task>" via PasClaw.Platform.RunOneShot. After every run, reads the persisted session JSON to harvest per-turn stats (TSessionStats: input/output tokens, cache reads, turn count, tool-call count). Aggregates per profile across runs, prints a comparison table: bench summary profile runs err wall(s) in_tok out_tok turns t_call baseline 3 0 11.2 1240 380 4 7 low-token 3 0 9.8 820 360 4 7 max-build 3 0 12.4 910 420 5 9 Token / turn / tool-call columns require stats_collection_enabled to be on in the active profile (max-build / all-on have it; the others don't by default). Wall time and exit codes work regardless. Validates every profile name BEFORE running so a typo doesn't burn provider calls. --runs defaults to 3; --task and --profiles are required. The user's existing config.json (provider keys, etc.) is reused; --profile is layered on top per spawn. Not a benchmark in the academic sense -- no statistical-significance testing, no variance bounds. A "show me concrete numbers on this task across these profiles" comparison harness for eyeballing. --judge for an LLM-graded quality column is a future follow-up (Ralph-loop pattern from PR #223). Tests: src/tests/config_profile_tests.pas gains TestDiffMaterial, asserting the invariant the diff command depends on (baseline vs max-build differ on at least one tracked field; stock vs no-profile match on the tracked subset). DoDiff's table layout is exercised by smoke; DoBench is shell-spawn-bound so left for a manual / staging exercise. docs/configuration.md updated with diff + bench in the CLI block and a short "profile diff and profile bench" subsection explaining the mechanics and caveats. This finishes the original 6-stage profile-system plan modulo --judge.
This was referenced Jun 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds configuration profiles — Stages A + B + the foundation of C from the prior planning thread. A profile is a small JSON patch merged into
TConfigbeforeconfig.jsonis applied. Five built-ins ship, operators can drop more at$PASCLAW_HOME/profiles/<name>.json.This unlocks the comparison you actually asked about:
pasclaw agent --profile baseline -m "task"vspasclaw agent --profile max-build -m "task", with everything you've shipped on one side and nothing on the other.The five built-ins
baselinelow-tokensecurityweb_fetch/ vault, agent-authored skills stage for approval.max-build_inherits: ["low-token"]plusweb_fetch, vault, vector search, checkpoints, stats, 16 KB cap, 1 h prompt cache, all four self-improving-skill switches.all-on_inherits: ["max-build"]. Every boolean knob flipped on. Surface-area testing only.Selection precedence (highest wins)
pasclaw <cmd> --profile <name>— per-invocation.PASCLAW_PROFILE=<name>env var — process scope."profile": "<name>"field inconfig.json— persistent default (write viapasclaw profile use <name>).TConfig.Createdefaults flow straight through.Layering
TConfig.Createdefaults → profile (with_inheritschain) → operatorconfig.json. Each layer is applied viaFromJSON, which is merge-style (everyGetXcall defaults to the currentTConfigvalue), so unset fields preserve the lower layer and set fields override. The operator's explicitconfig.jsonalways wins, so you can pickmax-buildand still pin one field your own way._inheritsis a list — ancestors apply first in declared order, then the current profile. Cycles rejected, depth capped at 4. Meta keys (_description/_inherits/_name) stay in the body becauseFromJSONignores unknown keys, which letspasclaw profile showreuse the same blob.User shadow
A file at
$PASCLAW_HOME/profiles/<name>.jsonwith the same name as a built-in wins, mirroring the skills loader's convention. Useful for forking a built-in:{ "_description": "low-token but with a smaller cap", "_inherits": ["low-token"], "tool_output_cap": 4096 }CLI
--profile <name>is wired into all four CLI surfaces:agent,tui,gateway,serve.Files
src/pkg/config/PasClaw.Config.Profile.pas—TProfileSpec,ListAvailableProfiles,LookupProfile,ResolveProfileBodies(recursive_inheritsresolver + cycle guard),ExtractProfileField, the five built-in JSON blobs embedded as Pascal string constantssrc/pkg/config/PasClaw.Config.pas— newLoadConfig(ProfileOverride: string)overload; the parameterlessLoadConfigdelegates so every existing caller still workssrc/cmd/PasClaw.Cmd.Profile.pas—pasclaw profile {list, show, use}subcommandsrc/cmd/PasClaw.Cmd.Root.pas— register the subcommandsrc/cmd/PasClaw.Cmd.{Agent,TUI,Gateway,Serve}.pas—--profile <name>flag, threaded intoLoadConfigTest plan
make test-config-profile— new test (src/tests/config_profile_tests.pas):LookupProfile+ResolveProfileBodieshandle base case,_inherits, unknown name, and parent-first orderingmax-buildresolves to two layers (low-token then max-build),all-onto three (low-token → max-build → all-on)$PASCLAW_HOME/profiles/<name>.jsonwins over a built-in with the same nameExtractProfileField: present / absent / malformed / empty bodyLoadConfig: profile fields apply; an operatorconfig.jsonvalue wins over the profile; theconfig.json"profile" field is honoured when no CLI flag is setmax-buildround-trip:tool_output_cap=16384(override wins over the parent's 8192),orient_task_awareinherited from low-token,progressive_disclosureinherited,self_manageset by max-buildmake clean && make— full FPC 3.2.2 build cleantest-config-secret-merge,test-config-env-subst,test-plan-build-mode,test-loop-shaping-defaultsall greenpasclaw profile listshows all five;profile showprints layers;profile use securitywrites{"profile":"security"}intoconfig.json; unknown names return a clean✗ profile "X" not found;agent --profile low-tokenlogs[info] config: profile "low-token" applied (1 layer(s))The test uses
PASCLAW_HOMEvia the Makefile target wrapper (mktemp -d+trap rm) so it touches no real state.Docs
docs/configuration.mdgains a "Profiles" section covering the five built-ins, precedence, layering, the CLI commands, and the user-shadow pattern.Out of scope (separate PRs)
pasclaw profile diff <a> <b>— field-level differencespasclaw profile bench --task ... --profiles ...— the A/B test harness against a fixed taskhttps://claude.ai/code/session_01TBcLtmpj7dqA5tyFbGnQon
Generated by Claude Code