Tooling Prep for Increase in Configurations Supported

.agents/skills/edg-agent-temp-files/SKILL.md

@@ -0,0 +1,103 @@
+---
+name: edg-agent-temp-files
+description: >-
+  Where agents put temporary files and directories (scratchpad, the project's
+  .tmp/agent-* directory, or the container's /tmp), how to share them with the
+  docker container via edg-exec, and how to run edgy ad hoc with --runs-dir
+  (and a scratch EDG_TEST_HOME) instead of edg-docker-test. Use when creating
+  scratch inputs or outputs, running throwaway tests or recordings, or cleaning
+  up temporary state.
+---
+
+# Temporary state and the docker boundary
+
+Tools run either on the host or in the docker container (`edg-exec`). The
+container sees **only the project root**, mounted at `/edg/workspace`; the host
+`/tmp` and your scratchpad are not visible to it. Project root = directory
+containing `src/cfe.c` and `dev_tools/`.
+
+## Where to put temporary state
+
+| Need | Put it in | Notes |
+| --- | --- | --- |
+| Host-only (scripts, outputs of host tools) | your scratchpad directory | no permission prompts |
+| Shared with the container | `<root>/.tmp/agent-<purpose>` | seen as `/edg/workspace/.tmp/agent-<purpose>`; `.tmp/` is gitignored |
+| Only needed inside the container | the container's `/tmp` | ephemeral, nothing is left on the host |
+
+- Everything you create in `.tmp/` **must** be named `agent-<purpose>` (the
+  rest of `.tmp/` is the user's host-to-docker bridge; see `.tmp/README.md`).
+- Do not use `build/`, `tests/runs/`, or the host `/tmp` for scratch state.
+- Files created in the container are owned by the host user, so they are
+  directly usable from the host.
+
+## edg-exec rules
+
+- Run `edg-exec` from the project root (the current directory is mounted as
+  `/edg/workspace`).
+- `edg-exec` joins its arguments with spaces and **does not preserve
+  quoting**: `edg-exec -- sh -c '. env.sh; cmd'` runs `sh -c .` and drops the
+  rest. Pass a compound command as ONE quoted string:
+
+  ```bash
+  edg-exec -- "mkdir -p .tmp/agent-foo && echo hi > .tmp/agent-foo/x"
+  edg-exec --dry-run -- "..."     # shows exactly what the container will run
+  ```
+
+- `edgy` and the other `dev_tools/bin` tools are on the container `PATH`.
+
+## Running edgy outside the standard workflow
+
+`edg-docker-test` is for the standard front-end change workflow, where the
+user wants results kept in `tests/runs/` (see **edg-cpfe-build-test**). For
+anything else (experiments, verification of tooling, comparisons), run `edgy`
+with `--runs-dir` so nothing lands in `tests/runs/`:
+
+```bash
+# In docker: results appear on the host under .tmp/agent-foo/runs/<timestamp>/
+edg-exec -- ". build/gcc/environment.sh && edgy --runs-dir=/edg/workspace/.tmp/agent-foo/runs regressions/foo.sft.cpp"
+```
+
+- `. build/<dir>/environment.sh` provides `CPFE`/`CPFE_CP`, which `edgy` uses
+  to find the front end (use the build directory you are testing). The default
+  configs need both `cpfe` and `cpfe-cp` built; `-c edg_x86_64` runs the C
+  config alone.
+- Use an **absolute** container path for `--runs-dir`. `edgy` copies the front
+  end under the runs directory and a relative path breaks tools that change
+  directory (e.g. `cpfe-cp-gen-be` fails with "No such file or directory").
+- Host-side tools that take `--runs-dir` (e.g. `edg-rematerialize-test-run`)
+  have no docker constraint: point them at the scratchpad.
+- If `edg-docker-test` is still used for a throwaway run, pass
+  `--discard-test-results` (results stay in the container's `/tmp`).
+
+## Scratch test home (for `-W` and other mutating experiments)
+
+`edg-docker-test` does not forward a host `EDG_TEST_HOME`, but setting it
+**inside** the container command works. Build a small test home and point at
+it; never run `-W` against the real tree:
+
+```bash
+H=.tmp/agent-foo/home
+mkdir -p $H/tests
+cp -r tests/.edgy $H/.edgy
+cp -r tests/expectations $H/expectations
+cp -r tests/tests/<suite> $H/tests/<suite>      # only the suites you need
+CMD='. build/gcc/environment.sh && EDG_TEST_HOME=/edg/workspace/.tmp/agent-foo/home edgy --runs-dir=/edg/workspace/.tmp/agent-foo/runs <suite>/<test>'
+edg-exec -- "$CMD"                    # read-only first: logs must land under .tmp/agent-foo
+edg-exec -- "${CMD/edgy /edgy -W }"   # then record (expectations and .rto files in the scratch home only)
+```
+
+Check `git status --short tests` and that `tests/runs/` did not gain an entry.
+
+## Cleanup
+
+- Remove the `.tmp/agent-*` paths **you created** in a batch step as the LAST
+  step before reporting results to the user (not after each sub-task), in ONE
+  command that lists them (one approval instead of many):
+
+  ```bash
+  rm -rf .tmp/agent-foo .tmp/agent-bar
+  ```
+
+- Never glob other entries, never remove non-`agent-` names in `.tmp/`, and
+  do not delete `tests/runs/` content (it is gitignored and may be the user's).
+- Say in the final summary what you created and removed.

.agents/skills/edg-changes-example/SKILL.md

@@ -64,8 +64,9 @@ To rebuild the real example bytes (for local reproduction, not for commit):
 ```bash
 edg-changes-example decode utf8-fragment.txt > raw-fragment.txt
 edg-changes-example decode -o raw-fragment.txt utf8-fragment.txt
-# Whole file:
-edg-changes-example decode src/Changes > /tmp/Changes.raw
+# Whole file (write host-only output to your scratchpad, see
+# edg-agent-temp-files):
+edg-changes-example decode src/Changes > <scratchpad>/Changes.raw
 ```
 
 Decode replaces each `«HHHH…»` (or legacy `«HH»«HH»`) with the corresponding

.agents/skills/edg-cmakedef-build-config/SKILL.md

@@ -0,0 +1,132 @@
+---
+name: edg-cmakedef-build-config
+description: >-
+  Explains EDG macro/build configurations: the .cmakedef files under
+  cmake/macro-conf, the edg-cmakedef resolver, EDG_MACRO_CONF selection, and
+  CMakePresets.json. Use when adding or changing a macro for a build, creating
+  a new configuration or variant, finding out why a macro is (not) set, or
+  choosing a preset. For running builds and tests, use edg-cpfe-build-test.
+---
+
+# EDG cmakedef build configurations
+
+A **macro configuration** is a named set of `.cmakedef` files that decide
+which preprocessor macros each binary is compiled with. CMake resolves them
+with `dev_tools/bin/edg-cmakedef`. Project root = directory containing
+`src/cfe.c` and `dev_tools/`.
+
+Source of truth (read these rather than trusting this summary):
+
+- `dev_tools/bin/edg-cmakedef` — the resolver / language.
+- `cmake/macro-conf.cmake` — CMake glue.
+- `cmake/host-defaults.cmake` — auto-selection when nothing is specified.
+- `cmake/macro-conf/` — the configurations.
+- `CMakePresets.json` — presets that pick a configuration.
+
+## The cmakedef language
+
+One statement per line; blank lines and `#` comments are ignored.
+
+| Line | Meaning |
+| --- | --- |
+| `NAME` | define `NAME` (emitted as `NAME=1` in `#define` output) |
+| `NAME=value` | define with a value (may be empty) |
+| `import <dir/file>` | process `cmake/macro-conf/dir/file.cmakedef` here (root-relative, no extension) |
+| `delete NAME` | remove a prior definition; **error** if not set |
+| `delete_if_set NAME` | remove a prior definition; message only if not set |
+
+- Later lines override earlier ones (including across imports), so order
+  matters: import the base first, then add overrides.
+- `@VAR@` in a value is expanded by CMake (`string(CONFIGURE)`).
+- Any unparsable line or failing `delete` makes the file invalid; the
+  CMake configure then fails with "Failed to process cmakedef."
+
+## Layout of `cmake/macro-conf/`
+
+- `<config>/<target>.cmakedef`, one file per target. Targets and who reads
+  them: `base` (`src/CMakeLists.txt`, emitted as a generated
+  `cmake_defines.h` plus `USE_CMAKE_DEFINES`), `cpfe`, `cpfe-cp`, `cdisp`
+  (`src/cmake/<target>/CMakeLists.txt`) and `cpprtlib`
+  (`lib_src/CMakeLists.txt`); the last four become `-D` flags.
+- `default/` — baseline each config imports first
+  (e.g. `default/cpfe` sets `BACK_END_IS_C_GEN_BE`, `DO_IL_LOWERING`).
+  `default/base.cmakedef` is a "should not be used" stub.
+- `support/` — reusable fragments: `platform/`, `arch/`, `compiler/`,
+  `build-type/`, `simple-conf/`, `feature/` (`on`/`off`), `target/`.
+  Fragments are per-target too (e.g. `support/build-type/debug/cpfe`).
+- Named configs (`linux-gcc`, `linux-gcc-release`, `linux-clang`,
+  `linux-i686-gcc`, `macos-arm-clang`, `windows-msvc`, …). Variants layer on
+  a parent: `linux-gcc-multi-thread-release/cpfe.cmakedef` imports
+  `<linux-gcc-release/cpfe>` and then appends its own macros.
+- A config directory missing a target file is auto-populated by CMake with
+  `import <default/<target>>`; don't rely on that for real configs.
+
+## Selection
+
+- Single-config generators (Ninja): cache var `EDG_MACRO_CONF`.
+- Multi-config generators: `EDG_DEBUG_MACRO_CONF` and
+  `EDG_RELEASE_MACRO_CONF`.
+- If empty, `host-defaults.cmake` picks one from OS, arch, compiler, and
+  build type (`detect_host_defaults`); otherwise CMake warns.
+- Presets set it explicitly, e.g. `linux-gcc-debug` →
+  `EDG_MACRO_CONF=linux-gcc`, binary dir `build/gcc`.
+- Every cmakedef file reached is added to `CMAKE_CONFIGURE_DEPENDS`, so
+  editing one re-runs configure on the next build.
+
+## Recipes
+
+Inspect what a config resolves to (host Python, read-only, no build):
+
+```bash
+edg-cmakedef cmake/macro-conf linux-gcc/cpfe.cmakedef print-values
+edg-cmakedef --verbose cmake/macro-conf linux-gcc/cpfe.cmakedef print-values
+edg-cmakedef cmake/macro-conf linux-gcc/base.cmakedef print-defines
+edg-cmakedef cmake/macro-conf linux-gcc/cpfe.cmakedef print-paths
+```
+
+- The target is resolved against the root if relative (CMake passes
+  absolute paths); never repeat the root in it.
+- `print-values` → `NAME=value` lines; `print-defines` → `#ifndef/#define`
+  blocks (the `base` form); `print-paths` → the files visited.
+- `--verbose` writes the import chain and each Setting / Overwriting /
+  Deleting step to **stderr**, which shows exactly which file last set a
+  macro. Use it to answer "why is X set?".
+- Compare two configs by diffing the `print-values` output (sort first).
+
+Change a macro for one build: edit the narrowest file that covers it.
+Shared by everything → `default/<target>`; one platform / build type /
+compiler → the matching `support/…` fragment; one named config or variant →
+that config's `<target>.cmakedef`. Check all targets that need it
+(`base` vs `cpfe` vs `cpprtlib` are separate files).
+
+Add a new configuration:
+
+1. Copy the nearest sibling directory under `cmake/macro-conf/` with all
+   its target files; keep imports of `default/` and `support/` fragments.
+2. If it is a variant, import the parent's file instead of copying it.
+3. Add a preset to `CMakePresets.json` (inherit `linux-common` or the
+   platform's common preset; set `EDG_MACRO_CONF`, `binaryDir`).
+4. If it should be auto-selected, add a branch in
+   `cmake/host-defaults.cmake` (and note release builds use the
+   `<name>-release` directory).
+5. Validate with the `print-values` recipe for every target, then
+   configure/build in Docker (see **edg-cpfe-build-test**):
+
+   ```bash
+   edg-exec -- cmake --preset <preset>
+   edg-exec --wdir build/<dir> -- ninja
+   ```
+
+## Pitfalls
+
+- `import <…>` paths are relative to `cmake/macro-conf`, with no extension.
+- `delete` of a never-set key fails the configure; use `delete_if_set` when
+  the key might legitimately be absent.
+- A macro in a cmakedef only matters if the source consumes it; the generated
+  `cmake_defines.h` is only used when `USE_CMAKE_DEFINES` is set. Editing
+  `src/defines.h` (BUILD.md) does not change what a CMake configuration
+  produces.
+- `print-paths` output order is unspecified (set iteration).
+- Windows uses `dev_tools/win-bin/edg-cmakedef.bat`; the Python is
+  type-checked by `dev_tools/typecheck-python.sh` (follow
+  **edg-python-style** if editing the tool).

.agents/skills/edg-cpfe-build-test/SKILL.md

@@ -24,6 +24,8 @@ edg-exec --wdir build/gcc -- ninja cpfe-cp   # C++ → C++ from IL
 ```
 
 - Default dirs: `build/gcc`, `bases/docker/dev-env/gcc` (`EDG_BASE`).
+- Changing which macros a build uses (`cmake/macro-conf/*.cmakedef`,
+  `EDG_MACRO_CONF`, new presets): skill **edg-cmakedef-build-config**.
 - Clang: preset `linux-clang-debug` → `build/clang`.
 - Shell in container (cwd → `/edg/workspace`): `edg-exec` or `edg-docker-shell`.
 - If `src/defines.h` is swapped for reproduction, one binary may not build;
@@ -45,7 +47,14 @@ edg-docker-test --dry-run rma
 - Postures: `day`, `night`, `quick-c-fe`, `extensive`, or `--posture=@file.json`.
 - Results: `tests/runs/<timestamp>/<config>/` (e.g. `changes.elog`,
   `executed.elist`). Empty / no unexpected diffs means a clean match.
-- Optional override: `EDG_TEST_HOME` (must contain `.edgy/config.json`).
+- `edg-docker-test` is for the standard front-end change workflow, where the
+  user wants these results kept. For ad-hoc runs and experiments, use
+  `edgy --runs-dir` instead (skill **edg-agent-temp-files**); if you still use
+  `edg-docker-test` for a throwaway run, pass `--discard-test-results`.
+- `EDG_TEST_HOME` (must contain `.edgy/config.json`) overrides the test home
+  but a host value is **not** forwarded into the container by
+  `edg-docker-test`; set it inside an `edg-exec` command string (see
+  **edg-agent-temp-files**).
 
 First-test walkthrough: [dev_annex/tutorials/FIRST_TEST.md](../../../dev_annex/tutorials/FIRST_TEST.md).
 
@@ -63,6 +72,12 @@ First-test walkthrough: [dev_annex/tutorials/FIRST_TEST.md](../../../dev_annex/t
    that are gitignored. See edgy helpers in `dev_tools/pylibs/edgtest`.
 4. Re-run without `-W` to confirm a clean match before asking to commit.
 
+Expectations inherit like recordings: a non-default config's
+`tests/expectations/<config>/<suite>.log` holds only differences from the
+default config's (an explicit `...:PASS:` line overrides an inherited
+failure). Record the default config first (`edgy -W` does this when both run).
+See "Expectation Inheritance" in `doc/source/testing.rst`.
+
 ## Finding a regressing or fixing commit
 
 Use skill **edg-cpfe-bisect** (`edg-docker-test-bisect` /

.github/workflows/build-and-test-ci.yml

@@ -46,7 +46,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 120
     container:
-      image: docker.io/edgcpp/ci-runner@sha256:69c63755ef518a982b09381b5f420c3da16f0c1eda2c22a6f73277e84a62c28f
+      image: docker.io/edgcpp/ci-runner@sha256:73be12d128b8507f5feb9e73c25f48f14e6b041940ef165bce5fda8efe2b605c
     defaults:
       run:
         shell: edg-container-exec "bash -eo pipefail {0}"
@@ -87,7 +87,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 120
     container:
-      image: docker.io/edgcpp/ci-runner@sha256:69c63755ef518a982b09381b5f420c3da16f0c1eda2c22a6f73277e84a62c28f
+      image: docker.io/edgcpp/ci-runner@sha256:73be12d128b8507f5feb9e73c25f48f14e6b041940ef165bce5fda8efe2b605c
     defaults:
       run:
         shell: edg-container-exec "bash -eo pipefail {0}"
@@ -134,7 +134,7 @@ jobs:
           - edg_x86_64
           - edg_x86_64_cp
     container:
-      image: docker.io/edgcpp/ci-runner@sha256:69c63755ef518a982b09381b5f420c3da16f0c1eda2c22a6f73277e84a62c28f
+      image: docker.io/edgcpp/ci-runner@sha256:73be12d128b8507f5feb9e73c25f48f14e6b041940ef165bce5fda8efe2b605c
       # Required for setarch ... -R (disable ASLR) used by the test harness.
       options: --privileged
     defaults:
@@ -202,7 +202,7 @@ jobs:
           - edg_x86_32
           - edg_x86_32_cp
     container:
-      image: docker.io/edgcpp/ci-runner@sha256:69c63755ef518a982b09381b5f420c3da16f0c1eda2c22a6f73277e84a62c28f
+      image: docker.io/edgcpp/ci-runner@sha256:73be12d128b8507f5feb9e73c25f48f14e6b041940ef165bce5fda8efe2b605c
       # Required for setarch ... -R (disable ASLR) used by the test harness.
       options: --privileged
     defaults:

.github/workflows/strict-build-check.yml

@@ -151,7 +151,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 120
     container:
-      image: docker.io/edgcpp/ci-runner@sha256:69c63755ef518a982b09381b5f420c3da16f0c1eda2c22a6f73277e84a62c28f
+      image: docker.io/edgcpp/ci-runner@sha256:73be12d128b8507f5feb9e73c25f48f14e6b041940ef165bce5fda8efe2b605c
     defaults:
       run:
         shell: edg-container-exec "bash -eo pipefail {0}"
@@ -176,7 +176,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 120
     container:
-      image: docker.io/edgcpp/ci-runner@sha256:69c63755ef518a982b09381b5f420c3da16f0c1eda2c22a6f73277e84a62c28f
+      image: docker.io/edgcpp/ci-runner@sha256:73be12d128b8507f5feb9e73c25f48f14e6b041940ef165bce5fda8efe2b605c
     defaults:
       run:
         shell: edg-container-exec "bash -eo pipefail {0}"
@@ -201,7 +201,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 120
     container:
-      image: docker.io/edgcpp/ci-runner@sha256:69c63755ef518a982b09381b5f420c3da16f0c1eda2c22a6f73277e84a62c28f
+      image: docker.io/edgcpp/ci-runner@sha256:73be12d128b8507f5feb9e73c25f48f14e6b041940ef165bce5fda8efe2b605c
     defaults:
       run:
         shell: edg-container-exec "bash -eo pipefail {0}"
@@ -226,7 +226,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 120
     container:
-      image: docker.io/edgcpp/ci-runner@sha256:69c63755ef518a982b09381b5f420c3da16f0c1eda2c22a6f73277e84a62c28f
+      image: docker.io/edgcpp/ci-runner@sha256:73be12d128b8507f5feb9e73c25f48f14e6b041940ef165bce5fda8efe2b605c
     defaults:
       run:
         shell: edg-container-exec "bash -eo pipefail {0}"

.tmp/README.md

@@ -9,3 +9,20 @@ This allows for the host to pass non-trivial empherial information to Docker
 for further processing (that if packaged entirely as command line arguments
 would exceed the maximum command line argument limit -- primarily a problem
 on Windows).
+
+## Notes for Agents
+
+This section is an instruction for AI coding agents that come across this
+file; it is not required reading for human users.
+
+Agents may use this directory for temporary files and directories that must be
+visible both on the host and inside the docker container (the project root is
+mounted at `/edg/workspace`, so `.tmp/agent-foo` on the host is
+`/edg/workspace/.tmp/agent-foo` in the container).  Everything an agent creates
+here must be named `agent-<purpose>` so it can be told apart from (and cleaned
+up separately from) anything else in this directory.  Agents remove these
+temporary files and directories in a batch step as the last step before
+reporting results to the user.
+
+See the `edg-agent-temp-files` skill (`.agents/skills/edg-agent-temp-files/`)
+for the full rules.