From b5eca6877b96065179a6dd511a99375e607d02e5 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 00:31:11 +0900 Subject: [PATCH 01/20] design 2026-10-29: target sets and legs, the packaging model revised, run coverage, no silent fallback, external builds, one mcppls package, the editors Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...arget-sets-packaging-and-editors-design.md | 567 ++++++++++++++++++ .agents/docs/README.md | 4 +- 2 files changed, 570 insertions(+), 1 deletion(-) create mode 100644 .agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md diff --git a/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md b/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md new file mode 100644 index 000000000..d9f67646c --- /dev/null +++ b/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md @@ -0,0 +1,567 @@ +--- +subject: platform +status: review +--- + +# 生态设计：目标集合与腿、打包模型 rev、运行覆盖、不回落、外部构建接入、mcppls 单包与编辑器 + +日期：2026-10-29（rev 2：2026-10-30）。基线：mcpp `db3b8159`（2026.10.29.1）。前文： +[platforms design](2026-10-28-platforms-design.md)（SPEC-013）、 +[part 2](2026-10-28-platforms-part2-packaging-and-distribution-design.md)（SPEC-014）、 +[openkal run/test/pack](2026-10-29-openkal-run-test-pack-design.md)（SPEC-002 §6 的执行宿主）。 +相关 issue：#849、#851、#852、#857、#860、openxlings/xlings#708。相关规范：SPEC-002、SPEC-007、SPEC-011、SPEC-013、SPEC-014， +mcppls S4；新增 SPEC-015（目标集合与腿）。 + +**修订记录** + +- rev 1：综合 2026-10-29 的讨论。已定：#849 的 APE 实现（openkal-cosmo）独立立项、先做 probe，本批只留位置；运行器的宿主列表由 + 平台声明；kit 移出程序包、包内带 openkal kit；VS Code 扩展与 mcpp-vscode 0.9.0 一模一样再微调；目标集合并入本设计。 +- rev 2（本稿，评审）：三个视角的自我评审（核心架构与规范、命令行与迁移、生态可行性）逐条处理，见 §14。主要变化：集合的展开 + 在 CLI 层、按平台选择后取交集（§3.1）；腿的构建复用工作空间多配置的机制（§3.3）；启动器统一为 POSIX 脚本加引擎的 Windows + 启动程序，取消"单系统本机启动器"与载体的新形式（§4.3）；`runs_on` 重定义（§4.4）；描述符键改为目标表的一列、不改变已有 + 行为的部分写明（§5.1）；`exceptions = true` 保留原义（§6）；#857 的方案按 openkal-linux 0.18.2 的实际代码重写，probe 前置 + （§7）；外部构建改为计划阶段的契约文件加扩展 `deps-cmake`（§8）；kit 的离线语义如实写出，S4 修订（§9.2）；VS Code 的"一模 + 一样"定义在渲染结果上，HTML 生成移入页面以与 CLion 共用（§10.1）；被替换的条款、下游迁移、完整级联（§12、§13）。 + +## 0. 摘要 + +`--target x86_64-openkal` 在 `pack` 里打出全部架构，在 `build`、`run` 里只是 x86_64：同一参数两种语义，一条与多条 `--target` 的结果 +不单调。根因有两个： + +1. **名字被静默放大或替换。** 架构被平台的 `universal` 放大；不认识的系统回落到宿主的描述符分段（#851）；有歧义的裸名绑到另一个包 + （#852）；分发条目把架构扩到所有系统（#860）。 +2. **"在哪里运行"没有一份数据。** 映像架构、运行器所在的系统、载体能构建的行、PATH 回落各处自行推算。 + +本批的模型： + +- **目标集合（SPEC-015）。** `--target` 精确；首段是系统名的写法（`openkal`、`linux-musl`）是集合，在 CLI 层展开为平台目标中 + 余下部分相同的全部目标。腿的规划与构建复用工作空间多配置的机制，`build`、`run`、`test`、`pack`、IDE 看到同一组腿。 +- **打包（SPEC-014 rev）。** 删除 `universal`：腿怎样合并由目标的种类与平台能力决定；openkal 的包只有一种布局，启动器是 POSIX + 脚本与引擎的 Windows 启动程序；`--pkg-format` 可写列表；`--systems` 接受 ``、`-` 与 `all`。 +- **运行覆盖。** 打包时计算一次 `runs_on`，分发条目、启动器、合并、摘要与 CI 都读它；单文件定义为"最少的载体覆盖腿 × 系统"， + APE 载体到来时命令不变。 +- **不回落。** 名字只解析到它确切指代的东西；显式声明的别名与展开不是回落；改变行为的地方逐条写出。 +- **外部构建接入。** 引擎在计划阶段写出目标的编译、链接、宿主三段契约；mcpp-plugins 的 `deps-cmake` 读它；静态库经 `link_flag` + 进入 mcpp 的链接与映像收尾。clangd 因此可以成为 ok 映像，mcppls 收敛为一个包。 +- **编辑器。** VS Code 扩展的界面与交互以 mcpp-vscode 0.9.0 为准，一致性定义在渲染结果上；数据经 mcpp-ide 后端的逐视图适配层 + 获得；0.9.0 的页面成为 VS Code 与 CLion 共用的组件库。 + +## 1. 问题 + +| 编号 | 观察 | 位置 / 来源 | +|---|---|---| +| P1 | 平台声明 `universal = true` 时，单个 `--target x86_64-openkal` 打出平台全部 `targets`；三元组的架构只决定第一条腿 | `src/pack/pipeline.cppm:410`、`:638` | +| P2 | 一个 `--target` 得到全部腿，两个得到恰好两条 | `src/cli/cmd_publish.cppm:905–915` | +| P3 | `universal` 同时决定"几个架构"与"什么布局"；单架构的跨系统布局、单架构 okb 写不出 | SPEC-014 §3 | +| P4 | 行为由依赖图里平台包的一个字段决定 | openkal-platform `mcpp.toml:32` | +| P5 | 腿的循环散在四处，各自 `prepare_build`：Android 的额外腿、通用包、库包、平台包的 `mcpp test` | `pipeline.cppm:162`、`:651`，`library_pipeline.cppm:174`，`cmd_build.cppm:1297–1320` | +| P6 | 分发条目给通用包写三个系统 × 全部映像架构，而包只带四个 `-` 的运行器（#860） | `src/pack/distribution.cppm:1537–1560` | +| P7 | 单文件每 `-` 一个；"一个文件到处运行"（#849）在模型里没有位置 | SPEC-014 §4 | +| P8 | `TargetPlatform::for_os` 对不认识的系统回落到宿主：`emscripten`（#851.1）、`openkal`、`ios`、`none`。Android 解析为 `os = linux`、`env = android`，不在此列。该值还用于 semver 解析、版本条目与 revision、索引刷新 | `platform.cppm:114–119`，`toolchain.cpp:1220`，`graph_load.cpp:320`、`:826`，`xlings.cpp:364` | +| P9 | 两个末段同名的构建依赖：裸名绑到其中一个，每次构建警告（#852） | `src/build/prepare/graph.cpp`（`fillDepDirs`）、`src/build/build_program.cppm` | +| P10 | emcc 链接不带 `-O`；异常只有 JS 模型（#851.2/.3） | 链接参数组装 | +| P11 | Android 应用沙箱（`run-as`）中 mcpp 启动的每个程序 ENOENT，2026.10.26.2 起（#857）。openkal-linux 0.18.2 的启动是 `openat(O_PATH)` 加 `execveat(AT_EMPTY_PATH)`，ENOENT 时已有带 `same_file` 守卫的 `execve` 重试，所以 ENOENT 来自别处；注释所说"首次 ENOSYS 后改用 execve"没有代码实现 | openkal-linux `src/process.cpp:684–712`、`:340–347` | +| P12 | CMake 项目无法为平台目标编译；`deps-cmake` 只传编译器路径，不传平台标志与层的头文件；`ldflags` 不展开 `${mcpp.out_dir}` | mcpp-plugins `deps/cmake.cppm:380–559`，`plan.cpp:1164–1212` | +| P13 | mcppls 每平台一个包：服务器已是一个 okb，但载荷带每系统的原生 clangd 与按宿主选的 kit | mcppls `packaging/README.md`、`src/engine/payload.cpp:118`、`:169–190` | +| P14 | mcpp-vscode 1.0 相对 0.9.0 大幅回退：扩展端 TypeScript 22 147 → 1 932 行，库界面 5 043 → 87 行，缓存界面 2 292 → 72 行，全部面板 583 行、样式 71 行 | mcpp-vscode `src/`、mcpp-ide `webui/` | +| P15 | mcpp-clion 0.1.1：1 处将移除 API（`CPPToolchains.Toolchain.getDebuggerKind()`）、4 处弃用、4 处实验 API；261 上 optional 依赖 `intellij.testRunner.plugin` 不可解析 | Marketplace 校验与本地 verifier 报告 | +| P16 | mcpp-ide 的 CI 打两次包，每次构建 aarch64 腿 | mcpp-ide `.github/workflows/ci.yml:62–64` | +| P17 | mcpp-vscode 1.0.1 在 Marketplace，Open VSX 仍为 0.9.0；仓库未归档 | 两个市场的查询 | + +## 2. 原则 + +1. **写什么做什么。** 参数精确；范围的扩大由用户写出（集合、多个目标、`all`）。 +2. **不静默。** 名字解析不到确切的对象就不绑定或拒绝并给出写法；显式声明的别名与展开不是回落；取交集、略去、改变行为都要说出来。 +3. **一维一参，默认在 manifest。** 平台声明能力，不声明项目的默认行为。 +4. **一个事实一处计算。** "在哪里运行"打包时算一次。 +5. **渐进披露。** 零参数做对的事；控制随层级出现；提示教下一层的写法。 +6. **核心只放通用契约。** 构建系统、工具与容器格式的细节在插件。 +7. **机器输出只增不改。** 单腿、单容器时输出逐字节不变（docs/50 §7）。 + +## 3. 目标集合与腿（SPEC-015，新） + +### 3.1 写法与展开 + +- **精确目标**：三元组，含义不变。 +- **目标集合**：首段属于系统词表（`linux`、`macos`、`windows`、`openkal`、`emscripten`、`ios`、`android`、`none`）的写法。`triple::parse` + 不校验首段（`unknown-linux-gnu` 会解析成架构 `unknown`），所以集合靠首段的词表识别，不靠解析失败。 +- **展开在 CLI 层、prepare 之前**： + 1. 按 docs/21 的规则补全环境（`linux` → `linux-gnu`；`windows` 补为 `windows-gnu`，不含 msvc）。 + 2. 候选：内置目标表中系统与环境相同、且不是 planned 的行。 + 3. 对每个候选按现有规则选平台（`prepare/platform.cpp:144–160`：目标行的平台列、`[target.] platform`、`MCPP__PLATFORM`）。 + 候选必须选到同一个平台，否则拒绝并写出分歧。 + 4. 与该平台的目标取交集：平台行用 `[platform] targets`，hosted 安排用 `[platform.hosted] targets`；无平台的行取候选本身。 + 5. 结果为空时拒绝，列出可用的目标。 +- 多个 `--target` 取并集、去重、按目标表的顺序。机器输出与摘要只写展开后的精确目标；`Packed`/`Finished` 的摘要列出各腿，展开随 + 平台版本变化时用户看得见。 +- `[build] target` 只接受精确目标；集合只出现在命令行与 `[pack] targets`。 + +| 写法 | 展开（当前内置表与 openkal-platform 0.17.3） | +|---|---| +| `openkal` | `x86_64-openkal`、`aarch64-openkal` | +| `linux-musl` | `x86_64-linux-musl`、`aarch64-linux-musl`（planned 的 `riscv64-linux-musl` 不在内） | +| `linux` | `x86_64-linux-gnu`、`aarch64-linux-gnu` | +| `none-elf` | 内置表中全部非 planned 的裸机 elf 行 | + +### 3.2 命令的语义 + +| 命令 | 精确目标 | 目标集合 | +|---|---|---| +| `build` | 构建这条腿 | 构建全部腿，每条腿在 `target//` 下 | +| `run` | 本机架构经运行器；另一架构按 SPEC-002 §6 经执行宿主，否则拒绝（2026.10.29.1 的 `runner_reach`） | 选一条能到达的腿，本机架构优先；只构建这一条；写出所选的腿；都不能到达则拒绝 | +| `test` | 同 `run` 的到达规则；到达不了即拒绝（SPEC-013 §10） | 构建全部腿，运行能到达的腿，其余逐条报告"已构建、未运行"并给出原因；**一个测试都没运行时失败** | +| `pack` | 一个包，一条腿 | 按 §4.2 | +| `self env` | 不变 | `mcpp.env` 增加 `legs[]`，每项是一份精确目标的报告；顶层取第一条腿 | +| IDE（`targets/list`） | 每个精确目标一行 | 集合一行，如 `openkal (x86_64, aarch64)` | + +平台包里不带 `--target` 的 `mcpp test` 今天就遍历平台的 `targets`（`cmd_build.cppm:1297–1320`），即"集合"语义的一个特例；本设计 +把它统一到上表，行为不变。 + +### 3.3 架构：腿的规划串行、构建并行 + +- 复用工作空间多配置构建的机制（`cmd_build.cppm` 约 395–430 行）：**逐腿串行规划**（`prepare_build` 一次一腿，`mcpp.lock` 只在 + 规划阶段写，避免 SPEC-011 §4 的乐观写入冲突），**再并行构建**（作业数按腿静态分摊 `[build] jobs`，`progress::configurations`、 + `defer_finished` 给出一个 `Finished`）。多把构建目录锁按目标名排序获取。 +- 四处腿循环（P5）收拢到这一处：Android 的额外腿、通用包、库包、平台包的 `mcpp test`。`pack` 只暂存构建层给出的产物。 +- 单值的地方扩展：`BuildOverrides.target_triple` 改为腿列表（单腿时与今天相同）；快速路径只用于单腿；`.last-result.json` 与结果 + 记录的 `configuration` 每腿一份，单腿时文件与内容不变。 +- 默认：`[pack] targets` 是打包的默认目标集合；`build`、`run`、`test` 的默认仍是 `[build] target`。`mcpp new` 的 openkal 模板写上 + `[pack] targets = ["openkal"]`。 + +### 3.4 构建一次，各处验证 + +- 写：`mcpp test --target openkal --no-run --archive `：构建各腿的测试程序，与清单 `mcpp-tests.json`（每个测试的腿、参数、 + 工作目录、期望）一起写成引擎的 tar（核心不认识 okb）。 +- 读：`mcpp test --archive `：在另一台机器上用本机的运行器或系统运行归档中本机能到达的测试，不构建；工作目录是项目检出； + 报告与 `mcpp test` 相同；一个测试都没运行时失败。 +- 同样适用于原生交叉：Linux 上构建的 `x86_64-windows-gnu` 测试拿到 Windows 上运行。 +- mcppls 与 mcpp-ide 的 CI：Linux 上构建一次，macOS 与 Windows 上 `--archive` 运行。 + +### 3.5 机器输出 + +- `mcpp.build`：单腿逐字节不变。多腿时顶层字段取第一条腿，`status` 汇总全部腿，新增 `legs[]`（每项一份完整结果记录）；`artifact` + 与 `diagnostic` 事件新增 `triple`。 +- `mcpp.pack`：新增 `pkgFormats[]`（`pkgFormat` 保留为第一个容器）、`legs[]`、`runsOn`（驼峰，同 §4.4）；`artifacts[]` 照旧。 + +## 4. 打包模型（SPEC-014 rev） + +### 4.1 维度 + +| 问题 | 参数 | `[pack]` | 说明 | +|---|---|---|---| +| 为哪些目标 | `--target`（可重复，精确或集合） | `targets` | 不再被平台放大 | +| 运行的机器提供什么 | `--mode` | `default_mode`（新增接受规范名 `system`、`vendored`、`self-contained`、`static`） | 不新增 `mode` 键 | +| 库在程序里还是旁边 | `--linkage` | `linkage` | 不变 | +| 交付成什么 | `--pkg-format`（可写列表） | `format`（字符串或列表） | 见 §4.6 | +| 运行器带上哪些 | `--systems`：`linux`、`linux-aarch64`、`all` 的列表 | `systems` | `` 取腿的全部架构；`all` 取平台运行器能到达的全部；默认本机系统 | + +### 4.2 腿的合并 + +| 目标的种类与平台能力 | 多条腿的结果 | +|---|---| +| `lib` / `shared` | 一个库包（已有，12-binary-distribution） | +| 程序，平台声明 `runner`（openkal） | 一棵树（§4.3），包名 `--<平台名>`（今天的 `universalName`，如 `openkal`） | +| 程序，Android 行 | 一个 APK（已有） | +| 其他程序 | 每腿一个包，各以三元组命名；结果多于一个时 `-o` 只能是目录 | + +- 单腿的包名是三元组，模式后缀规则不变。 +- 工作空间多成员打包接受集合：每个成员按上表（解除 `cmd_publish.cppm` 约 841 行的多三元组拒绝）。 +- 删除 `--no-universal`、`[pack] universal`、`[platform] universal` 的作用，按 §13.2 的迁移规则处理。 + +### 4.3 openkal 的包：一种布局，启动器统一 + +``` +--/ + bin/ POSIX 启动脚本 + bin/.exe Windows 启动程序（引擎的启动程序） + lib//.oke 每条腿一组映像（含 .okso） + libexec/okrun/-/ 仅 self-contained + share/… 资源一份 + mcpp-package.json +``` + +- 布局与腿数、`--systems` 都无关。`.cmd` 不再生成（Ctrl-C 的"终止批处理操作吗"、进程名是 cmd 的问题随之消失）。 +- **POSIX 脚本**：取包根，按 `uname -m` 选 `lib//`；`uname -s` 为 `MINGW*`/`MSYS*`/`CYGWIN*` 时 `exec "$top/bin/.exe"` + （Git Bash 优先选完全同名的文件，GitHub Actions 的 Windows `shell: bash` 即此情形）。 +- **Windows 启动程序**：引擎的启动程序（单文件所用的同一程序，基于 mcpplibs.os，为 `x86_64-windows-gnu` 构建），读包内 + `mcpp-package.json` 的 `launch`，按本机架构选映像，先找 `libexec//windows-/`，再在 `runs_on.with` 覆盖的范围内找 PATH。 + 自身位置取自 `GetModuleFileNameW`，不用 argv[0]；找不到映像时写出期望的路径。核心只知道 `launch`（运行器命令与每架构的入口）， + 不知道 okrun。 +- **启动器是派生文件**：`--merge` 合并后重新生成启动器与 `mcpp-package.json`，启动器不参与"同一路径两个摘要"的冲突检查。 +- **M3（映像嵌入载体）只属于单文件**。单腿 self-contained 包不再是"一个嵌入映像的本机程序"，而是上面的树；需要一个本机程序 + 的用户用 `--pkg-format single-file`。2026.10.29.1 rev 2 为 M3 写的 `emit xpkg` 规则改为读 §4.4 的 `runs_on`。 +- openkal 的 `--mode static`（今天是静态链接加 M3）：改为 `--linkage static --mode self-contained` 的树，即一个静态 `.oke` 每架构、 + 带运行器；单个嵌入文件用 `--linkage static --pkg-format single-file`。 +- `[pack] sign` 对 `bin/`、`libexec/` 中的每个程序执行，含启动程序。 + +### 4.4 运行覆盖（`runs_on`） + +打包时计算一次，写入 `mcpp-package.json`（schema 1 增加字段，不升级）： + +```json +"runs_on": { + "self": ["linux-aarch64", "linux-x86_64", "macos-aarch64", "windows-x86_64"], + "with": { "okrun": ["…"] } +} +``` + +- **`self`**：凭包内携带的东西与该系统自身的库即可运行（原生包需要的系统库照旧由 `HOST-REQUIREMENTS` 列出）。 +- **`with.`**：机器装有该运行器时可运行。 + +| 包 | `self` | `with.` | +|---|---|---| +| openkal `self-contained` | 腿的架构 × 构建出的运行器的 `-`，取交集 | 腿的架构 × 平台 `runner.hosts`，减去 `self` | +| openkal `vendored` / `system` | 空 | 腿的架构 × 平台 `runner.hosts` | +| 原生单腿（任何模式） | 三元组的 `-` | 空 | +| 单文件 | 每个文件各自的 `-` | 空 | +| WebAssembly、Android、iOS、裸机 | 空（`emit xpkg` 照旧拒绝） | 空 | + +- 平台声明运行器的宿主：`runner = { cmd = ["okrun", "{}"], host = "same-arch", hosts = [...] }`，即平台钉住的 openkal-image 构建的 + 行。`host = "any"` 时 `hosts` 列出运行器程序存在的 `-`，覆盖与腿的架构无关。 +- `self` 与 `with` 都为空时拒绝打包。交集略去的组合在摘要中写出：`Runs on: linux-x86_64, … / not covered: x86_64-macos (no runner + row)`。 +- 已有的 `systems` 字段保留、含义不变（携带的运行器，现写为 `-`），它是输入的记录，`runs_on` 是由它与腿、平台算出的答案。 +- 读者：`emit xpkg`（每个系统块只写 `self ∪ with` 覆盖到的架构，`macos` 写 `macosx`；只在 `with` 中时 `deps = { "xim:okrun" }`； + Windows 登记 `.exe`）、启动器在覆盖外的错误文字、`--merge`（并集；旧包没有 `runs_on` 时由 `systems`、`legs`、平台算出）、 + `Packed` 摘要、`mcpp.pack` 的 `runsOn`、各项目 CI 的断言。 +- 修复 #860；mcpp-ide 0.1.1 的索引条目重新生成。 + +### 4.5 单文件：最少的载体覆盖腿 × 系统 + +- 平台的 `carrier` 接受列表，每项增加 `hosts`（今天的单表仍接受，等于一项，`hosts` 同 `runner.hosts`）。执行宿主的运行器程序 + （SPEC-013 §10）取 `hosts` 覆盖"本机系统 × 目标架构"的那一项。 +- `--pkg-format single-file`：选出覆盖"腿 × `--systems`"的最少载体文件；结果多于一个时 `-o` 只能是目录。 +- **#849 的占位条款**：载体项可声明非按行构建的模型（`build = ""`）；APE 载体的模型随 openkal-cosmo 定义，届时 `--systems all` + 得到一个文件，命令不变。openkal-cosmo 独立立项，先做 probe（MS ABI 与 SysV/AAPCS 的转换、Cosmopolitan 在 Windows 上的进程、 + 终端、网络、各系统对 APE 的签名与放行）。 + +### 4.6 容器与列表 + +- okb 是平台插件提供的容器，含哪些架构由腿决定：`--target aarch64-openkal --pkg-format okb` 得单架构 okb。命名由插件定 + （`-.okb`，不变）。 +- `--pkg-format tar,okb`：一次构建、一棵暂存树、依次写出各容器。规则： + - 列表只能含读同一棵暂存树的容器（`tar`、`zip` 与插件格式）；`dir`、`single-file` 不进列表（拒绝）。 + - 列表时 `-o` 只能是目录（以 `/` 结尾或已存在的目录）；`--compress` 只作用于引擎容器。 + - `--mode self-contained` 而列表只有 okb：拒绝（okb v1 只含映像，运行器会被丢掉），提示加上 `tar`。 + - `mcpp run --pkg-format` 不接受列表。 + +### 4.7 拒绝表的变化 + +| 变化 | 行 | +|---|---| +| 删除 | 多个 `--target` 不在通用包里；平台没有 `runner` 不能打通用包 | +| 改写 | `--systems` 只在 `self-contained` 或单文件时有意义，其他组合拒绝；`[pack] systems` 不搭时提示后忽略（同 `[pack] compress` 的先例） | +| 新增 | 集合展开为空或候选选到不同平台；结果多于一个文件而 `-o` 是文件；列表中有 `dir`/`single-file`；self-contained 只配 okb；`runs_on` 为空 | + +### 4.8 渐进披露 + +| 层 | 写法 | 得到 | +|---|---|---| +| L0 | `mcpp pack` | `[pack] targets`（未写则 `[build] target`）的 vendored tar | +| L1 | `--target openkal`、`--target x86_64-openkal`、`--pkg-format single-file\|okb` | 目标与形态 | +| L2 | `--mode`、`--systems`、`--linkage` | 闭包、运行器、链接 | +| L3（多在 manifest） | `--compress`、`sign`、`--merge`、`strip`、`debug_symbols`、`[pack.bundle-project]` | 细节 | + +- `mcpp pack --help` 按 L0–L2 / L3 分组。 +- 提示：单腿 openkal 打包时，`Packed` 后写"只含 x86_64；`--target openkal` 或 `[pack] targets = ["openkal"]` 打 openkal-platform 的全部架构"。 + +## 5. 不回落 + +### 5.1 描述符键 + +- 描述符键成为**目标表的一列**（沿用目标表已有的平台列与 `kPlatformNames` 词表），每行声明它读哪些键、按什么顺序；宿主不再参与。 + 平台行由平台的 `[platform] descriptor_keys` 覆盖。 +- 各行的取值与行为变化： + +| 行 | 键（顺序） | 与今天相比 | +|---|---|---| +| Linux（含 android、musl） | `linux` | 不变 | +| macOS | `macosx` | 不变 | +| Windows | `windows` | 不变 | +| `wasm32-emscripten` | `emscripten`, `linux` | Windows、macOS 宿主上不再读宿主段；Linux 宿主上行为不变，taskflow 的 `-pthread` 由描述符增加 `emscripten` 段解决（mcpp-index：taskflow、freetype） | +| `-openkal` | 由 openkal-platform 声明，预计 `openkal`, `linux` | 先实测：compat 包的 openkal 构建依赖 `linux` 段的程度；Linux 与 macOS 宿主上各构建一次比较映像字节（同时检验与 #850 的关系） | +| iOS | `ios`, `macosx` | 不再读宿主段 | +| 裸机 `none` | `none` | 缺段时拒绝并写出包与目标 | + +- 这一列同时供给 semver 解析、版本条目与 revision、索引刷新（P8 的读者），所以"读哪一段"在整个准备过程中一致。 +- 宿主工具（`xim:` 的工具链、构建程序）仍按宿主键安装，这是宿主的事实，不是回落。 +- 旧 mcpp 读到 `descriptor_keys` 时警告并忽略（SPEC-002 §5.1）。 + +### 5.2 末段同名的构建依赖（#852） + +- 两个可见依赖末段相同、全名不同：裸名不绑定，不再每次构建警告。 +- mcpp 发布歧义的候选（`MCPP_DEP_AMBIGUOUS_=openmui.rules,gongfu.rules`）；mcpp-plugins 的 `dep_dir("rules")` 读到它时失败并 + 写出两个全名。 + +### 5.3 分发条目 + +即 §4.4（#860）。 + +## 6. Emscripten（#851.2、#851.3） + +- profile 的优化级别进入 emcc 的**链接**（wasm-opt 在链接时运行）。issue 实测：hello world 的 wasm 367 420 → 312 657 B，JS 65 920 → + 13 137 B。 +- `[target..abi] exceptions` 增加取值 `"wasm"`（`-fwasm-exceptions`）与 `"js"`；`true` 保留原义（JS 模型，2026.9.12.3 起， + docs/22:1281），并在 `mcpp why compat` 中建议 `"wasm"`。异常模型进入指纹；构建报告写出实际的模型。wasm 异常的运行环境要求 + （Safari ≥ 15.2、Node ≥ 17）写入文档。`mcpp new` 的 emscripten 模板写 `"wasm"`。 +- 栈与堆：文档给出 `ldflags = ["-sALLOW_MEMORY_GROWTH=1", "-sSTACK_SIZE=1048576"]` 的配方，不增加核心键。 + +## 7. 程序的启动（#857） + +openkal-linux 0.18.2 的启动（`src/process.cpp:684–712`）：`openat(base, name, O_PATH | O_CLOEXEC)`，再 +`execveat(exe, "", AT_EMPTY_PATH)`；ENOENT 且绝对名可得（`named`，来自 `/proc/self/fd` 的 readlink）且与 `exe` 是同一文件 +（`same_file`）时改用 `execve(绝对名)`；仍 ENOENT 时清除 CLOEXEC 再试；ENOSYS 且 `named` 时 `execve`。所以 run-as 下的 ENOENT 只能 +来自：`openat` 本身、`named` 或 `same_file` 为假（readlink 或 `newfstatat` 被拒）、或 `execve` 本身。 + +- **probe 前置且是决定性的**：模拟器 job 中，一个静态程序在 `run-as com.termux` 下走真实路径的每一步（`openat O_PATH`、readlink + `/proc/self/fd/N`、`newfstatat`、`execveat AT_EMPTY_PATH`、`execve`），记录每步的 errno 与 `logcat -b events` 的 avc。修复按测得的那 + 一步设计。 +- **不变的约束**：任何改用 `execve` 的重试都保留 `same_file` 守卫（`execve` 按名重新解析，rename、符号链接、命名空间、 + `(deleted)` 后缀都可能启动另一个文件）；守卫不成立时报告 `execveat` 的 errno。 +- **绝对名的来源**：调用方以路径给出程序时，绝对名由调用方的路径直接得到，不经 `/proc` 的 readlink（预计这正是 run-as 下 `named` + 为假的原因，待 probe 确认）。 +- **管道协议**：子进程由 `clone(SIGCHLD)` 创建，不共享地址空间，无法设置父进程的状态。管道改为带标签的消息：子进程在改用 + `execve` 之前写"已改用"，失败时写 errno；父进程据此记住此后直接用 `execve`。`:347` 的注释改为与实现一致。 +- 级联见 §12。 + +## 8. 外部构建系统接入 + +### 8.1 核心：计划阶段写出契约文件 + +- 引擎在计划阶段为每条腿写出 `${mcpp.contract}`（与 `${mcpp.compile_db}` 同一机制，供 SPEC-007 动作读取），三段： + - **编译**：每种语言的编译器程序、参数、包含目录（含目标侧包与层的头文件）、宏。 + - **链接**：链接驱动、参数、目标侧的库（供 CMake 的 try-compile；或由插件设 `CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY`）。 + - **宿主**：宿主图的编译器与参数，供需要宿主工具的项目（LLVM 的 `llvm-tblgen`、`clang-tblgen`）。 +- 动作在构建中读这个文件，不嵌套调用 mcpp（`self env --target` 会走完整的 `prepare_build`、取构建目录锁，嵌套会与父构建争锁）。 +- 核心不认识任何构建系统。 + +### 8.2 插件：扩展 mcpp-plugins 的 `deps-cmake` + +- `deps-cmake`（`deps/cmake.cppm:380–559`）读契约生成 toolchain 文件：平台标志与层的头文件（今天只传编译器路径）、链接段、宿主段 + （宿主工具先为宿主构建）；缓存键 `toolset_identity` 加入契约的摘要。 +- 产出的静态库在 `build.mcpp` 中经 `mcpp::link_flag` 给出（先例：`deps/deps.cppm:156–180`；`ldflags` 不展开 `${mcpp.out_dir}`）；一个 + `bin` 目标链接它们，mcpp 完成链接与平台的收尾（`okld note`、`okld check`、封装）。 +- openkal-musl 的 `features.h` 在 C++ 模式下对 `restrict` 的处理需要 `-Drestrict=restrict` 才能编过 LLVM（手工实验所得）：在 + openkal-musl 修复，进入 §12 的级联。 + +## 9. mcppls：一个包服务全部系统与架构 + +### 9.1 组成 + +| 部分 | 现在（每平台一个载荷） | 本设计（一个包） | +|---|---|---| +| 服务器 | `mcppls.okb`（已统一） | 不变 | +| okrun | 本系统的一个 | `runner.hosts` 的全部 | +| clangd | 每系统原生（Ubuntu 20.04 + Clang 12、macOS 14、Windows + MSVC） | `clangd.okb`：两个架构的映像（§8 构建；§9.4 的 probe 决定） | +| kit | 按宿主选一个 | 包内 openkal kit；目标 kit 按需（§9.2） | + +### 9.2 kit 按目标（S4 修订） + +- kit 回答"用户的代码按什么目标编译"。服务器改为多 kit、按项目的目标选择（今天启动时只选一个，`payload.cpp:169–190`；载荷与两个 + 编辑器端都拒绝非宿主平台的载荷，`payload.cpp:118`、`payload.ts:137`、`ServerLaunch.kt:67`）。 +- **包内 openkal kit**：libc++ 头文件与 `std`、`std.compat` 模块源码、openkal-musl 头文件，每架构一组生成头文件 + （`include/-openkal/`），生成 `libc++.modules.json`。版本校验改为与 kit 自己的 libc++ 对齐（openkal kit 的 libc++ 是 + 23.1.3 的补丁版，clangd 是 23.1.0；今天的校验要求与 clangd 版本相等，`payload.cpp:171`）。 +- **目标 kit**（`x86_64-linux-gnu`、`x86_64-w64-mingw32`、`arm64-apple-darwin`，macOS 仍要求本机 SDK）是 xlings 的 + `mcppls-kit-`，经同意后台安装；同时作为 release 资产，离线可手动安装。新增 kit 的同意与安装流程（今天的同意只覆盖构建 + 工具下载，`workspace.cpp:4003–4013`、`downloadPrompt.ts`），VS Code 与 CLion 两端都有。 +- **离线语义（行为变化，如实写出）**：没有编译器、离线、尚未装宿主 kit 时，宿主目标的文件以 openkal kit 的语义打开：`import std` + 与标准库可用，宿主的系统头文件（`windows.h`、Linux 内核头）与 `_WIN32` 等宿主宏不可用；状态栏写出缺什么与安装命令。在线时 + 第一次需要即安装宿主 kit，之后与今天相同。mcpp 项目由 mcpp 提供工具链，不受影响。 +- 要改的文件：mcppls `src/engine/payload.cpp`、`modules/pack/*`、`tools/devtools/src/{packaging,check,release,kit}.cpp`、CI 的 payload + action、`editors/vscode/src/payload.ts`、CLion 端 `ServerLaunch.kt`；S4 规范。 + +### 9.3 包与分发 + +| 产物 | 现在 | 本设计 | +|---|---|---| +| vsix | 4 个（target platform） | 1 个 | +| CLion 插件 | 4 个（`com.intellij.modules.os.*`/`arch.*`） | 1 个；mcpp-ide `mide-plugins/clion/build.gradle.kts:36–53` 不再按宿主下载 | +| 载荷 | 4 个 | 1 个 | +| xlings | 每系统一项 | 一个资产三系统，可装范围取自 `runs_on` | +| 构建 | 4 种主机与工具链 | 一台 Linux；测试在四个系统上（§3.4 的 `--archive`） | + +- 覆盖随 okrun 扩大：musl 系统、Alpine、Termux（#857 之后）可用，clangd 不再要求 glibc 2.31；Intel Mac、Windows arm64 在平台加入 + 对应运行器行后可用。 +- 体积（待实测）：vsix 约 50–60 MB（0.0.19 每平台 30–40 MB），前提是 clangd 映像的压缩率与原生相近；okb 以 stored 存放，安装后 + 含另一架构的 clangd（约 75 MB）。 + +### 9.4 clangd 映像的 probe（进行中） + +mcppls-clangd 的 draft PR（分支 `probe/clangd-openkal-image`），LLVM 源码经 xlings-res 的 GitCode 镜像获取。测量：编译错误清单、 +四个系统上 `clangd --check`、Windows 的路径风格（openkal 在 Windows 上给出原生名字；LLVM 的 `Style::native` 在编译期决定，需要 +补丁改为运行时读 `kal_env_system`）、与原生 clangd 的耗时与内存（musl 分配器的风险，必要时链接 mimalloc 或 scudo）、映像与 okb +大小。结论写入本节。 + +## 10. 编辑器 + +### 10.1 VS Code：与 mcpp-vscode 0.9.0 一模一样，再微调 + +**一致性定义在用户看得到的结果上**：命令、菜单、快捷键、设置、树视图、状态栏、通知与对话、webview 渲染出的 DOM 与样式、交互 +流程，与 0.9.0 相同。代码组织可以不同。 + +- **0.9.0 没有可替换的数据接缝**：`views/models.ts` 自己读 `mcpp.toml`、`mcpp.lock`；直接 spawn 10 处分布在 7 个文件，11 个文件直接读 + 文件系统。所以做法是**逐视图的数据适配层**：每个视图的视图模型由适配层从后端 RPC 得到，视图模型的形状与 0.9.0 相同。 +- **后端补齐的 RPC**：项目模型（0.9 的 `models.ts` 所读）、`xpkg` 详情（0.9 的 `xpkg parse`）、`cache info`/`cache dir`、库索引定位、 + 添加依赖。 +- **留在扩展端的**：VS Code 原生能力（Task 的 `ProcessExecution`、`problemMatchers`、`withProgress`、文件监视）与 mcppls 扩展的 + bridge（模块图沿用 0.9 的 mcppls `query module` 数据源，保证图相同）。 +- **HTML 生成移入页面**：0.9.0 的 HTML 写在扩展端的 TS 字符串里（如 `cachePanelHtml.ts`）。生成函数原样搬进 webview 的脚本包，扩展端 + 只发视图模型；渲染结果不变，并使 CLion 能运行同一份页面（§10.2）。 +- 1.0 新增而 0.9.0 没有的能力（调试、Task、Testing、打包、目标选择）按 0.9.0 的风格重做后加入。 +- **一致性的门**（CI 中）： + 1. **manifest**：0.9.0 `package.json` 的 `contributes` 是新版**生成后** manifest 的子集，id、标题、图标、`when`、默认值相同 + （mcpp-ide 的 manifest 由 `npm run gen` 生成，比较生成结果）。 + 2. **渲染**：同一视图模型夹具下，0.9.0 与新版各 webview 的 DOM 快照相同；原生视图（树、状态栏）在 xvfb 下截图比对。需要新建这一 + 基础设施（0.9.0 与 mcpp-ide 都没有）。 + 3. **流程**：按 0.9.0 的功能清单重写针对后端的 e2e（0.9.0 自己只有 7 个对 `fake-mcpp.js` 的冒烟测试）。 +- 通过三道门后进入微调与优化。 + +### 10.2 共用组件库 + +0.9.0 的页面（搬入页面的 HTML 生成函数、CSS、交互脚本）成为 VS Code 与 CLion 共用的 `webui/`： +- 宿主桥：页面通过一个桥对象通信，VS Code 上是 `acquireVsCodeApi`，CLion（JCEF）上由插件注入同名对象。 +- 主题：页面使用 `--vscode-*` 变量（0.9.0 有 248 处）；CLion 插件把 IntelliJ 主题映射为同名变量注入。 +- 于是 CLion 的面板与 VS Code 0.9.0 的面板一致，页面代码不分叉。 + +### 10.3 CLion 的 API + +- 替换 `isTrusted`、`startSession`、`runContentDescriptor`、`getDebuggerKind()`：替代 API 必须同时存在于 261（since-build）与 263，以两版 + jar 核对后定。 +- `plugin.xml`：261 上 test runner 属于平台核心，optional 依赖只在其存在的版本声明，注释中的"2025.2"改正。 + +### 10.4 其他 + +- 目标选择器显示集合的一行（§3.2）。 +- Open VSX 发布 1.0.x 之后的版本；mcpp-vscode 仓库在 Open VSX 同步后归档。 + +## 11. 测试 + +- SPEC-015：集合识别（首段词表；`unknown-linux-gnu` 仍是精确目标）、补全与展开（每行、planned 排除、空集、平台分歧、去重与顺序）； + `build`/`run`/`test` 对集合的行为，含执行宿主（qemu-aarch64）、"已构建、未运行"、一个都没运行时失败；规划串行与 `mcpp.lock`； + `--archive` 的写与读（CI 矩阵）；机器输出单腿逐字节不变（对照 2026.10.29.1 的输出）。 +- 打包：§13.1 与 §13.2 每一行；`tar,okb` 一次构建（构建计数断言）；启动脚本在 Git Bash 下转到 `.exe`；`.exe` 启动程序经符号链接 + 与改名后仍找到映像；`--merge` 重新生成启动器；`runs_on` 每种包与"未覆盖"一行；`emit xpkg` 按 `runs_on`（#860 的回归测试）； + 迁移规则的每条提示与拒绝。 +- 不回落：描述符键表的每一行；wasm 构建在三种宿主上读同一段；openkal 映像在 Linux 与 macOS 宿主上字节相同；#852 的两个 rules。 +- #857：probe 的逐步记录；Termux 应用沙箱 job 恢复通过。 +- 外部构建：一个 CMake 静态库经 `deps-cmake` 与契约链接成 ok 映像并在三系统运行；宿主工具的 CMake 项目。 +- 编辑器：§10.1 的三道门。 + +## 12. 交付（一轮）与级联 + +精确钉版本使一个修复逐层传递。顺序： + +| 步 | 仓库 | 内容 | +|---|---|---| +| 0 | openkal-linux（probe） | #857 的逐步 probe（§7），决定修复 | +| 1 | openkal-linux | #857 的修复、管道协议 | +| 2 | openkal-musl | 钉 openkal-linux（`mcpp.toml:181`）；`restrict` 修复（§8.2） | +| 3 | openkal-llvm-runtime | 钉 openkal-musl | +| 4 | xim-pkgindex | 以上的索引条目（openkal-image 与 mcpplibs.os 的 CI 之前） | +| 5 | openkal-image | ld、run、tools 同一版本，钉 openkal-linux（`run/mcpp.toml:18`）；okrun 与载体重建 | +| 6 | openkal-platform | `[platform.versions]` 与层；`runner.hosts`、`carrier` 列表与 `hosts`、`descriptor_keys`（实测后）；`mcpp = ">=<本轮 mcpp>"` | +| 7 | mcpplibs.os | 钉新版本组 | +| 8 | mcpp | SPEC-015、SPEC-014 rev、§5、§6、§8.1；同时接受平台的新旧两种写法；`triple.cppm:909` 钉 openkal-platform 新版 | +| 9 | mcpp-plugins | `deps-cmake` 读契约；`dep_dir` 的歧义失败 | +| 10 | mcppls-clangd | clangd 映像（§9.4 决定）、路径风格补丁、宿主 tblgen | +| 11 | mcppls | 单包、S4 修订、kit 流程、CI 改用 `--archive` | +| 12 | mcpp-ide | VS Code（§10.1）、组件库、CLion API、CI 迁移（§13.1） | +| 13 | xim-pkgindex | mcpp-ide、mcppls、okrun 的条目重新生成 | +| 14 | mcpp-vscode | Open VSX 同步后归档 | + +- **平台与 mcpp 的相互依赖**：平台的新键需要能读它们的 mcpp，而 mcpp 钉住平台。解法：本轮中平台的 CI 使用本轮 mcpp 发布分支构建 + 的 mcpp（probe PR 的做法）；mcpp 同时接受新旧两种写法；两者在同一轮发布，mcpp 的发布钉住新平台。旧 mcpp 读到列表形式的 + `carrier` 会报错（`platform_toml.cppm:460–463`，不是"未知键警告"），所以平台的 `mcpp` 下限必须提高。 +- #857 的 probe 在关键路径最前；其余在它的 CI 等待期间并行。 + +## 13. 迁移 + +### 13.1 下游 + +| 位置 | 今天 | 本设计 | +|---|---|---| +| mcpp-ide `ci.yml:62–64` | 两条 pack | `mcpp pack mcpp-ide --target openkal --mode self-contained --systems all --pkg-format tar,okb -o target/dist/` | +| mcpp-ide `ci.yml:67`、`release.yml:38–39` | 依赖 `mcpp-ide-$version.okb` | 名字不变 | +| mcpp-ide `ci.yml:70–75` | 读 `systems` | 改读 `runs_on.self`（`systems` 仍在，现写为 `-`） | +| mcpp-ide `ci.yml:285`、README（中英） | `mcpp-ide.cmd` | `mcpp-ide.exe` | +| mcppls `ci.yml:304` | macOS 行 `--target x86_64-openkal --systems macos` | `--target openkal --systems macos` | +| mcppls `ci.yml:315`、`:430–435` | 读 `systems` | 改读 `runs_on` | +| mcppls `ci.yml:328` | `mcppls.cmd` | `mcppls.exe` | +| mcppls `ci.yml:345` | `--target x86_64-openkal --pkg-format okb` | `--target openkal --pkg-format okb` | +| xim-pkgindex | Windows 别名 `.cmd` | `.exe`（重新生成） | +| mcpp 文档 | docs/10（81、116–121、206–207、734–817、900–905、935）、docs/35、docs/24、docs/specs/packaging.md、docs/specs/platforms.md、docs/zh/*、docs/50:733 | 按本设计改写 | +| mcpp 测试 | e2e 942、951、955、956、961，`test_pack_distribution.cpp` | 按本设计改写 | + +### 13.2 旧写法 + +| 写法 | 处理 | +|---|---| +| `--no-universal` + 精确目标 | 接受，提示布局的变化（M3 改为树；需要一个本机程序用 `--pkg-format single-file`） | +| `--no-universal` + 集合 | 拒绝（两者矛盾） | +| `[pack] universal = true` | 拒绝，给出 `[pack] targets = ["openkal"]` | +| `[pack] universal = false` | 提示后忽略 | +| `[platform] universal` | 提示后忽略 | +| 单个 `--target -openkal` | 打这一条腿，提示集合写法（§4.8） | + +以上都登记进 `mcpp why compat`。 + +### 13.3 被替换的条款 + +| 规范 | 条款 | 变化 | +|---|---|---| +| SPEC-013 | §1.1 `universal` | 删除作用（§13.2） | +| SPEC-013 | §1.1 `carrier`（"三键必须给出"） | 接受列表与 `hosts`；载体项可声明构建模型（§4.5） | +| SPEC-013 | §10 执行宿主的运行器程序、§15.4 `fields.carrier` | 按 `hosts` 选择载体项 | +| SPEC-013 | §11 载体的三种形式 | "嵌入一个映像"只属于单文件；判据 13（e2e 961）改写 | +| SPEC-013 | §16 平台包的 `mcpp test` | 归入 SPEC-015 §3.2 | +| SPEC-014 | §1.4、§1.5 的相关行 | §4.2、§4.7 | +| SPEC-014 | §2.1、§2.2（通用包） | §4.2、§4.3 | +| SPEC-014 | §4.1（ok 映像单腿用 `bin/`）、§4.2 命名、§4.3 启动器与 PATH | §4.2、§4.3、§4.4 | +| SPEC-014 | §4.5 `systems`、§5.2、§8（M3 的 `emit xpkg` 规则）、§9 | §4.4 | +| SPEC-002 | §6 执行宿主 | 不变，被 SPEC-015 §3.2 引用 | + +## 14. 自我评审的处理 + +三个视角（核心架构与规范、命令行与迁移、生态可行性）的意见与处理： + +| 意见 | 处理 | +|---|---| +| 集合的展开源循环依赖，`linux-musl` 的来源有歧义，`parse` 不校验首段 | §3.1：首段词表识别、CLI 层展开、按候选选平台后取交集 | +| P8 的事实：Android 不回落；`targetPlatform` 还供 semver、版本条目、索引刷新 | P8 改正；§5.1 改为目标表的一列，逐行写出行为变化 | +| 没写被替换的条款；旧 mcpp 读列表 `carrier` 报错 | §13.3；§12 的下限与相互依赖 | +| 腿下沉与 `mcpp.lock` 并发；四处腿循环；单值字段 | §3.3：规划串行、构建并行，四处收拢，单值扩展 | +| "单系统本机启动器"与 `-` 冲突；合并时启动器摘要冲突；`.cmd`/`.exe` 不一致 | §4.3：统一为 POSIX 脚本 + 引擎的 `.exe`，启动器是派生文件；取消载体的新形式 | +| `runs_on` 的 `carried` 定义与原生 `system` 不一致、与 `systems` 重复、`host = "any"`、PATH 回落 | §4.4：`self`/`with` 重定义、`systems` 保留为输入、`any` 的含义、PATH 只在 `with` 内 | +| 静默的不兼容改动；下游断点没列全 | §4.8 提示、§13.1、§13.2 | +| 机器输出违反 docs/50 §7 | §2 原则 7、§3.5 | +| 静默的交集与丢弃 | §4.4 的"未覆盖"、§4.6 的拒绝、§4.7 | +| `--pkg-format` 列表与 `-o`、`--compress`、`dir`、`single-file`、`run` 的交互 | §4.6 | +| Git Bash 选中 POSIX 脚本 | §4.3 | +| `test --archive/--from` 命名、okb、假绿 | §3.4：`--no-run --archive` 写、`--archive` 读、引擎 tar、无测试运行即失败 | +| `exceptions = true` 改义的风险 | §6：保留原义，新增取值 | +| `[pack] mode`、`[build] target` 的集合、`--systems` 的粒度、`run` 的选腿、工作空间、库目标 | §4.1、§3.1、§3.2、§4.2 | +| #857：代码已有带守卫的 ENOENT 重试；`clone` 不共享状态；`execve` 不等价 | §7 重写：probe 前置并决定修复、守卫保留、绝对名来源、带标签的管道协议 | +| `self env` 会构建、取锁；`ldflags` 不展开输出目录；缺链接与宿主契约；已有 `deps-cmake` | §8 重写 | +| 0.9.0 没有数据接缝；webview 是 VS Code 专用；三道门不现实 | §10.1、§10.2 重写：一致性定义在结果上、逐视图适配层、HTML 生成移入页面、门的重定义 | +| kit 的"默认行为不变"不成立；按目标选 kit 是新代码 | §9.2：如实写出离线语义；S4 修订与要改的文件 | +| 级联缺环节；平台与 mcpp 相互依赖 | §12 | + +## 15. 评审项 + +1. 目标集合：首段系统词表识别、CLI 层展开、按平台取交集；`[build] target` 只收精确目标（§3.1）。 +2. 腿的规划串行、构建并行，复用工作空间多配置机制（§3.3）。 +3. `test` 对集合：一个测试都没运行即失败；`--no-run --archive` 写、`--archive` 读（§3.2、§3.4）。 +4. 机器输出：单腿逐字节不变，多腿加 `legs[]`、事件加 `triple`、`pkgFormats[]`（§3.5）。 +5. 删除 `universal` 与旧写法的处理表（§4.2、§13.2）。 +6. openkal 唯一布局：POSIX 脚本 + 引擎的 `.exe`，取消 `.cmd`；M3 只属于单文件；openkal `--mode static` 的新含义（§4.3）。 +7. `runs_on` 的 `self`/`with` 与 `runner.hosts`（§4.4）。 +8. `carrier` 列表与 `hosts`、构建模型的占位（§4.5）。 +9. `--pkg-format` 列表的规则（§4.6）。 +10. 描述符键表的每一行，尤其 emscripten 的 `emscripten, linux` 与 openkal 先实测（§5.1）。 +11. `exceptions` 保留 `true` 的原义、新增 `"wasm"`（§6）。 +12. #857：probe 决定修复，守卫保留，带标签的管道协议（§7）。 +13. 契约文件三段 + 扩展 `deps-cmake`（§8）。 +14. kit 的离线语义（§9.2）。 +15. VS Code 的一致性定义、逐视图适配层、HTML 生成移入页面、三道门（§10.1、§10.2）。 +16. 交付顺序与平台、mcpp 同轮发布（§12）。 diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 4ac745e00..0a0111705 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -341 records. +342 records. ## By subject @@ -94,6 +94,7 @@ Records that declare one. Everything else is listed by date below. ### platform - [另一架构的 ok 映像的运行、测试与打包，以及 ok 映像与 Web 的关系（2026.10.29.1）](2026-10-29-openkal-run-test-pack-design.md) — active +- [生态设计：目标集合与腿、打包模型 rev、运行覆盖、不回落、外部构建接入、mcppls 单包与编辑器](2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md) — review - [平台 part 2：打包与分发——三维模型、通用包、单文件、okb、体积，以及 openkal 0.17.2 轮](2026-10-28-platforms-part2-packaging-and-distribution-design.md) — landed - [平台：目标体系的通用扩展；openkal 作为第一个平台包；IDE 一线与 mcppls 的同批升级](2026-10-28-platforms-design.md) — landed @@ -151,6 +152,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-10 - [另一架构的 ok 映像的运行、测试与打包，以及 ok 映像与 Web 的关系（2026.10.29.1）](2026-10-29-openkal-run-test-pack-design.md) — active +- [生态设计：目标集合与腿、打包模型 rev、运行覆盖、不回落、外部构建接入、mcppls 单包与编辑器](2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md) — review - [平台 part 2：打包与分发——三维模型、通用包、单文件、okb、体积，以及 openkal 0.17.2 轮](2026-10-28-platforms-part2-packaging-and-distribution-design.md) — landed - [平台：目标体系的通用扩展；openkal 作为第一个平台包；IDE 一线与 mcppls 的同批升级](2026-10-28-platforms-design.md) — landed - [生态设计 part 2：openkal 的最小原子与二进制映像、其上的 `mcpplibs.os`，以及 part 1 的收尾与回归记录](2026-10-26-ecosystem-part2-os-library-over-openkal-design.md) — active From 734d9c6d94f861869c68641fc85c2381ed882a36 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 01:40:46 +0900 Subject: [PATCH 02/20] descriptor keys by the target's row; runner hosts, carrier entries and descriptor_keys in a platform; an ambiguous dependency name is no package's; the Web link has the profile's level and the exception model; the build's contract - A target reads the descriptor blocks its row names, in order, and never the build machine's (#851): emscripten, linux; ios, macosx; openkal, linux; none. A platform states its own with [platform] descriptor_keys. - [platform] runner.hosts; carrier as a table or an array of entries with hosts and a build model; the entry serving an - is chosen. - A name two dependencies answer to is published for neither; the program reads MCPP_DEP_AMBIGUOUS_, and dep_dir() of it ends naming both (#852). - emcc's link carries the profile's optimisation level (#851); [target..abi] exceptions takes "wasm" or "js", true keeping the JavaScript model. - ${mcpp.contract}: the configuration's compilers, words and link, written beside its compile database for a foreign build to compile for the target. - Version 2026.10.30.1. Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...arget-sets-packaging-and-editors-design.md | 73 ++++++++------- .agents/docs/README.md | 4 +- ask.log | 1 + docs/11-publishing-a-library.md | 19 ++++ docs/22-target-side.md | 15 ++- docs/30-build-mcpp.md | 7 ++ docs/specs/platforms.md | 27 +++++- docs/zh/11-publishing-a-library.md | 14 +++ docs/zh/22-target-side.md | 10 +- docs/zh/30-build-mcpp.md | 4 + mcpp.toml | 2 +- modules/manifest/src/platform.cppm | 55 ++++++++++- modules/manifest/src/platform_toml.cppm | 83 +++++++++++++++-- modules/manifest/src/toml.cppm | 21 ++++- modules/manifest/src/types.cppm | 5 + modules/manifest/src/xpkg.cppm | 40 +++++--- modules/toolchain-model/src/platform.cppm | 68 +++++++++----- .../tests/test_platform_axis.cpp | 23 +++++ modules/versioning/src/version.cppm | 2 +- src/build/build_program.cppm | 8 ++ src/build/compile_commands.cppm | 61 +++++++++++++ src/build/hostprogram.cppm | 16 ++++ src/build/ninja_backend.cppm | 2 + src/build/prepare/config.cpp | 1 + src/build/prepare/graph.cpp | 43 ++++++--- src/build/prepare/plan.cpp | 3 + src/build/prepare/platform.cpp | 8 +- src/build/prepare/toolchain.cpp | 34 +++++-- src/pack/pack.cppm | 26 ++++-- src/platform/platform.cppm | 16 +++- src/project.cppm | 7 +- ...mbiguous_dependency_name_is_no_packages.sh | 91 +++++++++++++++++++ ...iles_level_and_the_wasm_exception_model.sh | 86 ++++++++++++++++++ ...982_an_action_reads_the_builds_contract.sh | 54 +++++++++++ tests/unit/test_descriptor_keys.cpp | 75 +++++++++++++++ tests/unit/test_platform.cpp | 78 +++++++++++++++- 36 files changed, 948 insertions(+), 134 deletions(-) create mode 100644 ask.log create mode 100755 tests/e2e/980_an_ambiguous_dependency_name_is_no_packages.sh create mode 100755 tests/e2e/981_a_web_link_has_the_profiles_level_and_the_wasm_exception_model.sh create mode 100755 tests/e2e/982_an_action_reads_the_builds_contract.sh create mode 100644 tests/unit/test_descriptor_keys.cpp diff --git a/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md b/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md index d9f67646c..93f2dc134 100644 --- a/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md +++ b/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md @@ -1,6 +1,6 @@ --- subject: platform -status: review +status: active --- # 生态设计：目标集合与腿、打包模型 rev、运行覆盖、不回落、外部构建接入、mcppls 单包与编辑器 @@ -41,7 +41,7 @@ mcppls S4；新增 SPEC-015（目标集合与腿）。 - **运行覆盖。** 打包时计算一次 `runs_on`，分发条目、启动器、合并、摘要与 CI 都读它；单文件定义为"最少的载体覆盖腿 × 系统"， APE 载体到来时命令不变。 - **不回落。** 名字只解析到它确切指代的东西；显式声明的别名与展开不是回落；改变行为的地方逐条写出。 -- **外部构建接入。** 引擎在计划阶段写出目标的编译、链接、宿主三段契约；mcpp-plugins 的 `deps-cmake` 读它；静态库经 `link_flag` +- **外部构建接入。** 引擎为每个配置写出编译、链接两段契约（宿主工具在宿主图构建）；mcpp-plugins 的 `deps-cmake` 读它；静态库经 `link_flag` 进入 mcpp 的链接与映像收尾。clangd 因此可以成为 ok 映像，mcppls 收敛为一个包。 - **编辑器。** VS Code 扩展的界面与交互以 mcpp-vscode 0.9.0 为准，一致性定义在渲染结果上；数据经 mcpp-ide 后端的逐视图适配层 获得；0.9.0 的页面成为 VS Code 与 CLion 共用的组件库。 @@ -302,43 +302,54 @@ mcppls S4；新增 SPEC-015（目标集合与腿）。 - profile 的优化级别进入 emcc 的**链接**（wasm-opt 在链接时运行）。issue 实测：hello world 的 wasm 367 420 → 312 657 B，JS 65 920 → 13 137 B。 - `[target..abi] exceptions` 增加取值 `"wasm"`（`-fwasm-exceptions`）与 `"js"`；`true` 保留原义（JS 模型，2026.9.12.3 起， - docs/22:1281），并在 `mcpp why compat` 中建议 `"wasm"`。异常模型进入指纹；构建报告写出实际的模型。wasm 异常的运行环境要求 - （Safari ≥ 15.2、Node ≥ 17）写入文档。`mcpp new` 的 emscripten 模板写 `"wasm"`。 + docs/22:1281）。`true` 不是弃用的写法，所以不进 `mcpp why compat` 的弃用表；文档建议新项目写 `"wasm"`。标志在方言标志集中， + 模型因此进入每个缓存键。wasm 异常的运行环境要求（Safari ≥ 15.2、Node ≥ 17）写入文档。依赖的 `requires_abi` 仍是布尔： + 依赖要求的是"有异常"，模型由根决定。 - 栈与堆：文档给出 `ldflags = ["-sALLOW_MEMORY_GROWTH=1", "-sSTACK_SIZE=1048576"]` 的配方，不增加核心键。 ## 7. 程序的启动（#857） -openkal-linux 0.18.2 的启动（`src/process.cpp:684–712`）：`openat(base, name, O_PATH | O_CLOEXEC)`，再 -`execveat(exe, "", AT_EMPTY_PATH)`；ENOENT 且绝对名可得（`named`，来自 `/proc/self/fd` 的 readlink）且与 `exe` 是同一文件 -（`same_file`）时改用 `execve(绝对名)`；仍 ENOENT 时清除 CLOEXEC 再试；ENOSYS 且 `named` 时 `execve`。所以 run-as 下的 ENOENT 只能 -来自：`openat` 本身、`named` 或 `same_file` 为假（readlink 或 `newfstatat` 被拒）、或 `execve` 本身。 - -- **probe 前置且是决定性的**：模拟器 job 中，一个静态程序在 `run-as com.termux` 下走真实路径的每一步（`openat O_PATH`、readlink - `/proc/self/fd/N`、`newfstatat`、`execveat AT_EMPTY_PATH`、`execve`），记录每步的 errno 与 `logcat -b events` 的 avc。修复按测得的那 - 一步设计。 -- **不变的约束**：任何改用 `execve` 的重试都保留 `same_file` 守卫（`execve` 按名重新解析，rename、符号链接、命名空间、 - `(deleted)` 后缀都可能启动另一个文件）；守卫不成立时报告 `execveat` 的 errno。 -- **绝对名的来源**：调用方以路径给出程序时，绝对名由调用方的路径直接得到，不经 `/proc` 的 readlink（预计这正是 run-as 下 `named` - 为假的原因，待 probe 确认）。 -- **管道协议**：子进程由 `clone(SIGCHLD)` 创建，不共享地址空间，无法设置父进程的状态。管道改为带标签的消息：子进程在改用 - `execve` 之前写"已改用"，失败时写 errno；父进程据此记住此后直接用 `execve`。`:347` 的注释改为与实现一致。 -- 级联见 §12。 +**实测（probe PR #861，run 40908273317，Android 14 x86_64 模拟器，`run-as com.termux`，SELinux `runas_app`，静态 strace）。** +逐步直接调用 openkal-linux 0.18.2 启动程序的每一步——`openat(O_PATH)`、readlink `/proc/self/fd`、`fstatat`、 +`execveat(AT_EMPTY_PATH)`、`execve`——对发布包内的 `xlings`、`mcpp` 与 app 自己写的静态程序全部成功（run 40904947962）。 +失败的 `mcpp build` 在 strace 下只有一个失败的调用，而且发生在任何子进程创建之前： + +``` +openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_DIRECTORY) = -1 EACCES (Permission denied) +``` + +app 的 SELinux 域不允许**读** `/`。openkal-linux 以只读方式打开预开放目录（`.` 与 `/`）与每个 `kal_dir`，根目录的句柄因此 +不存在，以绝对名启动的每个程序在根下解析名字时失败，报告为"找不到"。#857 与 `execveat` 无关；rev 1、rev 2 的推测均被实测否定。 + +**修复（openkal-linux 0.18.3）。** 目录句柄只用于命名：在其下解析名字（`*at`）、在其中启动程序（`fchdir`）、描述它 +（`fstat`、`fstatfs`），都不读目录。句柄改以 `O_PATH | O_DIRECTORY | O_CLOEXEC` 打开，不要求读权限；列举目录时 +`kal_fs_list_begin` 本就经句柄另开一个可读的描述符，读权限只在列举时需要。新测试 `tests/search_only_base.cpp` 以 +0111 权限的目录复现同一情形（可搜索不可读的目录作为基准目录：打开其中的文件、在其中启动程序成功；列举被拒绝）。 + +`process.cpp` 中"首次 ENOSYS 之后改用 execve"的注释与实现不符（子进程由 `clone` 复制、不共享内存，所学不传回父进程）： +改为如实陈述——每次遇到 ENOSYS 的启动在子进程中各自回退，代价是每次多一个被拒绝的系统调用。rev 2 设想的带标签管道协议 +因此不需要。 + +级联见 §12。 ## 8. 外部构建系统接入 ### 8.1 核心：计划阶段写出契约文件 -- 引擎在计划阶段为每条腿写出 `${mcpp.contract}`（与 `${mcpp.compile_db}` 同一机制，供 SPEC-007 动作读取），三段： - - **编译**：每种语言的编译器程序、参数、包含目录（含目标侧包与层的头文件）、宏。 - - **链接**：链接驱动、参数、目标侧的库（供 CMake 的 try-compile；或由插件设 `CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY`）。 - - **宿主**：宿主图的编译器与参数，供需要宿主工具的项目（LLVM 的 `llvm-tblgen`、`clang-tblgen`）。 -- 动作在构建中读这个文件，不嵌套调用 mcpp（`self env --target` 会走完整的 `prepare_build`、取构建目录锁，嵌套会与父构建争锁）。 +- 引擎为每个配置写出 `mcpp-contract.json`（与配置的编译数据库同目录、同时写出），构建程序的动作以 `${mcpp.contract}` 引用它： + - **编译**：C、C++、汇编各自的编译器程序与参数——构建中每个单元都带的词（目标、sysroot、目标侧的头文件、平台与 profile 的 + 标志）。包自己的包含目录不在其中：外部构建写它自己的。 + - **链接**：链接驱动、参数、归档工具（供 CMake 的 try-compile；或由插件设 `CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY`）。 +- **宿主工具**（LLVM 的 `llvm-tblgen`、`clang-tblgen`）在宿主图中构建：一个包把它作为 `[build-dependencies]` 的工具引用，工具包 + 的构建在宿主图的配置里，读的是那个配置自己的契约。契约因此只有本配置的两段，没有"宿主段"。 +- 动作在 ninja 运行中读这个文件，不嵌套调用 mcpp（`self env --target` 会走完整的 `prepare_build`、取构建目录锁，嵌套会与父构建 + 争锁）。 - 核心不认识任何构建系统。 ### 8.2 插件：扩展 mcpp-plugins 的 `deps-cmake` -- `deps-cmake`（`deps/cmake.cppm:380–559`）读契约生成 toolchain 文件：平台标志与层的头文件（今天只传编译器路径）、链接段、宿主段 - （宿主工具先为宿主构建）；缓存键 `toolset_identity` 加入契约的摘要。 +- `deps-cmake`（`deps/cmake.cppm:380–559`）读契约生成 toolchain 文件：平台标志与层的头文件（今天只传编译器路径）与链接段； + 宿主工具由宿主图中的工具包提供（§8.1）；缓存键 `toolset_identity` 加入契约的摘要。 - 产出的静态库在 `build.mcpp` 中经 `mcpp::link_flag` 给出（先例：`deps/deps.cppm:156–180`；`ldflags` 不展开 `${mcpp.out_dir}`）；一个 `bin` 目标链接它们，mcpp 完成链接与平台的收尾（`okld note`、`okld check`、封装）。 - openkal-musl 的 `features.h` 在 C++ 模式下对 `restrict` 的处理需要 `-Drestrict=restrict` 才能编过 LLVM（手工实验所得）：在 @@ -454,8 +465,8 @@ mcppls-clangd 的 draft PR（分支 `probe/clangd-openkal-image`），LLVM 源 | 步 | 仓库 | 内容 | |---|---|---| -| 0 | openkal-linux（probe） | #857 的逐步 probe（§7），决定修复 | -| 1 | openkal-linux | #857 的修复、管道协议 | +| 0 | mcpp（probe PR #861） | #857 的逐步 probe 与 strace（§7） | +| 1 | openkal-linux 0.18.3 | 目录句柄以 `O_PATH` 打开；注释改正 | | 2 | openkal-musl | 钉 openkal-linux（`mcpp.toml:181`）；`restrict` 修复（§8.2） | | 3 | openkal-llvm-runtime | 钉 openkal-musl | | 4 | xim-pkgindex | 以上的索引条目（openkal-image 与 mcpplibs.os 的 CI 之前） | @@ -541,7 +552,7 @@ mcppls-clangd 的 draft PR（分支 `probe/clangd-openkal-image`），LLVM 源 | `test --archive/--from` 命名、okb、假绿 | §3.4：`--no-run --archive` 写、`--archive` 读、引擎 tar、无测试运行即失败 | | `exceptions = true` 改义的风险 | §6：保留原义，新增取值 | | `[pack] mode`、`[build] target` 的集合、`--systems` 的粒度、`run` 的选腿、工作空间、库目标 | §4.1、§3.1、§3.2、§4.2 | -| #857：代码已有带守卫的 ENOENT 重试；`clone` 不共享状态；`execve` 不等价 | §7 重写：probe 前置并决定修复、守卫保留、绝对名来源、带标签的管道协议 | +| #857：代码已有带守卫的 ENOENT 重试；`clone` 不共享状态；`execve` 不等价 | §7：probe 前置；strace 测得真因（根目录句柄的读权限），修复为目录句柄改用 `O_PATH` | | `self env` 会构建、取锁；`ldflags` 不展开输出目录；缺链接与宿主契约；已有 `deps-cmake` | §8 重写 | | 0.9.0 没有数据接缝；webview 是 VS Code 专用；三道门不现实 | §10.1、§10.2 重写：一致性定义在结果上、逐视图适配层、HTML 生成移入页面、门的重定义 | | kit 的"默认行为不变"不成立；按目标选 kit 是新代码 | §9.2：如实写出离线语义；S4 修订与要改的文件 | @@ -560,8 +571,8 @@ mcppls-clangd 的 draft PR（分支 `probe/clangd-openkal-image`），LLVM 源 9. `--pkg-format` 列表的规则（§4.6）。 10. 描述符键表的每一行，尤其 emscripten 的 `emscripten, linux` 与 openkal 先实测（§5.1）。 11. `exceptions` 保留 `true` 的原义、新增 `"wasm"`（§6）。 -12. #857：probe 决定修复，守卫保留，带标签的管道协议（§7）。 -13. 契约文件三段 + 扩展 `deps-cmake`（§8）。 +12. #857：实测为根目录句柄的读权限；目录句柄改用 `O_PATH`（§7）。 +13. 契约文件两段（编译、链接；宿主工具在宿主图构建）+ 扩展 `deps-cmake`（§8）。 14. kit 的离线语义（§9.2）。 15. VS Code 的一致性定义、逐视图适配层、HTML 生成移入页面、三道门（§10.1、§10.2）。 16. 交付顺序与平台、mcpp 同轮发布（§12）。 diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 0a0111705..9c75a083e 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -94,7 +94,7 @@ Records that declare one. Everything else is listed by date below. ### platform - [另一架构的 ok 映像的运行、测试与打包，以及 ok 映像与 Web 的关系（2026.10.29.1）](2026-10-29-openkal-run-test-pack-design.md) — active -- [生态设计：目标集合与腿、打包模型 rev、运行覆盖、不回落、外部构建接入、mcppls 单包与编辑器](2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md) — review +- [生态设计：目标集合与腿、打包模型 rev、运行覆盖、不回落、外部构建接入、mcppls 单包与编辑器](2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md) — active - [平台 part 2：打包与分发——三维模型、通用包、单文件、okb、体积，以及 openkal 0.17.2 轮](2026-10-28-platforms-part2-packaging-and-distribution-design.md) — landed - [平台：目标体系的通用扩展；openkal 作为第一个平台包；IDE 一线与 mcppls 的同批升级](2026-10-28-platforms-design.md) — landed @@ -152,7 +152,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-10 - [另一架构的 ok 映像的运行、测试与打包，以及 ok 映像与 Web 的关系（2026.10.29.1）](2026-10-29-openkal-run-test-pack-design.md) — active -- [生态设计：目标集合与腿、打包模型 rev、运行覆盖、不回落、外部构建接入、mcppls 单包与编辑器](2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md) — review +- [生态设计：目标集合与腿、打包模型 rev、运行覆盖、不回落、外部构建接入、mcppls 单包与编辑器](2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md) — active - [平台 part 2：打包与分发——三维模型、通用包、单文件、okb、体积，以及 openkal 0.17.2 轮](2026-10-28-platforms-part2-packaging-and-distribution-design.md) — landed - [平台：目标体系的通用扩展；openkal 作为第一个平台包；IDE 一线与 mcppls 的同批升级](2026-10-28-platforms-design.md) — landed - [生态设计 part 2：openkal 的最小原子与二进制映像、其上的 `mcpplibs.os`，以及 part 1 的收尾与回归记录](2026-10-26-ecosystem-part2-os-library-over-openkal-design.md) — active diff --git a/ask.log b/ask.log new file mode 100644 index 000000000..ad3f15e50 --- /dev/null +++ b/ask.log @@ -0,0 +1 @@ +/bin/bash: line 32: : command not found diff --git a/docs/11-publishing-a-library.md b/docs/11-publishing-a-library.md index bc2bd8d1a..4937017ba 100644 --- a/docs/11-publishing-a-library.md +++ b/docs/11-publishing-a-library.md @@ -90,6 +90,25 @@ tarball is the same bytes on every platform, and an entry present in only one of them fails on the others as "no such version", which reads like a typo in the consumer's manifest. +A target that is none of the three reads the blocks its row names, in order, +and never the build machine's (2026.10.30.1+): + +| Target | Blocks, in order | +|---|---| +| Linux (Android and musl included), macOS, Windows | `linux`, `macosx`, `windows` respectively | +| `wasm32-emscripten` | `emscripten`, then `linux` | +| `-openkal` | `openkal`, then `linux` (a platform may state its own: `[platform] descriptor_keys`) | +| iOS | `ios`, then `macosx` | +| bare metal (`-none-`) | `none` | + +The first block present is the one read, both for the versions under `xpm` +and for the per-platform table of the `mcpp` segment. A library that builds +differently for the Web — without the `-pthread` its Linux block adds, say — +gives it an `emscripten` block; one that builds for bare metal gives it a +`none` block, and without one it has no version there. Before 2026.10.30.1 such +a target read the build machine's block, so the same project took different +flags on a Linux and on a Windows machine. + ## 4. Wait for the artifact **The index is an artifact, not a git clone.** Merging to `main` is not diff --git a/docs/22-target-side.md b/docs/22-target-side.md index aa521815e..e8ac04fd4 100644 --- a/docs/22-target-side.md +++ b/docs/22-target-side.md @@ -1278,12 +1278,19 @@ package needs. | Member | Type | Renders as | Reaches | |---|---|---|---| | `threads` | boolean | `-pthread` on a target that is neither PE nor freestanding; nothing on PE and on freestanding targets | the standard library module prebuild, the dependency scan, every C and C++ translation unit of every package, and the link | -| `exceptions` *(mcpp 2026.9.12.3+)* | boolean | `-fexceptions`, on `os = "emscripten"` only, on the compile line through the dialect flags and on the link; nothing on every other target, where exceptions are already the default | the same set `threads` reaches | +| `exceptions` *(mcpp 2026.9.12.3+)* | boolean, or `"wasm"` / `"js"` *(2026.10.30.1+)* | on `os = "emscripten"` only, on the compile line through the dialect flags and on the link: `"wasm"` is `-fwasm-exceptions`, WebAssembly's own exception handling; `"js"` and `true` are `-fexceptions`, the JavaScript-based model. Nothing on every other target, where exceptions are already the default | the same set `threads` reaches | Both members enter the dependency cache key through the dialect flags, so a -dependency built without one is never reused by a build with it. An unknown -member, and a member that is not a boolean, are refused naming both `threads` -and `exceptions`. +dependency built without one is never reused by a build with it, and one built +with one exception model is never reused by a build with the other. An unknown +member, and a value outside the table, are refused naming the values. + +**`"wasm"` is the model to choose for a new program.** Its code is smaller and +faster than the JavaScript model's, and it needs a host with WebAssembly +exception handling: Safari 15.2, Firefox 100, Chrome 95, Node 17 or later. +`true` keeps the meaning it has had since 2026.9.12.3, the JavaScript model. A +dependency's `requires_abi = { exceptions = true }` asks for exceptions and not +for a model; the root's choice serves it. **Without `exceptions`, the observed failure is at run time, not at link.** A Web program that throws across an `import std` boundary builds and links with diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index bd8d87a14..95a61f1ad 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -1089,6 +1089,7 @@ none to rely on), and the only interpolations are a closed set: | `${mcpp.out_dir}` | the build output directory | | `${mcpp.bin_dir}` | where produced binaries land: `bin/`, or, for an action a workspace member's build program declares, that member's product directory (2026.9.29.4+) | | `${mcpp.compile_db}` | path to `compile_commands.json` (what clang-tidy's `-p` wants) | +| `${mcpp.contract}` *(2026.10.30.1+)* | path to the configuration's contract, `mcpp-contract.json`, beside its compile database: the compiler of each language with the words every unit is compiled with (target, sysroot, target-side headers, platform and profile flags), and the link driver, its words and the archiver. A foreign build system reads it to compile for the target as the engine does; a package's own include directories are not in it, nor the engine's C++ dialect and module words (`-std=`, `-fmodules`), which are its choice for its own units | | `${mcpp.target_file:}` | the built file of target `` | | `${mcpp.stage_dir}` *(2026.9.11.1+)* | the tree `mcpp pack` staged, absolute. `artifact` role only, and only under `mcpp pack --pkg-format ` | | `${mcpp.self}` *(2026.9.13.1+)* | the engine's own executable, absolute. An action's command is an argv with no shell, so a build program has no portable way to copy a file; the engine is present wherever a build runs, and `${mcpp.self} stage --verify content --output ` copies one file, creates the destination's parent, and writes only when the bytes differ. That argument shape is a contract from 2026.9.13.1 on; a build program that names it declares that release as its floor | @@ -1937,6 +1938,12 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } `dep_bin("spike.installer", ...)` as well as `dep_bin("installer", ...)` (mcpp 2026.9.16.1+; before that release its tool was published under `installer` alone). +- **A name two dependencies answer to is no package's** (2026.10.30.1+). Two + packages whose manifests both say `rules` (`openmui.rules`, `gongfu.rules`) + are each addressed by their qualified name; `rules` itself is published for + neither, and `MCPP_DEP_AMBIGUOUS_RULES` lists the qualified names. + `dep_dir("rules")` ends the program with a message naming them. Before + 2026.10.30.1 the name went to one of the two and every build warned. #### Older mcpp reading a manifest that uses this diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 4d0a35320..cbb4fa11c 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -43,9 +43,10 @@ targets = ["", ...] toolchain = "" sysroot = "" | { path = "", provides = [...] } -runner = ["", "{}"] | { cmd = [...], host = "same-arch" | "any", exclusive = } +runner = ["", "{}"] | { cmd = [...], host = "same-arch" | "any", hosts = ["-", ...], exclusive = } runners = { = ["", ...] } -carrier = { package = "", section = "", main = "" } +carrier = { package = "", section = "", main = "", hosts = [...], build = "row" } | [ {...}, ... ] +descriptor_keys = ["", ...] plugin = "build.mcpp" [platform.layers] = | false @@ -66,7 +67,8 @@ plugin = "build.mcpp" | `sysroot` | 字符串或 `{ path, provides }` | 预构建的目标基,写法同 `[target.].sysroot`;`path` 相对于平台包 | | `runner` | 字符串数组或表 | §10 | | `runners` | ` = [argv]` | 命名的设备动作,每个 argv 非空 | -| `carrier` | 表 | §11;`package`、`section`、`main` 三键都**必须**给出 | +| `carrier` | 表或表的数组 | §11;每项的 `package`、`section`、`main` 三键都**必须**给出;`hosts` 为该项服务的 `-`(省略:服务每一对);`build` 为载体程序的构建模型,缺省 `"row"`(每个 `-` 一个程序,经平台的 hosted 安排构建),本 mcpp 不认识的模型的项被跳过并警告。**mcpp >= 2026.10.30.1**;更早的 mcpp 读到数组即拒绝,写数组的平台**必须**声明相应的 `mcpp` 下限 | +| `descriptor_keys` | 非空字符串数组 | 平台目标读取描述符的哪些按平台块及其顺序(第一个存在的块被读取),取值为 `linux`、`macosx`、`windows`、`android`、`ios`、`emscripten`、`openkal`、`none`;未声明时为目标行自己的键(§14)。**mcpp >= 2026.10.30.1** | | `plugin` | 字符串 | 只能为 `"build.mcpp"`:平台包自身的构建程序(SPEC-007)在平台阶段运行(§12);其他值拒绝 | | `universal` | 布尔 | `true` 时本平台任一目标的打包缺省为覆盖全部 `targets` 的通用包(SPEC-014 §1.4、§4.2) | | `layers` | 表 | §7.2;键只能是 §0 的四个层名 | @@ -295,6 +297,7 @@ plugin = "build.mcpp" | `cmd` | argv,非空;`{}` 为产物 | | `host` | `"same-arch"`:产物只在其架构的机器上运行;`"any"`:运行器可以从任何机器到达产物(模拟器、探针) | | `exclusive` | 为真时该目标的运行不重叠,`mcpp test` 串行运行 | +| `hosts` | 运行器程序存在的 `-`(`os` 为 `linux`、`macos`、`windows`):需要该运行器的包能在哪里运行(SPEC-014 §4.4 的 `runs_on.with`)。`host = "any"` 时与产物的架构无关。**mcpp >= 2026.10.30.1** | `host = "same-arch"` 而产物的架构不是本机架构时,`mcpp run` 与 `mcpp test` 拒绝运行,消息给出 `[target.] run_under` 的写法;`run_under` 存在或给出 `--no-runner` 时不拒绝。运行器的合并顺序与 §6 @@ -345,6 +348,10 @@ run_under = ["qemu-aarch64"] 需要载体的打包被拒绝并写出该键;本 mcpp 不构建程序的行(SPEC-014 §4.4)上的载体在生成工程之前被拒绝,写出系统、架构与行 (2026.10.29.1)。 +**载体项的选择(mcpp >= 2026.10.30.1)。** 为 `-` 构建载体程序时(上述三处与 §10 的执行宿主),所用的是 +`hosts` 列出该对的第一项,否则为第一个未写 `hosts` 的项;两者都没有时,如同平台未声明 `carrier`,打包或运行被拒绝,写出 +该对。`build` 不是 `"row"` 的项留给将来的构建模型(一个文件服务多个系统与架构的载体,mcpp#849),本 mcpp 读到时跳过并警告。 + ## 12. 平台阶段与协议 17 ### 12.1 平台阶段 @@ -384,6 +391,20 @@ run_under = ["qemu-aarch64"] 选择来源报告为 `built-in`。项目可以不等引擎更新,以 `[target.] platform` 选择另一个版本。 **已实现** +**描述符键(mcpp >= 2026.10.30.1)。** 每个目标行声明它读取依赖描述符的哪些按平台块及其顺序,第一个存在的块被读取; +`xpm` 下的版本与 `mcpp` 段的按平台表都按此读取,构建机不参与: + +| 目标 | 依次读取的块 | +|---|---| +| Linux(含 Android 与 musl)、macOS、Windows | `linux`、`macosx`、`windows` 分别 | +| `wasm32-emscripten` | `emscripten`,其次 `linux` | +| `-openkal` | `openkal`,其次 `linux`;平台的 `descriptor_keys`(§1.1)取代之 | +| iOS | `ios`,其次 `macosx` | +| 裸机(`os = none`) | `none` | + +平台在首次准备时尚未读到(须先取得平台包)而又声明了 `descriptor_keys` 时,准备以读到的平台重新开始,与 `sysroot` +相同。宿主工具(宿主图)按宿主的键安装。此前,不是三者之一的目标读构建机的块(mcpp#851)。 + ok 映像是 x86_64 与 aarch64 上位置无关的 ELF,只导入 `kal_*`,运行时需要加载器发布的可执行内存;okb 只装 ok 映像。 openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscripten` 目标(宿主目标,无平台包)到达,打包为 页面经 `dist-web` 提供者;以 openkal API 写的代码经 openkal 的 Emscripten 实现(openkal-emscripten)到达 Web。 diff --git a/docs/zh/11-publishing-a-library.md b/docs/zh/11-publishing-a-library.md index c12a89565..5b8d20581 100644 --- a/docs/zh/11-publishing-a-library.md +++ b/docs/zh/11-publishing-a-library.md @@ -82,6 +82,20 @@ cmp cn.tar.gz pkg-0.0.48.tar.gz # must be identical, not merely present 同样的字节；只写在其中一个平台块里，会在另外两个平台上以 `no such version` 失败， 读起来像是消费方 manifest 里打错了字。 +不属于这三者的目标，按其目标行列出的块依次读取，与构建机无关（2026.10.30.1+）： + +| 目标 | 依次读取的块 | +|---|---| +| Linux（含 Android 与 musl）、macOS、Windows | 分别为 `linux`、`macosx`、`windows` | +| `wasm32-emscripten` | `emscripten`，其次 `linux` | +| `-openkal` | `openkal`，其次 `linux`（平台可声明自己的：`[platform] descriptor_keys`） | +| iOS | `ios`，其次 `macosx` | +| 裸机（`-none-`） | `none` | + +读取的是第一个存在的块，`xpm` 下的版本与 `mcpp` 段的按平台表都如此。一个为 Web 构建时有所不同的库——例如不要 +Linux 块添加的 `-pthread`——写一个 `emscripten` 块；为裸机构建的库写一个 `none` 块，没有它就在裸机上没有版本。 +2026.10.30.1 之前，这类目标读构建机的块，同一个项目在 Linux 与 Windows 机器上会得到不同的标志。 + ## 4. 等待 artifact **索引是一个 artifact，不是一次 git clone。** 合进 `main` 还不够：必须等 diff --git a/docs/zh/22-target-side.md b/docs/zh/22-target-side.md index 9b48cd2ce..465570b79 100644 --- a/docs/zh/22-target-side.md +++ b/docs/zh/22-target-side.md @@ -1074,11 +1074,15 @@ WebAssembly 上，每个目标文件、预编译的标准库模块与链接必 | 成员 | 类型 | 渲染为 | 到达 | |---|---|---|---| | `threads` | 布尔 | 在既非 PE 也非 freestanding 的目标上为 `-pthread`；在 PE 与 freestanding 目标上不产生任何 flag | 标准库模块的预构建、依赖扫描、所有包的每个 C 与 C++ 翻译单元，以及链接 | -| `exceptions` *(mcpp 2026.9.12.3+)* | 布尔 | `-fexceptions`，只在 `os = "emscripten"` 上，经方言 flag 进入编译行，也进入链接行；在其余每个目标上什么都不产生，因为那里异常本来就是默认开启的 | 与 `threads` 相同的那一套 | +| `exceptions` *(mcpp 2026.9.12.3+)* | 布尔，或 `"wasm"` / `"js"` *(2026.10.30.1+)* | 只在 `os = "emscripten"` 上，经方言 flag 进入编译行，也进入链接行：`"wasm"` 为 `-fwasm-exceptions`，即 WebAssembly 自身的异常处理；`"js"` 与 `true` 为 `-fexceptions`，即基于 JavaScript 的模型。在其余每个目标上什么都不产生，因为那里异常本来就是默认开启的 | 与 `threads` 相同的那一套 | 两个成员都经由方言 flag 进入依赖缓存键，因此没有启用某个成员的构建，不会 -被启用它的构建复用。未知成员，以及不是布尔值的成员，都会被拒绝，并同时 -列出 `threads` 与 `exceptions`。 +被启用它的构建复用，以一种异常模型构建的依赖也不会被另一种模型的构建复用。 +未知成员，以及表外的取值，都会被拒绝，并列出可取的值。 + +**新程序应选 `"wasm"`。** 它的代码比 JavaScript 模型更小、更快，要求宿主支持 WebAssembly 异常处理：Safari 15.2、 +Firefox 100、Chrome 95、Node 17 及以上。`true` 保持自 2026.9.12.3 起的含义，即 JavaScript 模型。依赖的 +`requires_abi = { exceptions = true }` 要求的是"有异常"而不是某种模型，由根的选择满足。 **没有 `exceptions`，观察到的失败在运行时，不在链接时。** 一个跨 `import std` 边界抛出异常的 Web 程序能正常编译并链接——Emscripten 的 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index b6057e40d..3d53cfa82 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -925,6 +925,7 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件，永不 | `${mcpp.out_dir}` | 构建输出目录 | | `${mcpp.bin_dir}` | 产出的二进制所在目录:`bin/`;工作区成员的构建程序声明的 action 中为该成员的产物目录(2026.9.29.4+) | | `${mcpp.compile_db}` | `compile_commands.json` 的路径（clang-tidy 的 `-p` 要的就是它） | +| `${mcpp.contract}` *(2026.10.30.1+)* | 配置的契约 `mcpp-contract.json` 的路径，与该配置的编译数据库同目录：每种语言的编译器与每个单元都带的词（目标、sysroot、目标侧的头文件、平台与 profile 的标志），以及链接驱动、它的词与归档工具。外部构建系统读它，以引擎的方式为目标编译；包自己的包含目录不在其中，引擎为自己的单元选择的 C++ 方言与模块词（`-std=`、`-fmodules`）也不在其中 | | `${mcpp.target_file:}` | target `` 构建出的文件 | | `${mcpp.stage_dir}` *(2026.9.11.1+)* | `mcpp pack` 暂存出的那棵树，绝对路径。仅 `artifact` role 可用，且仅在 `mcpp pack --pkg-format ` 下可用 | | `${mcpp.self}` *(2026.9.13.1+)* | 引擎自己的可执行文件，绝对路径。action 的命令是没有 shell 的 argv，构建程序因此没有可移植的拷贝手段；引擎在构建运行的每台机器上都在，`${mcpp.self} stage --verify content --output ` 拷贝一个文件、创建目标的父目录、只在字节不同时写入。这个参数形状自 2026.9.13.1 起是契约；写下它的构建程序即以该版本为下限 | @@ -1614,6 +1615,9 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } 因此把 `namespace = "spike"` 与 `name = "installer"` 分开写的包，既是 `dep_bin("installer", ...)`，也是 `dep_bin("spike.installer", ...)`（mcpp 2026.9.16.1+; 此前它的工具只以 `installer` 发布）。 +- **两个依赖共有的名字不属于任何一个包**（2026.10.30.1+）。清单都写 `rules` 的两个包（`openmui.rules`、 + `gongfu.rules`）各以限定名寻址；`rules` 本身不为任何一个发布，`MCPP_DEP_AMBIGUOUS_RULES` 列出两个限定名。 + `dep_dir("rules")` 结束程序，并写出这两个名字。2026.10.30.1 之前，这个名字归两者之一，且每次构建都警告。 #### 旧版 mcpp 读到用了这些键的 manifest diff --git a/mcpp.toml b/mcpp.toml index 20130043b..7c8fe1d25 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.10.29.1" +version = "2026.10.30.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/manifest/src/platform.cppm b/modules/manifest/src/platform.cppm index 53fa649de..9b5680418 100644 --- a/modules/manifest/src/platform.cppm +++ b/modules/manifest/src/platform.cppm @@ -113,6 +113,11 @@ struct Flag { struct Runner { std::vector cmd; std::string host; + // The `-` pairs the runner program exists for (design 2026-10-29 + // §4.4): where a package that needs it can run. Unset: unknown, and a + // package's coverage names no pair through the runner. + std::vector hosts; + bool hostsSet = false; bool exclusive = false; bool exclusiveSet = false; Origin origin; @@ -166,14 +171,40 @@ struct Link { // the section the carried artifact is placed in, and the source of the // program's entry, a file of the stating package that runs the artifact it // finds between `mcpp_carried_begin` and `mcpp_carried_end`. +// +// A platform may state several (an array of such tables), each for the +// `-` pairs in its `hosts`; an entry without `hosts` serves every +// pair (design 2026-10-29 §4.5). `build` names how the carrier program is +// built: "row" (the default), one program per `-` over the +// platform's hosted arrangement. An entry of a model this mcpp does not know +// is skipped with a warning, so a platform can add one without breaking an +// older mcpp. struct Carrier { PackageRef package; std::string section; std::string main; // absolute + std::vector hosts; + std::string build = "row"; Origin origin; bool set = false; }; +// The `-` vocabulary of `runner.hosts`, `carrier.hosts` and a +// package's coverage: a system mcpp builds programs for, and an architecture. +inline bool is_host_pair(std::string_view s) { + const auto dash = s.find('-'); + if (dash == std::string_view::npos || dash + 1 == s.size()) return false; + const auto os = s.substr(0, dash); + return (os == "linux" || os == "macos" || os == "windows") + && s.find('-', dash + 1) == std::string_view::npos; +} + +// The keys of a descriptor's per-platform blocks that `descriptor_keys` may +// name (the xpkg vocabulary, `macosx` for macOS). +inline constexpr std::string_view kDescriptorKeys[] = { + "linux", "macosx", "windows", "android", "ios", "emscripten", "openkal", "none", +}; + struct LayerEntry { PackageRef ref; Origin origin; @@ -205,7 +236,11 @@ struct Statement { std::vector sysrootProvides; Runner runner; std::map runners; - Carrier carrier; + // Every carrier entry, in the order stated; `carrier_for` picks one. + std::vector carriers; + // The descriptor blocks the platform's targets read, in order (design + // 2026-10-29 §5.1); unset: the rows' own. + Words descriptorKeys; Text plugin; // `universal = true`: `mcpp pack` of any of the targets packs all of them // into one package (SPEC-014 §3). @@ -227,7 +262,7 @@ struct Statement { bool empty() const { return !targets.set && !toolchain.set && !sysroot.set && !runner.set - && runners.empty() && !carrier.set && !plugin.set && !universal.set && layers.empty() + && runners.empty() && carriers.empty() && !descriptorKeys.set && !plugin.set && !universal.set && layers.empty() && tools.empty() && build.empty() && buildCfg.empty() && !link.driver.set && !link.flags.set && !link.start.set && !link.script.set && !link.inputsSet && !link.exe.set && !link.shared.set && !link.sealSet @@ -260,7 +295,7 @@ constexpr bool is_layer(std::string_view k) { // list per table, read by the parser's unknown-key sweep and by the tests. inline constexpr std::string_view kPlatformKeys[] = { "targets", "toolchain", "sysroot", "runner", "runners", "carrier", "plugin", - "universal", "layers", "tools", "build", "link", "versions", "hosted", + "universal", "descriptor_keys", "layers", "tools", "build", "link", "versions", "hosted", }; inline constexpr std::string_view kBuildKeys[] = { "cflags", "cxxflags", "asmflags", "ldflags" }; inline constexpr std::string_view kLinkKeys[] = { @@ -275,6 +310,17 @@ inline constexpr std::string_view kOverrideKeys[] = { "platform", "layers", "tools", "carrier", "link", }; +// The carrier entry of `s` that serves `pair` (`-`): the first whose +// `hosts` names it, else the first without `hosts`. Null when none does. +inline const Carrier* carrier_for(const Statement& s, std::string_view pair) { + const Carrier* open = nullptr; + for (auto const& c : s.carriers) { + if (c.hosts.empty()) { if (!open) open = &c; continue; } + if (std::ranges::find(c.hosts, pair) != c.hosts.end()) return &c; + } + return open; +} + // `"[.]@"` -> the ref; the error names what is // missing. `none` is not a ref; callers test for it first. inline std::expected parse_ref_text(std::string_view text) { @@ -327,7 +373,8 @@ inline std::string identity(const Override& o) { for (auto const& [k, e] : f.layers) add(e.origin); for (auto const& [k, e] : f.tools) add(e.origin); for (auto const& [k, w] : f.runners) add(w.origin); - if (f.carrier.set) add(f.carrier.origin); + for (auto const& c : f.carriers) add(c.origin); + if (f.descriptorKeys.set) add(f.descriptorKeys.origin); if (f.runner.set) add(f.runner.origin); for (auto const* t : { &f.link.driver, &f.link.start, &f.link.script, &f.link.exe, &f.link.shared }) if (t->set) add(t->origin); diff --git a/modules/manifest/src/platform_toml.cppm b/modules/manifest/src/platform_toml.cppm index 6e821e9c0..0e8c2727f 100644 --- a/modules/manifest/src/platform_toml.cppm +++ b/modules/manifest/src/platform_toml.cppm @@ -59,6 +59,12 @@ struct Reader { std::map& out) const; std::expected carrier(const mcpp::libs::toml::Value& v, std::string_view where) const; + std::expected, std::string> + carriers(const mcpp::libs::toml::Value& v, std::string_view where) const; + + // A list of `-` pairs (`runner.hosts`, `carrier.hosts`). + std::expected, std::string> + host_pairs(const mcpp::libs::toml::Value& v, std::string_view where) const; // `[platform]` of a platform package. std::expected statement(const mcpp::libs::toml::Table& t) const; @@ -255,9 +261,14 @@ std::expected Reader::runner(const toml::Value& v, std::str if (!val.is_bool()) return std::unexpected(std::format("{} must be true or false", key)); r.exclusive = val.as_bool(); r.exclusiveSet = true; + } else if (k == "hosts") { + auto h = host_pairs(val, key); + if (!h) return std::unexpected(h.error()); + r.hosts = std::move(*h); + r.hostsSet = true; } else { warn(std::format( - "{}: '{}' is not a key of a runner (ignored); the keys are cmd, host, exclusive", + "{}: '{}' is not a key of a runner (ignored); the keys are cmd, host, hosts, exclusive", where, k)); } } @@ -457,6 +468,46 @@ Reader::tools(const toml::Value& v, std::string_view where, return {}; } +std::expected, std::string> +Reader::host_pairs(const toml::Value& v, std::string_view where) const { + auto w = strings(v, where); + if (!w) return std::unexpected(w.error()); + for (auto const& p : *w) + if (!mcpp::manifest::platform::is_host_pair(p)) + return std::unexpected(std::format( + "{}: '{}' is not a system and an architecture; write \"-\" with " + " one of linux, macos, windows, for example \"linux-x86_64\"", where, p)); + return w; +} + +// `carrier` is one table or an array of them (design 2026-10-29 §4.5). +std::expected, std::string> +Reader::carriers(const toml::Value& v, std::string_view where) const { + std::vector out; + if (v.is_table()) { + auto c = carrier(v, where); + if (!c) return std::unexpected(c.error()); + out.push_back(std::move(*c)); + return out; + } + if (!v.is_array()) + return std::unexpected(std::format( + "{} must be a table {{ package, section, main }} or an array of them", where)); + std::size_t i = 0; + for (auto const& e : v.as_array()) { + auto c = carrier(e, std::format("{}[{}]", where, i++)); + if (!c) return std::unexpected(c.error()); + if (c->build != "row") { + warn(std::format( + "{}[{}]: build = \"{}\" is a carrier model this mcpp does not build (skipped); " + "it builds \"row\"", where, i - 1, c->build)); + continue; + } + out.push_back(std::move(*c)); + } + return out; +} + std::expected Reader::carrier(const toml::Value& v, std::string_view where) const { if (!v.is_table()) return std::unexpected(std::format( @@ -478,10 +529,18 @@ std::expected Reader::carrier(const toml::Value& v, std::s if (!val.is_string() || val.as_string().empty()) return std::unexpected(std::format("{} must be a source file of the package", key)); c.main = absolute(val.as_string()).generic_string(); + } else if (k == "hosts") { + auto h = host_pairs(val, key); + if (!h) return std::unexpected(h.error()); + c.hosts = std::move(*h); + } else if (k == "build") { + if (!val.is_string() || val.as_string().empty()) + return std::unexpected(std::format("{} must name a carrier model, for example \"row\"", key)); + c.build = val.as_string(); } else { warn(std::format( - "{}: '{}' is not a key of a carrier (ignored); the keys are package, section, main", - where, k)); + "{}: '{}' is not a key of a carrier (ignored); the keys are package, section, main, " + "hosts, build", where, k)); } } if (c.package.empty() || c.section.empty() || c.main.empty()) @@ -540,9 +599,19 @@ std::expected Reader::statement(const toml::Table& t) co s.runners[rk] = Words{ std::move(*w), at(rv, rkey), true }; } } else if (k == "carrier") { - auto c = carrier(v, key); + auto c = carriers(v, key); if (!c) return std::unexpected(c.error()); - s.carrier = std::move(*c); + s.carriers = std::move(*c); + } else if (k == "descriptor_keys") { + auto w = strings(v, key); + if (!w) return std::unexpected(w.error()); + if (w->empty()) return std::unexpected(std::format("{} names no key", key)); + for (auto const& d : *w) + if (std::ranges::find(kDescriptorKeys, std::string_view(d)) == std::end(kDescriptorKeys)) + return std::unexpected(std::format( + "{}: '{}' is not a descriptor platform; the keys are {}", key, d, + list_of(kDescriptorKeys))); + s.descriptorKeys = Words{ std::move(*w), at(v, key), true }; } else if (k == "plugin") { // The platform's program is its package's build program, which a // package names `build.mcpp` (SPEC-007); no other file is run. @@ -634,9 +703,9 @@ Reader::override_(const toml::Table& t, std::string_view selector) const { } else if (k == "tools") { if (auto r = tools(v, key, o.fields.tools); !r) return std::unexpected(r.error()); } else if (k == "carrier") { - auto c = carrier(v, key); + auto c = carriers(v, key); if (!c) return std::unexpected(c.error()); - o.fields.carrier = std::move(*c); + o.fields.carriers = std::move(*c); } else if (k == "link") { if (!v.is_table()) return std::unexpected(std::format("{} must be a table", key)); if (auto r = link(v.as_table(), key, o.fields.link); !r) return std::unexpected(r.error()); diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 605be26c4..ea56facd2 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -1049,6 +1049,10 @@ std::expected parse_document(const t::Document& documen // §2.5): the platform reader type-checks every entry. "platform.link.inputs", "target.*.link.inputs", + // `carrier = [{ package, section, main, hosts }, ...]` (design + // 2026-10-29 §4.5): the platform reader type-checks every entry. + "platform.carrier", + "target.*.carrier", }; if (auto badPath = find_disallowed_array_of_tables(doc->root(), "", kAllowedArraysOfTables)) { return std::unexpected(error(origin, std::format( @@ -1056,7 +1060,7 @@ std::expected parse_document(const t::Document& documen "array-of-tables syntax is only supported for [[build.flags]], " "[[features..flags]], [[runtime.requirements]], " "[[runtime.artifacts]], [[runtime.deploy]], [[platform.link.inputs]], " - "and [xlings] deps entries", + "[[platform.carrier]] and [xlings] deps entries", *badPath, *badPath))); } @@ -4217,14 +4221,21 @@ std::expected parse_document(const t::Document& documen return std::unexpected(error(origin, std::format( "[target.{}.abi] has no member '{}'; the members are: " "threads, exceptions", triple, ak))); - if (!av.is_bool()) - return std::unexpected(error(origin, std::format( - "[target.{}.abi].{} must be true or false", triple, ak))); + // `exceptions` also takes the model on Emscripten (design + // 2026-10-29 §6): "wasm" or "js"; `true` keeps "js". + const bool model = ak == "exceptions" && av.is_string() + && (av.as_string() == "wasm" || av.as_string() == "js"); + if (!av.is_bool() && !model) + return std::unexpected(error(origin, ak == "exceptions" + ? std::format("[target.{}.abi].exceptions must be true, false, " + "\"wasm\" or \"js\"", triple) + : std::format("[target.{}.abi].{} must be true or false", triple, ak))); if (ak == "threads") { cc.abiThreads = av.as_bool(); cc.abiThreadsDeclared = true; } else { - cc.abiExceptions = av.as_bool(); + cc.abiExceptions = model || av.as_bool(); + cc.abiExceptionsModel = model ? av.as_string() : std::string(); cc.abiExceptionsDeclared = true; } } diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 84a2a5541..79c5e9561 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1128,6 +1128,10 @@ struct BuildConfig : BuildInputs { // a statement. bool abiExceptions = false; bool abiExceptionsDeclared = false; + // The model `exceptions` names on Emscripten (design 2026-10-29 §6): + // "wasm" (`-fwasm-exceptions`), "js" (`-fexceptions`), or empty for + // `exceptions = true`, which keeps its meaning since 2026.9.12.3: "js". + std::string abiExceptionsModel; std::string cStandard; // Escape hatch for the hermetic link check: a sandbox toolchain whose // CRT/loader resolve OUTSIDE the sandbox is a hard error by default @@ -1683,6 +1687,7 @@ struct ConditionalConfig { // `[target..abi] exceptions` -- see BuildConfig::abiExceptions. bool abiExceptions = false; bool abiExceptionsDeclared = false; + std::string abiExceptionsModel; // `[target..build] dialect_cxxflags` -- a GRAPH-WIDE switch under a // target condition (issue #717, design 2026-09-27 §6). Like `abiThreads`/ // `abiExceptions` above, only the ROOT's value is rendered; unlike them it diff --git a/modules/manifest/src/xpkg.cppm b/modules/manifest/src/xpkg.cppm index 338e7a9ee..c2b0571f1 100644 --- a/modules/manifest/src/xpkg.cppm +++ b/modules/manifest/src/xpkg.cppm @@ -273,6 +273,15 @@ inline constexpr std::string_view kKnownXpkgKeys[] = { "linux", "macosx", "modules", "provides", "requires", "runtime", "scan_overrides", "schema", "sources", "target_cfg", "targets", "windows", + // The per-platform blocks of the rows that read their own (design + // 2026-10-29 §5.1): the Web, Apple's phones, ok images, bare metal. + "android", "emscripten", "ios", "openkal", "none", +}; + +// The keys of a segment's per-platform blocks: the three systems and the rows +// that read a block of their own (TargetPlatform::for_os). +inline constexpr std::string_view kDescriptorPlatformKeys[] = { + "linux", "macosx", "windows", "android", "emscripten", "ios", "openkal", "none", }; // Well-known confusables an edit-distance metric cannot bridge (the intended @@ -995,8 +1004,8 @@ xpkg_name_form_violation_from_lua(std::string_view luaContent) std::vector list_xpkg_version_entries(std::string_view luaContent, const mcpp::toolchain::platform::PlatformKey& platformAxis) { - const std::string_view platform = platformAxis.key(); - // Locate `xpm = { ... = { ["X.Y.Z"] = {...}, ... } ... }`. + // Locate `xpm = { ... = { ["X.Y.Z"] = {...}, ... } ... }`, the + // platform being the first of the axis's keys the table has a block for. // We work on a sanitized copy so quoted version keys remain locatable // by their offsets in the original text. auto sanitized = strip_lua_comments_and_strings(luaContent); @@ -1052,8 +1061,13 @@ list_xpkg_version_entries(std::string_view luaContent, auto xpm_end = find_table_end(xpm_open); if (xpm_end == std::string_view::npos) return versions; - auto plat_pos = find_word_at_lhs(platform, xpm_open + 1); - if (plat_pos == std::string_view::npos || plat_pos >= xpm_end) return versions; + auto plat_pos = std::string_view::npos; + for (auto const& key : platformAxis.keys()) { + plat_pos = find_word_at_lhs(key, xpm_open + 1); + if (plat_pos != std::string_view::npos && plat_pos < xpm_end) break; + plat_pos = std::string_view::npos; + } + if (plat_pos == std::string_view::npos) return versions; auto plat_open = skip_to_open_brace(plat_pos); if (plat_open == std::string_view::npos || plat_open >= xpm_end) return versions; auto plat_end = find_table_end(plat_open); @@ -1334,12 +1348,16 @@ synthesize_from_xpkg_lua(std::string_view luaContent, std::format("xpkg-lua of {}@{}", packageName, packageVersion), 0, 0}); } - const std::string_view osKey = platform.key(); - if (auto platformBody = top_level_table_body_for_key(body, osKey); - !platformBody.empty()) { - body += "\n"; - body += platformBody; - } + // The first of the axis's keys the segment has a table for (design + // 2026-10-29 §5.1): a target reads the blocks its row names, in order, and + // never the build machine's. + for (auto const& osKey : platform.keys()) + if (auto platformBody = top_level_table_body_for_key(body, osKey); + !platformBody.empty()) { + body += "\n"; + body += platformBody; + break; + } Manifest m; m.sourcePath = std::format("xpkg-lua://{}@{}", packageName, packageVersion); @@ -2418,7 +2436,7 @@ synthesize_from_xpkg_lua(std::string_view luaContent, } cur.consume('}'); } - else if (key == "linux" || key == "macosx" || key == "windows") { + else if (std::ranges::find(kDescriptorPlatformKeys, key) != std::end(kDescriptorPlatformKeys)) { // Per-platform sub-table. The CURRENT platform's body was already // appended to the segment before this loop (see above), so every // spelling here is a known key: skip the table itself. diff --git a/modules/toolchain-model/src/platform.cppm b/modules/toolchain-model/src/platform.cppm index bd47178ef..efa59cbdc 100644 --- a/modules/toolchain-model/src/platform.cppm +++ b/modules/toolchain-model/src/platform.cppm @@ -65,9 +65,9 @@ export namespace mcpp::toolchain::platform { // descriptor and xlings' own project file say `macosx`. Every place that has // to accept both — a target's `os` token, a `[xlings.workspace]` platform // table, a descriptor's per-OS section — asks here, so the alias cannot be -// half-known. `nullopt` means the token names no platform at all, which the -// caller decides how to treat: `for_os` falls back to the host, a manifest -// parser reports it. +// half-known. `nullopt` means the token names none of the three, which the +// caller decides how to treat: `descriptor_keys_for_os` gives the row's own +// keys, a manifest parser reports it. inline std::optional xpkg_platform_key_for(std::string_view os) { if (os == "macos" || os == "macosx") return "macosx"; if (os == "windows") return "windows"; @@ -78,45 +78,69 @@ inline std::optional xpkg_platform_key_for(std::string_view os // The three keys a descriptor has a block for, in that vocabulary. inline constexpr std::string_view xpkg_platforms[] = {"linux", "macosx", "windows"}; -// A resolved xpkg platform key. Only obtainable through one of the axis -// types below, so possession of one implies somebody decided which axis it -// came from. +// A resolved xpkg platform axis: the descriptor blocks a build reads, in the +// order it reads them. Only obtainable through one of the axis types below, so +// possession of one implies somebody decided which axis it came from. +// +// A descriptor is read through the FIRST of `keys()` it has a block for. The +// primary key, `key()`, is the one messages name. class PlatformKey { public: - std::string_view key() const { return key_; } + std::string_view key() const { return keys_.front(); } + std::span keys() const { return keys_; } - bool operator==(const PlatformKey& other) const { return key_ == other.key_; } + bool operator==(const PlatformKey& other) const { return keys_ == other.keys_; } protected: - explicit PlatformKey(std::string key) : key_(std::move(key)) {} + explicit PlatformKey(std::vector keys) : keys_(std::move(keys)) {} private: - std::string key_; + std::vector keys_; }; // The machine mcpp itself is running on. class HostPlatform : public PlatformKey { public: static HostPlatform current() { - return HostPlatform(std::string(xpkg_platform)); + return HostPlatform({std::string(xpkg_platform)}); } private: - explicit HostPlatform(std::string key) : PlatformKey(std::move(key)) {} + explicit HostPlatform(std::vector keys) : PlatformKey(std::move(keys)) {} }; +// The descriptor blocks a target's row reads (design 2026-10-29 §5.1), from +// the triple's `os` token. The build machine takes no part: before 2026.10.30.1 +// a token naming no descriptor platform fell back to the HOST's block, so a +// WebAssembly build read `linux` on Linux and `windows` on Windows (#851) and +// an ok image read whichever system it was built on. +// +// linux (incl. android, musl) linux macos macosx windows windows +// emscripten emscripten, linux ios ios, macosx +// openkal openkal, linux (a platform may state its own: descriptor_keys) +// none none any other token: itself alone +inline std::vector descriptor_keys_for_os(std::string_view os) { + if (auto k = xpkg_platform_key_for(os)) return {std::string(*k)}; + if (os == "emscripten") return {"emscripten", "linux"}; + if (os == "ios") return {"ios", "macosx"}; + if (os == "openkal") return {"openkal", "linux"}; + return {std::string(os)}; +} + // The platform the artifacts being produced will run on. class TargetPlatform : public PlatformKey { public: - // From a resolved triple's `os` token. The triple vocabulary says - // "macos" where xpkg descriptors say "macosx"; that translation lives - // here so no caller has to remember it. + // From a resolved triple's `os` token, through the row's descriptor keys. + // The triple vocabulary says "macos" where xpkg descriptors say "macosx"; + // that translation lives here so no caller has to remember it. static TargetPlatform for_os(std::string_view tripleOs) { - // Unknown/absent os token: fall back to the host, which is what the - // whole code base did unconditionally before #254. - return TargetPlatform(std::string( - xpkg_platform_key_for(tripleOs) - .value_or(xpkg_platform))); + return TargetPlatform(descriptor_keys_for_os(tripleOs)); + } + + // The keys a platform package states (`[platform] descriptor_keys`), + // which replace its rows' built-in ones. + static TargetPlatform with_keys(std::vector keys) { + return TargetPlatform(std::move(keys)); } // For tooling that walks platforms it is not running on and is not @@ -124,11 +148,11 @@ public: // Named awkwardly on purpose: it should be obvious at the call site that // this is neither axis. static TargetPlatform for_lint_of(std::string_view xpkgKey) { - return TargetPlatform(std::string(xpkgKey)); + return TargetPlatform({std::string(xpkgKey)}); } private: - explicit TargetPlatform(std::string key) : PlatformKey(std::move(key)) {} + explicit TargetPlatform(std::vector keys) : PlatformKey(std::move(keys)) {} }; } // namespace mcpp::toolchain::platform diff --git a/modules/toolchain-model/tests/test_platform_axis.cpp b/modules/toolchain-model/tests/test_platform_axis.cpp index bb0aa272a..f508437b8 100644 --- a/modules/toolchain-model/tests/test_platform_axis.cpp +++ b/modules/toolchain-model/tests/test_platform_axis.cpp @@ -25,3 +25,26 @@ TEST(PlatformAxis, TwoKeysCompareByValue) { auto b = mcpp::toolchain::platform::HostPlatform::current(); EXPECT_TRUE(a == b); } + +TEST(PlatformAxis, ATargetReadsItsRowsBlocksAndNeverTheHosts) { + // Design 2026-10-29 §5.1. Before 2026.10.30.1 a token naming none of the + // three fell back to the HOST's block (#851). + using mcpp::toolchain::platform::TargetPlatform; + auto keys = [](std::string_view os) { + auto t = TargetPlatform::for_os(os); + return std::vector(t.keys().begin(), t.keys().end()); + }; + EXPECT_EQ(keys("linux"), (std::vector{"linux"})); + EXPECT_EQ(keys("macos"), (std::vector{"macosx"})); + EXPECT_EQ(keys("windows"), (std::vector{"windows"})); + EXPECT_EQ(keys("emscripten"), (std::vector{"emscripten", "linux"})); + EXPECT_EQ(keys("ios"), (std::vector{"ios", "macosx"})); + EXPECT_EQ(keys("openkal"), (std::vector{"openkal", "linux"})); + EXPECT_EQ(keys("none"), (std::vector{"none"})); + // The primary key is the one messages name. + EXPECT_EQ(TargetPlatform::for_os("emscripten").key(), "emscripten"); + // A platform's own keys replace the row's. + auto stated = TargetPlatform::with_keys({"openkal"}); + EXPECT_EQ(stated.key(), "openkal"); + EXPECT_FALSE(stated == TargetPlatform::for_os("openkal")); +} diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 14086adb5..a1cf7c107 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.10.29.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.10.30.1"; } // namespace mcpp diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index a8b0e068b..76a5d4d14 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -307,6 +307,12 @@ struct BuildProgramEnv { // only: the root decides every dependency's form, and a dependency's // program runs before facts that decide it are known (see prepare.cppm). std::vector> depLinkages; + // #852: a name that more than one visible dependency answers to (two + // packages whose manifests both say `rules`), with the qualified names of + // those packages, comma-separated. Such a name is in neither list above: + // no package can be it, so none is published under it, and the program + // reads MCPP_DEP_AMBIGUOUS_ to say which names to use. + std::vector> depAmbiguous; // Payload dirs of the packages this build declared in `[xlings] deps`, // as (env var name → dir). Resolved by the caller through the xlings path // helpers, so a build.mcpp asks `xpkg_dir("xim", "picolibc-riscv")` @@ -757,6 +763,8 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv it->second, dir.string())); } } + for (auto const& [name, which] : env.depAmbiguous) + e.emplace_back("MCPP_DEP_AMBIGUOUS_" + sanitize_feature_env(name), which); // #642 E2: the form each dependency takes, beside its directory and under // the same sanitised name. Its own collision map, because the variable // names differ from the `_DIR` ones; the rule is the same: keep the first, diff --git a/src/build/compile_commands.cppm b/src/build/compile_commands.cppm index facde555d..6a3a5d759 100644 --- a/src/build/compile_commands.cppm +++ b/src/build/compile_commands.cppm @@ -47,6 +47,7 @@ import mcpplibs.os.platform; import mcpp.home.files; import mcpp.manifest.flag_words; import mcpp.toolchain.model; +import mcpp.toolchain.dialect; import mcpp.ui; // the one-time note about a root database no longer asked for export namespace mcpp::build { @@ -191,6 +192,24 @@ publish_root_compile_commands( std::expected write_compile_commands(const BuildPlan& plan, const CompileFlags& flags); +// THE BUILD'S CONTRACT (design 2026-10-29 §8.1): what a foreign build system +// needs to compile and link for this configuration as the engine does -- +// the compiler, archiver and link driver, and the words every unit of the +// build is compiled and linked with (target, sysroot, target-side headers, +// platform and profile flags). A package's own include directories are not +// in it: a foreign build states its own; nor are the engine's C++ dialect and +// module words (`-std=`, `-fmodules`), which are its choice for its own units. Written beside the configuration's +// compile database as `mcpp-contract.json` and named by `${mcpp.contract}` +// in a build program's action, which reads it while ninja runs; no action +// calls mcpp back. A tool for the host is built in the host graph, whose +// configuration has its own contract. +// +// {"schema": 1, "target": "", "dialect": "", +// "compile": {"c": {"program", "args"}, "cxx": {...}, "asm": {...}}, +// "link": {"program", "args", "archiver"}} +std::filesystem::path contract_path(const BuildPlan& plan); +std::expected write_build_contract(const BuildPlan& plan, const CompileFlags& flags); + // The one sentence both publishers of the root file (the full build and the // fast path) print when the file they replaced held `n` entries mcpp did not // write. One function, so the two paths cannot word it differently. @@ -829,4 +848,46 @@ std::string foreign_entries_warning(std::size_t n) { "the file now holds mcpp's configuration", n, n == 1 ? "y" : "ies"); } +std::filesystem::path contract_path(const BuildPlan& plan) { + return plan.outputDir / "mcpp-contract.json"; +} + +std::expected write_build_contract(const BuildPlan& plan, const CompileFlags& flags) { + // The engine's own C++ dialect and module words are its choice for ITS + // units, not the target's: a foreign build chooses its own standard, and + // `-fmodules` names Clang modules to clang. Everything else -- target, + // sysroot, target-side headers, platform and profile flags -- is what + // compiling for this configuration means, and stays. + auto engine_only = [](std::string_view w) { + return w.starts_with("-std=") || w.starts_with("/std:") || w == "-fmodules" + || w == "-fmodules-ts" || w.starts_with("-fmodule-") || w.starts_with("-fprebuilt-module-path") + || w.starts_with("/experimental:module"); + }; + auto tool = [&](const std::filesystem::path& program, std::string_view words, bool compile) { + nlohmann::json t; + t["program"] = program.generic_string(); + auto args = split_flags(words); + if (compile) std::erase_if(args, [&](const std::string& w) { return engine_only(w); }); + t["args"] = std::move(args); + return t; + }; + nlohmann::json doc; + doc["schema"] = 1; + doc["target"] = plan.toolchain.targetTriple; + doc["dialect"] = std::string(mcpp::toolchain::dialect_for(plan.toolchain).id); + doc["compile"]["c"] = tool(flags.ccBinary, flags.cc, true); + doc["compile"]["cxx"] = tool(flags.cxxBinary, flags.cxx, true); + doc["compile"]["asm"] = tool(flags.ccBinary, flags.as, true); + auto link = tool(flags.ldDriver.empty() ? flags.cxxBinary : flags.ldDriver, flags.ld, false); + link["archiver"] = flags.arBinary.generic_string(); + doc["link"] = std::move(link); + const auto path = contract_path(plan); + std::error_code ec; + std::filesystem::create_directories(path.parent_path(), ec); + std::ofstream out(path, std::ios::binary | std::ios::trunc); + if (!out) return std::unexpected(std::format("cannot write {}", path.string())); + out << doc.dump(2) << '\n'; + return {}; +} + } // namespace mcpp::build diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 1bd400cf6..6ed67c073 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -787,6 +787,12 @@ inline bool has_feature(const char* name) { // mcpp#241: resolved install dir of a declared dependency (by its package // name), or "" if not found. Same sanitize as has_feature; wraps // MCPP_DEP__DIR. +// +// #852: a name two visible dependencies answer to (two packages whose +// manifests both say `rules`) is no package's, and the engine publishes it as +// MCPP_DEP_AMBIGUOUS_ with the qualified names instead. Asking +// for it ends the program, naming them: the answer otherwise would be one of +// the two packages' directories or none, and either is a wrong build. inline const char* dep_dir(const char* name) { char buf[256] = "MCPP_DEP_"; unsigned long o = 9; @@ -795,6 +801,16 @@ inline const char* dep_dir(const char* name) { buf[o] = (c >= 'a' && c <= 'z') ? char(c - 'a' + 'A') : ((c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) ? c : '_'; } + char ambiguous[288] = "MCPP_DEP_AMBIGUOUS_"; + unsigned long a = 19; + for (unsigned long i = 9; i < o && a + 1 < sizeof ambiguous; ++i, ++a) ambiguous[a] = buf[i]; + ambiguous[a] = 0; + if (const char* which = std::getenv(ambiguous)) { + std::fprintf(stderr, + "build.mcpp: dep_dir(\"%s\") names more than one dependency (%s); ask by one of " + "those names\n", name, which); + std::exit(2); + } buf[o++] = '_'; buf[o++] = 'D'; buf[o++] = 'I'; buf[o++] = 'R'; buf[o] = 0; return env_or(buf); } diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 1f32a5a40..d5b831a40 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -4580,6 +4580,8 @@ std::expected NinjaBackend::build(const BuildPlan& plan stage("compute-flags"); auto cdb = write_compile_commands(plan, flags); stage("compile-commands"); + if (auto c = write_build_contract(plan, flags); !c) + return std::unexpected(BuildError{c.error(), contract_path(plan)}); if (!cdb) { if (opts.requireCompileDatabase) { return std::unexpected(BuildError{ diff --git a/src/build/prepare/config.cpp b/src/build/prepare/config.cpp index f2345818a..e0f60ba3b 100644 --- a/src/build/prepare/config.cpp +++ b/src/build/prepare/config.cpp @@ -339,6 +339,7 @@ void merge_conditional_config(mcpp::manifest::Manifest& m, } if (cc.abiExceptionsDeclared) { m.buildConfig.abiExceptions = cc.abiExceptions; + m.buildConfig.abiExceptionsModel = cc.abiExceptionsModel; m.buildConfig.abiExceptionsDeclared = true; } // `[target..build] dialect_cxxflags` (#717): recorded for every diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index 26926c464..82e32c3cf 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -2116,23 +2116,38 @@ step4b_define_provisioning_closures(PrepareState& state) { if (auto note = prov::contest_note(tail, b); !note.empty()) mcpp::diag::warning("provisions/ambiguous", note); } + // Every spelling of `publishedNamesFor`: the manifest's name, the + // qualified name a manifest writing `namespace = "ns"` and + // `name = "fw"` is addressed by (#642: the framework's rule asks + // `dep_linkage("huxerui.huxerui")`), and the bound tail. A spelling two + // providers share (#852: `openmui.rules` and `gongfu.rules` are both + // `rules` by their manifests' names) is no package's: it is published + // as ambiguous, with the qualified names that do work. + std::map> byName; for (auto const& pr : state.provisionGraph.visible[consumer]) { - if (pr.kind != prov::Kind::DepDir) continue; - if (pr.provider >= state.packages.size()) continue; - auto const& depPkg = state.packages[pr.provider]; - auto const& canon = depPkg.manifest.package.name; - const std::string* form = nullptr; - if (linkForms) - if (auto f = linkForms->find(pr.provider); f != linkForms->end()) - form = &f->second; - // Every spelling of `publishedNamesFor`: the manifest's name, the - // qualified name a manifest writing `namespace = "ns"` and - // `name = "fw"` is addressed by (#642: the framework's rule asks - // `dep_linkage("huxerui.huxerui")`), and the bound tail. + if (pr.kind != prov::Kind::DepDir || pr.provider >= state.packages.size()) continue; for (auto const& n : state.publishedNamesFor(pr.provider, bind)) { - e.depDirs.emplace_back(n, depPkg.root); - if (form) e.depLinkages.emplace_back(n, *form); + auto& who = byName[n]; + if (std::ranges::find(who, pr.provider) == who.end()) who.push_back(pr.provider); + } + } + for (auto const& [n, who] : byName) { + if (who.size() > 1) { + std::string which; + for (auto p : who) { + auto const& pk = state.packages[p].manifest.package; + const bool qualified = pk.namespace_.empty() || pk.name.starts_with(pk.namespace_ + "."); + which += (which.empty() ? "" : ",") + + (qualified ? pk.name : pk.namespace_ + "." + pk.name); + } + e.depAmbiguous.emplace_back(n, std::move(which)); + continue; } + const auto provider = who.front(); + e.depDirs.emplace_back(n, state.packages[provider].root); + if (linkForms) + if (auto f = linkForms->find(provider); f != linkForms->end()) + e.depLinkages.emplace_back(n, f->second); } }; diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 03fedb74e..ac32ecc35 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -1180,6 +1180,9 @@ static std::expected step13_build_graph_actions(PrepareState& rep("${mcpp.out_dir}", ctx.plan.outputDir.generic_string()); rep("${mcpp.bin_dir}", binDir.generic_string()); rep("${mcpp.compile_db}", ctx.plan.compileDbPath.generic_string()); + // The build's contract (design 2026-10-29 §8.1), beside the + // configuration's compile database; see `write_build_contract`. + rep("${mcpp.contract}", (ctx.plan.outputDir / "mcpp-contract.json").generic_string()); // The engine's own executable, absolute (2026.9.13.1+). An action // whose command is an argv with no shell has no portable way to // copy, touch or compare a file, and the engine is the one diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index ea080a5f6..2aef79243 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -275,9 +275,11 @@ std::expected step4b_apply_platform(PrepareState& state) { mcpp::platform::merge(P.fields, o->fields); P.active = true; state.platformInjected.insert(P.package); - // A base the platform states, which this pass decided without: the - // next pass reads the package before deciding it (§7). - if (P.fields.sysroot.set && !state.platformEarly && P.fields.sysroot.origin.tier == mp::Tier::Platform) { + // A base, or descriptor keys, the platform states, which this pass + // decided without: the next pass reads the package before deciding + // them (§7; design 2026-10-29 §5.1). + const auto platformStated = [](auto const& f) { return f.set && f.origin.tier == mp::Tier::Platform; }; + if ((platformStated(P.fields.sysroot) || platformStated(P.fields.descriptorKeys)) && !state.platformEarly) { g_platformRestart = P.root; return std::unexpected(std::string("mcpp:platform-restart")); } diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index 1ed8c41e7..01c669c71 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -1217,7 +1217,12 @@ static std::expected step1_device_axis_and_layer_merge(Prepar c.accelerators.push_back(set.backend); return c; }; - state.targetPlatform = mcpp::toolchain::platform::TargetPlatform::for_os(state.cfgCtx().os); + // The descriptor blocks the target reads (design 2026-10-29 §5.1): the + // row's own, or those the platform states when it was read early (a + // platform fetched in the first pass restarts the prepare so that it is). + state.targetPlatform = state.platformEarly && state.platformData.descriptorKeys.set + ? mcpp::toolchain::platform::TargetPlatform::with_keys(state.platformData.descriptorKeys.value) + : mcpp::toolchain::platform::TargetPlatform::for_os(state.cfgCtx().os); // ── L1: merge conditional [target.'cfg(...)'] sections ─────────────────── // Evaluated now (target resolved) against the resolved target — the @@ -1275,12 +1280,29 @@ static std::expected step1_device_axis_and_layer_merge(Prepar // already link with exceptions on, so a host build with the member // declared is byte-identical to one without -- the same property // `threads` has on PE. - const bool abiExceptionsRendered = state.m->buildConfig.abiExceptions - && state.cfgCtx().os == "emscripten"; + // + // The MODEL (design 2026-10-29 §6): "wasm" is WebAssembly's own exception + // handling (`-fwasm-exceptions`: smaller and faster, and it needs Safari + // 15.2 / Node 17 or later); "js" and `true` are the JavaScript-based model + // `true` has meant since 2026.9.12.3. The flag is in the dialect set, so the + // model is part of every cache key. + const bool emscripten = state.cfgCtx().os == "emscripten"; + const bool abiExceptionsRendered = state.m->buildConfig.abiExceptions && emscripten; if (abiExceptionsRendered) { - state.add_once(state.m->buildConfig.dialectCxxflags, "-fexceptions"); - state.add_once(state.m->buildConfig.ldflags, "-fexceptions"); - state.add_once(state.m->buildConfig.graphLdflags, "-fexceptions"); + const char* flag = state.m->buildConfig.abiExceptionsModel == "wasm" + ? "-fwasm-exceptions" : "-fexceptions"; + state.add_once(state.m->buildConfig.dialectCxxflags, flag); + state.add_once(state.m->buildConfig.ldflags, flag); + state.add_once(state.m->buildConfig.graphLdflags, flag); + } + // #851: emcc runs wasm-opt and minifies the JavaScript launcher AT THE + // LINK, from the optimisation level the link is given; a link without + // one is -O0 whatever the compiles were. The profile's level reaches it. + if (emscripten) { + const auto& lvl = state.m->buildConfig.optLevel; + const std::string opt = "-O" + (lvl.empty() ? std::string("0") : lvl); + state.add_once(state.m->buildConfig.ldflags, opt); + state.add_once(state.m->buildConfig.graphLdflags, opt); } // `[build].defines` must reach the scanner (P1689) and the compile edge, // and must participate in the fingerprint. Fold before dependency diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 2823894a3..d5597b7b5 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -2448,12 +2448,17 @@ std::expected carrier_build(const Plan& plan, const CarrierBuild& b) { namespace fs = std::filesystem; - auto const& c = plan.platform.fields.carrier; - if (!plan.platform.active || !c.set) + // The carrier entry serving this program's system and architecture + // (design 2026-10-29 §4.5). + const auto* entry = plan.platform.active + ? mcpp::manifest::platform::carrier_for(plan.platform.fields, std::format("{}-{}", b.system, b.arch)) + : nullptr; + if (!entry) return std::unexpected(Error{std::format( - "--mode self-contained of {} builds a program of the machine over its platform's " - "carrier, and {} states none ([platform] carrier = {{ package, section, main }}).", - plan.binaryName, plan.platform.display())}); + "--mode self-contained of {} builds a program of {}-{} over its platform's carrier, and " + "{} states none for it ([platform] carrier = {{ package, section, main, hosts }}).", + plan.binaryName, b.system, b.arch, plan.platform.display())}); + auto const& c = *entry; std::error_code ec; if (c.main.empty() || !fs::is_regular_file(c.main, ec)) return std::unexpected(Error{std::format( @@ -3295,13 +3300,14 @@ std::vector carried_systems(const Plan& plan) { std::expected runner_program(const mcpp::platform::Resolved& platform, const std::filesystem::path& projectRoot, std::string_view name, std::string_view arch, bool build) { - if (!platform.active || !platform.fields.carrier.set) + const std::string system(detail::host_system()); + if (!platform.active + || !mcpp::manifest::platform::carrier_for(platform.fields, std::format("{}-{}", system, arch))) return std::unexpected(Error{std::format( "the runner `{}` of {} runs on a machine of the artifact's architecture, {}, and its " - "program for that architecture is the platform's carrier, which {} does not state " - "([platform] carrier = {{ package, section, main }})", - name, platform.display(), arch, platform.display())}); - const std::string system(detail::host_system()); + "program for that architecture is the platform's carrier, which {} does not state for " + "{}-{} ([platform] carrier = {{ package, section, main, hosts }})", + name, platform.display(), arch, platform.display(), system, arch)}); if (auto why = detail::unbuilt_carrier(platform, system, arch, false)) return std::unexpected(Error{*why}); const auto program = detail::packed_program(detail::carrier_dir(projectRoot, name, system, arch), diff --git a/src/platform/platform.cppm b/src/platform/platform.cppm index 2034f5d9e..d327199be 100644 --- a/src/platform/platform.cppm +++ b/src/platform/platform.cppm @@ -330,8 +330,13 @@ void merge(mp::Statement& b, const mp::Statement& o) { b.runner.exclusive = o.runner.exclusive; b.runner.exclusiveSet = true; } + if (o.runner.hostsSet) { + b.runner.hosts = o.runner.hosts; + b.runner.hostsSet = true; + } for (auto const& [k, v] : o.runners) b.runners[k] = v; - if (o.carrier.set) b.carrier = o.carrier; + if (!o.carriers.empty()) b.carriers = o.carriers; + words(b.descriptorKeys, o.descriptorKeys); text(b.plugin, o.plugin); if (o.universal.set) b.universal = o.universal; for (auto const& [k, v] : o.layers) b.layers[k] = v; @@ -580,7 +585,14 @@ std::string to_json(const Resolved& r) { if (f.link.start.set) field("link.start", q(f.link.start.value), f.link.start.origin); if (f.link.exe.set) field("link.exe", q(f.link.exe.value), f.link.exe.origin); if (f.link.shared.set) field("link.shared", q(f.link.shared.value), f.link.shared.origin); - if (f.carrier.set) field("carrier", q(mp::display(f.carrier.package)), f.carrier.origin); + for (auto const& c : f.carriers) + field(c.hosts.empty() ? std::string("carrier") + : std::format("carrier[{}]", [&] { + std::string j; + for (auto const& h : c.hosts) j += (j.empty() ? "" : ",") + h; + return j; + }()), + q(mp::display(c.package)), c.origin); o += "}}"; return o; } diff --git a/src/project.cppm b/src/project.cppm index 0d0b4e1d2..b5006b1b3 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -1093,7 +1093,8 @@ export std::string root_position_key( auto const& b = m.buildConfig; auto dialect = b.dialectCxxflags; auto abiThreads = b.abiThreadsDeclared ? (b.abiThreads ? "1" : "0") : "-"; - auto abiExceptions = b.abiExceptionsDeclared ? (b.abiExceptions ? "1" : "0") : "-"; + std::string abiExceptions = b.abiExceptionsDeclared + ? (b.abiExceptions ? "1" + b.abiExceptionsModel : "0") : "-"; if (applies) for (auto const& cc : m.conditionalConfigs) { if (cc.dialectCxxflags.empty() && !cc.abiThreadsDeclared && !cc.abiExceptionsDeclared) @@ -1101,7 +1102,7 @@ export std::string root_position_key( if (!applies(cc.predicate)) continue; dialect.insert(dialect.end(), cc.dialectCxxflags.begin(), cc.dialectCxxflags.end()); if (cc.abiThreadsDeclared) abiThreads = cc.abiThreads ? "1" : "0"; - if (cc.abiExceptionsDeclared) abiExceptions = cc.abiExceptions ? "1" : "0"; + if (cc.abiExceptionsDeclared) abiExceptions = cc.abiExceptions ? "1" + cc.abiExceptionsModel : "0"; } list("dialect_cxxflags", dialect); field("cxx_runtime", b.cxxRuntime); @@ -1193,6 +1194,7 @@ virtual_workspace_root(const mcpp::manifest::Manifest& workspace, b.abiThreads = f.abiThreads; b.abiThreadsDeclared = f.abiThreadsDeclared; b.abiExceptions = f.abiExceptions; + b.abiExceptionsModel = f.abiExceptionsModel; b.abiExceptionsDeclared = f.abiExceptionsDeclared; b.allowHostLibs = f.allowHostLibs; b.macosDeploymentTarget = f.macosDeploymentTarget; @@ -1216,6 +1218,7 @@ virtual_workspace_root(const mcpp::manifest::Manifest& workspace, row.abiThreads = cc.abiThreads; row.abiThreadsDeclared = cc.abiThreadsDeclared; row.abiExceptions = cc.abiExceptions; + row.abiExceptionsModel = cc.abiExceptionsModel; row.abiExceptionsDeclared = cc.abiExceptionsDeclared; v.conditionalConfigs.push_back(std::move(row)); } diff --git a/tests/e2e/980_an_ambiguous_dependency_name_is_no_packages.sh b/tests/e2e/980_an_ambiguous_dependency_name_is_no_packages.sh new file mode 100755 index 000000000..de8724f05 --- /dev/null +++ b/tests/e2e/980_an_ambiguous_dependency_name_is_no_packages.sh @@ -0,0 +1,91 @@ +#!/usr/bin/env bash +# requires: elf gcc +# #852: two dependencies whose manifests both say `rules` (namespaces openmui +# and gongfu). The bare name is no package's: it is not published, the build +# does not warn on every run, MCPP_DEP_AMBIGUOUS_RULES names the qualified +# names, and each qualified name still reaches its own directory. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +for ns in openmui gongfu; do + mkdir -p "$ns-rules/src" + echo "int ${ns}_rules_touch() { return 1; }" > "$ns-rules/src/r.cpp" + cat > "$ns-rules/mcpp.toml" < app/src/main.cpp <<'EOF' +import std; +extern const char* seen(); +int main() { std::println("{}", seen()); } +EOF +cat > app/build.mcpp <<'EOF' +#include +#include +#include +import mcpp; +int main() { + auto env = [](const char* n) { const char* v = std::getenv(n); return std::string(v ? v : ""); }; + std::string line = "bare=" + env("MCPP_DEP_RULES_DIR") + + " ambiguous=" + env("MCPP_DEP_AMBIGUOUS_RULES") + + " openmui=" + env("MCPP_DEP_OPENMUI_RULES_DIR") + + " gongfu=" + env("MCPP_DEP_GONGFU_RULES_DIR"); + std::string out = "src/seen.cpp"; + std::FILE* f = std::fopen(out.c_str(), "w"); + std::fprintf(f, "const char* seen() { return R\"(%s)\"; }\n", line.c_str()); + std::fclose(f); + mcpp::generated("src/seen.cpp"); + return 0; +} +EOF +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +[modules] +sources = ["src/**/*.cpp"] +[dependencies] +openmui.rules = { path = "../openmui-rules" } +gongfu.rules = { path = "../gongfu-rules" } +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF + +cd app +"$MCPP" build > build.log 2>&1 || { cat build.log; echo "FAIL: build failed"; exit 1; } +if grep -q "collides on" build.log; then + cat build.log; echo "FAIL: an ambiguous name still warns"; exit 1 +fi +seen="$("$MCPP" run 2>/dev/null | tail -1)" +echo "$seen" +[[ "$seen" == *"bare="* ]] || { echo "FAIL: the bare name is bound"; exit 1; } +[[ "$seen" == *"ambiguous=gongfu.rules,openmui.rules"* || "$seen" == *"ambiguous=openmui.rules,gongfu.rules"* ]] \ + || { echo "FAIL: MCPP_DEP_AMBIGUOUS_RULES does not name both"; exit 1; } +[[ "$seen" == *"openmui="*"openmui-rules"* ]] || { echo "FAIL: openmui.rules lost its directory"; exit 1; } +[[ "$seen" == *"gongfu="*"gongfu-rules"* ]] || { echo "FAIL: gongfu.rules lost its directory"; exit 1; } + +# A build program that asks for the ambiguous name ends, naming both. +cat > build.mcpp <<'EOF' +import mcpp; +int main() { return mcpp::dep_dir("rules")[0] ? 0 : 1; } +EOF +rm -f src/seen.cpp +if "$MCPP" build > ask.log 2>&1; then cat ask.log; echo "FAIL: dep_dir(\"rules\") answered"; exit 1; fi +grep -q 'dep_dir("rules") names more than one dependency' ask.log \ + || { cat ask.log; echo "FAIL: the program did not say the name is ambiguous"; exit 1; } +grep -q 'openmui.rules' ask.log && grep -q 'gongfu.rules' ask.log \ + || { cat ask.log; echo "FAIL: the refusal does not name both packages"; exit 1; } +echo "OK" diff --git a/tests/e2e/981_a_web_link_has_the_profiles_level_and_the_wasm_exception_model.sh b/tests/e2e/981_a_web_link_has_the_profiles_level_and_the_wasm_exception_model.sh new file mode 100755 index 000000000..3ab66267d --- /dev/null +++ b/tests/e2e/981_a_web_link_has_the_profiles_level_and_the_wasm_exception_model.sh @@ -0,0 +1,86 @@ +#!/usr/bin/env bash +# requires: elf +# #851 (design 2026-10-29 §6): emcc runs wasm-opt and minifies the launcher +# AT THE LINK, so the profile's optimisation level reaches the link; and +# `[target..abi] exceptions = "wasm"` selects WebAssembly's own exception +# handling (`-fwasm-exceptions`), while `true` keeps the JavaScript model. +# Same self-probe gate as 653: no emsdk payload, no test. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +have_emsdk=0 +for d in "${MCPP_HOME:-$HOME/.mcpp}"/registry/data/xpkgs/xim-x-emsdk/*/emscripten \ + "$HOME"/.xlings/data/xpkgs/xim-x-emsdk/*/emscripten; do + [[ -x "$d/em++" ]] && have_emsdk=1 +done +if [ "$have_emsdk" -ne 1 ]; then + echo "SKIP: 981 -- no emsdk payload installed" + echo "PASS: 981 (skipped)" + exit 0 +fi + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +write() { # $1 = the abi table's value, or empty for none + cat > mcpp.toml <> mcpp.toml; fi +} +cat > src/main.cpp <<'CPP' +import std; +int main() { + try { throw std::runtime_error("boom"); } + catch (std::exception const& e) { std::println("caught: {}", e.what()); } +} +CPP +ninja_of() { dirname "$(ls -t $(find target/wasm32-emscripten -name build.ninja -not -path '*/.build-mcpp/*') | head -1)"; } + +echo "== 981/1: a release link carries the profile's level, a dev link -O0 ==" +write '' +"$MCPP" build --release --target wasm32-emscripten > rel.log 2>&1 || fail "release build failed" rel.log +grep -E '^ldflags' "$(ninja_of)/build.ninja" | grep -qE -- '(^| )-O2( |$)' \ + || fail "the release link carries no -O2" "$(ninja_of)/build.ninja" +"$MCPP" build --target wasm32-emscripten > dev.log 2>&1 || fail "dev build failed" dev.log +grep -E '^ldflags' "$(ninja_of)/build.ninja" | grep -qE -- '(^| )-O0( |$)' \ + || fail "the dev link carries no -O0" "$(ninja_of)/build.ninja" +echo " ok" + +echo "== 981/2: exceptions = \"wasm\" compiles and links with -fwasm-exceptions, and catches ==" +write '"wasm"' +"$MCPP" build --target wasm32-emscripten > wasm.log 2>&1 || fail "the wasm-model build failed" wasm.log +n="$(ninja_of)/build.ninja" +grep -E '^cxxflags' "$n" | grep -q -- '-fwasm-exceptions' || fail "-fwasm-exceptions is not compiled with" "$n" +grep -E '^ldflags' "$n" | grep -q -- '-fwasm-exceptions' || fail "-fwasm-exceptions is not linked with" "$n" +grep -E '^cxxflags' "$n" | grep -qE -- '(^| )-fexceptions( |$)' && fail "the JS model is also selected" "$n" +"$MCPP" run --target wasm32-emscripten > wasm-run.log 2>&1 || fail "the wasm-model program did not run" wasm-run.log +grep -q '^caught: boom$' wasm-run.log || fail "the wasm-model program did not catch" wasm-run.log +echo " ok" + +echo "== 981/3: true keeps the JavaScript model ==" +write 'true' +"$MCPP" build --target wasm32-emscripten > js.log 2>&1 || fail "the js-model build failed" js.log +n="$(ninja_of)/build.ninja" +grep -E '^cxxflags' "$n" | grep -qE -- '(^| )-fexceptions( |$)' || fail "true no longer selects -fexceptions" "$n" +grep -E '^cxxflags' "$n" | grep -q -- '-fwasm-exceptions' && fail "true selects the wasm model" "$n" +echo " ok" + +echo "== 981/4: another value is refused, naming the values ==" +write '"seh"' +if "$MCPP" build --target wasm32-emscripten > bad.log 2>&1; then fail "exceptions = \"seh\" was accepted" bad.log; fi +grep -qF '"wasm" or "js"' bad.log || fail "the refusal does not name the values" bad.log +echo " ok" + +echo "PASS: 981" diff --git a/tests/e2e/982_an_action_reads_the_builds_contract.sh b/tests/e2e/982_an_action_reads_the_builds_contract.sh new file mode 100755 index 000000000..145157b2a --- /dev/null +++ b/tests/e2e/982_an_action_reads_the_builds_contract.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +# requires: elf +# Design 2026-10-29 §8.1: a build program's action names `${mcpp.contract}`, +# the configuration's contract (`mcpp-contract.json`, beside its compile +# database): the compiler of each language with the words every unit is +# compiled with, and the link driver, its words and the archiver. The action +# reads it while ninja runs; nothing calls mcpp back. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" +mkdir -p src +cat > mcpp.toml <<'EOF' +[package] +name = "contract" +version = "0.1.0" +EOF +printf 'int main() { return 0; }\n' > src/main.cpp +cat > build.mcpp <<'EOF' +#include +import mcpp; +int main() { + const std::string out = std::string(mcpp::out_dir()) + "/seen-contract.json"; + mcpp::action a; + a.id = "copy-contract"; + a.role = mcpp::roles::check; + a.arg("cp").arg("${mcpp.contract}").arg(out.c_str()) + .output(out.c_str()) + .submit(); +} +EOF + +"$MCPP" build > b.log 2>&1 || { cat b.log; echo "FAIL: build failed"; exit 1; } +seen=$(find target -name seen-contract.json | head -1) +[[ -n "$seen" ]] || { cat b.log; echo "FAIL: the action did not run"; exit 1; } +python3 - "$seen" <<'PY' +import json, os, sys +d = json.load(open(sys.argv[1])) +assert d["schema"] == 1, d +assert d["target"], d +for lang in ("c", "cxx", "asm"): + t = d["compile"][lang] + assert os.path.isabs(t["program"]) and isinstance(t["args"], list), (lang, t) +cxx = d["compile"]["cxx"]["args"] +assert any(a.startswith("-O") for a in cxx), cxx +assert not any(a.startswith("-std=") or a == "-fmodules" or a.startswith("-fmodule-") for a in cxx), cxx +assert os.path.isabs(d["link"]["program"]) and isinstance(d["link"]["args"], list), d["link"] +assert "archiver" in d["link"], d["link"] +print("contract:", d["target"], d["dialect"], d["compile"]["cxx"]["program"]) +PY +cfg=$(dirname "$(find target -name compile_commands.json -not -path '*/.build-mcpp/*' | head -1)") +[[ -f "$cfg/mcpp-contract.json" ]] || { echo "FAIL: no contract beside the compile database"; exit 1; } +echo "PASS: 982" diff --git a/tests/unit/test_descriptor_keys.cpp b/tests/unit/test_descriptor_keys.cpp new file mode 100644 index 000000000..93acf3650 --- /dev/null +++ b/tests/unit/test_descriptor_keys.cpp @@ -0,0 +1,75 @@ +// A descriptor is read through the blocks the target's row names, in order +// (design 2026-10-29 §5.1; #851): the first key with a block wins, and the +// build machine takes no part. Both readers -- the version table under `xpm` +// and the per-platform table of the `mcpp` segment -- follow the same order. + +#include + +import std; +import mcpp.manifest; +import mcpp.toolchain.platform; + +namespace { + +using mcpp::toolchain::platform::TargetPlatform; + +constexpr std::string_view kBoth = R"( +package = { + name = "taskflow", + xpm = { + linux = { ["4.1.0"] = { url = "https://example.org/linux.tar.gz" } }, + emscripten = { ["4.0.0"] = { url = "https://example.org/web.tar.gz" } }, + }, +} +mcpp = { + cxxflags = { "-DCOMMON" }, + linux = { cxxflags = { "-pthread" } }, + emscripten = { cxxflags = { "-DWEB" } }, +} +)"; + +constexpr std::string_view kLinuxOnly = R"( +package = { + name = "taskflow", + xpm = { linux = { ["4.1.0"] = { url = "https://example.org/linux.tar.gz" } } }, +} +mcpp = { + cxxflags = { "-DCOMMON" }, + linux = { cxxflags = { "-pthread" } }, +} +)"; + +std::vector versions(std::string_view lua, std::string_view os) { + return mcpp::manifest::list_xpkg_versions(lua, TargetPlatform::for_os(os)); +} + +} // namespace + +TEST(DescriptorKeys, AWebBuildReadsItsOwnBlockBeforeLinux) { + EXPECT_EQ(versions(kBoth, "emscripten"), (std::vector{"4.0.0"})); + EXPECT_EQ(versions(kBoth, "linux"), (std::vector{"4.1.0"})); +} + +TEST(DescriptorKeys, AWebBuildFallsToLinuxOnlyByItsRowsOrderNotTheHost) { + EXPECT_EQ(versions(kLinuxOnly, "emscripten"), (std::vector{"4.1.0"})); + EXPECT_EQ(versions(kLinuxOnly, "openkal"), (std::vector{"4.1.0"})); +} + +TEST(DescriptorKeys, ABareMetalBuildReadsNoHostedBlock) { + EXPECT_TRUE(versions(kLinuxOnly, "none").empty()); + EXPECT_TRUE(versions(kLinuxOnly, "windows").empty()); +} + +TEST(DescriptorKeys, TheSegmentTableFollowsTheSameOrder) { + auto web = mcpp::manifest::synthesize_from_xpkg_lua(kBoth, "taskflow", "4.0.0", + TargetPlatform::for_os("emscripten")); + ASSERT_TRUE(web) << web.error().format(); + auto const& f = web->buildConfig.cxxflags; + EXPECT_NE(std::ranges::find(f, "-DWEB"), f.end()); + EXPECT_EQ(std::ranges::find(f, "-pthread"), f.end()); + + auto ok = mcpp::manifest::synthesize_from_xpkg_lua(kLinuxOnly, "taskflow", "4.1.0", + TargetPlatform::for_os("openkal")); + ASSERT_TRUE(ok) << ok.error().format(); + EXPECT_NE(std::ranges::find(ok->buildConfig.cxxflags, "-pthread"), ok->buildConfig.cxxflags.end()); +} diff --git a/tests/unit/test_platform.cpp b/tests/unit/test_platform.cpp index 94bed6001..a3336c579 100644 --- a/tests/unit/test_platform.cpp +++ b/tests/unit/test_platform.cpp @@ -89,9 +89,9 @@ TEST(PlatformManifest, EveryGroupIsRead) { EXPECT_EQ(p.toolchain.value, "llvm@23.1.3"); EXPECT_EQ(p.runner.cmd, (std::vector{"okrun", "{}"})); EXPECT_EQ(p.runner.host, "same-arch"); - EXPECT_EQ(p.carrier.package.spec.shortName, "openkal-run"); - EXPECT_EQ(p.carrier.section, ".oke"); - EXPECT_TRUE(p.carrier.main.ends_with("carrier/main.cpp")); + EXPECT_EQ(p.carriers.at(0).package.spec.shortName, "openkal-run"); + EXPECT_EQ(p.carriers.at(0).section, ".oke"); + EXPECT_TRUE(p.carriers.at(0).main.ends_with("carrier/main.cpp")); ASSERT_TRUE(p.layers.contains("c++-abi")); EXPECT_EQ(p.layers.at("c++-abi").ref.spec.version, "0.16.3"); EXPECT_EQ(p.layers.at("c-abi").ref.spec.shortName, "openkal-musl"); @@ -487,3 +487,75 @@ TEST(PlatformReport, SourcesAndJson) { EXPECT_NE(json.find("\"layers.c++-abi\""), std::string::npos); EXPECT_EQ(pf::to_json(pf::Resolved{}), "null"); } + +// ── Runner hosts, carrier entries, descriptor keys (design 2026-10-29) ────── + +TEST(PlatformManifest, RunnerHostsCarrierEntriesAndDescriptorKeysAreRead) { + auto m = parse(R"( +[package] +name = "p" +version = "1.0.0" + +[platform] +targets = ["x86_64-openkal"] +runner = { cmd = ["okrun", "{}"], host = "same-arch", hosts = ["linux-x86_64", "macos-aarch64"] } +carrier = [ + { package = "openkal-run@0.1.3", section = ".oke", main = "carrier/main.cpp", hosts = ["linux-x86_64"] }, + { package = "openkal-run@0.1.3", section = ".oke", main = "carrier/main.cpp" }, + { package = "openkal-cosmo@0.1.0", section = ".oke", main = "carrier/ape.cpp", build = "ape" }, +] +descriptor_keys = ["openkal", "linux"] +)"); + auto const& p = m.platform; + EXPECT_TRUE(p.runner.hostsSet); + EXPECT_EQ(p.runner.hosts, (std::vector{"linux-x86_64", "macos-aarch64"})); + // The entry of a model this mcpp does not build is skipped, not refused. + ASSERT_EQ(p.carriers.size(), 2u); + EXPECT_EQ(p.carriers[0].hosts, (std::vector{"linux-x86_64"})); + EXPECT_EQ(p.carriers[0].build, "row"); + EXPECT_EQ(p.descriptorKeys.value, (std::vector{"openkal", "linux"})); + // The entry naming the pair serves it; another pair takes the open entry. + EXPECT_EQ(mp::carrier_for(p, "linux-x86_64"), &p.carriers[0]); + EXPECT_EQ(mp::carrier_for(p, "windows-x86_64"), &p.carriers[1]); +} + +TEST(PlatformManifest, ACarrierWithoutHostsServesEveryPairAndNoneServesNothing) { + auto m = parse(kPlatform); + EXPECT_EQ(mp::carrier_for(m.platform, "macos-aarch64"), &m.platform.carriers.at(0)); + mp::Statement none; + EXPECT_EQ(mp::carrier_for(none, "linux-x86_64"), nullptr); +} + +TEST(PlatformManifest, HostPairsAndDescriptorKeysAreAClosedVocabulary) { + const auto base = std::string(R"( +[package] +name = "p" +version = "1.0.0" +[platform] +targets = ["x86_64-openkal"] +)"); + EXPECT_NE(parse_error(base + R"(runner = { cmd = ["okrun"], hosts = ["linux"] })" "\n"), ""); + EXPECT_NE(parse_error(base + R"(runner = { cmd = ["okrun"], hosts = ["freebsd-x86_64"] })" "\n"), ""); + EXPECT_NE(parse_error(base + R"(descriptor_keys = ["macos"])" "\n").find("macosx"), std::string::npos); + EXPECT_NE(parse_error(base + R"(descriptor_keys = [])" "\n"), ""); + EXPECT_EQ(parse_error(base + R"(descriptor_keys = ["openkal", "linux"])" "\n"), ""); +} + +TEST(PlatformMerge, AnOverrideReplacesHostsCarriersAndDescriptorKeys) { + auto m = parse(kPlatform); + mp::Statement over; + over.runner.hosts = {"windows-x86_64"}; + over.runner.hostsSet = true; + over.descriptorKeys = mp::Words{{"openkal"}, {}, true}; + mp::Carrier c; + c.section = ".x"; + c.set = true; + over.carriers = {c}; + auto merged = m.platform; + pf::merge(merged, over); + EXPECT_EQ(merged.runner.hosts, (std::vector{"windows-x86_64"})); + EXPECT_EQ(merged.runner.cmd, m.platform.runner.cmd); + ASSERT_EQ(merged.carriers.size(), 1u); + EXPECT_EQ(merged.carriers[0].section, ".x"); + EXPECT_EQ(merged.descriptorKeys.value, (std::vector{"openkal"})); +} From 6eb85d61a697cc3f06cbd4ef65dbe86b0ffcc8e6 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 02:56:50 +0900 Subject: [PATCH 03/20] runs_on in mcpp-package.json and the index entry; the Windows launcher is a program - A package states where it runs, once, when it is packed: runs_on.self (with what it carries) and runs_on.with. (given the machine's runner, from the platform's runner.hosts). --merge unites it; an older package's is derived from its systems and images. - mcpp emit xpkg writes each system's block from runs_on: a self-contained universal package names the pairs whose runner it carries, so an Intel Mac is no longer offered a package with no runner for it (#860). - bin/.exe replaces bin/.cmd: the engine's launcher program, built for x86_64 Windows over the platform's hosted arrangement, finds the package from its own path and becomes the runner; the POSIX script hands a Windows shell (Git Bash, MSYS, Cygwin) to it. xvm registers .exe. - dep_dir's refusal of an ambiguous name is printed on standard output: a FILE binding in the build-program module collided with a program's . Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 63 ++++ src/build/hostprogram.cppm | 8 +- src/pack/distribution.cppm | 322 +++++++++++++----- src/pack/pack.cppm | 53 ++- ...versal_package_holds_every_architecture.sh | 4 +- tests/unit/test_abi.cpp | 2 +- tests/unit/test_descriptor_keys.cpp | 2 + tests/unit/test_manifest.cpp | 7 +- tests/unit/test_pack_distribution.cpp | 50 ++- 9 files changed, 402 insertions(+), 109 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5bf63c335..af8111cb3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,69 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. +## [2026.10.30.1] - 2026-10-30 + +### Added + +- **The build's contract, `${mcpp.contract}`.** Every configuration writes + `mcpp-contract.json` beside its compile database: the compiler of each + language with the words every unit is compiled with (target, sysroot, + target-side headers, platform and profile flags), and the link driver, its + words and the archiver. A build program's action names it with + `${mcpp.contract}` and reads it while ninja runs, so a foreign build system + compiles for the target as the engine does; the engine's own C++ dialect and + module words are not in it (e2e 982). mcpp-plugins 0.21.0's `deps-cmake` + compiles a cross build's subproject from it. +- **`[platform] runner.hosts`, carrier entries and `descriptor_keys`.** A + platform states the `-` pairs its runner exists for, any number of + carrier entries each for the pairs in its `hosts` (an entry of a carrier + model this mcpp does not build is skipped with a warning), and the + descriptor blocks its targets read (SPEC-013 §1.1, §10, §11, §14). +- **`runs_on` in `mcpp-package.json`.** Where a package runs is computed once + when it is packed: `self`, the pairs it runs on with what it carries, and + `with.`, the pairs it runs on given the machine's runner. + +### Changed + +- **A target reads the descriptor blocks its row names, never the build + machine's (#851).** A target that is not Linux, macOS or Windows read the + block of the system mcpp ran on, so a Web build took a dependency's `linux` + block on Linux and its `windows` block on Windows, and an ok image the block + of whichever system built it. Each row now names its blocks in order: + `wasm32-emscripten` reads `emscripten`, then `linux`; `-openkal` + `openkal`, then `linux` (a platform may state its own); iOS `ios`, then + `macosx`; bare metal `none`. Version resolution, the synthesised manifest + and the index refresh read the same block (unit `DescriptorKeys`). +- **The Web link carries the profile's optimisation level (#851).** emcc runs + wasm-opt and minifies the JavaScript launcher at the link, which had no + `-O`: a release Web program was linked at `-O0`. The hello world's + JavaScript is 13 KB rather than 66 KB (e2e 981). +- **`[target..abi] exceptions` takes the model (#851).** `"wasm"` selects + WebAssembly's exception handling (`-fwasm-exceptions`), `"js"` the + JavaScript model; `true` keeps the JavaScript model it has selected since + 2026.9.12.3. The model is part of every cache key (e2e 981). +- **`mcpp emit xpkg` names where a package runs (#860).** A + self-contained universal package's entry named every system with every + image architecture, so an Intel Mac installed a package that carries no + runner for it. Each system's block names the architectures `runs_on` gives: + the carried runners for a self-contained package, the pairs the platform's + runner exists for otherwise. + +### Fixed + +- **A name two dependencies answer to is no package's (#852).** Two + dependencies whose manifests both say `rules` warned on every build, and the + bare name went to one of them. It is now published for neither: + `MCPP_DEP_AMBIGUOUS_RULES` lists the qualified names, each qualified name + keeps its own directory, and `dep_dir("rules")` ends the build program naming + both (e2e 980). +- **Programs start in the Android app sandbox again (#857).** Under `run-as` + (SELinux `runas_app`) every program mcpp started failed with "not found" + since 2026.10.26.2. A static strace showed one failing call, before any + process was created: the root directory opened for reading, which the + domain may search but not read. openkal-linux 0.18.3 opens every directory + handle with `O_PATH`. + ## [2026.10.29.1] - 2026-10-29 ### Fixed diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 6ed67c073..94c704dc3 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -805,10 +805,12 @@ inline const char* dep_dir(const char* name) { unsigned long a = 19; for (unsigned long i = 9; i < o && a + 1 < sizeof ambiguous; ++i, ++a) ambiguous[a] = buf[i]; ambiguous[a] = 0; + // Said on standard output, which the engine prints when the program exits + // non-zero; `std::fprintf(stderr, ...)` here would bind `FILE` in this + // module and collide with a build program's own . if (const char* which = std::getenv(ambiguous)) { - std::fprintf(stderr, - "build.mcpp: dep_dir(\"%s\") names more than one dependency (%s); ask by one of " - "those names\n", name, which); + std::printf("build.mcpp: dep_dir(\"%s\") names more than one dependency (%s); ask by one of " + "those names\n", name, which); std::exit(2); } buf[o++] = '_'; buf[o++] = 'D'; buf[o++] = 'I'; buf[o++] = 'R'; buf[o] = 0; diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 0f85bf267..7af8ed548 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -57,7 +57,7 @@ unpack(const std::filesystem::path& package, const std::filesystem::path& into); // ─── launchers (SPEC-014 §4.3) ──────────────────────────────────────────── struct Launch { - std::string name; // bin/, bin/.cmd + std::string name; // bin/, bin/.exe // The platform's `runner.cmd`, expanded as `mcpp run` expands it: the // first `{}` of a word is the image (`{}`, `--image={}`), and the image is // appended when no word holds one. @@ -68,14 +68,20 @@ struct Launch { std::string loader; }; -// Whether both launchers can state `l`: a `.cmd` keeps a word literal inside -// double quotes, so a word holding a double quote or a line break cannot be -// written in one. Checked before a launcher is written; `windows_launcher` of -// a `Launch` this refuses is a `.cmd` that says so and runs nothing. -std::expected launchable(const Launch& l); std::string posix_launcher(const Launch& l); -std::string windows_launcher(const Launch& l); + +// THE WINDOWS LAUNCHER IS A PROGRAM (design 2026-10-29 §4.3), `bin/.exe`: +// a `.cmd` asked "Terminate batch job (Y/N)?" at Ctrl-C, showed `cmd` as the +// process, and could not state every word. The engine writes its source with +// the launch facts as constants and builds it for `-windows-gnu` like +// the single-file launcher: it finds the package from its own path +// (`env::image`, not argv[0]), picks the image of this machine's architecture, +// looks for the runner in `libexec//windows-/` and then on PATH, +// and becomes it. A POSIX shell on Windows (Git Bash, MSYS, Cygwin) picks the +// script of the same name, which hands over to this program. +std::string exe_launcher_manifest(std::string_view name, std::string_view version); +std::string exe_launcher_main(const Launch& l); // ─── mcpp-package.json (SPEC-014 §4.5) ──────────────────────────────────── @@ -98,9 +104,29 @@ struct Package { std::vector runner; // empty for a native package std::map entry; // arch -> path of the entry (universal) std::string program; // the program's path (one native leg) + // WHERE THE PACKAGE RUNS (design 2026-10-29 §4.4), computed once when it + // is packed: `self` the `-` pairs it runs on with what it + // carries and the system's own libraries; `with` the pairs it runs on + // given a runner the machine provides (`okrun` -> pairs). A reader asks + // here and computes nothing. + std::vector runsOnSelf; + std::map> runsOnWith; + bool runsOnSet = false; std::vector files; }; +// `runs_on` for `p`, from its legs, the runners it carries (`systems`), its +// mode, and the platform's statement of where its runner exists (`runnerHosts`; +// `runnerAnyArch` for a runner that reaches an artifact from any machine). +// A native package of one leg runs where its triple's program runs; a +// WebAssembly, Android, iOS or bare-metal package is no program of a system and +// runs nowhere an index entry could name. +void compute_runs_on(Package& p, const std::vector& runnerHosts, bool runnerAnyArch); + +// The `-` a triple's program runs on (`linux-x86_64`), or nothing +// for a target that is no program of Linux, macOS or Windows. +std::optional host_pair_of(std::string_view triple); + // What a caller knows of a file's role and leg; every other file's role is // read from where it lies (`libexec/` a loader, `lib/` a library, ...). struct Stated { @@ -167,7 +193,7 @@ struct Asset { // one system and architecture each are that system's entry, the architecture // in the file name a `${arch}` of one URL. The install moves the archive's // one top-level directory into place; the program is registered from its -// launcher (`bin/`, `bin/.cmd` on Windows) or the program itself. +// launcher (`bin/`, `bin/.exe` on Windows) or the program itself. std::expected xpkg_descriptor(const std::vector& assets, std::string_view urlPrefix); @@ -928,29 +954,6 @@ std::string sh_word(std::string_view w) { return out; } -// Text of a .cmd inside double quotes: `%` doubled, the one character a batch -// file reads there. -std::string cmd_quoted(std::string_view w) { - std::string out; - for (char ch : w) { - if (ch == '%') out += "%%"; - else out.push_back(ch); - } - return out; -} - -// Text of a .cmd outside quotes (an `echo`): `%` doubled and every character -// the command line reads escaped with `^`. -std::string cmd_bare(std::string_view w) { - std::string out; - for (char ch : w) { - if (ch == '%') out += "%%"; - else if (std::string_view("&|<>^()").find(ch) != std::string_view::npos) out += {'^', ch}; - else out.push_back(ch); - } - return out; -} - // How a launcher's language writes the runner's words: `literal` a piece of // a word as text, `image` the image's variable, `around` what encloses each // word. @@ -992,20 +995,6 @@ std::string arch_list(const Launch& l) { } // namespace -std::expected launchable(const Launch& l) { - std::vector words{l.name, l.loader}; - words.insert(words.end(), l.runner.begin(), l.runner.end()); - for (auto const& [arch, file] : l.entry) words.insert(words.end(), {arch, file}); - for (auto w : words) - // A word ending in `\` would make the quote that closes it a quote of - // the word for the program reading its arguments. - if (w.find_first_of(std::string_view("\"\r\n\0", 4)) != std::string_view::npos || w.ends_with('\\')) - return std::unexpected(std::format( - "the launcher of {} cannot state '{}': a word of a .cmd holds no double quote and no " - "line break, and does not end in a backslash", l.name, w)); - return {}; -} - std::string posix_launcher(const Launch& l) { std::string cases; for (auto const& [arch, file] : l.entry) @@ -1050,6 +1039,7 @@ std::string posix_launcher(const Launch& l) { "case \"$(uname -s)\" in\n" " Linux) os=linux ;;\n" " Darwin) os=macos ;;\n" + " MINGW*|MSYS*|CYGWIN*) exec \"$top/bin/\"{0}.exe \"$@\" ;;\n" " *) os=$(uname -s) ;;\n" "esac\n" "case \"$arch\" in\n" @@ -1062,47 +1052,96 @@ std::string posix_launcher(const Launch& l) { sh_word(l.name), cases, sh_word(arch_list(l)), find, args); } -std::string windows_launcher(const Launch& l) { - if (!launchable(l)) - return "@echo off\r\n" - "rem Generated by mcpp pack (SPEC-014 §4.3).\r\n" - "echo This package's runner has a word a .cmd cannot state; run it from a POSIX shell. 1>&2\r\n" - "exit /b 126\r\n"; - std::string picks; - for (auto const& [arch, file] : l.entry) - picks += std::format("if \"%arch%\"==\"{0}\" set \"image=%top%\\lib\\{0}\\{1}\"\r\n", - cmd_quoted(arch), cmd_quoted(file)); - std::string find = "set \"runner=%image%\"\r\n", args; - if (!l.runner.empty()) { - args = runner_args(l.runner, { cmd_quoted, "%image%", "\"" }); - const auto program = cmd_quoted(l.runner.front()); - find = std::format("set \"runner={}\"\r\n", program); - if (!l.loader.empty()) - find = std::format( - "set \"runner=%top%\\libexec\\{0}\\windows-%arch%\\{0}.exe\"\r\n" - "if not exist \"%runner%\" set \"runner={1}\"\r\n", cmd_quoted(l.loader), program); - } +std::string exe_launcher_manifest(std::string_view name, std::string_view version) { return std::format( - "@echo off\r\n" - "rem Generated by mcpp pack (SPEC-014 §4.3): runs the package's program on this\r\n" - "rem machine's architecture through the platform's runner.\r\n" - "setlocal DisableDelayedExpansion\r\n" - "for %%I in (\"%~dp0..\") do set \"top=%%~fI\"\r\n" - "set \"machine=%PROCESSOR_ARCHITECTURE%\"\r\n" - "if defined PROCESSOR_ARCHITEW6432 set \"machine=%PROCESSOR_ARCHITEW6432%\"\r\n" - "set \"arch=%machine%\"\r\n" - "if /i \"%machine%\"==\"AMD64\" set \"arch=x86_64\"\r\n" - "if /i \"%machine%\"==\"ARM64\" set \"arch=aarch64\"\r\n" - "set \"image=\"\r\n" - "{1}" - "if not defined image (\r\n" - " echo {0}: this machine is %arch%; the package holds {2} 1>&2\r\n" - " exit /b 126\r\n" - ")\r\n" - "{3}" - "\"%runner%\"{4} %*\r\n" - "exit /b %ERRORLEVEL%\r\n", - cmd_bare(l.name), picks, cmd_bare(arch_list(l)), find, args); + "# Generated by `mcpp pack`: the Windows launcher of {0} (design 2026-10-29 §4.3).\n" + "[package]\nname = \"{0}\"\nversion = \"{1}\"\n\n" + "[targets.{0}]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n" + "# 0.2.1: the release whose Windows half builds over a platform's hosted\n" + "# arrangement, which the launcher is built over.\n" + "[dependencies.mcpplibs]\nos = \"0.2.1\"\n", + name, version.empty() ? std::string_view("0.0.0") : version); +} + +std::string exe_launcher_main(const Launch& l) { + auto cxx = [](std::string_view w) { // a C++ raw string of `w` + return std::format("R\"mcpp({})mcpp\"", w); + }; + std::string entries, words; + for (auto const& [arch, file] : l.entry) entries += std::format(" {{ {}, {} }},\n", cxx(arch), cxx(file)); + for (auto const& w : l.runner) words += std::format(" {},\n", cxx(w)); + return std::format(R"MCPP(// The Windows launcher of {0} (design 2026-10-29 §4.3), written by mcpp pack. +import std; +import mcpplibs.os; + +namespace os = mcpplibs::os; +namespace fs = std::filesystem; + +constexpr std::string_view kName = {1}; +constexpr std::string_view kLoader = {2}; +constexpr std::pair kEntries[] = {{ +{3}}}; +constexpr std::string_view kRunner[] = {{ +{4} std::string_view{{}}, +}}; + +int main(int argc, char** argv) {{ + // The package is the parent of the directory this program is in, found + // from the program's own path and not from argv[0]. + const auto self = os::env::image(); + const fs::path top = self ? self->parent_path().parent_path() : fs::current_path(); + std::string machine = os::env::get("PROCESSOR_ARCHITEW6432").value_or( + os::env::get("PROCESSOR_ARCHITECTURE").value_or("")); + for (auto& c : machine) c = static_cast(std::toupper(static_cast(c))); + const std::string arch = machine == "AMD64" || machine == "X86_64" ? "x86_64" + : machine == "ARM64" || machine == "AARCH64" ? "aarch64" : machine; + std::string_view entry; + for (auto const& [a, file] : kEntries) + if (a == arch) entry = file; + if (entry.empty()) {{ + std::string all; + for (auto const& [a, file] : kEntries) all += (all.empty() ? "" : ", ") + std::string(a); + std::println(std::cerr, "{{}}: this machine is {{}}; the package holds {{}}", kName, arch, all); + return 126; + }} + const fs::path image = top / "lib" / arch / std::string(entry); + if (!fs::exists(image)) {{ + std::println(std::cerr, "{{}}: the package has no {{}}", kName, image.string()); + return 126; + }} + os::process::spawn_options o; + std::vector args; + bool placed = false; + std::size_t first = 0; + if (kRunner[0].empty()) {{ + o.program = image; + }} else {{ + fs::path carried; + if (!kLoader.empty()) + carried = top / "libexec" / std::string(kLoader) / ("windows-" + arch) / (std::string(kLoader) + ".exe"); + std::error_code ec; + o.program = !carried.empty() && fs::exists(carried, ec) ? carried : fs::path(std::string(kRunner[0])); + first = 1; + }} + for (std::size_t i = first; !kRunner[i].empty(); ++i) {{ + std::string w(kRunner[i]); + // A word's first `{{}}` is the image, as `mcpp run` expands it. + if (auto at = w.find("{{}}"); at != std::string::npos) {{ + w.replace(at, 2, image.string()); + placed = true; + }} + args.push_back(std::move(w)); + }} + if (!kRunner[0].empty() && !placed) args.push_back(image.string()); + for (int i = 1; i < argc; ++i) args.emplace_back(argv[i]); + o.args = std::move(args); + o.in = o.out = o.err = os::process::stdio::inherit; + const std::string program = o.program.string(); + (void)os::process::exec(std::move(o)); + std::println(std::cerr, "{{}}: cannot start {{}}", kName, program); + return 127; +}} +)MCPP", l.name, cxx(l.name), cxx(l.loader), entries, words); } // ─── mcpp-package.json ──────────────────────────────────────────────────── @@ -1139,6 +1178,42 @@ std::string role_of(std::string_view path, bool executable) { } // namespace +std::optional host_pair_of(std::string_view triple) { + auto t = mcpp::toolchain::triple::parse(triple); + if (!t || t->env == "android") return std::nullopt; + if (t->os != "linux" && t->os != "macos" && t->os != "windows") return std::nullopt; + return std::format("{}-{}", t->os, t->arch); +} + +void compute_runs_on(Package& p, const std::vector& runnerHosts, bool runnerAnyArch) { + p.runsOnSelf.clear(); + p.runsOnWith.clear(); + p.runsOnSet = true; + auto arch_of_pair = [](std::string const& pair) { return pair.substr(pair.find('-') + 1); }; + std::set archs; + for (auto const& l : p.legs) + if (auto t = mcpp::toolchain::triple::parse(l)) archs.insert(t->arch); + for (auto const& [arch, _] : p.entry) archs.insert(arch); + if (p.runner.empty()) { + // A program of a system: the carrier's system when it carries an + // image, else the leg's own. + if (!p.systems.empty()) p.runsOnSelf = p.systems; + else if (p.legs.size() == 1) + if (auto h = host_pair_of(p.legs.front())) p.runsOnSelf.push_back(*h); + return; + } + const auto runner = std::filesystem::path(p.runner.front()).stem().string(); + if (p.mode == "self-contained") + for (auto const& s : p.systems) + if (archs.contains(arch_of_pair(s))) p.runsOnSelf.push_back(s); + std::vector with; + for (auto const& h : runnerHosts) + if ((runnerAnyArch || archs.contains(arch_of_pair(h))) + && std::ranges::find(p.runsOnSelf, h) == p.runsOnSelf.end()) + with.push_back(h); + if (!with.empty()) p.runsOnWith[runner] = std::move(with); +} + std::string to_json(const Package& p) { nlohmann::json files = nlohmann::json::array(); for (auto const& f : p.files) @@ -1164,6 +1239,11 @@ std::string to_json(const Package& p) { j["legs"] = p.legs; j["systems"] = p.systems; j["launch"] = std::move(launch); + if (p.runsOnSet) { + nlohmann::ordered_json with = nlohmann::ordered_json::object(); + for (auto const& [runner, pairs] : p.runsOnWith) with[runner] = pairs; + j["runs_on"] = {{"self", p.runsOnSelf}, {"with", std::move(with)}}; + } j["files"] = std::move(files); return j.dump(2) + "\n"; } @@ -1203,6 +1283,14 @@ std::expected read(const fs::path& root) { p.mode = j.value("mode", std::string{}); p.legs = j.value("legs", std::vector{}); p.systems = j.value("systems", std::vector{}); + if (j.contains("runs_on") && j["runs_on"].is_object()) { + auto const& r = j["runs_on"]; + p.runsOnSelf = r.value("self", std::vector{}); + if (r.contains("with") && r["with"].is_object()) + for (auto it = r["with"].begin(); it != r["with"].end(); ++it) + p.runsOnWith[it.key()] = it.value().get>(); + p.runsOnSet = true; + } if (j.contains("launch")) { auto const& l = j["launch"]; if (l.contains("runner")) p.runner = l["runner"].get>(); @@ -1543,20 +1631,52 @@ std::expected xpkg_descriptor(const std::vector return std::unexpected(std::string("a universal package is one asset; merge its legs first " "(mcpp pack --merge)")); auto const& a = assets.front(); - for (auto const& [arch, _] : first.entry) archs.push_back(arch); + // WHERE IT RUNS IS WHAT IT SAYS (design 2026-10-29 §4.4, #860): a + // self-contained package names the pairs whose runner it carries, and + // a package that needs the machine's runner the pairs that runner + // exists for. A package written before `runs_on` existed says the + // first through `systems`, and the second not at all: every system, + // every architecture of its images. + Package pkg = first; + if (!pkg.runsOnSet) { + std::vector every; + for (auto system : { "linux", "macos", "windows" }) + for (auto const& [arch, _] : pkg.entry) every.push_back(std::format("{}-{}", system, arch)); + compute_runs_on(pkg, every, false); + } + const bool carried = pkg.mode == "self-contained"; + std::vector pairs = carried ? pkg.runsOnSelf : std::vector{}; + if (!carried) + for (auto const& [r, hosts] : pkg.runsOnWith) pairs.insert(pairs.end(), hosts.begin(), hosts.end()); // The runner is a package of the index's own namespace; a bare name // turns ambiguous as soon as a second index the home lists has one // (xim-pkgindex refuses it: check-dep-namespace). std::string deps; - if (first.mode != "self-contained") + if (!carried) deps = std::format(" deps = {{ \"xim:{}\" }},\n", lua_text(std::filesystem::path(first.runner.front()).stem().string())); - for (auto system : { "linux", "macosx", "windows" }) - blocks[system] = std::format( + for (auto [system, key] : { std::pair{"linux", "linux"}, std::pair{"macos", "macosx"}, + std::pair{"windows", "windows"} }) { + std::vector systemArchs; + for (auto const& pr : pairs) + if (pr.starts_with(std::string(system) + "-")) systemArchs.push_back(pr.substr(std::strlen(system) + 1)); + if (systemArchs.empty()) continue; + std::ranges::sort(systemArchs); + std::string shas; + for (auto const& arch : systemArchs) { + if (std::ranges::find(archs, arch) == archs.end()) archs.push_back(arch); + shas += std::format(" [\"{}\"] = \"{}\",\n", lua_text(arch), lua_text(a.sha256)); + } + blocks[key] = std::format( "{0} [\"latest\"] = {{ ref = \"{1}\" }},\n" - " [\"{1}\"] = {{\n url = \"{2}/{3}\",\n sha256 = \"{4}\",\n }},\n", - deps, lua_text(first.version), lua_text(prefix), lua_text(a.file.filename().string()), - lua_text(a.sha256)); + " [\"{1}\"] = {{\n url = \"{2}/{3}\",\n sha256 = {{\n{4}" + " }},\n }},\n", + deps, lua_text(first.version), lua_text(prefix), lua_text(a.file.filename().string()), shas); + } + if (blocks.empty()) + return std::unexpected(std::format("{} runs on no system an index entry names " + "(mcpp-package.json runs_on is empty)", + a.file.filename().string())); } else { // system -> (arch -> asset) std::map> bySystem; @@ -1616,7 +1736,7 @@ std::expected xpkg_descriptor(const std::vector // each system's directory. std::string bindir = "\"bin\"", alias, bindirs; if (universal) { - alias = std::format("if os.host() == \"windows\" then opts.alias = \"{}.cmd\" end\n ", + alias = std::format("if os.host() == \"windows\" then opts.alias = \"{}.exe\" end\n ", lua_text(command)); } else { std::map dirOf; // system -> the program's directory @@ -1835,6 +1955,9 @@ std::expected merge(const MergeRequest& req) { Package merged = first; merged.legs.clear(); merged.systems.clear(); + merged.runsOnSelf.clear(); + merged.runsOnWith.clear(); + merged.runsOnSet = false; merged.entry.clear(); Stated stated; std::map> seen; // path -> (digest, input) @@ -1844,6 +1967,14 @@ std::expected merge(const MergeRequest& req) { if (std::ranges::find(merged.legs, l) == merged.legs.end()) merged.legs.push_back(l); for (auto const& s : pkg.systems) if (std::ranges::find(merged.systems, s) == merged.systems.end()) merged.systems.push_back(s); + // Where the merged package runs is where any of its inputs ran. + for (auto const& s : pkg.runsOnSelf) + if (std::ranges::find(merged.runsOnSelf, s) == merged.runsOnSelf.end()) merged.runsOnSelf.push_back(s); + for (auto const& [r, pairs] : pkg.runsOnWith) + for (auto const& h : pairs) + if (std::ranges::find(merged.runsOnWith[r], h) == merged.runsOnWith[r].end()) + merged.runsOnWith[r].push_back(h); + merged.runsOnSet = merged.runsOnSet || pkg.runsOnSet; for (auto const& [arch, e] : pkg.entry) merged.entry[arch] = e; // Each file as it is, not as its package says: a file changed after the // package was written is refused, and two inputs agree on a file when @@ -1873,6 +2004,11 @@ std::expected merge(const MergeRequest& req) { } std::ranges::sort(merged.legs); std::ranges::sort(merged.systems); + std::ranges::sort(merged.runsOnSelf); + for (auto& [r, pairs] : merged.runsOnWith) { + std::ranges::sort(pairs); + std::erase_if(pairs, [&](std::string const& h) { return std::ranges::find(merged.runsOnSelf, h) != merged.runsOnSelf.end(); }); + } if (auto d = describe(out, merged, stated); !d) return std::unexpected(d.error()); if (format == "dir") { diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index d5597b7b5..7829f27d3 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -1609,7 +1609,7 @@ std::expected finalize_tree(const Plan& plan) { pkg.runner = plan.platform.fields.runner.cmd; const auto stem = fs::path(plan.binaryName).stem().string(); stated.role["bin/" + stem] = "launcher"; - stated.role["bin/" + stem + ".cmd"] = "launcher"; + stated.role["bin/" + stem + ".exe"] = "launcher"; std::vector> legs{ { canonical(plan.triple), [&] { auto t = mcpp::toolchain::triple::parse(plan.triple); return t ? t->arch : plan.triple; }() } }; @@ -1637,6 +1637,18 @@ std::expected finalize_tree(const Plan& plan) { // of; an index entry reads it there (SPEC-014 §8). pkg.systems = carried_systems(plan); } + // Where it runs, once (design 2026-10-29 §4.4): every reader of the + // package -- an index entry, a merge, the summary -- asks here. + d::compute_runs_on(pkg, plan.platform.active ? plan.platform.fields.runner.hosts : std::vector{}, + plan.platform.active && plan.platform.fields.runner.host == "any"); + // A package of images starts on Windows through its `.exe`; without one + // it does not run there. + if (!pkg.runner.empty() + && !fs::is_regular_file(plan.stagingRoot / "bin" / (fs::path(plan.binaryName).stem().string() + ".exe"), ec)) { + auto windows = [](std::string const& p) { return p.starts_with("windows-"); }; + std::erase_if(pkg.runsOnSelf, windows); + for (auto& [r, pairs] : pkg.runsOnWith) std::erase_if(pairs, windows); + } // What `[pack] include` named is the package's resource, whatever its name. for (auto const& rel : *included) stated.role[rel] = "resource"; auto described = d::describe(plan.stagingRoot, pkg, stated); @@ -2614,9 +2626,6 @@ run_loaded(const Plan& plan) launch.runner = runner.cmd; for (auto const& leg : legs) launch.entry[leg.arch] = leg.artifact.filename().string(); if (plan.opts.mode == Mode::BundleAll) launch.loader = fs::path(runner.cmd.front()).stem().string(); - // A word a `.cmd` cannot hold is refused now, before anything is staged. - if (auto ok = mcpp::pack::distribution::launchable(launch); !ok) - return std::unexpected(Error{ok.error()}); for (auto const& leg : legs) { const auto dir = plan.stagingRoot / "lib" / leg.arch; if (auto r = copy(leg.artifact, dir / leg.artifact.filename()); !r) return r; @@ -2662,8 +2671,40 @@ run_loaded(const Plan& plan) fs::permissions(posix, fs::perms::owner_all | fs::perms::group_read | fs::perms::group_exec | fs::perms::others_read | fs::perms::others_exec, fs::perm_options::replace, ec); - std::ofstream(plan.stagingRoot / "bin" / (launch.name + ".cmd"), std::ios::binary | std::ios::trunc) - << mcpp::pack::distribution::windows_launcher(launch); + // The Windows launcher, a program (design 2026-10-29 §4.3): built for + // x86_64 Windows, which an arm64 Windows machine also runs. A host + // that builds no program for that row leaves it out, says so, and + // the package's `runs_on` names no Windows pair. + // Over the platform's hosted arrangement where it serves the row, as + // the runners are: the same program whichever machine packs it. + const auto row = program_row("windows", "x86_64", true, &plan.platform); + if (!row_is_built(row)) { + mcpp::ui::warning(std::format( + "no Windows launcher in the package: this mcpp builds no program for {}; the package " + "does not run on Windows", row.triple)); + } else { + const auto dir = plan.projectRoot / "target" / "launcher-pack" / launch.name; + fs::create_directories(dir / "src", ec); + fs::remove(dir / "mcpp.lock", ec); + auto write = [&](const fs::path& p, std::string_view text) -> bool { + std::ofstream out(p, std::ios::binary | std::ios::trunc); + out.write(text.data(), static_cast(text.size())); + return bool(out); + }; + if (!write(dir / "mcpp.toml", mcpp::pack::distribution::exe_launcher_manifest( + launch.name, plan.packageVersion) + + static_row_pin(row, plan.platform.fields.toolchain.set + ? plan.platform.fields.toolchain.value : std::string())) + || !write(dir / "src" / "main.cpp", mcpp::pack::distribution::exe_launcher_main(launch))) + return std::unexpected(Error{std::format("cannot write the launcher's sources under {}", + dir.string())}); + auto packed = pack_generated_program(dir, row); + if (!packed) return std::unexpected(packed.error()); + fs::copy_file(*packed / (launch.name + ".exe"), plan.stagingRoot / "bin" / (launch.name + ".exe"), + fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format("cannot place the Windows launcher: {}", + ec.message())}); + } } copy_if_exists(plan.projectRoot / "README.md", plan.stagingRoot); diff --git a/tests/e2e/955_a_universal_package_holds_every_architecture.sh b/tests/e2e/955_a_universal_package_holds_every_architecture.sh index d8cf44513..5569e8fb9 100755 --- a/tests/e2e/955_a_universal_package_holds_every_architecture.sh +++ b/tests/e2e/955_a_universal_package_holds_every_architecture.sh @@ -7,7 +7,7 @@ # # Criteria: # A. `--openkal/` holds `lib/x86_64/` and `lib/aarch64/` (each -# the image of its architecture), `bin/` and `bin/.cmd`, and +# the image of its architecture), `bin/` and `bin/.exe`, and # `mcpp-package.json` naming both legs, the runner and each entry. # B. `bin/` runs the image of this machine's architecture through the # runner found on PATH, with its arguments. @@ -48,7 +48,7 @@ CPP # A "$MCPP" pack --target "$T" --pkg-format dir > a.log 2>&1 || fail "A: pack" a.log P=target/dist/uni-0.1.0-openkal -for f in "lib/x86_64/uni.oke" "lib/aarch64/uni.oke" "bin/uni" "bin/uni.cmd" "mcpp-package.json"; do +for f in "lib/x86_64/uni.oke" "lib/aarch64/uni.oke" "bin/uni" "bin/uni.exe" "mcpp-package.json"; do [ -f "$P/$f" ] || fail "A: the package has no $f" a.log done python3 - "$P" <<'PY' || fail "A: the package's metadata" "$P/mcpp-package.json" diff --git a/tests/unit/test_abi.cpp b/tests/unit/test_abi.cpp index 927c88d9f..75ec7ae59 100644 --- a/tests/unit/test_abi.cpp +++ b/tests/unit/test_abi.cpp @@ -189,7 +189,7 @@ exceptions = 1 )"; auto m = mcpp::manifest::parse_string(src); ASSERT_FALSE(m.has_value()); - EXPECT_NE(m.error().message.find(".exceptions must be true or false"), + EXPECT_NE(m.error().message.find(".exceptions must be true, false, \"wasm\" or \"js\""), std::string::npos) << m.error().message; } diff --git a/tests/unit/test_descriptor_keys.cpp b/tests/unit/test_descriptor_keys.cpp index 93acf3650..f90f4139d 100644 --- a/tests/unit/test_descriptor_keys.cpp +++ b/tests/unit/test_descriptor_keys.cpp @@ -22,6 +22,7 @@ package = { }, } mcpp = { + sources = { "src/*.cpp" }, cxxflags = { "-DCOMMON" }, linux = { cxxflags = { "-pthread" } }, emscripten = { cxxflags = { "-DWEB" } }, @@ -34,6 +35,7 @@ package = { xpm = { linux = { ["4.1.0"] = { url = "https://example.org/linux.tar.gz" } } }, } mcpp = { + sources = { "src/*.cpp" }, cxxflags = { "-DCOMMON" }, linux = { cxxflags = { "-pthread" } }, } diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index cd10ee218..75f097fc7 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -3835,9 +3835,10 @@ TEST(PlatformAxis, HostAndTargetAreNotInterchangeable) { EXPECT_EQ(TargetPlatform::for_os("macosx").key(), "macosx"); EXPECT_EQ(TargetPlatform::for_os("windows").key(), "windows"); EXPECT_EQ(TargetPlatform::for_os("linux").key(), "linux"); - // An absent/unknown os token keeps the pre-#254 behaviour: host. - EXPECT_EQ(TargetPlatform::for_os("").key(), - HostPlatform::current().key()); + // A token naming none of the three reads its own block, never the + // host's (design 2026-10-29 §5.1, #851). + EXPECT_EQ(TargetPlatform::for_os("emscripten").key(), "emscripten"); + EXPECT_EQ(TargetPlatform::for_os("none").key(), "none"); } // The behaviour half: a descriptor's per-OS section must be spliced for the diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index b6d7cc0e3..d5dfde009 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -620,6 +620,54 @@ TEST(PackMerge, TheCompressionFitsTheContainer) { ASSERT_TRUE(fits) << fits.error(); } +// #860 (design 2026-10-29 §4.4): a self-contained universal package runs where +// it carries a runner, and its entry names exactly those pairs -- macOS aarch64 +// only, Windows x86_64 only -- with no runner to install. A package that needs +// the machine's runner names the pairs the platform's runner exists for. +TEST(PackDistribution, AnEntryNamesWhereThePackageRuns) { + d::Asset a; + a.file = "mcpp-ide-0.1.1-openkal.tar.gz"; + a.sha256 = std::string(64, 'c'); + a.top = "mcpp-ide-0.1.1-openkal"; + a.pkg.name = "mcpp-ide"; + a.pkg.version = "0.1.1"; + a.pkg.mode = "self-contained"; + a.pkg.runner = { "okrun", "{}" }; + a.pkg.legs = { "x86_64-openkal", "aarch64-openkal" }; + a.pkg.entry = { { "x86_64", "lib/x86_64/mcpp-ide.oke" }, { "aarch64", "lib/aarch64/mcpp-ide.oke" } }; + a.pkg.systems = { "linux-aarch64", "linux-x86_64", "macos-aarch64", "windows-x86_64" }; + const std::vector hosts = { "linux-x86_64", "linux-aarch64", "macos-aarch64", + "macos-x86_64", "windows-x86_64" }; + d::compute_runs_on(a.pkg, hosts, false); + EXPECT_EQ(a.pkg.runsOnSelf, a.pkg.systems); + ASSERT_TRUE(a.pkg.runsOnWith.contains("okrun")); + EXPECT_EQ(a.pkg.runsOnWith.at("okrun"), (std::vector{"macos-x86_64"})); + auto r = d::xpkg_descriptor({ a }, "https://example.org/v0.1.1"); + ASSERT_TRUE(r) << r.error(); + auto block = [&](std::string_view system) { + const auto at = r->find(std::format(" {} = {{", system)); + return at == std::string::npos ? std::string() : r->substr(at, r->find(" },", at) - at); + }; + EXPECT_NE(block("macosx").find("[\"aarch64\"]"), std::string::npos) << *r; + EXPECT_EQ(block("macosx").find("[\"x86_64\"]"), std::string::npos) << *r; + EXPECT_NE(block("windows").find("[\"x86_64\"]"), std::string::npos) << *r; + EXPECT_EQ(block("windows").find("[\"aarch64\"]"), std::string::npos) << *r; + EXPECT_EQ(r->find("deps = {"), std::string::npos) << *r; + + a.pkg.mode = "vendored"; + a.pkg.systems.clear(); + d::compute_runs_on(a.pkg, hosts, false); + EXPECT_TRUE(a.pkg.runsOnSelf.empty()); + EXPECT_EQ(a.pkg.runsOnWith.at("okrun").size(), hosts.size()); + auto v = d::xpkg_descriptor({ a }, "https://example.org/v0.1.1"); + ASSERT_TRUE(v) << v.error(); + EXPECT_NE(v->find("deps = { \"xim:okrun\" }"), std::string::npos) << *v; + + // The document carries it, and a reader takes it back. + auto j = d::to_json(a.pkg); + EXPECT_NE(j.find("\"runs_on\""), std::string::npos) << j; +} + // The entry is named after the command a person runs, not the package, and // moves the archive's own top-level directory into place. TEST(PackDistribution, XpkgDescriptorNamesTheCommandAndTheArchiveTop) { @@ -635,7 +683,7 @@ TEST(PackDistribution, XpkgDescriptorNamesTheCommandAndTheArchiveTop) { ASSERT_TRUE(u) << u.error(); EXPECT_NE(u->find("name = \"mcppls\""), std::string::npos) << *u; EXPECT_NE(u->find("xvm.add(\"mcppls\", opts)"), std::string::npos) << *u; - EXPECT_NE(u->find("opts.alias = \"mcppls.cmd\""), std::string::npos) << *u; + EXPECT_NE(u->find("opts.alias = \"mcppls.exe\""), std::string::npos) << *u; EXPECT_NE(u->find("for _, ext in ipairs({ \".zip\" }) do"), std::string::npos) << *u; EXPECT_NE(u->find("local top = pkginfo.install_file()"), std::string::npos) << *u; EXPECT_NE(u->find("os.mv(top, pkginfo.install_dir())"), std::string::npos) << *u; From 8bea56de970ad7e784fc22e4896dd26957f3ecff Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 03:35:07 +0900 Subject: [PATCH 04/20] tests: the launchers' unit tests read the Windows launcher program, and the POSIX script's hand-off to it Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- tests/unit/test_pack_distribution.cpp | 45 +++++++++++++-------------- 1 file changed, 21 insertions(+), 24 deletions(-) diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index d5dfde009..6806411ae 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -175,7 +175,6 @@ TEST(PackDistribution, LaunchersPickTheArchitectureAndTheRunner) { l.runner = { "okrun", "{}" }; l.entry = { { "x86_64", "hello.oke" }, { "aarch64", "hello.oke" } }; l.loader = "okrun"; - ASSERT_TRUE(d::launchable(l)); const auto sh = d::posix_launcher(l); EXPECT_TRUE(sh.starts_with("#!/bin/sh\n")); EXPECT_NE(sh.find("x86_64|amd64|AMD64) arch=x86_64"), std::string::npos); @@ -186,10 +185,8 @@ TEST(PackDistribution, LaunchersPickTheArchitectureAndTheRunner) { // The package is found through a link to the launcher, without readlink -f. EXPECT_NE(sh.find("readlink -- \"$self\""), std::string::npos); EXPECT_EQ(sh.find("readlink -f"), std::string::npos); - const auto cmd = d::windows_launcher(l); - EXPECT_NE(cmd.find("PROCESSOR_ARCHITEW6432"), std::string::npos); - EXPECT_NE(cmd.find("libexec\\okrun\\windows-%arch%\\okrun.exe"), std::string::npos); - EXPECT_NE(cmd.find("\"%runner%\" \"%image%\" %*"), std::string::npos); + // A POSIX shell on Windows hands over to the program of the same name. + EXPECT_NE(sh.find("MINGW*|MSYS*|CYGWIN*) exec \"$top/bin/\"'hello'.exe \"$@\""), std::string::npos) << sh; } // The runner's words expand as `mcpp run` expands them: a word's `{}` is the @@ -201,32 +198,32 @@ TEST(PackDistribution, LaunchersExpandTheRunnerAsMcppRunDoes) { l.runner = { "okrun", "--image={}", "it's" }; EXPECT_NE(d::posix_launcher(l).find("exec \"$runner\" '--image='\"$image\" 'it'\\''s' \"$@\""), std::string::npos) << d::posix_launcher(l); - EXPECT_NE(d::windows_launcher(l).find("\"%runner%\" \"--image=%image%\" \"it's\" %*"), std::string::npos) - << d::windows_launcher(l); l.runner = { "okrun", "--quiet" }; EXPECT_NE(d::posix_launcher(l).find("exec \"$runner\" '--quiet' \"$image\" \"$@\""), std::string::npos) << d::posix_launcher(l); - EXPECT_NE(d::windows_launcher(l).find("\"%runner%\" \"--quiet\" \"%image%\" %*"), std::string::npos) - << d::windows_launcher(l); } -// A .cmd doubles `%` in what it states literally, and a word it cannot state -// (a double quote) is refused: the .cmd written for it runs nothing. -TEST(PackDistribution, AWindowsLauncherEscapesWhatItStates) { +// The Windows launcher is a program (design 2026-10-29 §4.3): its source states +// the launch as constants, finds the package from its own path, reads the +// machine's architecture, and expands the runner's words as `mcpp run` does. +// Any word is stated, quotes and percent signs included: a raw string holds it. +TEST(PackDistribution, TheWindowsLauncherIsAProgram) { d::Launch l; l.name = "hello"; - l.entry = { { "x86_64", "hello.oke" } }; - l.runner = { "ok%run", "--load=50%", "{}" }; - const auto cmd = d::windows_launcher(l); - EXPECT_NE(cmd.find("set \"runner=ok%%run\""), std::string::npos) << cmd; - EXPECT_NE(cmd.find("\"%runner%\" \"--load=50%%\" \"%image%\" %*"), std::string::npos) << cmd; - l.runner = { "okrun", "--say=\"hi\"", "{}" }; - auto ok = d::launchable(l); - ASSERT_FALSE(ok); - EXPECT_NE(ok.error().find("double quote"), std::string::npos) << ok.error(); - const auto refused = d::windows_launcher(l); - EXPECT_EQ(refused.find("%runner%"), std::string::npos) << refused; - EXPECT_NE(refused.find("exit /b 126"), std::string::npos) << refused; + l.entry = { { "x86_64", "hello.oke" }, { "aarch64", "hello.oke" } }; + l.runner = { "okrun", "--say=\"50%\"", "{}" }; + l.loader = "okrun"; + const auto src = d::exe_launcher_main(l); + EXPECT_NE(src.find("import mcpplibs.os;"), std::string::npos) << src; + EXPECT_NE(src.find("os::env::image()"), std::string::npos) << src; + EXPECT_NE(src.find("PROCESSOR_ARCHITEW6432"), std::string::npos) << src; + EXPECT_NE(src.find("R\"mcpp(--say=\"50%\")mcpp\""), std::string::npos) << src; + EXPECT_NE(src.find("R\"mcpp(aarch64)mcpp\""), std::string::npos) << src; + EXPECT_NE(src.find("(\"windows-\" + arch)"), std::string::npos) << src; + EXPECT_NE(src.find("os::process::exec"), std::string::npos) << src; + const auto manifest = d::exe_launcher_manifest("hello", "1.0.0"); + EXPECT_NE(manifest.find("os = \"0.2.1\""), std::string::npos) << manifest; + EXPECT_NE(manifest.find("[targets.hello]"), std::string::npos) << manifest; } // The POSIX launcher, run through a link to it from another directory, finds From 87ab15328269b024c57a402333155edbc8baab61 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 03:06:59 +0900 Subject: [PATCH 05/20] SPEC-015: target sets and legs -- a set expands to the table's rows of a system, the legs of one command are planned serially and built at once, run takes the first reachable leg, test runs every reachable leg and fails when none ran, tests are archived once and run elsewhere - target sets (`openkal`, `linux-musl`, `none-eabihf`) are recognised by their first segment and expanded in the CLI against the target table; members must select one platform; a member the platform does not serve is left out, stated - mcpp::build::legs: plan (serial, mcpp.lock written by one planner), build and run_by_directory (concurrent across build directories, static job share, one Finished); the Android, universal, library and platform-test leg loops use it - build/run/test/self env take several --target; [pack] targets is parsed; [build] target refuses a set - mcpp test --no-run --archive / --archive - machine output: one leg unchanged; several legs add data.legs, the legs' status, triple on artifact/diagnostic events, legs in .last-result.json - unit tests (target sets, legs, manifest); e2e 963, 964 Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- modules/manifest/src/toml.cppm | 30 +- modules/manifest/src/types.cppm | 4 + modules/toolchain-model/src/triple.cppm | 215 +++++ src/build/execute.cppm | 749 +++++++++++++++- src/build/prepare.cppm | 92 ++ src/build/prepare/legs.cpp | 105 +++ src/build/prepare/platform.cpp | 31 + src/build/result.cppm | 49 ++ src/cli.cppm | 22 +- src/cli/cmd_build.cppm | 816 ++++++++++++------ src/pack/library_pipeline.cppm | 87 +- src/pack/pipeline.cppm | 71 +- ...ery_member_and_runs_where_it_is_reached.sh | 119 +++ ...64_tests_built_once_run_from_an_archive.sh | 92 ++ tests/unit/test_build_legs.cpp | 50 ++ tests/unit/test_manifest.cpp | 55 ++ tests/unit/test_target_set.cpp | 170 ++++ 17 files changed, 2397 insertions(+), 360 deletions(-) create mode 100644 src/build/prepare/legs.cpp create mode 100755 tests/e2e/963_a_target_set_builds_every_member_and_runs_where_it_is_reached.sh create mode 100755 tests/e2e/964_tests_built_once_run_from_an_archive.sh create mode 100644 tests/unit/test_build_legs.cpp create mode 100644 tests/unit/test_target_set.cpp diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index ea56facd2..0dc6a1ad9 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -12,6 +12,7 @@ import std; import mcpp.invocation; import mcpp.source_kind; import mcpp.libs.toml; +import mcpp.toolchain.triple; import mcpp.pm.dep_spec; import mcpp.version_req; import mcpp.pm.dependency_selector; @@ -2950,7 +2951,17 @@ std::expected parse_document(const t::Document& documen m.buildConfig.buildProgramTimeoutSecs = static_cast(secs); } if (auto v = doc->get_string("build.c_standard")) m.buildConfig.cStandard = *v; - if (auto v = doc->get_string("build.target")) m.buildConfig.target = *v; + if (auto v = doc->get_string("build.target")) { + // One target: `build`, `run` and `test` act on it by default, and a + // default that built every architecture would charge each build for + // all of them (SPEC-015 §3.3). + if (mcpp::toolchain::target_set::is_set_spelling(*v)) + return std::unexpected(error(origin, std::format( + "[build].target = '{}' names a target set; [build] target names one target " + "(`x86_64-openkal`). A set is written in [pack] targets, or as `--target {}`", + *v, *v))); + m.buildConfig.target = *v; + } // `jobs` accepts a number or "auto"; both arrive as text and are validated // where they are used, so a bad value warns at build time instead of making // the whole manifest unloadable. (A published package carrying an unknown @@ -3826,6 +3837,23 @@ std::expected parse_document(const t::Document& documen m.packConfig.sign = *v; } if (auto v = doc->get_bool("pack.universal")) m.packConfig.universal = *v; + // `[pack] targets` (SPEC-015 §3.3): a string or a list; each a target or a + // target set. A set's members are known only with the platform the + // command selects, so a set is checked for its form here and expanded + // where the command reads it. + { + std::vector targets; + if (auto v = doc->get_string("pack.targets")) targets.push_back(*v); + else if (auto a = doc->get_string_array("pack.targets")) targets = *a; + for (auto const& t : targets) { + if (mcpp::toolchain::target_set::is_set_spelling(t) || mcpp::toolchain::triple::parse(t)) + continue; + return std::unexpected(error(origin, std::format( + "[pack].targets: '{}' is neither a target (`x86_64-linux-musl`) nor a target set " + "(`openkal`, `linux-musl`)", t))); + } + m.packConfig.targets = std::move(targets); + } // [target.] — per-target overrides. We accept both GCC // (x86_64-linux-musl) and Rust-style (x86_64-unknown-linux-musl) diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 79c5e9561..f2f523df7 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1880,6 +1880,10 @@ struct PackConfig { // `false`: a pack of a platform's target packs that target alone, where // the platform states `universal = true` (SPEC-014 §3). std::optional universal; + // The targets `mcpp pack` acts on when the command names none (SPEC-015 + // §3.3): targets and target sets, as `--target` takes them (`openkal`, + // `x86_64-linux-musl`). Empty: unstated, the build's target. + std::vector targets; }; // `[workspace]` — multi-package workspace support (0.0.11+). diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 8457fac74..da981d5a9 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -1584,3 +1584,218 @@ std::optional did_you_mean(std::string_view input) { } } // namespace mcpp::toolchain::triple + +// ── target sets (SPEC-015) ────────────────────────────────────────────────── +// +// A `--target` value is either a target or a set of targets. A target is a +// triple and means what it always meant. A SET is written by leaving the +// architecture out: `openkal`, `linux-musl`, `none-eabihf`. It names every row +// of the target table whose system and environment are the ones written, and +// a command over a set acts once per member, each member being one LEG. +// +// THE SET IS RECOGNISED BY ITS FIRST SEGMENT, NOT BY A FAILED PARSE. The +// triple parser does not validate the architecture: `unknown-linux-gnu` parses +// as a triple whose architecture is `unknown`, and the four-segment GNU forms +// put a vendor first. A spelling is a set exactly when its first segment is +// one of the system words below; everything else is a triple, refused or +// accepted where triples always were. +// +// THE ENVIRONMENT IS FILLED AS A WRITTEN TRIPLE FILLS IT. `linux` is +// `linux-gnu` and `windows` is `windows-gnu`, the fills docs/21 states for +// `-linux` and `-windows`; a system without an environment segment +// (`macos`, `openkal`, `emscripten`, `ios`) has nothing to fill. Bare metal has +// no default environment, so `none` alone names no set and the refusal lists +// the environments that do. +// +// WHICH PLATFORM SERVES THE MEMBERS IS ASKED, NOT ASSUMED. Each candidate is +// put to the platform selection the build uses (the flag, the environment, +// the project, this machine's configuration, the target table), through the +// callback the caller passes. A set whose members select different platforms +// is refused, naming the split: one command acting on two platforms at once +// is not what a set was written to mean. Whether the selected platform serves +// every member is a fact of the platform package, which is read when a leg is +// planned; a member it does not serve is left out there, and the leaving out +// is reported. + +export namespace mcpp::toolchain::target_set { + +// The words a set begins with. `android` is the one word that is an +// environment rather than a system (`-linux-android`); it is a word here +// because a user names the platform by it. +inline constexpr std::string_view kSystemWords[] = { + "linux", "macos", "windows", "openkal", "emscripten", "ios", "android", "none", +}; + +// Whether `spelling` names a set: its first segment is a system word. +bool is_set_spelling(std::string_view spelling); + +// The platform a candidate selects, as one comparable string ("" = the +// engine's own conventions), or the selection's own error. +using PlatformOf = + std::function(const mcpp::toolchain::triple::Triple&)>; + +struct Leg { + std::string triple; // canonical for a set's member; as written for a target + std::string fromSet; // the set spelling it came from; empty for a target +}; + +struct Expansion { + std::vector members; // canonical triples, in target-table order + std::string platform; // the platform every member selected +}; + +// The members of one set spelling. +std::expected expand_set(std::string_view spelling, + const PlatformOf& platformOf); + +// Every `--target` of one command: sets expanded, targets kept as written, the +// union in target-table order (targets the table does not hold keep the order +// they were written in, after the table's), a target named twice once. +std::expected, std::string> resolve(std::span requests, + const PlatformOf& platformOf); + +} // namespace mcpp::toolchain::target_set + +// ── implementation ────────────────────────────────────────────────────────── + +namespace mcpp::toolchain::target_set { + +namespace tt = mcpp::toolchain::triple; + +namespace { + +std::string_view first_segment(std::string_view s) { + auto d = s.find('-'); + return d == std::string_view::npos ? s : s.substr(0, d); +} + +// The table row index of a canonical triple; the table's size when absent. +std::size_t table_rank(std::string_view canonical) { + auto rows = tt::known_targets(); + for (std::size_t i = 0; i < rows.size(); ++i) + if (rows[i].canonical == canonical) return i; + return rows.size(); +} + +struct Wanted { + std::string os; + std::string env; +}; + +std::expected wanted_of(std::string_view spelling) { + std::vector seg; + for (std::size_t b = 0; b <= spelling.size();) { + auto d = spelling.find('-', b); + seg.push_back(spelling.substr(b, d == std::string_view::npos ? std::string_view::npos : d - b)); + if (d == std::string_view::npos) break; + b = d + 1; + } + const auto word = seg.front(); + if (seg.size() > 2 || (seg.size() == 2 && seg[1].empty())) + return std::unexpected(std::format( + "'{}' is not a target set: a set is a system and at most one environment " + "(`openkal`, `linux-musl`, `none-eabihf`)", spelling)); + const std::string written = seg.size() == 2 ? std::string(seg[1]) : std::string{}; + if (word == "android") { + if (!written.empty()) + return std::unexpected(std::format( + "'{}' is not a target set: `android` is the environment of `-linux-android` " + "and takes no further segment", spelling)); + return Wanted{"linux", "android"}; + } + if (word == "linux" || word == "windows") + return Wanted{std::string(word), written.empty() ? std::string("gnu") : written}; + if (word == "none") { + if (written.empty()) + return std::unexpected( + "`none` names no target set: bare metal has no default environment.\n" + " use: none-elf | none-eabi | none-eabihf"); + return Wanted{"none", written}; + } + if (word == "ios") return Wanted{"ios", written}; + if (!written.empty()) + return std::unexpected(std::format( + "'{}' is not a target set: `{}` has no environment segment", spelling, word)); + return Wanted{std::string(word), {}}; +} + +} // namespace + +bool is_set_spelling(std::string_view spelling) { + const auto w = first_segment(spelling); + return std::ranges::find(kSystemWords, w) != std::end(kSystemWords); +} + +std::expected expand_set(std::string_view spelling, + const PlatformOf& platformOf) { + auto want = wanted_of(spelling); + if (!want) return std::unexpected(want.error()); + + Expansion out; + std::vector> chosen; // member, platform + std::vector planned; + for (auto const& row : tt::known_targets()) { + auto t = tt::parse(row.canonical); + if (!t || t->os != want->os || t->env != want->env) continue; + if (row.tier == "planned") { planned.push_back(row.canonical); continue; } + auto p = platformOf ? platformOf(*t) : std::expected{}; + if (!p) return std::unexpected(std::format("{}: {}", row.canonical, p.error())); + chosen.emplace_back(std::string(row.canonical), *p); + } + if (chosen.empty()) { + std::string rows; + for (auto r : planned) rows += std::format("{}{}", rows.empty() ? "" : ", ", r); + return std::unexpected(std::format( + "the target set '{}' names no target this mcpp builds{}.\n" + " `mcpp toolchain list` lists the targets", spelling, + rows.empty() ? std::string{} : std::format(" (planned, not yet built: {})", rows))); + } + for (auto const& [member, platform] : chosen) { + if (platform == chosen.front().second) continue; + std::string split; + for (auto const& [m, p] : chosen) + split += std::format("\n {} {}", m, p.empty() ? "(the engine's conventions)" : p); + return std::unexpected(std::format( + "the members of the target set '{}' select different platforms:{}\n" + " a set acts on one platform; name the targets one by one, or select one " + "platform for all of them ([target.] platform, --platform)", + spelling, split)); + } + out.platform = chosen.front().second; + for (auto& [member, platform] : chosen) out.members.push_back(std::move(member)); + return out; +} + +std::expected, std::string> resolve(std::span requests, + const PlatformOf& platformOf) { + struct Ranked { std::size_t rank; std::size_t order; Leg leg; std::string key; }; + std::vector all; + std::set seen; + auto add = [&](Leg leg, std::string key) { + if (!seen.insert(key).second) return; + all.push_back({table_rank(key), all.size(), std::move(leg), std::move(key)}); + }; + for (auto const& r : requests) { + if (is_set_spelling(r)) { + auto e = expand_set(r, platformOf); + if (!e) return std::unexpected(e.error()); + for (auto& m : e->members) { + std::string key = m; + add(Leg{std::move(m), r}, std::move(key)); + } + continue; + } + // A target keeps its spelling: the leg is planned as the command + // would have planned it alone, completion and report included. + auto t = tt::parse(r); + add(Leg{r, {}}, t ? t->str() : r); + } + std::ranges::stable_sort(all, [](Ranked const& a, Ranked const& b) { + return a.rank != b.rank ? a.rank < b.rank : a.order < b.order; + }); + std::vector legs; + for (auto& r : all) legs.push_back(std::move(r.leg)); + return legs; +} + +} // namespace mcpp::toolchain::target_set diff --git a/src/build/execute.cppm b/src/build/execute.cppm index b9e67d880..7187a1e78 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -16,10 +16,13 @@ import mcpp.build.compile_commands; // C1: the fast path restores a deleted root import mcpp.build.prepare; import mcpp.pack; // #622 A10: mcpp::pack::Options / Format import mcpp.pack.pipeline; // #622 A10: build_and_pack, for `run --format` +import mcpp.pack.compress; // the test archive's tar (SPEC-015 §4) +import mcpp.pack.distribution; // its writer and reader import mcpp.build.test_targets; import mcpp.diag; import mcpp.build.plan; import mcpp.toolchain.triple; +import mcpp.build.refusal; // a set member its platform does not serve import mcpp.freestanding.runner; import mcpp.platform; import mcpp.libs.json; // the plan event's platform objects @@ -64,6 +67,204 @@ import mcpp.ui; import mcpp.toolchain.msvc_coroutines; import mcpp.build.progress; +// THE TEST ARCHIVE — tests built once and run elsewhere (SPEC-015 §4). +// +// `mcpp test --no-run --archive ` builds the tests of every leg and +// writes them as one archive; `mcpp test --archive `, in a checkout of +// the same project on another machine, runs the tests that machine reaches +// without building anything. An ok image is the same bytes on every system +// (openkal), and a native cross build is the same bytes as a build on the +// target's own system, so a CI that builds once on one machine and runs on the +// others measures the bytes it ships rather than a rebuild of them. +// +// THE ARCHIVE is the engine's own tar (written by the packer's writer, the same +// bytes on every host), with one top-level directory holding: +// +// mcpp-tests.json what follows, and how to run it +// legs//tests// each test program, and beside it the +// files it loads from its own directory +// (the runtime files `mcpp run` hands a +// runner: shared libraries, deployed +// files) +// legs//programs/ the package's own programs, which a +// test finds as MCPP_BIN_ (L5) +// +// `mcpp-tests.json` (schema 1): `package`, `cwd` (the directory the tests run +// in, relative to the project root: the package's own), and `legs`, each with +// its `triple`, `programs` (environment name to path) and `tests` (`name`, +// `program`, the `files` beside it). Paths are relative to the archive's +// directory and use `/`. +export namespace mcpp::build::test_archive { + +inline constexpr int kSchema = 1; +inline constexpr std::string_view kManifestName = "mcpp-tests.json"; + +struct Test { + std::string name; + std::string program; // relative to the archive's directory + std::vector files; // beside it, relative to the archive's directory +}; + +struct Leg { + std::string triple; + std::vector> programs; // MCPP_BIN_ -> path + std::vector tests; +}; + +struct Manifest { + std::string package; + std::string cwd; // relative to the project root; "" is the root + std::vector legs; +}; + +// A file the archive carries: where it is now, and where it goes. +struct Copy { + std::filesystem::path source; + std::string dest; // relative to the archive's directory +}; + +// Where each test of a leg goes, and the program a test finds. +std::string test_dest(std::string_view triple, std::string_view test, std::string_view file); +std::string program_dest(std::string_view triple, std::string_view file); + +// Copies `copies` into a fresh tree under `stage`, writes the manifest beside +// them and the tree as `archive` (gzip-compressed tar). +std::expected write(const Manifest& m, const std::vector& copies, + const std::filesystem::path& stage, + const std::filesystem::path& archive); + +struct Opened { + std::filesystem::path root; // the archive's directory, extracted + Manifest manifest; +}; + +// Extracts `archive` into `into` (new or emptied) and reads its manifest. +std::expected open(const std::filesystem::path& archive, + const std::filesystem::path& into); + +} // namespace mcpp::build::test_archive + +// ── implementation ────────────────────────────────────────────────────────── + +namespace mcpp::build::test_archive { + +namespace fs = std::filesystem; + +std::string test_dest(std::string_view triple, std::string_view test, std::string_view file) { + return std::format("legs/{}/tests/{}/{}", triple, test, file); +} + +std::string program_dest(std::string_view triple, std::string_view file) { + return std::format("legs/{}/programs/{}", triple, file); +} + +namespace { + +nlohmann::json to_json(const Manifest& m) { + nlohmann::json legs = nlohmann::json::array(); + for (auto const& l : m.legs) { + nlohmann::json programs = nlohmann::json::object(); + for (auto const& [env, path] : l.programs) programs[env] = path; + nlohmann::json tests = nlohmann::json::array(); + for (auto const& t : l.tests) + tests.push_back({{"name", t.name}, {"program", t.program}, {"files", t.files}}); + legs.push_back({{"triple", l.triple}, {"programs", std::move(programs)}, + {"tests", std::move(tests)}}); + } + return {{"schema", kSchema}, {"mcpp", std::string(mcpp::MCPP_VERSION)}, + {"package", m.package}, {"cwd", m.cwd}, {"legs", std::move(legs)}}; +} + +std::expected from_json(const nlohmann::json& d) { + if (!d.is_object() || d.value("schema", 0) != kSchema) + return std::unexpected(std::format("{} is not a schema-{} test archive manifest", + kManifestName, kSchema)); + Manifest m; + m.package = d.value("package", ""); + m.cwd = d.value("cwd", ""); + if (!d.contains("legs") || !d["legs"].is_array()) + return std::unexpected(std::format("{} names no legs", kManifestName)); + for (auto const& jl : d["legs"]) { + Leg l; + l.triple = jl.value("triple", ""); + if (jl.contains("programs") && jl["programs"].is_object()) + for (auto it = jl["programs"].begin(); it != jl["programs"].end(); ++it) + l.programs.emplace_back(it.key(), it.value().get()); + if (jl.contains("tests") && jl["tests"].is_array()) + for (auto const& jt : jl["tests"]) { + Test t{jt.value("name", ""), jt.value("program", ""), {}}; + if (jt.contains("files") && jt["files"].is_array()) + for (auto const& f : jt["files"]) t.files.push_back(f.get()); + l.tests.push_back(std::move(t)); + } + m.legs.push_back(std::move(l)); + } + return m; +} + +// A destination stays inside the archive's directory. +bool contained(std::string_view dest) { + if (dest.empty() || dest.front() == '/' || dest.find(':') != std::string_view::npos) return false; + for (auto const& part : fs::path(dest)) if (part == "..") return false; + return true; +} + +} // namespace + +std::expected write(const Manifest& m, const std::vector& copies, + const fs::path& stage, const fs::path& archive) { + std::error_code ec; + fs::remove_all(stage, ec); + auto stem = archive.filename().string(); + for (std::string_view ext : {".tar.gz", ".tgz", ".tar"}) + if (stem.ends_with(ext)) { stem.resize(stem.size() - ext.size()); break; } + if (stem.empty()) stem = "mcpp-tests"; + const auto root = stage / stem; + fs::create_directories(root, ec); + if (ec) return std::unexpected(std::format("cannot create {}: {}", root.string(), ec.message())); + for (auto const& c : copies) { + if (!contained(c.dest)) + return std::unexpected(std::format("'{}' leaves the test archive", c.dest)); + const auto to = root / c.dest; + fs::create_directories(to.parent_path(), ec); + fs::copy_file(c.source, to, fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(std::format("cannot copy {} into the test archive: {}", + c.source.string(), ec.message())); + } + { + std::ofstream out(root / std::string(kManifestName), std::ios::binary | std::ios::trunc); + out << to_json(m).dump(2) << '\n'; + if (!out) return std::unexpected(std::format("cannot write {}", kManifestName)); + } + if (archive.has_parent_path()) fs::create_directories(archive.parent_path(), ec); + auto choice = mcpp::pack::compress::parse("gzip"); + if (!choice) return std::unexpected(choice.error()); + return mcpp::pack::distribution::write_tar(root, archive, *choice); +} + +std::expected open(const fs::path& archive, const fs::path& into) { + std::error_code ec; + if (!fs::is_regular_file(archive, ec)) + return std::unexpected(std::format("no test archive at {}", archive.string())); + fs::remove_all(into, ec); + fs::create_directories(into, ec); + auto root = mcpp::pack::distribution::unpack(archive, into); + if (!root) return std::unexpected(root.error()); + std::ifstream in(*root / std::string(kManifestName), std::ios::binary); + auto doc = nlohmann::json::parse(in, nullptr, false); + if (doc.is_discarded()) + return std::unexpected(std::format("{} has no readable {}", archive.string(), kManifestName)); + auto m = from_json(doc); + if (!m) return std::unexpected(m.error()); + for (auto const& l : m->legs) + for (auto const& t : l.tests) + if (!contained(t.program)) + return std::unexpected(std::format("'{}' leaves the test archive", t.program)); + return Opened{*root, std::move(*m)}; +} + +} // namespace mcpp::build::test_archive + namespace mcpp::build { // ─── The exit status of a run ──────────────────────────────────────── @@ -1455,8 +1656,13 @@ void finish_record(mcpp::build::result::Record& r, bool ok, std::string_view fai res::write(r); if (res::events_enabled()) { auto json = res::to_json(r); - for (auto const& d : json["diagnostics"]) res::emit("diagnostic", {{"diagnostic", d}}); - for (auto const& a : json["artifacts"]) res::emit("artifact", {{"artifact", a}}); + // Several legs at once: each event names its leg (SPEC-015 §3.5). + auto with_leg = [&](nlohmann::json fields) { + if (res::legs()) fields["triple"] = r.triple; + return fields; + }; + for (auto const& d : json["diagnostics"]) res::emit("diagnostic", with_leg({{"diagnostic", d}})); + for (auto const& a : json["artifacts"]) res::emit("artifact", with_leg({{"artifact", a}})); } } @@ -3071,6 +3277,56 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, // coherent "run"). Threaded straight to prepare_build's BuildOverrides, // which already does the member switch (basename OR member path — the same // rule mcpp::project::resolve_member_dir documents for build/test). +// THE LEG `mcpp run` RUNS (SPEC-015 §3.2). One program runs, so of several +// legs one is chosen: the first this machine reaches -- by the rule `mcpp +// run` refuses an unreachable target with (`unreachable_execution`) -- +// trying this machine's architecture first. Each leg is planned to be asked, +// and only the chosen one is kept and built. A member of a set its platform +// does not serve is passed over, as the build of the set leaves it out. +struct ChosenLeg { + std::string triple; + BuildContext ctx; +}; +std::optional choose_run_leg(std::span legs, + const BuildOverrides& base, bool noRunner) { + std::vector order; + for (int pass = 0; pass < 2; ++pass) + for (auto const& l : legs) { + auto t = mcpp::toolchain::triple::parse(l.triple); + const bool native = t && t->arch == mcpp::toolchain::triple::host_triple().arch; + if (native == (pass == 0)) order.push_back(&l); + } + std::vector passed; + for (auto const* leg : order) { + BuildOverrides o = base; + o.target_triple = leg->triple; + o.will_run = true; + (void)refusal::take(); + auto ctx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, {}, o); + if (!ctx) { + if (!leg->fromSet.empty() && refusal::take() == refusal::Code::PlatformTargetUnserved) { + passed.push_back(std::format("{}: its platform does not serve it", leg->triple)); + continue; + } + mcpp::ui::error(std::format("the {} leg: {}", leg->triple, ctx.error())); + return std::nullopt; + } + const auto key = choose_runner(*ctx, noRunner).tripleKey; + if (auto why = unreachable_execution(*ctx, key, noRunner)) { + passed.push_back(std::format("{}: {}", leg->triple, why->substr(0, why->find('\n')))); + continue; + } + for (auto const& p : passed) mcpp::ui::note(std::format("not run here -- {}", p)); + mcpp::ui::info("Leg", std::format("{} (of {} targets)", leg->triple, legs.size())); + return ChosenLeg{leg->triple, std::move(*ctx)}; + } + std::string why; + for (auto const& p : passed) why += "\n " + p; + mcpp::ui::error(std::format("no leg of this run is reached on this {} machine:{}", + mcpp::toolchain::platform::host_arch, why)); + return std::nullopt; +} + export int build_run_target(const std::optional& targetName, std::span passthrough, const std::string& package_filter = {}, @@ -3107,7 +3363,11 @@ export int build_run_target(const std::optional& targetName, // --format` — same value space, same refusal // naming what the resolved graph provides. Empty // is the ordinary run of the link output. - const std::string& format = {}) { + const std::string& format = {}, + // SEVERAL LEGS (SPEC-015 §3.2): `--target` named + // more than one target, or a set. One is chosen + // and run; `target_triple` is then ignored. + std::span legs = {}) { // --format WITH --no-runner: refused before anything is built. A // distributable (an `.apk`, an `.msi`, this record's own `blob` fixture) // is not the link output, and "run it directly, ignoring the runner" has @@ -3119,6 +3379,23 @@ export int build_run_target(const std::optional& targetName, return 2; } + std::string target = target_triple; + std::optional legCtx; + if (legs.size() > 1) { + BuildOverrides base; + base.package_filter = package_filter; + base.cache_mode = cache_mode; + base.features = features; + base.profile = profile; + base.accel = accel; + auto chosen = choose_run_leg(legs, base, no_runner); + if (!chosen) return kRunBuildFailed; + target = chosen->triple; + // A pack plans the leg again, in the same directory: its lock is not + // held across that plan. + if (format.empty()) legCtx.emplace(std::move(chosen->ctx)); + } + // mcpp#225 (E2): reuse the resolved build cache when it's still fresh, // skipping prepare_build's toolchain resolution + modgraph scan // entirely — mirrors cmd_build's try_fast_build fast path. The cached @@ -3134,7 +3411,7 @@ export int build_run_target(const std::optional& targetName, // so with the flag there is nothing for it to ignore — and it has no // manifest to print the note against. if (package_filter.empty() && cache_mode.empty() && !no_cache - && target_triple.empty() && !no_runner + && target.empty() && !no_runner // AND NEITHER NEW AXIS IS SET. The cached entry was written for // whichever feature set and profile the last build used; taking it // here would silently ignore the flag, which is the same reason @@ -3172,7 +3449,7 @@ export int build_run_target(const std::optional& targetName, // the request. if (!format.empty()) { mcpp::pack::Options popts; - popts.targetTriple = target_triple; + popts.targetTriple = target; popts.profile = profile; popts.format = mcpp::pack::Format::Dispatched; popts.formatName = format; @@ -3202,7 +3479,7 @@ export int build_run_target(const std::optional& targetName, mcpp::build::BuildOverrides ov2; ov2.package_filter = package_filter; ov2.cache_mode = cache_mode; - ov2.target_triple = target_triple; + ov2.target_triple = target; ov2.features = features; ov2.profile = profile; ov2.accel = accel; @@ -3210,7 +3487,7 @@ export int build_run_target(const std::optional& targetName, auto ctx2 = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, /*extraTargets=*/{}, ov2); if (!ctx2) { mcpp::ui::error(std::format("{}", ctx2.error())); return kRunBuildFailed; } - if (run_build_plan(*ctx2, /*verbose=*/false, no_cache, target_triple) != 0) + if (run_build_plan(*ctx2, /*verbose=*/false, no_cache, target) != 0) return kRunBuildFailed; // ONE DISTRIBUTABLE, OR A SENTENCE. The pack pipeline reports the // terminal artifacts of the request (outputs no other introduced @@ -3253,7 +3530,7 @@ export int build_run_target(const std::optional& targetName, mcpp::build::BuildOverrides ov; ov.package_filter = package_filter; ov.cache_mode = cache_mode; - ov.target_triple = target_triple; + ov.target_triple = target; ov.features = features; ov.profile = profile; ov.accel = accel; @@ -3261,8 +3538,10 @@ export int build_run_target(const std::optional& targetName, // part of what has to exist. `mcpp build` does not set it, which is the // whole of the difference the tier buys. ov.will_run = true; - auto ctx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, - /*extraTargets=*/{}, ov); + // The chosen leg was planned to be chosen; it is not planned twice. + auto ctx = legCtx ? std::expected(std::move(*legCtx)) + : prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, + /*extraTargets=*/{}, ov); if (!ctx) { mcpp::ui::error(std::format("{}", ctx.error())); return kRunBuildFailed; } // `target_triple` IS PASSED, AND OMITTING IT WROTE A CROSS BUILD INTO // THE HOST'S CACHE SLOT. @@ -3282,7 +3561,7 @@ export int build_run_target(const std::optional& targetName, // The comment on `try_fast_run` records the same defect reached through the // MANIFEST's default target, and guards that door alone. This is the other // door: the flag. Measured on 2026.8.24.3, from a clean `target/`. - if (run_build_plan(*ctx, /*verbose=*/false, no_cache, target_triple) != 0) + if (run_build_plan(*ctx, /*verbose=*/false, no_cache, target) != 0) return kRunBuildFailed; // The program run is the selected member's (workspace design 2026-09-29 // §15), with its closure's runtime. @@ -3385,6 +3664,10 @@ export struct TestOptions { // property of the project, so the project says it; mcpp only has to make // saying it possible, which is what was missing. int buildTimeoutSecs = 0; + // `--archive ` (SPEC-015 §4). With `--no-run`: the tests built are + // written to it, with what each needs beside it. Without: nothing is + // built, and the tests it holds run here. + std::filesystem::path archive; }; // What one member's `run_tests` actually did. `--workspace` fans out over @@ -4322,6 +4605,206 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, return rc; } +// ─── the test archive (SPEC-015 §4) ───────────────────────────────────────── + +// One leg's built tests, for the archive. +struct ArchiveLeg { + std::string triple; + const BuildContext* ctx = nullptr; + const std::vector>* programEnv = nullptr; +}; + +// THE TESTS OF THE LEGS, WRITTEN AS ONE ARCHIVE (`mcpp test --no-run +// --archive`). Every test program that built, with the files it loads from its +// own directory -- the runtime files `mcpp run` hands a runner -- beside it, and +// the package's programs a test finds as MCPP_BIN_. The directory the +// tests run in is the one this command was run in, relative to the project. +int write_test_archive(const TestOptions& o, std::span legs) { + namespace ta = mcpp::build::test_archive; + namespace fs = std::filesystem; + if (legs.empty()) return 0; + auto const& first = *legs.front().ctx; + ta::Manifest m; + m.package = first.manifest.package.name; + std::error_code ec; + auto cwd = fs::current_path(ec).lexically_normal(); + auto rel = cwd.lexically_relative(first.projectRoot.lexically_normal()); + if (!rel.empty() && *rel.begin() != ".." && rel != ".") m.cwd = rel.generic_string(); + std::vector copies; + std::size_t count = 0; + for (auto const& l : legs) { + ta::Leg leg; + leg.triple = l.triple; + for (auto const& lu : l.ctx->plan.linkUnits) { + if (lu.kind != LinkUnit::TestBinary) continue; + if (!o.filter.empty() && lu.targetName.find(o.filter) == std::string::npos) continue; + const auto exe = l.ctx->outputDir / lu.output; + if (!fs::is_regular_file(exe, ec)) continue; + ta::Test t{lu.targetName, ta::test_dest(l.triple, lu.targetName, exe.filename().string()), {}}; + copies.push_back({exe, t.program}); + for (auto const& [dest, source] : runtime_files_for(*l.ctx, exe)) { + // A file the program reads above its own directory goes beside + // it under its own name: the archive keeps one directory per test. + const bool above = dest.starts_with("../"); + const auto where = above ? fs::path(dest).filename().generic_string() : dest; + t.files.push_back(ta::test_dest(l.triple, lu.targetName, where)); + copies.push_back({source, t.files.back()}); + } + leg.tests.push_back(std::move(t)); + ++count; + } + if (l.programEnv) + for (auto const& [env, path] : *l.programEnv) { + auto dest = ta::program_dest(l.triple, fs::path(path).filename().string()); + copies.push_back({path, dest}); + leg.programs.emplace_back(env, dest); + } + m.legs.push_back(std::move(leg)); + } + const auto archive = fs::absolute(o.archive, ec).lexically_normal(); + const auto stage = first.projectRoot / "target" / ".test-archive" / "stage"; + if (auto w = ta::write(m, copies, stage, archive); !w) { + mcpp::ui::error(std::format("--archive {}: {}", o.archive.string(), w.error())); + return 1; + } + fs::remove_all(stage, ec); + mcpp::ui::info("Archived", std::format("{} test(s) of {} leg(s) -> {}", count, legs.size(), + archive.generic_string())); + return 0; +} + +// THE TESTS OF AN ARCHIVE, RUN HERE (`mcpp test --archive`). Nothing is built: +// each leg's configuration is prepared to find how this machine starts its +// programs (the runner, the execution host) by the rule `mcpp test` uses, and +// the tests of a leg this machine does not reach are reported not run, with the +// reason. They run in the directory the archive names, inside this checkout of +// the project. No test run on any leg is a failure. +export int run_test_archive(std::span passthrough, BuildOverrides base, + TestOptions testOpts) { + namespace ta = mcpp::build::test_archive; + namespace fs = std::filesystem; + const bool json = testOpts.format == TestMessageFormat::Json; + if (json) mcpp::ui::set_quiet(true); + auto root = mcpp::project::find_manifest_root(fs::current_path()); + if (!root) { + mcpp::ui::error("mcpp test --archive runs in a checkout of the project the archive was built from: " + "no mcpp.toml found in the current directory or any parent"); + return 2; + } + std::error_code ec; + const auto into = *root / "target" / ".test-archive" / "run"; + auto opened = ta::open(fs::absolute(testOpts.archive, ec), into); + if (!opened) { mcpp::ui::error(std::format("--archive {}: {}", testOpts.archive.string(), opened.error())); return 2; } + const auto cwd = opened->manifest.cwd.empty() ? *root : *root / opened->manifest.cwd; + int passed = 0, failed = 0, notRun = 0, notReached = 0; + std::vector reasons; + const auto deadline = std::chrono::milliseconds(static_cast(testOpts.timeoutSecs) * 1000); + for (auto const& leg : opened->manifest.legs) { + if (json) std::println("{{\"leg\":{{\"triple\":\"{}\"}}}}", test_json_escape(leg.triple)); + else mcpp::ui::info("Testing", std::format("on {} (from {})", leg.triple, testOpts.archive.string())); + auto ctx = prepare_configuration(leg.triple); + if (!ctx) { + reasons.push_back(std::format("{}: {}", leg.triple, ctx.error().substr(0, ctx.error().find('\n')))); + notRun += static_cast(leg.tests.size()); + mcpp::ui::error(std::format("the {} leg: {}", leg.triple, ctx.error())); + continue; + } + TestBuild tb; + tb.ctx.emplace(std::move(*ctx)); + auto o = testOpts; + o.unreachableIsBuilt = true; + test_resolve_runner(tb, o, json); + // A leg this machine does not reach is another machine's to run, as + // `mcpp test` reports a leg it does not reach built and not run; a + // runner that is missing is an answer not established (exit 2). + if (tb.runnerFatal || tb.builtNotRunHere || !tb.invocationNotRunReason.empty()) { + const auto why = tb.builtNotRunHere ? std::format("not reached on this {} machine", + mcpp::toolchain::platform::host_arch) + : tb.invocationNotRunReason; + reasons.push_back(std::format("{}: {}", leg.triple, why.substr(0, why.find('\n')))); + (tb.builtNotRunHere ? notReached : notRun) += static_cast(leg.tests.size()); + if (!json) mcpp::ui::note(std::format("the tests of {} are not run here: {}", leg.triple, why)); + continue; + } + std::vector> legEnv; + for (auto const& [env, path] : leg.programs) + legEnv.emplace_back(env, (opened->root / path).lexically_normal().string()); + legEnv.emplace_back("MCPP_RUN_TRIPLE", tb.runnerChoice.tripleKey); + legEnv.emplace_back("MCPP_RUN_MODE", "test"); + for (auto const& t : leg.tests) { + if (!testOpts.filter.empty() && t.name.find(testOpts.filter) == std::string::npos) continue; + const auto exe = (opened->root / t.program).lexically_normal(); + std::vector argv; + if (tb.runnerTmpl.empty()) argv.push_back(exe.string()); + else argv = mcpp::freestanding::expand(tb.runnerTmpl, exe); + for (auto const& a : passthrough) argv.push_back(a); + argv.insert(argv.begin(), tb.runUnderPrefix.begin(), tb.runUnderPrefix.end()); + auto env = legEnv; + // The files beside the program are its own: the loader searches its + // directory, and a runner reads the list it is handed. + std::array dir{exe.parent_path()}; + env.emplace_back(mcpp::runtime::launch_env::runtime_library_path_key(), + mcpp::runtime::launch_env::prepend_path_list( + mcpp::runtime::launch_env::runtime_library_path_key(), dir)); + { + const auto list = exe.parent_path() / ".mcpp-runtime-files.tsv"; + std::ofstream os(list, std::ios::binary | std::ios::trunc); + for (auto const& f : t.files) + os << fs::path(f).filename().generic_string() << '\t' + << (opened->root / f).lexically_normal().string() << '\n'; + env.emplace_back(std::string(kRuntimeFilesEnv), list.string()); + } + const auto t0 = std::chrono::steady_clock::now(); + bool timedOut = false; + int spawnErr = 0; + const auto cwdText = cwd.string(); + auto rr = mcpp::run::capture_exec_deadline(argv, env, deadline, &timedOut, cwdText, &spawnErr); + const auto ms = std::chrono::duration_cast( + std::chrono::steady_clock::now() - t0).count(); + const bool ok = spawnErr == 0 && !timedOut && rr.exit_code == 0; + if (spawnErr != 0) { + ++notRun; + reasons.push_back(std::format("{}: {}", leg.triple, + mcpp::build::runner_lookup::spawn_failed_message(argv.front(), spawnErr))); + } else if (ok) { + ++passed; + } else { + ++failed; + } + if (json) { + std::println("{{\"test\":\"{}\",\"triple\":\"{}\",\"status\":\"{}\",\"ms\":{}}}", + test_json_escape(t.name), test_json_escape(leg.triple), + spawnErr ? "not_run" : ok ? "pass" : timedOut ? "timeout" : "fail", ms); + } else { + mcpp::ui::plain(std::format("{} ... {} ({:.2f}s)", t.name, + spawnErr ? "NOT RUN" : ok ? "ok" : timedOut + ? std::format("FAIL (timeout after {}s)", testOpts.timeoutSecs) + : std::format("FAIL (exit {})", rr.exit_code), + static_cast(ms) / 1000.0)); + if (!ok && spawnErr == 0 && !rr.output.empty()) mcpp::ui::block(rr.output); + } + } + } + if (json) { + std::println("{{\"archive_summary\":{{\"passed\":{},\"failed\":{},\"not_run\":{}," + "\"not_reached\":{}}}}}", passed, failed, notRun, notReached); + std::fflush(stdout); + } + if (passed + failed == 0) { + std::string why; + for (auto const& r : reasons) why += "\n " + r; + mcpp::ui::error(std::format("no test of {} ran on this machine{}{}", testOpts.archive.string(), + why.empty() ? std::string{} : ":", why)); + return 1; + } + const auto counts = std::format("{} passed; {} failed{}{}", passed, failed, + notReached ? std::format("; {} not run here", notReached) : std::string{}, + notRun ? std::format("; {} not run", notRun) : std::string{}); + if (failed) { mcpp::ui::error(std::format("test result: FAILED. {}", counts)); return 1; } + if (!json) mcpp::ui::result("test result", std::format("ok. {}", counts)); + return notRun ? 2 : 0; +} + // `mcpp test` driver: discover tests/**/*.cpp, synthesize targets, build // with dev-deps, run each test binary, summarize. export int run_tests(std::span passthrough, @@ -4455,6 +4938,13 @@ export int run_tests(std::span passthrough, // does to `mcpp run` — `[xlings.workspace]` has no separate `test` tier // because there is no separate need. overrides.will_run = true; + // The leg an archive records: the target as its canonical name, the host's + // when none was asked for. + const std::string legTriple = [&] { + if (overrides.target_triple.empty()) return mcpp::toolchain::triple::host_triple().str(); + auto t = mcpp::toolchain::triple::parse(overrides.target_triple); + return t ? t->str() : overrides.target_triple; + }(); auto prepared = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, std::move(testTargets), @@ -4530,8 +5020,241 @@ export int run_tests(std::span passthrough, test_bulk(tb, testOpts, filter_match); test_resolve_runner(tb, testOpts, json); summary.buildMs = tb.buildMs; - return test_run_member(tb, testOpts, passthrough, json, memberName, /*owner=*/"", - member_ms(), summary); + const int rc = test_run_member(tb, testOpts, passthrough, json, memberName, /*owner=*/"", + member_ms(), summary); + if (rc != 0 || testOpts.archive.empty()) return rc; + const ArchiveLeg leg{legTriple, ctx, &tb.programEnv}; + return write_test_archive(testOpts, std::span(&leg, 1)); +} + +// `mcpp test` OVER SEVERAL LEGS (SPEC-015 §3.2): a target set, several +// `--target`, or a platform package's own targets (SPEC-013 §16). +// +// Every leg is planned first, one after another -- the package's programs +// (L5) and then its tests with the dev-dependencies -- and the legs are then +// built at the same time, each leg on its own thread (mcpp.build.legs): the +// programs, the package's own build (Phase A) and the keep-going pass over +// the test goals. The tests then run leg by leg. A leg this machine does not +// reach is built and reported not run, with the reason, once; a set member +// its platform does not serve is left out. A command whose legs ran no test at +// all fails: a green exit that executed nothing is the false pass #544 +// records, and `--no-run` is how a build-only result is asked for. +export struct TestLeg { + std::string triple; + std::string fromSet; // empty: named on its own, or the platform's own target +}; + +export int run_test_legs(std::span passthrough, + BuildOverrides base, TestOptions testOpts, + std::vector requested) { + const bool json = (testOpts.format == TestMessageFormat::Json); + if (json) mcpp::ui::set_quiet(true); + mcpp::build::progress::open(mcpp::log::is_verbose()); + const std::string memberName = base.package_filter; + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); + if (!root) { + mcpp::ui::error("no mcpp.toml found in current directory or any parent"); + return 2; + } + auto discovered = mcpp::build::discover_test_targets(*root, base.package_filter); + if (!discovered) { mcpp::ui::error(discovered.error()); return 2; } + // Nothing to build, a listing, or a filter that selects nothing: the + // answer does not depend on the target, and is the one-leg answer. + const bool anyMatch = testOpts.filter.empty() + || std::ranges::any_of(discovered->targets, [&](auto const& t) { + return t.name.find(testOpts.filter) != std::string::npos; }); + if (discovered->targets.empty() || testOpts.list || !anyMatch) { + base.target_triple = requested.front().triple; + return run_tests(passthrough, std::move(base), testOpts); + } + bool hasPrograms = false; + if (auto m = mcpp::manifest::load(discovered->packageRoot / "mcpp.toml")) { + hasPrograms = std::ranges::any_of(m->targets, [](auto const& t) { + return t.kind == mcpp::manifest::Target::Binary + || t.kind == mcpp::manifest::Target::Application; + }); + std::error_code ec; + if (m->targets.empty() && std::filesystem::exists(discovered->packageRoot / "src" / "main.cpp", ec)) + hasPrograms = true; + } + + struct Leg { + std::string triple; + std::optional programs; + std::vector> programEnv; + TestBuild tb; + std::optional packageError; + bool programsFailed = false; + TestRunSummary summary; + }; + std::vector> legs; + std::vector leftOut; + + // ── planned, one leg after another ── + for (auto const& r : requested) { + auto leg = std::make_unique(); + leg->triple = r.triple; + BuildOverrides o = base; + o.target_triple = r.triple; + auto refused = [&](const std::string& why) -> int { + if (!r.fromSet.empty() && refusal::take() == refusal::Code::PlatformTargetUnserved) { + leftOut.push_back(std::format("{}: {}", r.triple, why)); + return 0; + } + mcpp::ui::error(std::format("the {} leg: {}", r.triple, why)); + return 2; + }; + (void)refusal::take(); + if (hasPrograms) { + auto bin = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, {}, o); + if (!bin) { if (int rc = refused(bin.error())) return rc; continue; } + if (std::ranges::any_of(bin->plan.linkUnits, + [](auto const& lu) { return lu.kind == LinkUnit::Binary; })) + leg->programs.emplace(std::move(*bin)); + } + o.will_run = true; + auto prepared = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, + discovered->targets, std::move(o)); + if (!prepared) { if (int rc = refused(prepared.error())) return rc; continue; } + leg->tb.ctx.emplace(std::move(*prepared)); + leg->tb.backend = mcpp::build::make_ninja_backend(); + leg->tb.ninjaLog = leg->tb.ctx->outputDir / ".ninja_log"; + std::error_code ec; + leg->tb.logFrom = std::filesystem::file_size(leg->tb.ninjaLog, ec); + if (ec) leg->tb.logFrom = 0; + legs.push_back(std::move(leg)); + } + for (auto const& why : leftOut) + mcpp::ui::note(std::format("left out of the target set, its platform does not serve it -- {}", why)); + if (legs.empty()) { + mcpp::ui::error("no member of the target set is served by its platform"); + return 2; + } + + auto filter_match = [&](const mcpp::build::LinkUnit& lu) { + return lu.kind == mcpp::build::LinkUnit::TestBinary + && (testOpts.filter.empty() + || lu.targetName.find(testOpts.filter) != std::string::npos); + }; + + // ── built at the same time, a thread per leg ── + for (auto& l : legs) test_announce(*l->tb.ctx); + mcpp::build::progress::programs_done(); + const bool reporting = mcpp::build::progress::is_open() && legs.size() > 1; + if (reporting) { + mcpp::build::progress::configurations(legs.size()); + mcpp::build::progress::defer_finished(); + } + std::vector dirs; + for (auto& l : legs) dirs.push_back(l->tb.ctx->outputDir); + const std::size_t distinct = std::set(dirs.begin(), dirs.end()).size(); + for (auto& l : legs) { + l->tb.ctx->plan.publishRootCompileDb = false; + mcpp::build::legs::share_jobs(*l->tb.ctx, distinct); + if (l->programs) mcpp::build::legs::share_jobs(*l->programs, distinct); + } + auto built = mcpp::build::legs::run_by_directory(dirs, [&](std::size_t i) -> int { + auto& l = *legs[i]; + if (l.programs) { + if (run_build_plan(*l.programs, mcpp::log::is_verbose(), false, l.triple) != 0) { + l.programsFailed = true; + return 1; + } + for (auto const& lu : l.programs->plan.linkUnits) { + if (lu.kind != LinkUnit::Binary) continue; + std::string key = "MCPP_BIN_"; + for (char c : lu.targetName) + key += (c >= 'a' && c <= 'z') ? char(c - 'a' + 'A') + : ((c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) ? c : '_'; + l.programEnv.emplace_back(std::move(key), + (l.programs->outputDir / lu.output).lexically_normal().string()); + } + } + if (auto a = test_phase_a(l.tb, testOpts, json)) { + l.packageError = std::move(*a); + return 1; + } + test_bulk(l.tb, testOpts, filter_match); + return 0; + }); + if (reporting && mcpp::build::legs::first_failure(built) == 0) + mcpp::build::progress::finish_deferred(); + mcpp::build::progress::close(); + mcpp::build::publish_workspace_compile_commands(*root, {legs.front()->tb.ctx->outputDir}, + legs.front()->tb.ctx->plan.rootCompileDb); + + // ── run, one leg after another ── + int rc = 0; + int passed = 0, failed = 0, notRun = 0, builtNotRun = 0; + std::vector reasons; + auto worse = [&](int r) { + // 1 (a test failed) outranks 2 (an answer not established), which + // outranks 0. + if (r == 1 || (r != 0 && rc == 0)) rc = r; + }; + auto legOpts = testOpts; + legOpts.unreachableIsBuilt = true; + for (auto& lp : legs) { + auto& l = *lp; + if (json) std::println("{{\"leg\":{{\"triple\":\"{}\"}}}}", test_json_escape(l.triple)); + else mcpp::ui::info("Testing", std::format("on {}", l.triple)); + if (l.programsFailed) { + mcpp::ui::error(std::format("the {} leg: the package's programs did not build", l.triple)); + worse(1); + continue; + } + if (l.packageError) { + if (json) + std::println("{{\"error\":\"package\",\"compile_output\":\"{}\"}}", + test_json_escape(l.packageError->diagnosticOutput)); + if (!l.packageError->reported) mcpp::ui::error(l.packageError->message); + mcpp::ui::block(l.packageError->diagnosticOutput); + worse(1); + continue; + } + l.tb.programEnv = std::move(l.programEnv); + test_resolve_runner(l.tb, legOpts, json); + if (l.tb.builtNotRunHere) { + reasons.push_back(std::format("{}: built, not run on this {} machine", + l.triple, mcpp::toolchain::platform::host_arch)); + } + worse(test_run_member(l.tb, legOpts, passthrough, json, memberName, /*owner=*/"", + l.tb.buildMs, l.summary)); + passed += l.summary.passed; + failed += l.summary.failed; + notRun += l.summary.notRun; + builtNotRun += l.summary.built; + if (l.summary.notRun) + reasons.push_back(std::format("{}: {}", l.triple, + l.summary.notRunReason.substr(0, l.summary.notRunReason.find('\n')))); + } + // NO TEST RAN ON ANY LEG: a failure, unless a build was all that was asked. + const bool noneRan = !testOpts.noRun && passed + failed == 0 && rc != 1; + if (json) { + std::println("{{\"legs_summary\":{{\"legs\":{},\"passed\":{},\"failed\":{}," + "\"not_run\":{},\"built\":{},\"none_ran\":{}}}}}", + legs.size(), passed, failed, notRun, builtNotRun, noneRan ? "true" : "false"); + std::fflush(stdout); + } + if (noneRan) { + std::string why; + for (auto const& r : reasons) why += "\n " + r; + mcpp::ui::error(std::format("no test ran on any of the {} legs{}{}", legs.size(), + why.empty() ? std::string{} : ":", why)); + return 1; + } + if (!json) + mcpp::ui::result("legs result", std::format("{} legs; {} passed; {} failed{}{}", + legs.size(), passed, failed, + builtNotRun ? std::format("; {} built, not run", builtNotRun) : std::string{}, + notRun ? std::format("; {} not run", notRun) : std::string{})); + if (rc != 0 || testOpts.archive.empty()) return rc; + std::vector archived; + for (auto& l : legs) { + auto t = mcpp::toolchain::triple::parse(l->triple); + archived.push_back({t ? t->str() : l->triple, &*l->tb.ctx, &l->tb.programEnv}); + } + return write_test_archive(testOpts, archived); } // `mcpp test` over several members: the members are planned once per diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 47e68d2af..26afcb179 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -1075,6 +1075,16 @@ prepare_build(bool print_fingerprint, bool includeDevDeps = false, export std::expected prepare_configuration(std::string_view targetTriple); +// THE PLATFORM A TARGET SELECTS, ASKED WITHOUT PREPARING IT (SPEC-015 §1): the +// selection the first phase makes -- `--platform`, the target's environment +// variable, the project's `[target.]` tables, this machine's +// configuration, the target table's column -- put to one target, so the +// members of a target set can be checked to select one platform before any +// of them is planned. "" when no platform is selected. +export std::expected +platform_selection_for(const mcpp::toolchain::triple::Triple& target, + const BuildOverrides& overrides); + // The PlanNotes a failed call recorded before the phase that failed it. // // On success, `prepare_build` copies `PrepareState::planNotes` into @@ -1108,3 +1118,85 @@ export std::vector take_notes_on_failure(); export void populate_dependency_cache(BuildContext& ctx); } // namespace mcpp::build + +// THE LEGS OF ONE COMMAND, PLANNED AND BUILT (SPEC-015 §3; legs.cpp). +// +// A command over several targets acts once per target; each is a LEG. The +// legs of a command are planned one after another and then built at the same +// time, as the configuration groups of a workspace are (workspace design +// 2026-09-29 §15), and by the same machinery: +// +// PLANNING IS SERIAL. `prepare_build` resolves dependencies, may write +// `mcpp.lock` (SPEC-011 §4 writes it optimistically and refuses a second +// writer), runs build programs and narrates what it resolves. One leg at a +// time keeps the lock file one writer's, the narration readable, and the +// build directories' locks taken in one order -- the legs' order, which is +// the target table's -- so two commands over overlapping legs cannot each +// hold the lock the other waits for. +// +// BUILDING IS CONCURRENT ACROSS BUILD DIRECTORIES. Each leg's plan names its +// own directory (`target//`); the builds run on one +// thread per directory, each with a static share of the jobs one build +// would use, and one `Finished` follows them all. Two legs that resolve to +// one directory (one target spelled two ways) are built one after the other +// in it, since one ninja owns a directory. +// +// A LEG THAT CAME FROM A SET MAY BE LEFT OUT. A target set names the target +// table's rows of a system; whether the platform those rows select serves each +// of them is a fact of the platform package, which is read when the leg is +// planned. A member the platform does not serve is left out and reported; a +// target named on its own is refused, as it always was. +export namespace mcpp::build::legs { + +// One leg to plan: the overrides `prepare_build` takes for it +// (`target_triple` is the leg), and what the command plans besides the +// package's own targets. +struct Spec { + BuildOverrides overrides; + bool includeDevDeps = false; + std::vector extraTargets; + // The set the leg is a member of; empty for a target named on its own. + std::string fromSet; +}; + +struct Planned { + std::string triple; // as the leg was requested + std::string fromSet; + BuildContext ctx; +}; + +struct Plan { + std::vector legs; + // Members of a set the selected platform does not serve: ": ". + std::vector leftOut; +}; + +// Plans every leg, in order; the first leg that cannot be planned ends it, +// its error naming the leg (a leg of the host, unnamed, is not prefixed). +std::expected plan(std::vector specs, bool printFingerprint = false); + +// Builds the contexts concurrently across their build directories, each with +// `buildOne`, which returns the exit status of its build. One context is +// built on the calling thread with nothing changed. Returns each context's +// status, in order; the deferred `Finished` is stated when every one is 0. +std::vector build(std::span contexts, + const std::function& buildOne); + +// Runs `work(i)` for every index, concurrently across distinct directories +// and one after another within a directory; returns each status, in order. +// The primitive `build` is made of, for a command whose leg is more than one +// build (`mcpp test`: the package's programs, then its tests). +std::vector run_by_directory(std::span directories, + const std::function& work); + +// Gives each build plan a static share of the jobs one build would use, for +// `n` directories built at the same time. +void share_jobs(BuildContext& ctx, std::size_t n); + +// The first non-zero status, or 0. +int first_failure(std::span statuses); + +// The legs' order: native architecture first, the rest as they were. +std::vector native_first(std::span triples); + +} // namespace mcpp::build::legs diff --git a/src/build/prepare/legs.cpp b/src/build/prepare/legs.cpp new file mode 100644 index 000000000..95479c3e1 --- /dev/null +++ b/src/build/prepare/legs.cpp @@ -0,0 +1,105 @@ +// legs.cpp -- the legs of one command, planned and built (SPEC-015 §3). +// Declared in the primary interface (prepare.cppm), where the comment that +// states the model is. + +module mcpp.build.prepare; + +import std; +import mcpp.build.progress; +import mcpp.build.refusal; +import mcpp.manifest; +import mcpp.toolchain.triple; +import mcpp.ui; + +namespace mcpp::build::legs { + +std::expected plan(std::vector specs, bool printFingerprint) { + Plan out; + for (auto& s : specs) { + const std::string triple = s.overrides.target_triple; + (void)refusal::take(); + auto ctx = prepare_build(printFingerprint, s.includeDevDeps, std::move(s.extraTargets), + std::move(s.overrides)); + if (!ctx) { + if (!s.fromSet.empty() && refusal::take() == refusal::Code::PlatformTargetUnserved) { + out.leftOut.push_back(std::format("{}: {}", triple, ctx.error())); + continue; + } + if (triple.empty()) return std::unexpected(ctx.error()); + return std::unexpected(std::format("the {} leg: {}", triple, ctx.error())); + } + out.legs.push_back({triple, s.fromSet, std::move(*ctx)}); + } + for (auto const& why : out.leftOut) + mcpp::ui::note(std::format("left out of the target set, its platform does not serve it -- {}", + why)); + if (out.legs.empty() && !out.leftOut.empty()) + return std::unexpected("no member of the target set is served by its platform"); + return out; +} + +std::vector run_by_directory(std::span directories, + const std::function& work) { + std::vector results(directories.size(), 0); + std::map> byDirectory; + for (std::size_t i = 0; i < directories.size(); ++i) + byDirectory[directories[i].lexically_normal()].push_back(i); + std::vector runs; + for (auto const& [dir, indices] : byDirectory) + runs.emplace_back([&, indices] { + for (auto i : indices) results[i] = work(i); + }); + runs.clear(); // joins + return results; +} + +void share_jobs(BuildContext& ctx, std::size_t n) { + const std::size_t hw = std::max(1u, std::thread::hardware_concurrency()); + const std::size_t want = ctx.plan.scheduleNinjaJobs > 0 + ? static_cast(ctx.plan.scheduleNinjaJobs) : hw + 2; + ctx.plan.scheduleNinjaJobs = static_cast(std::max(1, want / std::max(1, n))); +} + +std::vector build(std::span contexts, + const std::function& buildOne) { + if (contexts.empty()) return {}; + if (contexts.size() == 1) return {buildOne(*contexts.front())}; + // One report for every leg: each package line names its configuration, + // and one `Finished` follows them all. + const bool reporting = mcpp::build::progress::is_open(); + if (reporting) { + mcpp::build::progress::configurations(contexts.size()); + mcpp::build::progress::defer_finished(); + } + std::vector dirs; + for (auto* c : contexts) dirs.push_back(c->outputDir); + const std::size_t distinct = std::set( + dirs.begin(), dirs.end()).size(); + // The root compile database is the command's, published by the caller + // from every leg's, not by each concurrent build. + for (auto* c : contexts) { + c->plan.publishRootCompileDb = false; + share_jobs(*c, distinct); + } + auto results = run_by_directory(dirs, [&](std::size_t i) { return buildOne(*contexts[i]); }); + if (reporting && first_failure(results) == 0) mcpp::build::progress::finish_deferred(); + return results; +} + +int first_failure(std::span statuses) { + for (int r : statuses) + if (r != 0) return r; + return 0; +} + +std::vector native_first(std::span triples) { + std::vector native, other; + for (auto const& t : triples) { + auto p = mcpp::toolchain::triple::parse(t); + (p && p->arch == mcpp::toolchain::triple::host_triple().arch ? native : other).push_back(t); + } + native.insert(native.end(), other.begin(), other.end()); + return native; +} + +} // namespace mcpp::build::legs diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index 2aef79243..139e9f55a 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -26,6 +26,7 @@ import mcpp.invocation; import mcpp.manifest; import mcpp.platform; import mcpp.pm.compat; +import mcpp.project; import mcpp.targetside; import mcpp.toolchain.triple; import mcpp.toolchain.platform; @@ -118,6 +119,36 @@ mcpp::platform::Matches matcher(cfgpred::Ctx c) { } // namespace +std::expected +platform_selection_for(const mcpp::toolchain::triple::Triple& t, const BuildOverrides& overrides) { + const auto flag = !overrides.platform_choice.empty() + ? overrides.platform_choice + : mcpp::invocation::get("MCPP_PLATFORM").value_or(""); + const auto envKey = target_env_name(t, "PLATFORM"); + const auto env = mcpp::invocation::get(envKey).value_or(""); + std::vector project; + std::error_code ec; + auto root = !overrides.project_root.empty() + ? std::optional(overrides.project_root) + : mcpp::project::find_manifest_root(fs::current_path(ec)); + if (root) + if (auto m = mcpp::manifest::load(*root / "mcpp.toml")) project = project_tables(*m); + // Read, never initialised: asking which platform a target selects writes + // nothing, and a machine without a configuration has no tables to apply. + std::vector config; + if (auto c = mcpp::config::load_or_init(/*quiet=*/true, {}, {}, + mcpp::config::Initialization::ReadOnly)) + config = c->platformOverrides; + std::string column; + if (auto* row = mcpp::toolchain::triple::find_known_target(t)) column = std::string(row->platform); + mcpp::platform::ChoiceInputs in{ flag, env, envKey, project, config, column }; + if (overrides.platform_choice.empty() && !stated_on_command_line("MCPP_PLATFORM")) + in.flagKey = "MCPP_PLATFORM"; + auto sel = mcpp::platform::select_target(in, matcher(cfgpred::context_for(t.str()))); + if (!sel) return std::unexpected(sel.error()); + return sel->none ? std::string{} : mp::display(sel->ref); +} + cfgpred::Ctx platform_cfg_context(const PrepareState& state) { return cfgpred::context_for(target_of(state).str(), state.platformPackageShortName); } diff --git a/src/build/result.cppm b/src/build/result.cppm index 3c272b5c7..8cf3f47c4 100644 --- a/src/build/result.cppm +++ b/src/build/result.cppm @@ -111,6 +111,21 @@ bool events_enabled(); // One NDJSON line on stdout: {"schemaVersion":1,"kind":"mcpp.event","event":, ...fields}. void emit(std::string_view event, nlohmann::json fields = nlohmann::json::object()); +// ── several legs (SPEC-015 §3.5) ── +// +// A command over several targets builds them at the same time, so its +// `artifact` and `diagnostic` events carry the target they belong to. One +// leg's stream is unchanged: the field is written only while this is on. +void set_legs(bool on); +bool legs(); + +// After the legs of one command: `target/.last-result.json` names the first +// leg's record, as it names the one record of a single build, states the +// legs' status together (`failed` if any failed, else `cancelled` if any was, +// else `ok`), and lists every leg's record under `legs`. +void write_last_of_legs(const std::filesystem::path& projectRoot, + std::span outputDirs); + } // namespace mcpp::build::result namespace mcpp::build::result { @@ -386,6 +401,40 @@ Record begin(const std::filesystem::path& projectRoot, const std::filesystem::pa void enable_events(bool on) { g_events.store(on); } bool events_enabled() { return g_events.load(); } +namespace { +std::atomic g_legs{false}; +} // namespace +void set_legs(bool on) { g_legs.store(on); } +bool legs() { return g_legs.load(); } + +void write_last_of_legs(const std::filesystem::path& projectRoot, + std::span outputDirs) { + if (projectRoot.empty() || outputDirs.empty()) return; + nlohmann::json legs = nlohmann::json::array(); + std::string status = "ok"; + std::int64_t finishedAt = 0; + for (auto const& dir : outputDirs) { + auto r = read(dir); + const std::string s = r ? r->status : std::string("failed"); + if (s == "failed" || (s == "cancelled" && status != "failed") || (s == "running" && status == "ok")) + status = s; + if (r) finishedAt = std::max(finishedAt, r->finishedAt); + legs.push_back({{"triple", r ? r->triple : std::string{}}, + {"result", (dir / "result.json").generic_string()}, + {"status", s}}); + } + nlohmann::json last{ + {"schemaVersion", 1}, + {"kind", "mcpp.last-result"}, + {"kindVersion", 1}, + {"data", {{"result", (outputDirs.front() / "result.json").generic_string()}, + {"status", status}, + {"finishedAt", finishedAt == 0 ? nlohmann::json(nullptr) : nlohmann::json(finishedAt)}, + {"legs", std::move(legs)}}}, + }; + write_atomically(projectRoot / "target" / ".last-result.json", last.dump(2) + "\n"); +} + void emit(std::string_view event, nlohmann::json fields) { if (!events_enabled()) return; nlohmann::json line{{"schemaVersion", 1}, {"kind", "mcpp.event"}, {"event", std::string(event)}}; diff --git a/src/cli.cppm b/src/cli.cppm index 7c366fb5d..7eb1f13c4 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -620,8 +620,9 @@ int run(int argc, char** argv) { .help("The host graph's platform for this invocation (= MCPP_HOST_PLATFORM)")) .option(cl::Option("no-cache") .help("Deprecated alias for --cache=off (also clears the build dir)")) - .option(cl::Option("target").takes_value().help( - "Build for (e.g. x86_64-linux-musl); looks up [target.] in mcpp.toml")) + .option(cl::Option("target").takes_value().multiple().help( + "Build for (e.g. x86_64-linux-musl), or for a target set (e.g. openkal, " + "linux-musl); repeatable: one leg per target")) .option(cl::Option("accel").takes_value().value_name("SPEC") .help("Device backends and architectures, e.g. 'cuda12.8+{sm_89}'; overrides [build] accel")) .option(cl::Option("no-accel") @@ -668,8 +669,9 @@ int run(int argc, char** argv) { // lets `run` spell the flag `--target` like every other // subcommand, instead of being the one command that cannot. .arg(cl::Arg("bin").help("Binary name (optional)")) - .option(cl::Option("target").takes_value().value_name("TRIPLE") - .help("Cross target triple (same axis as `mcpp build --target`)")) + .option(cl::Option("target").takes_value().multiple().value_name("TRIPLE") + .help("Cross target triple or target set (same axis as `mcpp build --target`); " + "of several, the first this host reaches, its own architecture first")) .option(cl::Option("accel").takes_value().value_name("SPEC") .help("Device backends and architectures (same axis as `mcpp build --accel`)")) .option(cl::Option("no-accel") @@ -762,8 +764,9 @@ int run(int argc, char** argv) { .description("Build + run all tests/**/*.cpp (after `--`, args go to each test binary)") .arg(cl::Arg("pattern") .help("Run only tests whose name contains PATTERN (optional)")) - .option(cl::Option("target").takes_value().value_name("TRIPLE") - .help("Cross target triple (same axis as `mcpp build --target`)")) + .option(cl::Option("target").takes_value().multiple().value_name("TRIPLE") + .help("Cross target triple or target set (same axis as `mcpp build --target`); " + "repeatable: every leg is built, and run where this host reaches it")) .option(cl::Option("accel").takes_value().value_name("SPEC") .help("Device backends and architectures (same axis as `mcpp build --accel`)")) .option(cl::Option("no-accel") @@ -776,6 +779,8 @@ int run(int argc, char** argv) { .help("Run test binaries directly, ignoring any [target.].runner (a host that runs them natively)")) .option(cl::Option("no-run") .help("Build the tests for --target and stop; a build that succeeds is the result (for a target this host cannot execute)")) + .option(cl::Option("archive").takes_value().value_name("FILE") + .help("With --no-run: write the built tests to FILE. Without: run the tests FILE holds, building nothing (in a checkout of the same project)")) .option(cl::Option("timeout").takes_value().value_name("SECS") .help("Kill a test still RUNNING after SECS seconds (default 300; 0 = no limit)")) .option(cl::Option("build-timeout").takes_value().value_name("SECS") @@ -1208,8 +1213,9 @@ int run(int argc, char** argv) { .description("Diagnose mcpp environment health")) .subcommand(cl::App("env") .description("Print mcpp paths and configuration; with --target, one configuration: toolchain, platform, tools, runner") - .option(cl::Option("target").takes_value().value_name("TRIPLE") - .help("Report this target's configuration (prepared, not built; the project's when run in one)")) + .option(cl::Option("target").takes_value().multiple().value_name("TRIPLE") + .help("Report this target's configuration (prepared, not built; the project's when run in one); " + "a target set or several reports each leg")) .option(cl::Option("host") .help("Report the host graph's configuration: what build programs and tools are built for")) .option(cl::Option("platform").takes_value().value_name("SPEC") diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index cb7c09d55..6a3ac1f32 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -191,6 +191,28 @@ mcpp::build::BuildOverrides overrides_from_selectors( return ov; } +// THE LEGS `--target` NAMES (SPEC-015 §1): every value of the option, a set +// (`openkal`, `linux-musl`) expanded to the target table's rows, a target +// kept as written, the union in the table's order. Empty without `--target`: +// the host, as always. `run` passes `alsoTargetTriple`: it also accepts +// `--target-triple`. +namespace { +std::expected, std::string> +target_legs(const mcpplibs::cmdline::ParsedArgs& parsed, const mcpp::build::BuildOverrides& ov, + bool alsoTargetTriple = false) { + std::vector requests; + if (auto o = parsed.option("target")) + for (auto const& v : o->get().values) requests.push_back(v); + if (alsoTargetTriple) + if (auto o = parsed.option("target-triple")) + for (auto const& v : o->get().values) requests.push_back(v); + if (requests.empty()) return std::vector{}; + return mcpp::toolchain::target_set::resolve(requests, [&](const mcpp::toolchain::triple::Triple& t) { + return mcpp::build::platform_selection_for(t, ov); + }); +} +} // namespace + int cmd_build_body(const mcpplibs::cmdline::ParsedArgs& parsed); // `mcpp build --message-format json` (design 2026-10-26 §5.1). @@ -242,14 +264,28 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { if (auto root = mcpp::project::find_manifest_root(std::filesystem::current_path(ec))) { std::ifstream in(*root / "target" / ".last-result.json", std::ios::binary); auto last = nlohmann::json::parse(in, nullptr, false); - if (!last.is_discarded() && last.contains("data") && last["data"].contains("result") - && last["data"]["result"].is_string()) { - const std::filesystem::path recordPath = last["data"]["result"].get(); + // The record this process wrote, at `path`; null otherwise. + auto record_of = [](const std::filesystem::path& recordPath) -> nlohmann::json { if (auto record = res::read(recordPath.parent_path()); record && record->pid == mcpp::build::dirlock::process_id()) { std::ifstream rin(recordPath, std::ios::binary); auto doc = nlohmann::json::parse(rin, nullptr, false); - if (!doc.is_discarded() && doc.contains("data")) env.data = doc["data"]; + if (!doc.is_discarded() && doc.contains("data")) return doc["data"]; + } + return nullptr; + }; + if (!last.is_discarded() && last.contains("data") && last["data"].contains("result") + && last["data"]["result"].is_string()) { + env.data = record_of(last["data"]["result"].get()); + // SEVERAL LEGS (SPEC-015 §3.5): the first leg's record, the legs' + // status together, and every leg's record under `legs`. + if (last["data"].contains("legs") && last["data"]["legs"].is_array() && !env.data.is_null()) { + nlohmann::json legs = nlohmann::json::array(); + for (auto const& l : last["data"]["legs"]) + if (l.contains("result") && l["result"].is_string()) + legs.push_back(record_of(l["result"].get())); + env.data["status"] = last["data"].value("status", "failed"); + env.data["legs"] = std::move(legs); } } } @@ -269,6 +305,103 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { return rc; } +// `mcpp build` OVER SEVERAL LEGS (SPEC-015 §3). Every leg is planned, one after +// another, as the build of that one target would be planned -- in a +// workspace, each leg's configuration groups -- and then every plan is built +// at once (mcpp.build.legs). The root compile database is the first leg's: an +// editor reads one target's view of the sources. `target/.last-result.json` +// names the first leg's record and lists every leg's. +namespace { +int build_several_legs(const mcpplibs::cmdline::ParsedArgs& parsed, + const mcpp::build::BuildOverrides& ov, + const std::vector& legs, + bool verbose, bool print_fp, bool no_cache, bool configure_only) { + namespace L = mcpp::build::legs; + auto selection = mcpp::cli::select_members(member_request(parsed)); + if (!selection) { mcpp::ui::error(std::format("{}", selection.error())); return 2; } + std::error_code ec; + std::filesystem::path root; + if (*selection) root = (*selection)->root; + else if (auto r = mcpp::project::find_manifest_root(std::filesystem::current_path(ec))) root = *r; + else { mcpp::ui::error("no mcpp.toml found in current directory or any parent"); return 2; } + + // The specs: one per leg, or in a workspace one per configuration group + // of each leg. + std::vector specs; + for (auto const& leg : legs) { + if (*selection) { + auto const& members = (*selection)->members; + auto groups = workspace_groups(root, members, leg.triple); + if (!groups) { mcpp::ui::error(std::format("{}", groups.error())); return 2; } + std::vector request; + for (auto const& g : *groups) + for (auto const& mp : g) request.push_back(mp); + for (auto const& g : *groups) { + L::Spec spec{ov}; + spec.fromSet = leg.fromSet; + auto& mo = spec.overrides; + mo.target_triple = leg.triple; + mo.package_filter.clear(); + mo.project_root = root; + mo.workspace_members = g; + mo.workspace_request = request; + if (configure_only) { + auto tests = group_tests(root, g); + if (!tests) { mcpp::ui::error(std::format("{}", tests.error())); return 2; } + spec.includeDevDeps = !tests->targets.empty(); + mo.member_targets = std::move(tests->targets); + } + specs.push_back(std::move(spec)); + } + continue; + } + L::Spec spec{ov}; + spec.fromSet = leg.fromSet; + spec.overrides.target_triple = leg.triple; + if (configure_only) { + // The surface an editor needs, as for one target: the tests and + // the dev-dependencies. + auto discovered = mcpp::build::discover_test_targets(root, ov.package_filter); + if (!discovered) { mcpp::ui::error(discovered.error()); return 2; } + spec.includeDevDeps = !discovered->targets.empty(); + spec.extraTargets = std::move(discovered->targets); + } + specs.push_back(std::move(spec)); + } + auto planned = L::plan(std::move(specs), print_fp); + if (!planned) { mcpp::ui::error(planned.error()); return 2; } + std::vector contexts; + for (auto& p : planned->legs) contexts.push_back(&p.ctx); + const auto firstTriple = planned->legs.front().triple; + std::vector firstDirs; + for (auto& p : planned->legs) + if (p.triple == firstTriple) firstDirs.push_back(p.ctx.outputDir); + const std::string rootDb = contexts.front()->plan.rootCompileDb; + + int rc = 0; + if (configure_only) { + for (auto* c : contexts) { + c->plan.publishRootCompileDb = false; + if (int r = mcpp::build::run_configure_plan(*c, verbose); r != 0 && rc == 0) rc = r; + } + } else { + std::map tripleOf; + for (auto& p : planned->legs) tripleOf[&p.ctx] = p.triple; + mcpp::build::result::set_legs(true); + auto results = L::build(contexts, [&](mcpp::build::BuildContext& c) { + return run_build_with_hooks(c, verbose, no_cache, tripleOf[&c]); + }); + mcpp::build::result::set_legs(false); + rc = L::first_failure(results); + std::vector dirs; + for (auto* c : contexts) dirs.push_back(c->outputDir); + mcpp::build::result::write_last_of_legs(contexts.front()->projectRoot, dirs); + } + mcpp::build::publish_workspace_compile_commands(root, firstDirs, rootDb); + return rc; +} +} // namespace + int cmd_build_body(const mcpplibs::cmdline::ParsedArgs& parsed) { bool verbose = parsed.is_flag_set("verbose") || mcpp::log::is_verbose(); bool print_fp = parsed.is_flag_set("print-fingerprint"); @@ -285,6 +418,15 @@ int cmd_build_body(const mcpplibs::cmdline::ParsedArgs& parsed) { if (auto c = parsed.value("cache")) ov.cache_mode = *c; else if (no_cache) ov.cache_mode = "off"; + // THE LEGS `--target` NAMES (SPEC-015 §3). One leg is the build it always + // was, spelled as written; several are planned one after another and + // built at once. + auto legs = target_legs(parsed, ov); + if (!legs) { mcpp::ui::error(legs.error()); return 2; } + if (legs->size() == 1) ov.target_triple = legs->front().triple; + if (legs->size() > 1) + return build_several_legs(parsed, ov, *legs, verbose, print_fp, no_cache, configure_only); + // Fan-out prefixes every diagnostic with the member it came from; the // single-package path has nothing to disambiguate and passes "". // `configured`, when given, receives the plan's output directory and the @@ -956,6 +1098,14 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, std::string target_triple; if (auto tt = parsed.value("target")) target_triple = *tt; if (auto tt = parsed.value("target-triple")) target_triple = *tt; + // SEVERAL TARGETS, OR A SET (SPEC-015 §3.2): one leg runs, chosen in + // build_run_target; one leg is the target as written. + mcpp::build::BuildOverrides legOv; + legOv.package_filter = package_filter; + auto legs = target_legs(parsed, legOv, /*alsoTargetTriple=*/true); + if (!legs) { mcpp::ui::error(legs.error()); return 2; } + if (!legs->empty()) target_triple = legs->front().triple; + if (legs->size() < 2) legs->clear(); // --no-runner: "this host can execute the artifact" is a fact about the // host, and the manifest has no host axis to state it on (#544, D3). const bool no_runner = parsed.is_flag_set("no-runner"); @@ -994,7 +1144,7 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, return mcpp::build::build_run_target(targetName, passthrough, package_filter, cache_mode, no_cache, target_triple, no_runner, runner_name, features, - profile, accel, format); + profile, accel, format, *legs); } export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, @@ -1016,6 +1166,14 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, if (auto tt = parsed.value("target")) ov.target_triple = *tt; if (parsed.is_flag_set("no-accel")) ov.accel = "(none)"; else if (auto a = parsed.value("accel")) ov.accel = *a; + // THE LEGS `--target` NAMES (SPEC-015 §3.2). One leg is the test run it + // always was; several are planned, built at once, and run leg by leg. + auto legs = target_legs(parsed, ov); + if (!legs) { mcpp::ui::error(legs.error()); return 2; } + if (legs->size() == 1) ov.target_triple = legs->front().triple; + std::vector testLegs; + if (legs->size() > 1) + for (auto const& l : *legs) testLegs.push_back({l.triple, l.fromSet}); mcpp::build::TestOptions to; if (parsed.positional_count() > 0) to.filter = parsed.positional(0); @@ -1031,6 +1189,11 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, "--no-run does not run them."); return 2; } + // BUILT ONCE, RUN ELSEWHERE (SPEC-015 §4): `--no-run --archive` writes the + // tests it built; `--archive` alone runs an archive's tests, building + // nothing. + if (auto a = parsed.value("archive")) to.archive = *a; + const bool runArchive = !to.archive.empty() && !to.noRun; // The three deadlines share one parser: they differ only in what they // bound, not in how they are spelled. 0 always means "no limit" — for // --timeout that now has to be asked for rather than being the default. @@ -1057,6 +1220,14 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, return 2; } } + if (runArchive) { + if (!testLegs.empty() || !ov.target_triple.empty()) { + mcpp::ui::error("mcpp test --archive runs the legs the archive holds; --target " + "chooses legs when the archive is written (--no-run --archive)"); + return 2; + } + return mcpp::build::run_test_archive(passthrough, ov, to); + } // The members this command tests, by the one selection every command reads // (`mcpp::cli::select_members`). A selection of one member, named or implied @@ -1068,231 +1239,250 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, // selection design 2026-09-30, S4). auto selection = mcpp::cli::select_members(member_request(parsed)); if (!selection) { mcpp::ui::error(std::format("{}", selection.error())); return 2; } + if (*selection && ((*selection)->whole || (*selection)->members.size() > 1) + && !to.archive.empty()) { + mcpp::ui::error("--archive holds the tests of one package: select one member (-p)"); + return 2; + } if (*selection && ((*selection)->whole || (*selection)->members.size() > 1)) { - auto const& sel = **selection; - auto const& members = sel.members; - const bool json = (to.format == mcpp::build::TestMessageFormat::Json); - // Silence the ui BEFORE the first member, not inside run_tests. The - // quiet flag used to be set by run_tests itself, so the fan-out's own - // "testing member" line escaped for member #1 and was suppressed from - // #2 on — one stdout stream, two behaviors, and the stray line broke - // NDJSON for any consumer that parsed it strictly. - if (json) mcpp::ui::set_quiet(true); + // The members, tested on each leg in turn: a leg's members are planned + // by its configuration groups and built once, as for one target. + auto test_members = [&](mcpp::build::BuildOverrides ov) -> int { + auto const& sel = **selection; + auto const& members = sel.members; + const bool json = (to.format == mcpp::build::TestMessageFormat::Json); + // Silence the ui BEFORE the first member, not inside run_tests. The + // quiet flag used to be set by run_tests itself, so the fan-out's own + // "testing member" line escaped for member #1 and was suppressed from + // #2 on — one stdout stream, two behaviors, and the stray line broke + // NDJSON for any consumer that parsed it strictly. + if (json) mcpp::ui::set_quiet(true); - int rc = 0; - std::vector failed; - std::vector notRun; // --workspace-timeout reached - std::vector unrunnable; // tests built, none executed (#544) - std::vector> memberTimes; - int totalPassed = 0, totalFailed = 0, totalNotRun = 0; - // Carried for the same reason `totalNotRun` is: without it a - // `--no-run` workspace reports "0 passed; 0 failed", which is - // what a workspace with no tests at all reports. - int totalBuilt = 0; - auto tWs = std::chrono::steady_clock::now(); - auto ws_ms = [&tWs] { - return std::chrono::duration_cast( - std::chrono::steady_clock::now() - tWs).count(); - }; - const long long wsDeadlineMs = - static_cast(workspaceTimeoutSecs) * 1000; - - // Asked before a member's tests start. The deadline is checked BEFORE - // starting a member rather than after: stopping mid-member would leave - // a half-tested member reported as neither run nor skipped. It is - // measured from the start of the command, so the build the members - // share counts against it, and a member not started by then is listed - // as not run; the build itself is bounded by --build-timeout. - auto member_begin = [&](std::size_t i, const std::string& mp) -> bool { - if (wsDeadlineMs > 0 && ws_ms() >= wsDeadlineMs) { - notRun.push_back(mp); - return false; - } - mcpp::ui::status("Workspace", - std::format("testing member '{}' ({}/{})", mp, i + 1, members.size())); - return true; - }; - // The member's line: how it ended, and how long its own tests ran. The - // build is the group's, stated once by the group's line, so a member's - // line does not state it again. - auto member_end = [&](std::size_t i, const std::string& mp, int r, - const mcpp::build::TestRunSummary& sum) { - const auto idx = i + 1; - totalPassed += sum.passed; - totalFailed += sum.failed; - totalNotRun += sum.notRun; - totalBuilt += sum.built; - // Ranked by its run, which is the member's own: a build a group - // shares belongs to no one member. - memberTimes.emplace_back(mp, sum.buildGroup >= 0 ? sum.runMs : sum.elapsedMs); - auto secs = static_cast(sum.buildGroup >= 0 ? sum.runMs : sum.elapsedMs) / 1000.0; - const char* took = sum.buildGroup >= 0 ? "run " : ""; - const char* in = sum.buildGroup >= 0 ? ", " : " in "; - if (r == 2 && sum.failed == 0 && sum.notRun > 0) { - // Built and not executed (#544): the member did not fail, and - // it did not pass. 2 outranks 0 and yields to 1, as it does - // for a single member. - if (rc == 0) rc = 2; - unrunnable.push_back(mp); - mcpp::ui::status("Workspace", - std::format("member '{}' ({}/{}) NOT RUN — {} passed, {} not run{}{}{:.2f}s", - mp, idx, members.size(), sum.passed, sum.notRun, in, took, secs)); - } else if (r != 0) { - rc = r; - failed.push_back(mp); + int rc = 0; + std::vector failed; + std::vector notRun; // --workspace-timeout reached + std::vector unrunnable; // tests built, none executed (#544) + std::vector> memberTimes; + int totalPassed = 0, totalFailed = 0, totalNotRun = 0; + // Carried for the same reason `totalNotRun` is: without it a + // `--no-run` workspace reports "0 passed; 0 failed", which is + // what a workspace with no tests at all reports. + int totalBuilt = 0; + auto tWs = std::chrono::steady_clock::now(); + auto ws_ms = [&tWs] { + return std::chrono::duration_cast( + std::chrono::steady_clock::now() - tWs).count(); + }; + const long long wsDeadlineMs = + static_cast(workspaceTimeoutSecs) * 1000; + + // Asked before a member's tests start. The deadline is checked BEFORE + // starting a member rather than after: stopping mid-member would leave + // a half-tested member reported as neither run nor skipped. It is + // measured from the start of the command, so the build the members + // share counts against it, and a member not started by then is listed + // as not run; the build itself is bounded by --build-timeout. + auto member_begin = [&](std::size_t i, const std::string& mp) -> bool { + if (wsDeadlineMs > 0 && ws_ms() >= wsDeadlineMs) { + notRun.push_back(mp); + return false; + } mcpp::ui::status("Workspace", - std::format("member '{}' ({}/{}) FAILED — {} passed, {} failed{}{}{:.2f}s", - mp, idx, members.size(), sum.passed, sum.failed, in, took, secs)); + std::format("testing member '{}' ({}/{})", mp, i + 1, members.size())); + return true; + }; + // The member's line: how it ended, and how long its own tests ran. The + // build is the group's, stated once by the group's line, so a member's + // line does not state it again. + auto member_end = [&](std::size_t i, const std::string& mp, int r, + const mcpp::build::TestRunSummary& sum) { + const auto idx = i + 1; + totalPassed += sum.passed; + totalFailed += sum.failed; + totalNotRun += sum.notRun; + totalBuilt += sum.built; + // Ranked by its run, which is the member's own: a build a group + // shares belongs to no one member. + memberTimes.emplace_back(mp, sum.buildGroup >= 0 ? sum.runMs : sum.elapsedMs); + auto secs = static_cast(sum.buildGroup >= 0 ? sum.runMs : sum.elapsedMs) / 1000.0; + const char* took = sum.buildGroup >= 0 ? "run " : ""; + const char* in = sum.buildGroup >= 0 ? ", " : " in "; + if (r == 2 && sum.failed == 0 && sum.notRun > 0) { + // Built and not executed (#544): the member did not fail, and + // it did not pass. 2 outranks 0 and yields to 1, as it does + // for a single member. + if (rc == 0) rc = 2; + unrunnable.push_back(mp); + mcpp::ui::status("Workspace", + std::format("member '{}' ({}/{}) NOT RUN — {} passed, {} not run{}{}{:.2f}s", + mp, idx, members.size(), sum.passed, sum.notRun, in, took, secs)); + } else if (r != 0) { + rc = r; + failed.push_back(mp); + mcpp::ui::status("Workspace", + std::format("member '{}' ({}/{}) FAILED — {} passed, {} failed{}{}{:.2f}s", + mp, idx, members.size(), sum.passed, sum.failed, in, took, secs)); + } else { + // Under `--no-run` nothing passed and nothing was meant to: + // reporting "0 passed" for a member whose tests all built is + // the same sentence a member with no tests would produce. + mcpp::ui::status("Workspace", + sum.built + ? std::format("member '{}' ({}/{}) ok — {} built, not run{}{}{:.2f}s", + mp, idx, members.size(), sum.built, in, took, secs) + : std::format("member '{}' ({}/{}) ok — {} passed{}{}{:.2f}s", + mp, idx, members.size(), sum.passed, in, took, secs)); + } + }; + + if (to.list) { + // A listing builds nothing, so there is nothing to plan once: each + // member lists its own tests. + for (std::size_t i = 0; i < members.size(); ++i) { + if (!member_begin(i, members[i])) continue; + mcpp::build::BuildOverrides mo = ov; + mo.package_filter = members[i]; + mcpp::build::TestRunSummary sum; + int r = mcpp::build::run_tests(passthrough, mo, to, &sum); + member_end(i, members[i], r, sum); + } } else { - // Under `--no-run` nothing passed and nothing was meant to: - // reporting "0 passed" for a member whose tests all built is - // the same sentence a member with no tests would produce. - mcpp::ui::status("Workspace", - sum.built - ? std::format("member '{}' ({}/{}) ok — {} built, not run{}{}{:.2f}s", - mp, idx, members.size(), sum.built, in, took, secs) - : std::format("member '{}' ({}/{}) ok — {} passed{}{}{:.2f}s", - mp, idx, members.size(), sum.passed, in, took, secs)); + // Each member's own tests, discovered from the member's own + // directory: two members may each have a `tests/main.cpp`. + std::vector inputs; + for (auto const& mp : members) { + mcpp::build::WorkspaceTestMember wm; + wm.path = mp; + auto d = discover_member_tests(sel.root, mp); + if (!d) { + wm.error = std::format("{}: {}", mp, d.error()); + } else { + wm.targets = std::move(d->targets); + if (wm.targets.empty()) { + // Names where it looked when the manifest chose the + // place, so that a glob that matches nothing is not + // read as a project without tests. + if (d->discoverDeclared) { + std::string globs; + for (auto const& g : d->discover) + globs += std::format("{}\"{}\"", globs.empty() ? "" : ", ", g); + wm.noTests = std::format("no tests found ([test] discover = [{}])", globs); + } else { + wm.noTests = "no tests found in tests/"; + } + } + } + inputs.push_back(std::move(wm)); + } + // Members whose root-position values are equal are planned together, + // as `mcpp build` plans them. A member whose manifest cannot be read + // has no configuration: each member is then its own group, and it + // fails alone when it is planned. + std::vector> groups; + if (auto g = workspace_groups(sel.root, members, ov.target_triple)) groups = std::move(*g); + else for (auto const& mp : members) groups.push_back({mp}); + + mcpp::build::WorkspaceTestHooks hooks; + hooks.begin = member_begin; + hooks.end = member_end; + mcpp::build::run_workspace_tests(passthrough, ov, to, sel.root, groups, + std::move(inputs), hooks); } - }; - if (to.list) { - // A listing builds nothing, so there is nothing to plan once: each - // member lists its own tests. - for (std::size_t i = 0; i < members.size(); ++i) { - if (!member_begin(i, members[i])) continue; - mcpp::build::BuildOverrides mo = ov; - mo.package_filter = members[i]; - mcpp::build::TestRunSummary sum; - int r = mcpp::build::run_tests(passthrough, mo, to, &sum); - member_end(i, members[i], r, sum); - } - } else { - // Each member's own tests, discovered from the member's own - // directory: two members may each have a `tests/main.cpp`. - std::vector inputs; - for (auto const& mp : members) { - mcpp::build::WorkspaceTestMember wm; - wm.path = mp; - auto d = discover_member_tests(sel.root, mp); - if (!d) { - wm.error = std::format("{}: {}", mp, d.error()); - } else { - wm.targets = std::move(d->targets); - if (wm.targets.empty()) { - // Names where it looked when the manifest chose the - // place, so that a glob that matches nothing is not - // read as a project without tests. - if (d->discoverDeclared) { - std::string globs; - for (auto const& g : d->discover) - globs += std::format("{}\"{}\"", globs.empty() ? "" : ", ", g); - wm.noTests = std::format("no tests found ([test] discover = [{}])", globs); - } else { - wm.noTests = "no tests found in tests/"; + auto wsElapsed = ws_ms(); + if (!notRun.empty()) rc = rc ? rc : 1; + + if (json) { + // Member paths are manifest-authored strings, so escape rather + // than assume: one backslash in a member path would otherwise emit + // a stream that is not JSON at all. + auto join = [](const std::vector& v) { + std::string s; + for (auto& x : v) { + if (!s.empty()) s += ','; + s += '"'; + for (char c : x) { + if (c == '"' || c == '\\') s += '\\'; + s += c; } + s += '"'; } - } - inputs.push_back(std::move(wm)); + return s; + }; + // `not_run` keeps its meaning (members the --workspace-timeout + // stopped before they started); `tests_not_run` and + // `unrunnable_members` are #544's — tests that were built and not + // executed, and the members all of whose tests were. + std::println("{{\"workspace_summary\":{{\"members\":{},\"passed\":{},\"failed\":{}," + "\"tests_not_run\":{},\"tests_built\":{}," + "\"failed_members\":[{}],\"unrunnable_members\":[{}]," + "\"not_run\":[{}],\"elapsed_ms\":{}}}}}", + members.size(), totalPassed, totalFailed, totalNotRun, + totalBuilt, + join(failed), join(unrunnable), join(notRun), wsElapsed); + std::fflush(stdout); + return rc; } - // Members whose root-position values are equal are planned together, - // as `mcpp build` plans them. A member whose manifest cannot be read - // has no configuration: each member is then its own group, and it - // fails alone when it is planned. - std::vector> groups; - if (auto g = workspace_groups(sel.root, members, ov.target_triple)) groups = std::move(*g); - else for (auto const& mp : members) groups.push_back({mp}); - - mcpp::build::WorkspaceTestHooks hooks; - hooks.begin = member_begin; - hooks.end = member_end; - mcpp::build::run_workspace_tests(passthrough, ov, to, sel.root, groups, - std::move(inputs), hooks); - } - auto wsElapsed = ws_ms(); - if (!notRun.empty()) rc = rc ? rc : 1; + // Slowest members, printed unconditionally rather than only on failure: + // "which member ate the wall clock" is the question a green-but-slow CI + // run raises, and answering it used to mean reverse-engineering log + // timestamps. + std::ranges::sort(memberTimes, [](auto& a, auto& b) { return a.second > b.second; }); + std::string slowest; + for (std::size_t i = 0; i < memberTimes.size() && i < 3; ++i) { + if (memberTimes[i].second < 1000) break; + if (!slowest.empty()) slowest += ", "; + slowest += std::format("{} {:.1f}s", memberTimes[i].first, + static_cast(memberTimes[i].second) / 1000.0); + } - if (json) { - // Member paths are manifest-authored strings, so escape rather - // than assume: one backslash in a member path would otherwise emit - // a stream that is not JSON at all. - auto join = [](const std::vector& v) { + auto join_names = [](const std::vector& v) { std::string s; - for (auto& x : v) { - if (!s.empty()) s += ','; - s += '"'; - for (char c : x) { - if (c == '"' || c == '\\') s += '\\'; - s += c; - } - s += '"'; - } + for (auto& f : v) { if (!s.empty()) s += ", "; s += f; } return s; }; - // `not_run` keeps its meaning (members the --workspace-timeout - // stopped before they started); `tests_not_run` and - // `unrunnable_members` are #544's — tests that were built and not - // executed, and the members all of whose tests were. - std::println("{{\"workspace_summary\":{{\"members\":{},\"passed\":{},\"failed\":{}," - "\"tests_not_run\":{},\"tests_built\":{}," - "\"failed_members\":[{}],\"unrunnable_members\":[{}]," - "\"not_run\":[{}],\"elapsed_ms\":{}}}}}", - members.size(), totalPassed, totalFailed, totalNotRun, - totalBuilt, - join(failed), join(unrunnable), join(notRun), wsElapsed); - std::fflush(stdout); + // The not-run count is in the line whenever it is non-zero, at the + // same weight as the failure count (#544): a member whose tests were + // built and not executed must not read as a passing member. + std::string notRunCounts = totalNotRun + ? std::format("; {} not run", totalNotRun) : std::string{}; + if (totalBuilt) + notRunCounts += std::format("; {} built, not run", totalBuilt); + if (failed.empty() && notRun.empty() && unrunnable.empty()) + mcpp::ui::result("workspace result", + std::format("ok. {} member(s); {} passed; 0 failed{}; finished in {:.2f}s", + members.size(), totalPassed, notRunCounts, + static_cast(wsElapsed) / 1000.0)); + else + mcpp::ui::error(std::format( + "workspace test: {}/{} member(s) failed; {} passed; {} failed{}; " + "finished in {:.2f}s", + failed.size(), members.size(), totalPassed, totalFailed, notRunCounts, + static_cast(wsElapsed) / 1000.0)); + if (!failed.empty()) + mcpp::ui::plain(std::format(" failed members: {}", join_names(failed))); + if (!unrunnable.empty()) + mcpp::ui::plain(std::format(" not run (no runner on this host): {}", + join_names(unrunnable))); + if (!notRun.empty()) + mcpp::ui::plain(std::format( + " not run (--workspace-timeout {}s reached): {}", + workspaceTimeoutSecs, join_names(notRun))); + if (!slowest.empty()) + mcpp::ui::plain(std::format(" slowest: {}", slowest)); return rc; + }; + if (testLegs.empty()) return test_members(ov); + int legsRc = 0; + for (auto const& l : testLegs) { + auto lov = ov; + lov.target_triple = l.triple; + mcpp::ui::info("Testing", std::format("on {}", l.triple)); + const int r = test_members(std::move(lov)); + if (r == 1 || (r != 0 && legsRc == 0)) legsRc = r; } - - // Slowest members, printed unconditionally rather than only on failure: - // "which member ate the wall clock" is the question a green-but-slow CI - // run raises, and answering it used to mean reverse-engineering log - // timestamps. - std::ranges::sort(memberTimes, [](auto& a, auto& b) { return a.second > b.second; }); - std::string slowest; - for (std::size_t i = 0; i < memberTimes.size() && i < 3; ++i) { - if (memberTimes[i].second < 1000) break; - if (!slowest.empty()) slowest += ", "; - slowest += std::format("{} {:.1f}s", memberTimes[i].first, - static_cast(memberTimes[i].second) / 1000.0); - } - - auto join_names = [](const std::vector& v) { - std::string s; - for (auto& f : v) { if (!s.empty()) s += ", "; s += f; } - return s; - }; - // The not-run count is in the line whenever it is non-zero, at the - // same weight as the failure count (#544): a member whose tests were - // built and not executed must not read as a passing member. - std::string notRunCounts = totalNotRun - ? std::format("; {} not run", totalNotRun) : std::string{}; - if (totalBuilt) - notRunCounts += std::format("; {} built, not run", totalBuilt); - if (failed.empty() && notRun.empty() && unrunnable.empty()) - mcpp::ui::result("workspace result", - std::format("ok. {} member(s); {} passed; 0 failed{}; finished in {:.2f}s", - members.size(), totalPassed, notRunCounts, - static_cast(wsElapsed) / 1000.0)); - else - mcpp::ui::error(std::format( - "workspace test: {}/{} member(s) failed; {} passed; {} failed{}; " - "finished in {:.2f}s", - failed.size(), members.size(), totalPassed, totalFailed, notRunCounts, - static_cast(wsElapsed) / 1000.0)); - if (!failed.empty()) - mcpp::ui::plain(std::format(" failed members: {}", join_names(failed))); - if (!unrunnable.empty()) - mcpp::ui::plain(std::format(" not run (no runner on this host): {}", - join_names(unrunnable))); - if (!notRun.empty()) - mcpp::ui::plain(std::format( - " not run (--workspace-timeout {}s reached): {}", - workspaceTimeoutSecs, join_names(notRun))); - if (!slowest.empty()) - mcpp::ui::plain(std::format(" slowest: {}", slowest)); - return rc; + return legsRc; } // A PLATFORM PACKAGE IS TESTED ON THE TARGETS IT SERVES (SPEC-013 §16): // with no --target, its tests are built for each of its `targets` with the @@ -1308,18 +1498,18 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, std::unexpected(mcpp::manifest::ManifestError{})); if (m && m->platformDeclared && !m->platform.targets.value.empty()) { mcpp::invocation::set("MCPP_PLATFORM", "path:" + root->generic_string()); - int rc = 0; - for (auto const& t : m->platform.targets.value) { - auto tov = ov; - tov.target_triple = t; - auto tto = to; - tto.unreachableIsBuilt = true; - mcpp::ui::info("Testing", std::format("platform {} on {}", m->package.name, t)); - if (int r = mcpp::build::run_tests(passthrough, tov, tto); r != 0 && rc == 0) rc = r; - } - return rc; + mcpp::ui::info("Testing", std::format("platform {} on {} target(s)", m->package.name, + m->platform.targets.value.size())); + std::vector own; + for (auto const& t : m->platform.targets.value) own.push_back({t, {}}); + if (own.size() > 1) return mcpp::build::run_test_legs(passthrough, ov, to, std::move(own)); + ov.target_triple = own.front().triple; + to.unreachableIsBuilt = true; + return mcpp::build::run_tests(passthrough, ov, to); } } + if (!testLegs.empty()) + return mcpp::build::run_test_legs(passthrough, ov, to, std::move(testLegs)); return mcpp::build::run_tests(passthrough, ov, to); } @@ -1647,6 +1837,14 @@ export int cmd_platform_step(const mcpplibs::cmdline::ParsedArgs& parsed, // `mcpp self env --target ` / `--host` (SPEC-013 §9): one configuration, // prepared and not built -- the toolchain, the platform and its source, the // tools it builds for this machine, the runner and where its program is. +namespace { + +// One configuration of `self env --target`: its JSON object, or its lines. +nlohmann::json env_data(const mcpp::build::BuildContext& ctx, bool host); +void env_print(const mcpp::build::BuildContext& ctx, bool host); + +} // namespace + export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { bool json = false; if (auto f = parsed.value("format")) { @@ -1656,78 +1854,126 @@ export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { } json = true; } - const std::string target = parsed.value("target").value_or(""); const bool host = parsed.is_flag_set("host"); + // THE LEGS (SPEC-015 §3.2): one configuration per leg; several are + // reported in order, the first at the top of the object as one is. + std::vector targets; + if (!host) { + auto legs = target_legs(parsed, {}); + if (!legs) { mcpp::ui::error(legs.error()); return 2; } + for (auto const& l : *legs) targets.push_back(l.triple); + } + if (targets.empty()) targets.push_back({}); const bool wasQuiet = mcpp::ui::is_quiet(); if (json) mcpp::ui::set_quiet(true); - auto ctx = mcpp::build::prepare_configuration(host ? std::string_view{} : std::string_view(target)); + std::vector contexts; + std::string failure; + for (auto const& t : targets) { + auto ctx = mcpp::build::prepare_configuration(host ? std::string_view{} : std::string_view(t)); + if (!ctx) { + failure = targets.size() > 1 ? std::format("the {} leg: {}", t, ctx.error()) : ctx.error(); + break; + } + contexts.push_back(std::move(*ctx)); + } mcpp::ui::set_quiet(wasQuiet); - if (!ctx) { + if (!failure.empty()) { if (json) { mcpp::wire::Envelope env{.kind = "mcpp.env", .effects = {}}; env.data = nullptr; env.diagnostics.push_back({"MCPP_CONFIGURATION_FAILED", mcpp::wire::Severity::Error, - ctx.error()}); + failure}); std::println("{}", mcpp::wire::to_json(env).dump()); } else { - mcpp::ui::error(ctx.error()); + mcpp::ui::error(failure); } return 1; } - auto const& P = host ? ctx->hostPlatform : ctx->plan.platform; - // The graph reported: the host graph's triple and compiler with --host. - const std::string triple = host ? ctx->hostTriple : ctx->tc.targetTriple; - const std::string compiler = host ? ctx->hostCompiler : ctx->tc.binaryPath.generic_string(); - const std::string compilerOrigin = host ? ctx->hostCompilerOrigin : ctx->compilerChoice.origin; - auto const& runner = P.fields.runner; - std::string runnerProgram; - // The target's: what `mcpp run` and `mcpp test` would start here, by - // their own rule (SPEC-013 §10) -- the runner's program and its - // architecture, the execution host's program, or why runs are refused. + if (json) { + mcpp::wire::Envelope env{.kind = "mcpp.env", .effects = {}}; + env.data = env_data(contexts.front(), host); + if (contexts.size() > 1) { + nlohmann::json legs = nlohmann::json::array(); + for (auto const& c : contexts) legs.push_back(env_data(c, host)); + env.data["legs"] = std::move(legs); + } + for (auto const& c : contexts) + for (auto const& n : c.planNotes) + env.diagnostics.push_back({n.code, n.severity, n.message}); + mcpp::wire::emit(env); + return 0; + } + for (std::size_t i = 0; i < contexts.size(); ++i) { + if (i > 0) std::println(""); + env_print(contexts[i], host); + } + return 0; +} + +namespace { + +// The target's: what `mcpp run` and `mcpp test` would start here, by their +// own rule (SPEC-013 §10) -- the runner's program and its architecture, the +// execution host's program, or why runs are refused. With --host, the +// platform's own tool when it is one, else the program the runner names. +struct EnvReach { mcpp::build::RunnerOnThisMachine reach; - reach.arch = std::string(mcpp::toolchain::platform::host_arch); + std::string runnerProgram; +}; +EnvReach env_reach(const mcpp::build::BuildContext& ctx, bool host) { + EnvReach out; + auto const& P = host ? ctx.hostPlatform : ctx.plan.platform; + auto const& runner = P.fields.runner; + out.reach.arch = std::string(mcpp::toolchain::platform::host_arch); if (!host) { - reach = mcpp::build::runner_on_this_machine(*ctx); - runnerProgram = reach.program; + out.reach = mcpp::build::runner_on_this_machine(ctx); + out.runnerProgram = out.reach.program; } else if (runner.set && !runner.cmd.empty()) { - // The platform's own tool when it is one (located by the build that - // used it), else the program the runner names on this machine. if (auto it = P.tools.find(runner.cmd.front()); it != P.tools.end()) { - runnerProgram = it->second; + out.runnerProgram = it->second; } else { auto found = mcpp::build::runner_lookup::locate( - runner.cmd.front(), ctx->xlingsDepBinDirs, + runner.cmd.front(), ctx.xlingsDepBinDirs, mcpp::invocation::get("PATH").value_or("")); - if (found.program) runnerProgram = found.program->generic_string(); + if (found.program) out.runnerProgram = found.program->generic_string(); } } + return out; +} + +nlohmann::json env_data(const mcpp::build::BuildContext& ctx, bool host) { + auto const& P = host ? ctx.hostPlatform : ctx.plan.platform; + auto const& runner = P.fields.runner; + const auto r = env_reach(ctx, host); auto or_null = [](const std::string& v) { return v.empty() ? nlohmann::json(nullptr) : nlohmann::json(v); }; - if (json) { - nlohmann::json tools = nlohmann::json::object(); - for (auto const& [name, path] : P.tools) tools[name] = path; - nlohmann::json data{ - {"triple", triple}, - {"graph", host ? "host" : "target"}, - {"toolchain", {{"compiler", compiler}, {"origin", compilerOrigin}}}, - {"platform", nlohmann::json::parse(mcpp::platform::to_json(P), nullptr, false)}, - {"tools", std::move(tools)}, - {"runner", !runner.set ? nlohmann::json(nullptr) : nlohmann::json{ - {"argv", runner.cmd}, {"host", runner.host}, {"exclusive", runner.exclusive}, - {"program", or_null(runnerProgram)}, - {"arch", reach.arch}, {"built", reach.built}, - {"refusal", or_null(reach.refusal)}, - {"source", mcpp::platform::origin_text(runner.origin)}}}, - {"runUnder", ctx->runUnder}, - {"runUnderProgram", or_null(reach.runUnderProgram)}, - }; - mcpp::wire::Envelope env{.kind = "mcpp.env", .effects = {}}; - env.data = std::move(data); - for (auto const& n : ctx->planNotes) - env.diagnostics.push_back({n.code, n.severity, n.message}); - mcpp::wire::emit(env); - return 0; - } - for (auto const& n : ctx->planNotes) mcpp::ui::warning(n.message); + nlohmann::json tools = nlohmann::json::object(); + for (auto const& [name, path] : P.tools) tools[name] = path; + return nlohmann::json{ + {"triple", host ? ctx.hostTriple : ctx.tc.targetTriple}, + {"graph", host ? "host" : "target"}, + {"toolchain", {{"compiler", host ? ctx.hostCompiler : ctx.tc.binaryPath.generic_string()}, + {"origin", host ? ctx.hostCompilerOrigin : ctx.compilerChoice.origin}}}, + {"platform", nlohmann::json::parse(mcpp::platform::to_json(P), nullptr, false)}, + {"tools", std::move(tools)}, + {"runner", !runner.set ? nlohmann::json(nullptr) : nlohmann::json{ + {"argv", runner.cmd}, {"host", runner.host}, {"exclusive", runner.exclusive}, + {"program", or_null(r.runnerProgram)}, + {"arch", r.reach.arch}, {"built", r.reach.built}, + {"refusal", or_null(r.reach.refusal)}, + {"source", mcpp::platform::origin_text(runner.origin)}}}, + {"runUnder", ctx.runUnder}, + {"runUnderProgram", or_null(r.reach.runUnderProgram)}, + }; +} + +void env_print(const mcpp::build::BuildContext& ctx, bool host) { + auto const& P = host ? ctx.hostPlatform : ctx.plan.platform; + auto const& runner = P.fields.runner; + const auto r = env_reach(ctx, host); + const std::string triple = host ? ctx.hostTriple : ctx.tc.targetTriple; + const std::string compiler = host ? ctx.hostCompiler : ctx.tc.binaryPath.generic_string(); + const std::string compilerOrigin = host ? ctx.hostCompilerOrigin : ctx.compilerChoice.origin; + for (auto const& n : ctx.planNotes) mcpp::ui::warning(n.message); std::println("{:<10} {}", host ? "host" : "target", triple); std::println("{:<10} {} ({})", "compiler", compiler, compilerOrigin); std::println("{:<10} {}{}", "platform", P.display(), @@ -1742,30 +1988,30 @@ export int cmd_env_configuration(const mcpplibs::cmdline::ParsedArgs& parsed) { for (auto const& a : runner.cmd) argv += (argv.empty() ? "" : " ") + a; // Another architecture's program (for an execution host) says so. std::string arch; - if (reach.arch != mcpp::toolchain::platform::host_arch) - arch = std::format(" ({}{})", reach.arch, reach.built ? "" : "; built by the first run"); + if (r.reach.arch != mcpp::toolchain::platform::host_arch) + arch = std::format(" ({}{})", r.reach.arch, r.reach.built ? "" : "; built by the first run"); std::println("{:<10} {} ({}){}{}", "runner", argv, mcpp::platform::origin_text(runner.origin), - runnerProgram.empty() ? std::string(" -- not found on this machine") - : " -> " + runnerProgram, arch); + r.runnerProgram.empty() ? std::string(" -- not found on this machine") + : " -> " + r.runnerProgram, arch); } - if (!ctx->runUnder.empty()) { + if (!ctx.runUnder.empty()) { std::string argv; - for (auto const& a : ctx->runUnder) argv += (argv.empty() ? "" : " ") + a; - std::println("{:<10} {} ({}){}", "run_under", argv, ctx->runUnderOrigin, - reach.runUnderProgram.empty() ? std::string() - : " -> " + reach.runUnderProgram); + for (auto const& a : ctx.runUnder) argv += (argv.empty() ? "" : " ") + a; + std::println("{:<10} {} ({}){}", "run_under", argv, ctx.runUnderOrigin, + r.reach.runUnderProgram.empty() ? std::string() + : " -> " + r.reach.runUnderProgram); } // Runs refused on this machine are said here, with the key that changes // it, as `mcpp run` would say it. - if (!reach.refusal.empty()) { - std::string text = reach.refusal; + if (!r.reach.refusal.empty()) { + std::string text = r.reach.refusal; for (std::size_t at = text.find('\n'); at != std::string::npos; at = text.find('\n', at + 12)) text.insert(at + 1, std::string(11, ' ')); std::println("{:<10} refused on this machine: {}", "run", text); } - return 0; } +} // namespace // `mcpp toolchain install --target ` with no compiler named installs THE // CONFIGURATION (SPEC-013 §7): the toolchain (already installed by the diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 0fcb96b9c..71e2f0ccf 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -133,10 +133,6 @@ export int build_and_pack_library(const std::string& targetName, // packages' lines and the status row as it runs, and one `Finished` for // the legs together, after the last of them. const bool reporting = mcpp::build::progress::is_open(); - if (reporting && legs.size() > 1) { - mcpp::build::progress::configurations(legs.size()); - mcpp::build::progress::defer_finished(); - } // A library's package is a tree, a tar or a zip; a single file runs a // program, and a library has none. @@ -161,8 +157,12 @@ export int build_and_pack_library(const std::string& targetName, // are printable. std::vector declaredPlatforms; + // EVERY LEG IS PLANNED, ONE AFTER ANOTHER, AND THEN BUILT AT ONCE + // (mcpp.build.legs); each leg's artifact is then taken from its plan. + std::vector specs; for (auto const& want : legs) { - mcpp::build::BuildOverrides ov; + mcpp::build::legs::Spec spec; + auto& ov = spec.overrides; ov.target_triple = want; // A packaged artifact leaves this machine, so the fallback is release // rather than the interactive "dev". `[build] default-profile` still @@ -171,18 +171,59 @@ export int build_and_pack_library(const std::string& targetName, ov.no_lto_in_archives = true; ov.profile_fallback = "release"; ov.features = opts.features; - auto ctx = mcpp::build::prepare_build(false, /*includeDevDeps=*/false, {}, ov); - if (!ctx) { mcpp::ui::error(ctx.error()); return 2; } - // P5 (mcpp#797): a build plugin is distributed as source; its BMIs - // are the build program's toolchain's and travel nowhere. - if (ctx->buildPluginRoot) { - mcpp::ui::error(std::format( - "'{}' is a build plugin (its library imports mcpp.core) and is distributed as " - "source: build programs compile it with their own toolchain. Publish it " - "(`mcpp publish`) instead of packing it.", ctx->manifest.package.name)); + specs.push_back(std::move(spec)); + } + auto planned = mcpp::build::legs::plan(std::move(specs)); + if (!planned) { mcpp::ui::error(planned.error()); return 2; } + // P5 (mcpp#797): a build plugin is distributed as source; its BMIs + // are the build program's toolchain's and travel nowhere. + if (auto const& first = planned->legs.front().ctx; first.buildPluginRoot) { + mcpp::ui::error(std::format( + "'{}' is a build plugin (its library imports mcpp.core) and is distributed as " + "source: build programs compile it with their own toolchain. Publish it " + "(`mcpp publish`) instead of packing it.", first.manifest.package.name)); + return 2; + } + // The target is named before anything is built: a name the package does + // not have is refused as the package's, not after a build of every leg. + { + auto const& first = planned->legs.front().ctx; + const auto& subject = first.workspaceMembers.size() == 1 + ? first.workspaceMembers.front().manifest : first.manifest; + if (std::ranges::none_of(subject.targets, [&](auto const& t) { return t.name == targetName; })) { + std::string names; + for (auto const& t : subject.targets) names += std::format("{}{}", names.empty() ? "" : ", ", t.name); + mcpp::ui::error(std::format("no target named '{}' in this package{}{}", + targetName, names.empty() ? "" : "; available: ", names)); return 2; } + } + { + std::vector contexts; + for (auto& l : planned->legs) contexts.push_back(&l.ctx); + mcpp::build::progress::programs_done(); + auto built = mcpp::build::legs::build(contexts, [&](mcpp::build::BuildContext& c) { + auto be = mcpp::build::make_ninja_backend(); + mcpp::build::BuildOptions bo; + if (auto br = be->build(c.plan, bo); !br) { + // The compiler's own output, not just "build failed". `mcpp + // build` has always printed this; `mcpp pack` dropped it, so a + // failure inside the packer's build arrived as three words and + // CI logs had nothing to go on. + mcpp::build::report_failed_drive(br.error()); + return 1; + } + mcpp::build::populate_dependency_cache(c); + if (reporting) + mcpp::build::progress::finished( + c.profile, mcpp::build::profile_descriptor(c.plan.manifest.buildConfig)); + return 0; + }); + if (int r = mcpp::build::legs::first_failure(built)) return r; + } + for (auto& plannedLeg : planned->legs) { + auto* ctx = &plannedLeg.ctx; // ── the target, and what its kind means here ────────────────── // In a workspace plan the package is its one selected member // (workspace design 2026-09-29 §15). @@ -211,26 +252,11 @@ export int build_and_pack_library(const std::string& targetName, return t ? t->str() : ctx->tc.targetTriple; }(); - // ── build, then take the artifact FROM THE PLAN ─────────────── + // ── the artifact, FROM THE PLAN ─────────────────────────────── // // Never a glob over `target/`: that directory holds one subtree per // fingerprint, and picking one by name or by mtime silently selects a // stale binary. The link unit knows its own output. - auto be = mcpp::build::make_ninja_backend(); - mcpp::build::progress::programs_done(); - mcpp::build::BuildOptions bo; - if (auto br = be->build(ctx->plan, bo); !br) { - // The compiler's own output, not just "build failed". `mcpp build` - // has always printed this; `mcpp pack` dropped it, so a failure - // inside the packer's build arrived as three words and CI logs had - // nothing to go on. - mcpp::build::report_failed_drive(br.error()); - return 1; - } - mcpp::build::populate_dependency_cache(*ctx); - if (reporting) - mcpp::build::progress::finished( - ctx->profile, mcpp::build::profile_descriptor(ctx->plan.manifest.buildConfig)); // Everything below reads the package being packed. const auto kept = *target; mcpp::build::focus_on_member(*ctx); @@ -462,7 +488,6 @@ export int build_and_pack_library(const std::string& targetName, }); mcpp::ui::status("Packed leg", std::format("{} [{}]", triple, tag.str())); } - if (reporting && legs.size() > 1) mcpp::build::progress::finish_deferred(); // ── does the package cover what it claims? ───────────────────────── // diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index c4c72432c..6cec35820 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -165,8 +165,11 @@ build_extra_android_legs(const std::string& targetName, const std::string& features) { std::vector out; + // Planned one after another, then built at once (mcpp.build.legs). + std::vector specs; for (auto const& triple : triples) { - mcpp::build::BuildOverrides ov; + mcpp::build::legs::Spec spec; + auto& ov = spec.overrides; ov.target_triple = triple; // A packaged artifact leaves this machine — same fallback as every // other pack leg (`build_and_pack_library`, and the primary leg @@ -174,17 +177,27 @@ build_extra_android_legs(const std::string& targetName, ov.profile = profile; ov.profile_fallback = "release"; ov.features = features; - auto ctx = mcpp::build::prepare_build(false, false, {}, ov); - if (!ctx) { mcpp::ui::error(ctx.error()); return std::nullopt; } - + specs.push_back(std::move(spec)); + } + auto planned = mcpp::build::legs::plan(std::move(specs)); + if (!planned) { mcpp::ui::error(planned.error()); return std::nullopt; } + std::vector contexts; + for (auto& l : planned->legs) contexts.push_back(&l.ctx); + mcpp::build::progress::programs_done(); + auto built = mcpp::build::legs::build(contexts, [](mcpp::build::BuildContext& c) { auto be = mcpp::build::make_ninja_backend(); - mcpp::build::progress::programs_done(); mcpp::build::BuildOptions bo; - if (auto br = be->build(ctx->plan, bo); !br) { + if (auto br = be->build(c.plan, bo); !br) { mcpp::build::report_failed_drive(br.error()); - return std::nullopt; + return 1; } - mcpp::build::populate_dependency_cache(*ctx); + mcpp::build::populate_dependency_cache(c); + return 0; + }); + if (mcpp::build::legs::first_failure(built) != 0) return std::nullopt; + for (auto& planned_leg : planned->legs) { + const auto& triple = planned_leg.triple; + auto* ctx = &planned_leg.ctx; // FROM THE PLAN, never a glob — see the function comment. A // dependency's own `shared` target also contributes a `SharedLibrary` @@ -634,12 +647,15 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { opts.universalName = mcpp::manifest::platform::platform_name( P.package.substr(P.package.find('.') + 1)); // The legs: those the command named, else every target of the - // platform. + // platform. The other legs are planned one after another and + // built at once (mcpp.build.legs); this leg is the pack's own. auto const& legs = opts.legTargets.empty() ? P.fields.targets.value : opts.legTargets; + std::vector specs; for (auto const& target : legs) { auto lt = mcpp::toolchain::triple::parse(target); if (!lt || lt->str() == t->str()) continue; - mcpp::build::BuildOverrides ov = g.ov; + mcpp::build::legs::Spec spec{g.ov}; + auto& ov = spec.overrides; ov.target_triple = lt->str(); // The member's own plan, in a workspace; the project's otherwise. ov.workspace_members.clear(); @@ -648,28 +664,39 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { ov.workspace_members = { m.path }; ov.workspace_request = { m.path }; } - auto legCtx = mcpp::build::prepare_build(false, false, {}, ov); - if (!legCtx) return Refusal{1, about(run, m, std::format( - "the {} leg of the universal package: {}", lt->str(), legCtx.error()))}; + specs.push_back(std::move(spec)); + } + auto planned = mcpp::build::legs::plan(std::move(specs)); + if (!planned) return Refusal{1, about(run, m, planned.error())}; + std::vector contexts; + for (auto& l : planned->legs) contexts.push_back(&l.ctx); + auto built = mcpp::build::legs::build(contexts, [](mcpp::build::BuildContext& c) { auto be = mcpp::build::make_ninja_backend(); mcpp::build::BuildOptions bo; - if (auto br = be->build(legCtx->plan, bo); !br) { + if (auto br = be->build(c.plan, bo); !br) { mcpp::build::report_failed_drive(br.error()); - return Refusal{1, about(run, m, std::format("the {} leg did not build", lt->str()))}; + return 1; } - mcpp::build::populate_dependency_cache(*legCtx); + mcpp::build::populate_dependency_cache(c); + return 0; + }); + for (std::size_t i = 0; i < planned->legs.size(); ++i) { + auto const& leg = planned->legs[i]; + if (built[i] != 0) + return Refusal{1, about(run, m, std::format("the {} leg did not build", leg.triple))}; std::filesystem::path artifact; - for (auto const& lu : legCtx->plan.linkUnits) + for (auto const& lu : leg.ctx.plan.linkUnits) if (lu.targetName == chosenLu->targetName && lu.kind == chosenLu->kind && !lu.dependencyOwned - && (legCtx->workspaceMembers.size() < 2 || lu.memberOf == m.name)) { - artifact = legCtx->outputDir / lu.output; + && (leg.ctx.workspaceMembers.size() < 2 || lu.memberOf == m.name)) { + artifact = leg.ctx.outputDir / lu.output; break; } if (artifact.empty()) return Refusal{1, about(run, m, std::format( - "the {} leg of the universal package built no '{}'", lt->str(), chosenLu->targetName))}; - mcpp::ui::status("Packed leg", lt->str()); - loadedLegs.push_back({ lt->str(), lt->arch, artifact }); + "the {} leg of the universal package built no '{}'", leg.triple, chosenLu->targetName))}; + auto lt = mcpp::toolchain::triple::parse(leg.triple); + mcpp::ui::status("Packed leg", leg.triple); + loadedLegs.push_back({ leg.triple, lt ? lt->arch : std::string{}, artifact }); } } } diff --git a/tests/e2e/963_a_target_set_builds_every_member_and_runs_where_it_is_reached.sh b/tests/e2e/963_a_target_set_builds_every_member_and_runs_where_it_is_reached.sh new file mode 100755 index 000000000..718b557fa --- /dev/null +++ b/tests/e2e/963_a_target_set_builds_every_member_and_runs_where_it_is_reached.sh @@ -0,0 +1,119 @@ +#!/usr/bin/env bash +# requires: llvm python3 +# 963 -- a target set builds every member and runs where it is reached +# (SPEC-015 §1-§3; design 2026-10-29 rev 2 §3). +# +# `--target openkal` names every row of the target table whose system is +# openkal: `x86_64-openkal` and `aarch64-openkal`. A command over a set acts +# once per member, each member a leg: the legs are planned one after another +# and built at once. `mcpp run` runs one leg, the first this machine reaches, +# its own architecture first; `mcpp test` builds every leg, runs the legs this +# machine reaches and reports the others built, not run; a command whose legs +# ran no test fails. On 2026.10.29.1 `--target openkal` is an unknown target. +# +# Criteria: +# A. `mcpp build --target openkal` builds both legs, each under +# target//, and states one `Finished`. +# B. `--message-format json`: the envelope's `data` is the first leg's +# record with `legs` holding both, and every artifact event names its leg. +# C. `mcpp run --target openkal` runs this machine's leg and says which. +# D. `mcpp test --target openkal` runs this machine's leg, reports the other +# built and not run, and exits 0. +# E. With an execution host named for each leg that does not exist, `mcpp +# test --target openkal` runs no test on any leg and exits 1, saying so. +# F. `mcpp self env --target openkal --format json` reports both legs. +# G. Refusals: `none` (no environment), `linux-msvc` (no member), and a set +# in `[build] target`. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -m)" in + x86_64|amd64) HOST_ARCH=x86_64; OTHER=aarch64 ;; + aarch64|arm64) HOST_ARCH=aarch64; OTHER=x86_64 ;; + *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; +esac +HT="$HOST_ARCH-openkal" +OT="$OTHER-openkal" + +mkdir -p "$TMP/app/src" "$TMP/app/tests" +cd "$TMP/app" +printf '[package]\nname = "app"\nversion = "0.1.0"\n' > mcpp.toml +cat > src/main.cpp <<'CPP' +import std; +int main() { std::println("image of the set"); return 0; } +CPP +cat > tests/smoke.cpp <<'CPP' +import std; +int main() { std::vector v{2, 1}; std::ranges::sort(v); return v.front() == 1 ? 0 : 1; } +CPP + +# A +"$MCPP" build --target openkal > a.log 2>&1 || fail "A: mcpp build --target openkal" a.log +for t in "$HT" "$OT"; do + ls target/"$t"/*/bin/app.oke > /dev/null 2>&1 || fail "A: no image for $t" a.log +done +[ "$(grep -c 'Finished' a.log)" = 1 ] || fail "A: not exactly one Finished" a.log +echo "ok: a target set builds every member" + +# B +"$MCPP" build --target openkal --message-format json > b.json 2> b.err || fail "B: json build" b.err b.json +python3 - b.json x86_64-openkal aarch64-openkal <<'PY' || fail "B: the envelope or the events" b.json +import json, sys +lines = [json.loads(l) for l in open(sys.argv[1]) if l.strip()] +env = lines[-1] +assert env["kind"] == "mcpp.build", env["kind"] +legs = env["data"]["legs"] +assert len(legs) == 2, legs +triples = sorted(l["configuration"]["triple"] for l in legs) +assert triples == sorted(sys.argv[2:]), triples +assert env["data"]["configuration"]["triple"] == sys.argv[2], env["data"]["configuration"] +assert env["data"]["status"] == "ok", env["data"]["status"] +arts = [l for l in lines[:-1] if l.get("event") == "artifact"] +assert arts and all("triple" in a for a in arts), arts +PY +echo "ok: the envelope lists the legs, and each artifact event names its leg" + +# C +"$MCPP" run --target openkal > c.log 2>&1 || fail "C: mcpp run --target openkal" c.log +grep -q "image of the set" c.log || fail "C: the program did not run" c.log +grep -q "Leg.*$HT" c.log || fail "C: the run does not say which leg" c.log +echo "ok: run runs this machine's leg" + +# D +"$MCPP" test --target openkal > d.log 2>&1 || fail "D: mcpp test --target openkal" d.log +grep -q "smoke ... ok" d.log || fail "D: this machine's leg did not pass" d.log +grep -q "built, not run" d.log || fail "D: the other leg is not reported built, not run" d.log +grep -q "legs result" d.log || fail "D: no result over the legs" d.log +echo "ok: test runs the legs this machine reaches" + +# E +rc=0 +MCPP_TARGET_X86_64_OPENKAL_RUN_UNDER=/nonexistent/execution-host \ +MCPP_TARGET_AARCH64_OPENKAL_RUN_UNDER=/nonexistent/execution-host \ + "$MCPP" test --target openkal > e.log 2>&1 || rc=$? +[ "$rc" = 1 ] || fail "E: a test over legs that ran nothing exited $rc, not 1" e.log +grep -q "no test ran on any of the 2 legs" e.log || fail "E: the failure does not say so" e.log +echo "ok: no test run on any leg is a failure" + +# F +"$MCPP" self env --target openkal --format json > f.json 2> f.err || fail "F: self env" f.err f.json +python3 -c 'import json,sys; d=json.load(open(sys.argv[1]))["data"]; assert len(d["legs"]) == 2, d' f.json \ + || fail "F: self env does not report both legs" f.json +echo "ok: self env reports each leg" + +# G +rc=0; "$MCPP" build --target none > g1.log 2>&1 || rc=$? +[ "$rc" = 2 ] && grep -q "none-elf" g1.log || fail "G: \`none\` is not refused naming the environments" g1.log +rc=0; "$MCPP" build --target linux-msvc > g2.log 2>&1 || rc=$? +[ "$rc" = 2 ] && grep -q "names no target" g2.log || fail "G: an empty set is not refused" g2.log +mkdir -p "$TMP/bt/src" && cd "$TMP/bt" +printf '[package]\nname = "bt"\nversion = "0.1.0"\n[build]\ntarget = "openkal"\n' > mcpp.toml +printf 'int main() {}\n' > src/main.cpp +rc=0; "$MCPP" build > g3.log 2>&1 || rc=$? +[ "$rc" != 0 ] && grep -q "names a target set" g3.log || fail "G: a set in [build] target is not refused" g3.log +echo "ok: sets are refused where they name nothing, and in [build] target" + +echo "PASS: 963 a target set builds every member and runs where it is reached" diff --git a/tests/e2e/964_tests_built_once_run_from_an_archive.sh b/tests/e2e/964_tests_built_once_run_from_an_archive.sh new file mode 100755 index 000000000..59febe17e --- /dev/null +++ b/tests/e2e/964_tests_built_once_run_from_an_archive.sh @@ -0,0 +1,92 @@ +#!/usr/bin/env bash +# requires: llvm python3 +# 964 -- tests built once run from an archive (SPEC-015 §4; design 2026-10-29 +# rev 2 §3.4). +# +# `mcpp test --no-run --archive ` builds the tests of every leg and +# writes them, with what each needs beside it, as one archive; `mcpp test +# --archive ` in a checkout of the project runs the tests this machine +# reaches and builds nothing. An ok image is the same bytes on every system, +# so a CI builds on one machine and runs on the others. On 2026.10.29.1 +# `--archive` is an unknown option. +# +# Criteria: +# A. `mcpp test --target openkal --no-run --archive t.tar.gz` writes the +# archive; its mcpp-tests.json lists both legs, each with the test. +# B. In a copy of the project without target/, `mcpp test --archive` runs +# this machine's leg, reports the other not run, builds nothing and +# exits 0 (`2` would be a leg it could not establish; the other leg is +# not reached, which is a leg not run, and some test passed). +# C. A test that fails in the archive fails the run (exit 1). +# D. An archive none of whose legs this machine reaches runs nothing and +# exits 1, saying so. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -m)" in + x86_64|amd64) HOST_ARCH=x86_64; OTHER=aarch64 ;; + aarch64|arm64) HOST_ARCH=aarch64; OTHER=x86_64 ;; + *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; +esac +OT="$OTHER-openkal" + +mkdir -p "$TMP/app/src" "$TMP/app/tests" +cd "$TMP/app" +printf '[package]\nname = "app"\nversion = "0.1.0"\n' > mcpp.toml +printf 'import std;\nint main() { std::println("app"); }\n' > src/main.cpp +cat > tests/smoke.cpp <<'CPP' +import std; +int main() { std::vector v{2, 1}; std::ranges::sort(v); return v.front() == 1 ? 0 : 1; } +CPP + +# A +"$MCPP" test --target openkal --no-run --archive "$TMP/t.tar.gz" > a.log 2>&1 || fail "A: --no-run --archive" a.log +[ -s "$TMP/t.tar.gz" ] || fail "A: no archive written" a.log +mkdir -p "$TMP/x" && tar -xzf "$TMP/t.tar.gz" -C "$TMP/x" +python3 - "$TMP/x" <<'PY' || fail "A: the manifest" a.log +import json, pathlib, sys +m = next(pathlib.Path(sys.argv[1]).glob("*/mcpp-tests.json")) +d = json.load(open(m)) +assert d["schema"] == 1 and len(d["legs"]) == 2, d +for leg in d["legs"]: + names = [t["name"] for t in leg["tests"]] + assert "smoke" in names, (leg["triple"], names) + for t in leg["tests"]: + assert (m.parent / t["program"]).is_file(), t["program"] +PY +echo "ok: the archive holds each leg's tests" + +# B: a checkout with nothing built. +mkdir -p "$TMP/co" && cp -R "$TMP/app/src" "$TMP/app/tests" "$TMP/app/mcpp.toml" "$TMP/co/" +cd "$TMP/co" +"$MCPP" test --archive "$TMP/t.tar.gz" > b.log 2>&1 || fail "B: mcpp test --archive" b.log +grep -q "smoke ... ok" b.log || fail "B: this machine's leg did not pass" b.log +grep -q "not run here" b.log || fail "B: the other leg is not reported not run" b.log +# Nothing of the package is built here: no test program and no image of it. +[ -z "$(find target -path '*/.test-archive' -prune -o \( -name 'smoke*' -o -name 'app.oke' \) -print 2>/dev/null)" ] \ + || fail "B: the archive's tests were built again" b.log +echo "ok: an archive's tests run without a build" + +# C: a failing test. +cd "$TMP/app" +printf 'int main() { return 3; }\n' > tests/broken.cpp +"$MCPP" test --target openkal --no-run --archive "$TMP/c.tar.gz" > c1.log 2>&1 || fail "C: archive with a failing test" c1.log +cd "$TMP/co" +rc=0; "$MCPP" test --archive "$TMP/c.tar.gz" > c2.log 2>&1 || rc=$? +[ "$rc" = 1 ] || fail "C: a failing archived test exited $rc, not 1" c2.log +grep -q "broken ... FAIL" c2.log || fail "C: the failure is not named" c2.log +echo "ok: an archived test that fails fails the run" + +# D: only the other architecture. +cd "$TMP/app" && rm -f tests/broken.cpp +"$MCPP" test --target "$OT" --no-run --archive "$TMP/d.tar.gz" > d1.log 2>&1 || fail "D: archive of $OT" d1.log +cd "$TMP/co" +rc=0; "$MCPP" test --archive "$TMP/d.tar.gz" > d2.log 2>&1 || rc=$? +[ "$rc" = 1 ] || fail "D: an archive this machine reaches no leg of exited $rc, not 1" d2.log +grep -q "no test of .* ran on this machine" d2.log || fail "D: the failure does not say so" d2.log +echo "ok: an archive run that runs nothing fails" + +echo "PASS: 964 tests built once run from an archive" diff --git a/tests/unit/test_build_legs.cpp b/tests/unit/test_build_legs.cpp new file mode 100644 index 000000000..c1a452d44 --- /dev/null +++ b/tests/unit/test_build_legs.cpp @@ -0,0 +1,50 @@ +#include + +import std; +import mcpp.build.prepare; +import mcpp.toolchain.triple; + +namespace legs = mcpp::build::legs; + +TEST(BuildLegs, NativeArchitectureFirstTheRestInOrder) { + const auto host = mcpp::toolchain::triple::host_triple().arch; + const std::string other = host == "x86_64" ? "aarch64" : "x86_64"; + std::vector in{other + "-openkal", host + "-openkal", other + "-linux-musl", + host + "-linux-musl"}; + auto out = legs::native_first(in); + EXPECT_EQ(out, (std::vector{host + "-openkal", host + "-linux-musl", + other + "-openkal", other + "-linux-musl"})); +} + +TEST(BuildLegs, FirstFailureIsTheFirstNonZero) { + EXPECT_EQ(legs::first_failure(std::vector{0, 0}), 0); + EXPECT_EQ(legs::first_failure(std::vector{0, 2, 1}), 2); + EXPECT_EQ(legs::first_failure(std::vector{}), 0); +} + +// One directory is one ninja's: work in it runs one item after another, and +// distinct directories run at the same time. +TEST(BuildLegs, OneDirectoryIsWorkedOneItemAtATime) { + std::vector dirs{"a", "b", "a", "b", "a"}; + std::mutex m; + std::map inside, most; + std::atomic total{0}; + auto results = legs::run_by_directory(dirs, [&](std::size_t i) { + const auto key = dirs[i].string(); + { + std::scoped_lock l(m); + most[key] = std::max(most[key], ++inside[key]); + } + std::this_thread::sleep_for(std::chrono::milliseconds(5)); + { + std::scoped_lock l(m); + --inside[key]; + } + ++total; + return static_cast(i); + }); + EXPECT_EQ(total.load(), 5); + EXPECT_EQ(most["a"], 1); + EXPECT_EQ(most["b"], 1); + for (std::size_t i = 0; i < results.size(); ++i) EXPECT_EQ(results[i], static_cast(i)); +} diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 75f097fc7..34aba42dd 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -7069,3 +7069,58 @@ TEST(Manifest, OnlyEntryTargetsStillInferTheLibrary) { std::error_code ec; std::filesystem::remove_all(tmp, ec); } + +// ── [pack] targets and [build] target (SPEC-015 §3.3) ──────────────────────── + +TEST(Manifest, PackTargetsTakeTargetsAndSets) { + auto list = mcpp::manifest::parse_string(R"( +[package] +name = "p" +version = "0.1.0" +[pack] +targets = ["openkal", "x86_64-linux-musl"] +)"); + ASSERT_TRUE(list.has_value()) << list.error().format(); + EXPECT_EQ(list->packConfig.targets, (std::vector{"openkal", "x86_64-linux-musl"})); + auto one = mcpp::manifest::parse_string(R"( +[package] +name = "p" +version = "0.1.0" +[pack] +targets = "linux-musl" +)"); + ASSERT_TRUE(one.has_value()) << one.error().format(); + EXPECT_EQ(one->packConfig.targets, (std::vector{"linux-musl"})); +} + +TEST(Manifest, PackTargetsRefuseWhatIsNeitherATargetNorASet) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "p" +version = "0.1.0" +[pack] +targets = ["not a target"] +)"); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().format().find("neither a target"), std::string::npos); +} + +TEST(Manifest, BuildTargetIsOneTarget) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "p" +version = "0.1.0" +[build] +target = "openkal" +)"); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().format().find("names a target set"), std::string::npos); + auto ok = mcpp::manifest::parse_string(R"( +[package] +name = "p" +version = "0.1.0" +[build] +target = "x86_64-openkal" +)"); + ASSERT_TRUE(ok.has_value()) << ok.error().format(); +} diff --git a/tests/unit/test_target_set.cpp b/tests/unit/test_target_set.cpp new file mode 100644 index 000000000..dfe543eda --- /dev/null +++ b/tests/unit/test_target_set.cpp @@ -0,0 +1,170 @@ +#include + +import std; +import mcpp.toolchain.triple; + +namespace ts = mcpp::toolchain::target_set; +namespace tt = mcpp::toolchain::triple; + +namespace { + +// The engine's own conventions for every candidate, except where the target +// table's platform column names one. +std::expected column_platform(const tt::Triple& t) { + if (auto* row = tt::find_known_target(t)) return std::string(row->platform); + return std::string{}; +} + +std::vector legs_of(std::initializer_list requests) { + std::vector req(requests); + auto r = ts::resolve(req, column_platform); + EXPECT_TRUE(r.has_value()) << (r ? "" : r.error()); + std::vector out; + if (r) for (auto& l : *r) out.push_back(l.triple); + return out; +} + +} // namespace + +// ── recognition: the first segment, not a failed parse ────────────────────── + +TEST(TargetSet, RecognisesSystemWords) { + EXPECT_TRUE(ts::is_set_spelling("openkal")); + EXPECT_TRUE(ts::is_set_spelling("linux-musl")); + EXPECT_TRUE(ts::is_set_spelling("none-eabihf")); + EXPECT_TRUE(ts::is_set_spelling("android")); + EXPECT_FALSE(ts::is_set_spelling("x86_64-openkal")); + EXPECT_FALSE(ts::is_set_spelling("aarch64-linux-musl")); +} + +TEST(TargetSet, VendorFirstSpellingIsATriple) { + // `unknown-linux-gnu` parses as a triple (architecture `unknown`); it is + // not a set, because `unknown` is not a system word. + EXPECT_FALSE(ts::is_set_spelling("unknown-linux-gnu")); + EXPECT_FALSE(ts::is_set_spelling("apple-macos")); + auto legs = legs_of({"unknown-linux-gnu"}); + ASSERT_EQ(legs.size(), 1u); + EXPECT_EQ(legs.front(), "unknown-linux-gnu"); +} + +// ── expansion ──────────────────────────────────────────────────────────────── + +TEST(TargetSet, OpenkalNamesBothArchitectures) { + auto e = ts::expand_set("openkal", column_platform); + ASSERT_TRUE(e.has_value()) << e.error(); + EXPECT_EQ(e->members, (std::vector{"x86_64-openkal", "aarch64-openkal"})); + EXPECT_TRUE(e->platform.starts_with("mcpplibs.openkal-platform@")); +} + +TEST(TargetSet, LinuxMuslLeavesPlannedRowsOut) { + auto e = ts::expand_set("linux-musl", column_platform); + ASSERT_TRUE(e.has_value()) << e.error(); + EXPECT_EQ(e->members, (std::vector{"x86_64-linux-musl", "aarch64-linux-musl"})); + for (auto& m : e->members) EXPECT_NE(m, "riscv64-linux-musl"); +} + +TEST(TargetSet, LinuxFillsGnuAndLeavesAndroidOut) { + auto e = ts::expand_set("linux", column_platform); + ASSERT_TRUE(e.has_value()) << e.error(); + EXPECT_EQ(e->members, (std::vector{"x86_64-linux-gnu", "aarch64-linux-gnu"})); +} + +TEST(TargetSet, WindowsFillsGnu) { + auto e = ts::expand_set("windows", column_platform); + ASSERT_TRUE(e.has_value()) << e.error(); + EXPECT_EQ(e->members, (std::vector{"x86_64-windows-gnu"})); +} + +TEST(TargetSet, AndroidIsTheLinuxAndroidRows) { + auto e = ts::expand_set("android", column_platform); + ASSERT_TRUE(e.has_value()) << e.error(); + EXPECT_EQ(e->members, + (std::vector{"aarch64-linux-android", "x86_64-linux-android"})); +} + +TEST(TargetSet, IosAndItsSimulator) { + auto device = ts::expand_set("ios", column_platform); + ASSERT_TRUE(device.has_value()) << device.error(); + EXPECT_EQ(device->members, (std::vector{"aarch64-ios"})); + auto sim = ts::expand_set("ios-sim", column_platform); + ASSERT_TRUE(sim.has_value()) << sim.error(); + EXPECT_EQ(sim->members, (std::vector{"aarch64-ios-sim", "x86_64-ios-sim"})); +} + +TEST(TargetSet, BareMetalNeedsItsEnvironment) { + auto bare = ts::expand_set("none", column_platform); + ASSERT_FALSE(bare.has_value()); + EXPECT_NE(bare.error().find("none-elf"), std::string::npos); + auto elf = ts::expand_set("none-elf", column_platform); + ASSERT_TRUE(elf.has_value()) << elf.error(); + EXPECT_EQ(elf->members, (std::vector{ + "riscv64-none-elf", "riscv32-none-elf", "aarch64-none-elf", "x86_64-none-elf"})); +} + +TEST(TargetSet, EmptySetIsRefusedNamingThePlannedRows) { + auto e = ts::expand_set("macos-gnu", column_platform); + ASSERT_FALSE(e.has_value()); + auto none = ts::expand_set("linux-msvc", column_platform); + ASSERT_FALSE(none.has_value()); + EXPECT_NE(none.error().find("names no target"), std::string::npos); +} + +TEST(TargetSet, MalformedSetsAreRefused) { + EXPECT_FALSE(ts::expand_set("linux-musl-extra", column_platform).has_value()); + EXPECT_FALSE(ts::expand_set("linux-", column_platform).has_value()); + EXPECT_FALSE(ts::expand_set("openkal-gnu", column_platform).has_value()); + EXPECT_FALSE(ts::expand_set("android-x", column_platform).has_value()); +} + +TEST(TargetSet, MembersSelectingDifferentPlatformsAreRefused) { + auto split = [](const tt::Triple& t) -> std::expected { + return t.arch == "x86_64" ? std::string("a@1") : std::string("b@1"); + }; + auto e = ts::expand_set("openkal", split); + ASSERT_FALSE(e.has_value()); + EXPECT_NE(e.error().find("select different platforms"), std::string::npos); + EXPECT_NE(e.error().find("x86_64-openkal"), std::string::npos); + EXPECT_NE(e.error().find("aarch64-openkal"), std::string::npos); +} + +TEST(TargetSet, ASelectionErrorNamesTheMember) { + auto broken = [](const tt::Triple&) -> std::expected { + return std::unexpected(std::string("no such platform")); + }; + auto e = ts::expand_set("openkal", broken); + ASSERT_FALSE(e.has_value()); + EXPECT_NE(e.error().find("x86_64-openkal: no such platform"), std::string::npos); +} + +// ── several --target ───────────────────────────────────────────────────────── + +TEST(TargetSet, UnionInTableOrderWithoutDuplicates) { + auto legs = legs_of({"aarch64-openkal", "openkal", "x86_64-linux-musl"}); + EXPECT_EQ(legs, (std::vector{ + "x86_64-linux-musl", "x86_64-openkal", "aarch64-openkal"})); +} + +TEST(TargetSet, ATargetKeepsItsSpelling) { + // Planned as the command would plan it alone: `x86_64-linux` is completed + // and reported by the build, not here. + auto legs = legs_of({"x86_64-linux"}); + ASSERT_EQ(legs.size(), 1u); + EXPECT_EQ(legs.front(), "x86_64-linux"); + auto twice = legs_of({"x86_64-linux", "x86_64-linux-gnu"}); + EXPECT_EQ(twice.size(), 1u); +} + +TEST(TargetSet, UnknownTargetsFollowTheTableInWrittenOrder) { + auto legs = legs_of({"zz-custom-target", "x86_64-openkal", "yy-other"}); + EXPECT_EQ(legs, (std::vector{"x86_64-openkal", "zz-custom-target", "yy-other"})); +} + +TEST(TargetSet, LegsRecordTheirSet) { + std::vector req{"openkal", "x86_64-linux-musl"}; + auto r = ts::resolve(req, column_platform); + ASSERT_TRUE(r.has_value()); + for (auto& l : *r) { + if (l.triple.ends_with("openkal")) EXPECT_EQ(l.fromSet, "openkal"); + else EXPECT_TRUE(l.fromSet.empty()); + } +} From bdd9029b86a386ba0a8969376da37537efb62042 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 04:31:54 +0900 Subject: [PATCH 06/20] runs_on: a platform that states no runner hosts is read as every system of each architecture Under openkal-platform 0.17.3 a vendored universal package's runs_on came out empty and emit xpkg refused it (e2e 957). Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/distribution.cppm | 12 ++++++++++-- tests/unit/test_pack_distribution.cpp | 5 +++++ 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 7af8ed548..b79a5d438 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -117,7 +117,8 @@ struct Package { // `runs_on` for `p`, from its legs, the runners it carries (`systems`), its // mode, and the platform's statement of where its runner exists (`runnerHosts`; -// `runnerAnyArch` for a runner that reaches an artifact from any machine). +// `runnerAnyArch` for a runner that reaches an artifact from any machine). A +// platform that states no hosts is read as every system of each architecture. // A native package of one leg runs where its triple's program runs; a // WebAssembly, Android, iOS or bare-metal package is no program of a system and // runs nowhere an index entry could name. @@ -1206,8 +1207,15 @@ void compute_runs_on(Package& p, const std::vector& runnerHosts, bo if (p.mode == "self-contained") for (auto const& s : p.systems) if (archs.contains(arch_of_pair(s))) p.runsOnSelf.push_back(s); + // A platform that does not state where its runner exists is read as + // before runs_on existed: every system, for each architecture of the + // images. Unknown is not "nowhere". + std::vector every; + if (runnerHosts.empty()) + for (auto system : { "linux", "macos", "windows" }) + for (auto const& arch : archs) every.push_back(std::format("{}-{}", system, arch)); std::vector with; - for (auto const& h : runnerHosts) + for (auto const& h : runnerHosts.empty() ? every : runnerHosts) if ((runnerAnyArch || archs.contains(arch_of_pair(h))) && std::ranges::find(p.runsOnSelf, h) == p.runsOnSelf.end()) with.push_back(h); diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index 6806411ae..36db61850 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -660,6 +660,11 @@ TEST(PackDistribution, AnEntryNamesWhereThePackageRuns) { ASSERT_TRUE(v) << v.error(); EXPECT_NE(v->find("deps = { \"xim:okrun\" }"), std::string::npos) << *v; + // A platform that states no hosts for its runner is read as every system + // of each architecture, as before runs_on existed. + d::compute_runs_on(a.pkg, {}, false); + EXPECT_EQ(a.pkg.runsOnWith.at("okrun").size(), 6u); + // The document carries it, and a reader takes it back. auto j = d::to_json(a.pkg); EXPECT_NE(j.find("\"runs_on\""), std::string::npos) << j; From fcadaba3c1356c30f6e16ef890991c324e3956ca Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 05:21:58 +0900 Subject: [PATCH 07/20] SPEC-014 2.0: a pack holds the legs it names -- sets expand, images of a platform with a runner are one tree whatever the number of legs, other programs a package per leg; runs_on in the summary and the envelope; --systems , -, all; --pkg-format lists of one tree's containers; single files cover legs x systems; --merge writes the launchers again; the universal spellings migrated Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/10-pack-and-release.md | 224 ++++++---- docs/24-openkal-cross.md | 11 +- docs/35-platforms.md | 7 +- docs/50-machine-output.md | 6 +- docs/51-supported-versions.md | 3 + docs/specs/README.md | 2 +- docs/specs/packaging.md | 237 ++++++---- docs/specs/platforms.md | 23 +- docs/zh/10-pack-and-release.md | 160 ++++--- docs/zh/24-openkal-cross.md | 6 +- docs/zh/35-platforms.md | 7 +- docs/zh/50-machine-output.md | 6 +- docs/zh/51-supported-versions.md | 3 + modules/manifest/src/compat/registry.cppm | 19 + modules/manifest/src/toml.cppm | 14 +- modules/manifest/src/types.cppm | 26 +- src/cli.cppm | 71 ++- src/cli/cmd_build.cppm | 6 + src/cli/cmd_publish.cppm | 413 ++++++++++++------ src/pack/distribution.cppm | 140 ++++-- src/pack/pack.cppm | 259 ++++++++--- src/pack/pipeline.cppm | 349 ++++++++++++--- ...a_universal_apk_is_two_legs_in_one_tree.sh | 33 +- ...ok_image_is_built_once_and_run_by_okrun.sh | 32 +- .../951_an_ok_image_needs_no_declaration.sh | 14 +- ..._set_is_one_tree_of_every_architecture.sh} | 30 +- ...56_a_single_file_runs_without_unpacking.sh | 9 +- ..._packages_merge_and_describe_themselves.sh | 46 +- ...image_in_a_program_of_its_architecture.sh} | 70 +-- .../e2e/983_a_pack_holds_the_legs_it_names.sh | 123 ++++++ ..._a_set_of_programs_is_a_package_per_leg.sh | 68 +++ ..._list_writes_each_container_of_one_tree.sh | 79 ++++ ...single_files_cover_the_legs_and_systems.sh | 72 +++ ...87_the_universal_spellings_are_migrated.sh | 84 ++++ tests/unit/test_manifest.cpp | 27 ++ tests/unit/test_pack_distribution.cpp | 77 +++- tests/unit/test_pack_modes.cpp | 20 + 37 files changed, 2073 insertions(+), 703 deletions(-) rename tests/e2e/{955_a_universal_package_holds_every_architecture.sh => 955_the_openkal_set_is_one_tree_of_every_architecture.sh} (79%) rename tests/e2e/{961_a_one_target_package_carries_the_image_in_a_program_of_its_architecture.sh => 961_a_single_file_carries_the_image_in_a_program_of_its_architecture.sh} (52%) create mode 100644 tests/e2e/983_a_pack_holds_the_legs_it_names.sh create mode 100644 tests/e2e/984_a_set_of_programs_is_a_package_per_leg.sh create mode 100644 tests/e2e/985_a_format_list_writes_each_container_of_one_tree.sh create mode 100644 tests/e2e/986_single_files_cover_the_legs_and_systems.sh create mode 100644 tests/e2e/987_the_universal_spellings_are_migrated.sh diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index ab3c726ec..1ed89f6a7 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -63,8 +63,9 @@ A pack is decided by three independent choices and a set of legs (2026.10.27.1+) |---|---|---|---| | linkage | `--linkage` | are the libraries inside the program or beside it | `static`, `shared` | | closure | `--mode` | what the machine that runs the package provides | `system`, `vendored` (default), `self-contained` | -| container | `--pkg-format` | what is delivered | `tar`, `zip`, `dir`, `single-file`, a format a package provides; unstated: `zip` for a Windows program, `tar` elsewhere | -| legs | `--target`, repeatable | which targets one package covers | triples | +| container | `--pkg-format` | what is delivered | `tar`, `zip`, `dir`, `single-file`, a format a package provides, or a list of containers of one tree (`tar,okb`); unstated: `zip` for a Windows program, `tar` elsewhere | +| legs | `--target`, repeatable | which targets the pack covers | targets and target sets (`x86_64-openkal`, `openkal`, `linux-musl`) | +| systems | `--systems` | which systems a self-contained package's runners, or single files, are for | `linux`, `linux-aarch64`, `all` | - **Linkage** is a property of the program. `--linkage static|shared` (or `[pack] linkage`) is the build's dependency linkage for the pack, over @@ -77,8 +78,23 @@ A pack is decided by three independent choices and a set of legs (2026.10.27.1+) closure travels with the program. `--mode static` is shorthand for `--linkage static --mode self-contained`. - **Container** is the shape of the delivery, described below. -- **Legs**: each `--target` is one build, staged into the same tree. A platform - that declares `universal = true` covers all of its targets by default. +- **Legs** are the targets the command names (2026.10.30.1+): every `--target`, + else `[pack] targets`, else the build's target. A target set (`openkal`, + `linux-musl`) names the target table's rows of its system that the selected + platform serves. A platform never adds a leg. The legs are merged by what the + target is: + + | Target | Several legs | + |---|---| + | `lib`, `shared` | one library package ([12](12-binary-distribution.md)) | + | a program of a platform that states a runner (openkal) | one tree named after the platform, `--openkal` | + | an Android application | one APK tree, `lib//` per leg | + | any other program | one package per leg, each named by its triple | + + A pack that writes more than one file takes a directory for `-o`: a path + ending in `/`, or an existing directory, below which each file keeps its + default name. One openkal leg is said after `Packed`: `only x86_64; + --target openkal or [pack] targets = ["openkal"] packs every architecture`. | Mode | Host must provide | Size | Use case | |---|---|---|---| @@ -113,12 +129,15 @@ choices: | `--compress` with `--pkg-format dir` or a provided format | `--pkg-format tar`, `zip` or `single-file`: only a container is compressed | | `--compress zstd` or `xz` with `--pkg-format zip` | `--compress gzip` (DEFLATE) or `none`, or `--pkg-format tar` | | `--compress gzip` or `xz` with `--pkg-format single-file` | `--compress zstd[:]` or `none`: the launcher decompresses zstd | -| several `--target` for an executable outside a universal package | one pack per target | -| `--systems` on anything but a self-contained universal package or a single file of an image | `--mode self-contained` over a platform with `universal = true`, or `--pkg-format single-file` | -| an image's single file for a system whose row this mcpp builds no program for (`x86_64-macos`, `aarch64-windows-gnu`) | `--systems` naming the systems that are built | -| an image's single file for several `--systems` with `-o` | leave `-o` out: one file per system | -| a universal package of a platform that states no `runner` | `--no-universal` | -| `--target` outside the platform's `targets` in a universal package | a target the platform serves | +| `--systems` on anything but a self-contained tree of images or a single file of an image | `--mode self-contained`, or `--pkg-format single-file`; `[pack] systems` that does not fit is noted and left aside | +| an image's single file for a system named whose row this mcpp builds no program for (`x86_64-macos`, `aarch64-windows-gnu`) | `--systems` naming the pairs that are built; `--systems all` takes those and says the rest | +| more than one file (a package per leg, several single files) with `-o` naming one file | `-o /` | +| several images of a platform that states no `runner`, not as single files | one pack per target, or `--pkg-format single-file` | +| `--target` outside the platform's targets | a target the platform serves | +| a target set that expands to nothing, or whose rows select different platforms | a target, or another set | +| `dir` or `single-file` in a `--pkg-format` list; a list with `-o` naming one file; a list on `mcpp run` | one format, or `-o /` | +| `--mode self-contained` of an image with `okb` as the only format | `--pkg-format tar,okb`: okb holds the images alone, and the runner would be dropped | +| a tree that runs nowhere (`runs_on` empty) | `--mode self-contained --systems ` | | a dispatched format for a `lib` or `shared` target | the library route, [12](12-binary-distribution.md) | | `--pkg-format single-file` for a `lib` or `shared` target | `tar`, `zip` or `dir` | @@ -203,8 +222,10 @@ mcpp pack --pkg-format msi --features installer # activate root-package feature mcpp pack --linkage static # the libraries inside the program mcpp pack --pkg-format single-file # one executable file mcpp pack --pkg-format tar --compress zstd # .tar.zst -mcpp pack --target x86_64-openkal # a universal package: every architecture of the platform -mcpp pack --target x86_64-openkal --mode self-contained --systems linux,macos,windows +mcpp pack --target x86_64-openkal # one leg: the image's tree for x86_64 +mcpp pack --target openkal # every architecture of the platform, one tree +mcpp pack --target openkal --mode self-contained --systems all --pkg-format tar,okb -o dist/ +mcpp pack --target linux-musl -o dist/ # a static package per architecture mcpp pack --merge a.zip --merge b.zip -o all.zip # one package from packages built on several machines mcpp emit xpkg --package all.zip --url https://example.org/v1.0.0 # its xim-pkgindex entry ``` @@ -729,22 +750,16 @@ runs the package must provide: | Mode | The package holds | The machine provides | |---|---|---| -| `system` | `bin/.oke` | `okrun` and every image it names | -| `vendored` (default) | `bin/.oke` and the `.okso` images it names | `okrun` | -| `self-contained` | `bin/` — a program of this machine's system and the image's architecture that carries the image in the platform's carrier section (`.oke` on ELF and COFF, `__OKE,__carried` on Mach-O) and runs it through libokrun — and the `.okso` images beside it | nothing | -| `static` (`--linkage static --mode self-contained`) | `bin/`, a program of this machine's system and the image's architecture that carries one static `.oke` | nothing | - -These are the packages of one target (`--no-universal`, or a platform that is -not universal). `--linkage static` builds the image with no `.okso`. The image -is not recompiled for `self-contained`: mcpp builds the platform's carrier -(`[platform] carrier`, a package and its entry source) as a generated project -under `target/carrier-pack/--/`, places the image's bytes in -the carrier's section, and packs that program as it packs any other — static -(musl) on Linux, with only the system's libraries on macOS and Windows. The -program is of the image's architecture (2026.10.29.1+): on x86_64 Linux, -`--target aarch64-openkal` packs an aarch64 Linux program. A system and -architecture this mcpp builds no program for (`aarch64-windows-gnu`, on a -Windows machine) is refused, naming the row. +| `system` | the tree below: `lib//.oke` and the launchers | `okrun` and every image it names | +| `vendored` (default) | the tree with the `.okso` images the image names | `okrun` | +| `self-contained` | the tree with the runner of each system of `--systems` and each architecture under `libexec/okrun/-/` | nothing | +| `static` (`--linkage static --mode self-contained`) | the self-contained tree of one static `.oke` per architecture | nothing | + +The layout is the same for one leg and for several (2026.10.30.1+). +`--linkage static` builds the image with no `.okso`. A program of this +machine that carries the image in the platform's carrier section (`.oke` on ELF +and COFF, `__OKE,__carried` on Mach-O) is the single file +(`--pkg-format single-file`, below), and only the single file. ### Android: the application object and its closure under `lib/` @@ -772,60 +787,87 @@ own closure. `--mode` does not apply on this row: the device provides exactly the platform libraries, so there is no host set to leave out and no loader to carry. -### A universal package (`[platform] universal`) +### The tree of a platform's images -A platform that declares `universal = true` (openkal-platform does) makes a -pack of any of its targets cover all of them; `--no-universal` packs the -selected target alone. One tree serves every system and every architecture of -the platform: +An image of a platform that states a runner (openkal-platform does) is packed +as one tree, whatever the number of legs: `--target x86_64-openkal` is the tree +of one leg, `hello-0.1.0-x86_64-openkal/`; `--target openkal` is the tree of +both, named after the platform. One tree serves every system and every +architecture it holds: ``` target/dist/hello-0.1.0-openkal/ ├── bin/hello ← POSIX launcher -├── bin/hello.cmd ← Windows launcher -├── lib/x86_64/hello.oke ← one set of images per architecture +├── bin/hello.exe ← Windows launcher (a program) +├── lib/x86_64/hello.oke ← one set of images per leg ├── lib/aarch64/hello.oke ├── libexec/okrun/-/ ← only with --mode self-contained ├── share/… ← resources, once └── mcpp-package.json ``` -The other legs are built during the pack (`Packed leg `). The -launchers are generated from the platform's `runner`: each resolves the package -as the parent of its own directory, picks the architecture directory from -`uname -m` (`x86_64`, `amd64` → `x86_64`; `aarch64`, `arm64` → `aarch64`) or -from `%PROCESSOR_ARCHITECTURE%` / `%PROCESSOR_ARCHITEW6432%`, and starts the -runner on the image there with its own arguments, looking for the runner first -in `libexec//-/` and then on `PATH`. A machine of an -architecture the package does not hold gets exit status 126 and a sentence -naming both. Files `[runtime] deploy` places lie beside each leg's image. -`[pack] universal = false` is `--no-universal`; a platform that states no -`runner` cannot make a universal package, and the pack is refused with -`--no-universal` as the remedy. +The other legs are planned one after another and built at once +(`Packed leg `). The launchers are generated from the platform's +`runner`: the POSIX script resolves the package as the parent of its own +directory, picks the architecture directory from `uname -m` (`x86_64`, `amd64` +→ `x86_64`; `aarch64`, `arm64` → `aarch64`), hands a Git Bash, MSYS or Cygwin +shell to `bin/.exe`, and starts the runner on the image there with its +own arguments, looking for the runner first in `libexec//-/` +and then on `PATH`. The Windows launcher is the engine's program built for +`x86_64-windows-gnu`: it finds the package from its own path, picks the image +of the machine's architecture, and starts the runner the same way; no `.cmd` is +written. A machine of an architecture the package does not hold gets exit +status 126 and a sentence naming both. Files `[runtime] deploy` places lie +beside each leg's image. Under `--mode self-contained` the runner travels: for each system of -`--systems linux,macos,windows` (default: this machine's) and each architecture -of the package, mcpp builds the platform's carrier with nothing carried — a -program that runs the image its first argument names, as `okrun` does — into -`libexec//-/`. Each runner is built on its row -(`-linux-musl`, `-macos`, `-windows-gnu`) over the platform's -hosted arrangement, so the same program results whichever machine packs, with -no SDK or C run-time of that system; a Linux runner is static. A row this mcpp -builds no program for (`x86_64-macos`, planned; `aarch64-windows-gnu`, not in -its table) is left out with a warning, and the launcher there looks for the -runner on `PATH`. An image built on a Windows host is not the bytes of one -built elsewhere (#850): pack universal packages on Linux or macOS. +`--systems` (`linux`, `macos`, `windows`, each for every architecture of the +legs; `linux-aarch64` for one pair; `all`; default: this machine's) mcpp builds +the platform's carrier with nothing carried — a program that runs the image its +first argument names, as `okrun` does — into `libexec//-/`. +Each runner is built on its row (`-linux-musl`, `-macos`, +`-windows-gnu`) over the platform's hosted arrangement, so the same +program results whichever machine packs, with no SDK or C run-time of that +system; a Linux runner is static. A row this mcpp builds no program for +(`x86_64-macos`, planned; `aarch64-windows-gnu`, not in its table) is left out +with a warning. An image built on a Windows host is not the bytes of one built +elsewhere (#850): pack trees on Linux or macOS. + +**Where it runs.** The pack writes `runs_on` into `mcpp-package.json` and says +it after `Packed`: `self`, the `-` pairs the package runs on with +what it carries, and `with.`, the pairs it runs on where the machine +has the runner (the legs' architectures × the platform's `runner.hosts`). + +``` + Packed target/dist/hello-0.1.0-openkal-bundle-all + Runs on linux-aarch64, linux-x86_64, macos-aarch64, windows-x86_64 + Not covered macos-x86_64 (no runner row: this mcpp builds no program for x86_64-macos) +``` + +A tree that runs nowhere — no runner carried and none the platform names — is +refused; over a platform that states no `runner.hosts` this is a note instead. Packages built on several machines are joined with `mcpp pack --merge a.zip --merge b.zip` (also `--merge a.zip,b.zip`). The inputs are package directories or archives (`.zip`, `.tar`, `.tar.gz`, `.tgz`, `.tar.zst`, `.tar.xz`) this command wrote; nothing is built. Legs and systems -are united, each file is taken once, and the result is described anew. Inputs -of another name, version, platform, linkage or mode are refused, and so is one +and `runs_on` are united, each file is taken once, and the result is described +anew. The launchers are derived: written again for the merged legs, so two +one-leg trees merge into the tree of both, named after the platform when `-o` +is left out (`hello-0.1.0-x86_64-openkal` → `hello-0.1.0-openkal`). Inputs of +another name, version, platform, linkage or mode are refused, and so is one path with two digests, naming the path, both digests and both inputs. The container is `--pkg-format`, else the extension of `-o`, else the first input's. +**A list of containers.** `--pkg-format tar,okb` (or `[pack] format = ["tar", +"okb"]`) builds once, stages one tree and writes each container from it, in the +order named. A list holds the containers of one tree — `tar`, `zip` and a +provider's format — never `dir` or `single-file`; it takes a directory for `-o`; +`--compress` applies to its engine containers. okb holds the images alone, so a +self-contained tree whose only format is okb is refused (`--pkg-format +tar,okb`). `mcpp run --pkg-format` takes one format. + ### A single-file package `--pkg-format single-file` delivers one executable file: @@ -854,7 +896,11 @@ the root of a self-contained Linux tree, or `.exe` at the root of a Windows tree — with its arguments, and its exit status is the program's. An ok image's file is `target/dist/---` (`.exe` on -Windows), one per system of `--systems` (default: this machine's). The carrier +Windows), one per leg and system of the leg's architecture (`--systems`; +default: this machine's): `--target openkal --pkg-format single-file -o dist/` +writes one per architecture. `--systems all` makes the file of each row this +mcpp builds a program for and says the others (`Not covered`); more than one +file takes a directory for `-o`. The carrier carries the image alone when it names no other image, and otherwise a stored zip of all the images, each aligned to 64 KiB, which the carrier's entry reads through openkal-run's bundle interface (openkal-platform 0.17.2 and later). @@ -879,31 +925,30 @@ and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it The file is written for every pack, a provided format's included. Each file's `role` is `program`, `image`, `library`, `loader`, `launcher`, `resource` or -`metadata`; in a universal package every file of `lib//` is an image of -that leg. Symbolic links are not listed. `linkage` is `static`, `shared` or -`declared` (each library as it declares itself); `launch` is `program` for one -native leg and `runner` with an `entry` per architecture for a universal -package. +`metadata`; in a tree every file of `lib//` is an image of that leg. +Symbolic links are not listed. `linkage` is `static`, `shared` or `declared` +(each library as it declares itself); `launch` is `program` for one native leg +and `runner` with an `entry` per architecture for a tree; `runs_on` is where the +package runs (`{"self": [...], "with": {"okrun": [...]}}`), and `systems` the +runners it carries. `mcpp emit xpkg --package [--package …] --url [-o ]` writes the xim-pkgindex entry (spec 2) of packages this command wrote, from -their `mcpp-package.json`. A universal package is one asset that `linux`, -`macosx` and `windows` install alike, with the runner as a dependency -(`deps = { "xim:okrun" }`) unless the package carries it; packages of one system +their `mcpp-package.json`. A tree is one asset; each system's block names the +architectures its `runs_on` covers, with the runner as a dependency +(`deps = { "xim:okrun" }`) unless the package carries it, and a tree whose +`runs_on` is empty is refused; packages of one system and architecture each make that system's entry, one URL with `${arch}` and a sha256 per architecture. The install moves the archive's top directory into -place and registers the program with xvm (the `.cmd` launcher on Windows), +place and registers the program with xvm (`bin/.exe` on Windows), from each system's own directory when the packages place the program apart (a Windows program at the root, a POSIX one under `bin/`). Each archive is its -own `--package`. A package of one leg is described for the system and -architecture its program runs on (2026.10.29.1+): the triple's for a Linux, -macOS or Windows target, and for an ok image packed with its carrier -(`--no-universal --mode self-contained`) the carrier's, which the package -records in `systems` (`"systems": ["linux-aarch64"]` for an aarch64 image packed -on Linux). A package whose program is no program of a system is refused, naming -the route that distributes it: a `wasm32-emscripten` package is published as a -page (`dist-web`), an ok image without its carrier as its platform's universal -package, and an Android, iOS or bare-metal package has no entry. +own `--package`. A package of one leg of a Linux, macOS or Windows target is +described for the triple's system and architecture. A package whose program is +no program of a system is refused, naming the route that distributes it: a +`wasm32-emscripten` package is published as a page (`dist-web`), an ok image as +its platform's tree (`--target openkal`) or a single file, and an Android, iOS +or bare-metal package has no entry. ### Compression and signing @@ -916,7 +961,8 @@ stores them (`none`); a single file's payload is `zstd` or `none`; the images inside an okb are stored so that the loader maps them in place. `[pack] sign = ["", …, "{}"]` runs once on each program, image, library -and loader of the staged tree before the container is written, and on each +and loader of the staged tree and on the Windows launcher before the container +is written, and on each single file; `{}` is the file. A failure stops the pack, naming the argv and its output, and `mcpp-package.json` states the digests of the signed files. @@ -929,10 +975,11 @@ common fields are: [pack] default_mode = "static" # override the normal vendored default for bare `mcpp pack` linkage = "shared" # the default of --linkage -format = "tar" # the default of --pkg-format +format = "tar" # the default of --pkg-format; a list: ["tar", "okb"] +targets = ["openkal"] # the default of --target: targets and target sets +systems = ["linux", "macos"] # the default of --systems compress = "zstd" # the default of --compress sign = ["signtool", "sign", "{}"] # run on each program, image, library and loader before the container -universal = false # = --no-universal strip = true # default. false ships the artifacts as built debug_symbols = "dist/debug" # separate the debug info here instead of discarding it include = ["share/**", "config/*.toml"] # project files staged at the same path, once @@ -945,10 +992,15 @@ also_skip = ["libcustom.so"] # libraries assumed to exist on the target s force_bundle = ["libfoo.so"] # bundle even if matched by the PEP 600 list ``` -`[pack].default_mode` currently accepts the established manifest spellings -`static`, `bundle-project`, and `bundle-all`; the `system` mode is selected -explicitly with `mcpp pack --mode system`. CLI input accepts both the canonical -and compatibility names described above. +`[pack].default_mode` accepts the canonical names `system`, `vendored`, +`self-contained` and `static` (2026.10.30.1+) and the established spellings +`bundle-project` and `bundle-all`. + +`[pack] universal`, `[platform] universal` and `--no-universal` no longer decide +what a pack holds (2026.10.30.1+; [51](51-supported-versions.md)): +`[pack] universal = true` is refused with `[pack] targets = ["openkal"]` as the +remedy; `false` and `[platform] universal` are noted and ignored; +`--no-universal` with one target is noted, and with a target set refused. The `static` mode additionally requires a musl toolchain configured under `[target.]`; for the full setup, see the `mcpp.toml` in diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index 71c54480e..56941c93e 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -439,8 +439,8 @@ machine of another architecture `mcpp run` and `mcpp test` refuse the image and name `run_under`, the key that states the execution host. With one named, the runner runs there, and its program is the platform's runner for the image's architecture: mcpp builds the platform's carrier with nothing carried, for this -system and that architecture, which is the program a self-contained universal -package carries under `libexec/okrun/-/`. A user-mode emulator is an +system and that architecture, which is the program a self-contained package of +images carries under `libexec/okrun/-/`. A user-mode emulator is an execution host; on an x86_64 Linux machine with `qemu-aarch64`: ```toml @@ -464,9 +464,10 @@ $ mcpp run --target aarch64-openkal for one run, and `[target.aarch64-openkal] run_under` in `~/.mcpp/config.toml` for one machine. `mcpp self env --target aarch64-openkal` reports the runner's program and its architecture and the execution host's program, or why runs -are refused on this machine. A self-contained package of one image -(`--no-universal --mode self-contained`) is a program of this system and the -image's architecture ([10](10-pack-and-release.md)). +are refused on this machine. A single file of one image +(`--pkg-format single-file`) is a program of this system and the image's +architecture; a self-contained package of it is the image's tree with that +runner ([10](10-pack-and-release.md)). ### ok Images And The Web diff --git a/docs/35-platforms.md b/docs/35-platforms.md index fd852b66f..e0415a21e 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -71,8 +71,8 @@ for this machine and one for an ok image leave each other's entries in place. | the compile | `[platform.build]`, `[platform.build.'cfg(…)']` | flags for every unit of the target | | the link | `[platform.link]` | `driver`, `flags`, `start`, `script`, `inputs`, the names `exe` and `shared`, the `seal` steps | | the run | `runner`, `runners` | how `mcpp run` and `mcpp test` start an artifact, and named device actions | -| the carrier | `carrier` | the program `mcpp pack` builds of a loaded artifact for a system: the self-contained program, a universal package's runner, a single file | -| the package | `universal` | a pack of any target covers all of them, in one package ([10](10-pack-and-release.md)) | +| the carrier | `carrier` | the program `mcpp pack` builds of a loaded artifact for a system: a self-contained tree's runner, a single file; a table or a list of tables, each serving the `-` pairs of its `hosts` | +| where the runner exists | `runner.hosts` | the `-` pairs the runner's program exists for: where a package that needs it runs (`runs_on.with`, [10](10-pack-and-release.md)) | | the code | `plugin` | a build program run in the platform phase, for what depends on the graph | | the group | `[platform.versions]` | one version of each package of a family | | the host part | `[platform.hosted]` | what a host triple takes when it selects the platform | @@ -192,7 +192,6 @@ targets = ["x86_64-openkal", "aarch64-openkal"] toolchain = "llvm@23.1.3" runner = { cmd = ["okrun", "{}"], host = "same-arch" } carrier = { package = "openkal-run@0.1.3", section = ".oke", main = "carrier/main.cpp" } -universal = true plugin = "build.mcpp" [platform.layers] @@ -237,7 +236,7 @@ seal = [["okld", "note", "{}"], ["okld", "check", "{}", "--with", "{needed}"]] phase ([34](34-authoring-a-bsp.md)). - **The carrier's entry reads what it carries.** mcpp places the carried bytes between `mcpp_carried_begin` and `mcpp_carried_end` (the entry's name at - `mcpp_carried_name`): nothing, when the program is a universal package's + `mcpp_carried_name`): nothing, when the program is a self-contained tree's runner and runs the image its first argument names; the image itself; or a stored zip of the images (`PK\x03\x04`, each aligned to 64 KiB) for a single file of an image that names others ([SPEC-013](specs/platforms.md) diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index b559f79b5..b3c22eda4 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -727,10 +727,12 @@ to stderr, including what the build programs and tools the pack starts print. | field | | |---|---| -| `artifacts` | one record per produced artifact: `path` (absolute), `type` (`file` or `directory`), `format` (the container: the `--pkg-format` value; when none is given, `zip` for a Windows program and `tar` otherwise), `targets` (the canonical triple of each leg that went into it) and `sha256` (a file's digest; `null` for a directory; 2026.10.27.1+). A dispatched format reports the terminal outputs of the actions the request introduced; a several-`--target` Android pack reports one artifact whose `targets` lists every leg | +| `artifacts` | one record per produced artifact: `path` (absolute), `type` (`file` or `directory`), `format` (the container: the `--pkg-format` value; when none is given, `zip` for a Windows program and `tar` otherwise), `targets` (the canonical triple of each leg that went into it) and `sha256` (a file's digest; `null` for a directory; 2026.10.27.1+). A dispatched format reports the terminal outputs of the actions the request introduced; a several-`--target` Android pack reports one artifact whose `targets` lists every leg; a pack of a package per leg (2026.10.30.1+) reports each package with its own leg | | `stage` | the tree the artifact was made from: `dir`, `manifest` (the stage manifest below) and `closure` (`walked` or `not-walked`); `null` for a library package, when no tree was staged, and for a pack of several members | | `linkage`, `mode`, `pkgFormat` *(2026.10.27.1+)* | the three choices of the pack ([10](10-pack-and-release.md)): `linkage` is `static`, `shared` or `declared`; `pkgFormat` is the `format` of the artifacts (`tar`, `zip`, `dir`, `single-file`, or a provided format) | -| `legs`, `systems` *(2026.10.27.1+)* | the canonical triple of each leg (a universal package lists every target of its platform), and the `-` of each runner a self-contained universal package carries, or of the carrier that is the program of a self-contained package of one ok image (2026.10.29.1+) | +| `legs`, `systems` *(2026.10.27.1+)* | the canonical triple of each leg the command named (a target set expanded; 2026.10.30.1+), and the `-` of each runner a self-contained tree of images carries | +| `pkgFormats` *(2026.10.30.1+)* | every container written, in the order `--pkg-format` named them; `pkgFormat` is the first | +| `runsOn` *(2026.10.30.1+)* | where the package runs, as its `mcpp-package.json` `runs_on` says: `{"self": ["linux-x86_64", …], "with": {"okrun": […]}}`; a single file's pairs are its files' | | `stages` *(2026.10.1.1+)* | present only for a pack of several members (`--workspace`, or `-p` repeated): one record per member in `[workspace] members` order, with `member` (the qualified package name) and the `dir`, `manifest` and `closure` of `stage` | For a pack of several members `artifacts` lists every member's artifacts, in the diff --git a/docs/51-supported-versions.md b/docs/51-supported-versions.md index 28ecf4f27..fef07b562 100644 --- a/docs/51-supported-versions.md +++ b/docs/51-supported-versions.md @@ -80,6 +80,9 @@ the rows of this table (mcpp 2026.10.11.1+). | `abi-capability`: an `abi:` runtime capability (`abi:glibc`, `abi:libc=`, `abi:cxxstdlib=`), read as a requirement on a target-side layer | 2026.10.26.1 | 2027.1 | `requires = ["mcpp:c-abi="]` or `["mcpp:c++-abi="]` | | `undeclared-cxx-runtime`: a C++ runtime other than the build's own, named on its link line or needed by a library the line names, that no package declares | 2026.10.26.1 | 2027.1 | `requires = [{ layer = "mcpp:c++-abi=", visibility = "private" }]` in the package that needs it | | `plugin-imports-from-build-dependencies`: a build plugin's library importing a package it lists in its own `[build-dependencies]` with `host-module = true` | 2026.10.11.1 | 2027.1 | the package in the plugin's `[dependencies]` | +| `pack-universal`: `[pack] universal` -- `true` refused, `false` ignored; a pack's targets are its legs | 2026.10.30.1 | 2027.1 | `[pack] targets = ["openkal"]` (every architecture), or one `--target` | +| `platform-universal`: a platform's `[platform] universal`, ignored; a pack's targets are its legs | 2026.10.30.1 | 2027.1 | nothing in the platform; the pack names its targets (`--target openkal`) | +| `no-universal-flag`: `mcpp pack --no-universal`, which a pack of one target no longer needs (refused with a target set) | 2026.10.30.1 | 2027.1 | `--target -` (one leg) | ## Reproducing a build diff --git a/docs/specs/README.md b/docs/specs/README.md index 8c35ca1d0..bf45eeef3 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -43,7 +43,7 @@ | [SPEC-011](concurrency.md) | 并发运行：三段、构建目录锁、原子发布、`mcpp.lock` 的乐观写入、取消 | 评审中 v1.0 | 2026-10-26 | mcpp >= 2026.10.26.1 | | [SPEC-012](human-output.md) | 面向人的输出：两条流、两种介质、行的种类、颜色 | 评审中 v1.0 | 2026-10-26 | mcpp >= 2026.10.26.1 | | [SPEC-013](platforms.md) | 平台:`[platform]` 的七组键、版本组与 `hosted`,三种写法与层次,目标图与宿主图的选择,`cfg(platform)`,依赖图的补全,链接、命名与封装,运行器与载体,平台阶段与协议 17 | 评审中 v1.2 | 2026-10-29 | mcpp >= 2026.10.27.1;§10 的执行宿主与 §11 的载体架构 mcpp >= 2026.10.29.1 | -| [SPEC-014](packaging.md) | 打包与分发:链接、闭包、容器三个维度与构建腿,暂存树与 `mcpp-package.json`,通用包、单文件、签名、压缩、合并与分发描述 | 评审中 v1.3 | 2026-10-29 | mcpp >= 2026.10.29.1;逐条标注 | +| [SPEC-014](packaging.md) | 打包与分发:链接、闭包、容器三个维度与构建腿,腿的合并,暂存树与 `mcpp-package.json`,平台的树、运行覆盖、单文件、容器列表、签名、压缩、合并与分发描述 | 评审中 v2.0 | 2026-10-30 | mcpp >= 2026.10.30.1;逐条标注 | ## 文档约定 diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 2e64683a7..3c2fc8c1f 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -3,14 +3,14 @@ | 项 | 值 | |---|---| | 规范编号 | SPEC-014 | -| 标题 | 打包与分发:链接、闭包、容器三个维度与构建腿,暂存树与包元数据,通用包、单文件、签名、压缩、合并与分发描述 | -| 状态 | 评审中 v1.3 | -| 版本 | 1.3 | -| 最后修改 | 2026-10-29 | -| 对应实现 | mcpp >= 2026.10.29.1 | -| 相关设计文档 | `.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`(§2–§7、§14)、`.agents/docs/2026-10-28-platforms-design.md`(§4.7)、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md`(§2.2、§2.6) | -| 相关 issue | mcpp#847、mcpp#849 | -| 相关规范 | [SPEC-007](build-plugins.md)(包格式提供者、artifact action)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-013](platforms.md)(平台、载体) | +| 标题 | 打包与分发:链接、闭包、容器三个维度与构建腿,腿的合并,暂存树与包元数据,运行覆盖、单文件、容器列表、签名、压缩、合并与分发描述 | +| 状态 | 评审中 v2.0 | +| 版本 | 2.0 | +| 最后修改 | 2026-10-30 | +| 对应实现 | mcpp >= 2026.10.30.1 | +| 相关设计文档 | `.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md`(§4、§13)、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`(§2–§7、§14)、`.agents/docs/2026-10-28-platforms-design.md`(§4.7)、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md`(§2.2、§2.6) | +| 相关 issue | mcpp#847、mcpp#849、mcpp#860 | +| 相关规范 | [SPEC-007](build-plugins.md)(包格式提供者、artifact action)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-013](platforms.md)(平台、运行器、载体)、SPEC-015(目标集合与腿) | | 使用文档 | [10 —— 发布打包](../zh/10-pack-and-release.md) | 用语按 RFC 2119:**必须 / 禁止**为强制,**应当**为建议,**可以**为可选。 @@ -21,7 +21,7 @@ `mcpp run --pkg-format` 与 `mcpp emit xpkg` 读取的部分。库的预编译包(docs/12)不在本规范内。 ok 映像是 x86_64 与 aarch64 上位置无关的 ELF,只导入 `kal_*`,运行时需要加载器发布的可执行内存(SPEC-013 §14); -通用包、okb 与映像的单文件只装 ok 映像。Web 经 `wasm32-emscripten` 目标到达,其包为 WebAssembly 的同名文件族 +平台的树(§4.2)、okb 与映像的单文件只装 ok 映像。Web 经 `wasm32-emscripten` 目标到达,其包为 WebAssembly 的同名文件族 (§4.1),发布为页面经 `dist-web` 提供者(§6);以 openkal API 写的代码经 openkal-emscripten 到达 Web。 ## 1. 模型:三个维度与构建腿 @@ -32,8 +32,9 @@ ok 映像是 x86_64 与 aarch64 上位置无关的 ELF,只导入 `kal_*`,运行 |---|---|---|---| | 链接 `--linkage` | `static`、`shared` | 库在程序里,还是在程序旁边 | 程序属性 | | 闭包 `--mode` | `system`、`vendored`、`self-contained` | 运行的机器要提供什么 | 打包属性 | -| 容器 `--pkg-format` | `dir`、`tar`、`zip`、`single-file`,以及提供者的格式 | 交付成什么 | 打包属性 | -| 腿 `--target`(可重复) | 三元组 | 一次打包覆盖哪些目标 | 打包属性 | +| 容器 `--pkg-format` | `dir`、`tar`、`zip`、`single-file`,提供者的格式,或一棵树的容器列表(§5.4) | 交付成什么 | 打包属性 | +| 腿 `--target`(可重复) | 目标或目标集合(SPEC-015) | 一次打包覆盖哪些目标 | 打包属性 | +| 系统 `--systems` | ``、`-`、`all` | 自带的运行器与单文件为哪些系统而做 | 打包属性 | ### 1.1 链接 @@ -49,12 +50,15 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` |---|---|---|---| | `system` | 程序 | 映像 | 全部共享库;openkal 上为加载器与映像所需的 `.okso` | | `vendored`(缺省) | 程序与非系统共享库 | 映像与其 `.okso` | C 库、C++ 运行时与加载器;openkal 上为加载器 | -| `self-contained` | 程序、全部共享库、C 与 C++ 运行时、加载器 | 映像、`.okso` 与程序形式的加载器(载体,SPEC-013 §11):单腿包为本机系统、映像架构的程序(2026.10.29.1),通用包为 `--systems` 的每个系统与每个架构的运行器(§4.4) | 无 | +| `self-contained` | 程序、全部共享库、C 与 C++ 运行时、加载器 | 映像、`.okso` 与 `--systems` 的每个系统 × 腿的每个架构的运行器(载体的运行器形式,SPEC-013 §11;§4.4),腿数不论 | 无 | `bundle-project` 与 `bundle-all` 是 `vendored` 与 `self-contained` 的永久别名。 -`--mode static` 是 `--linkage static --mode self-contained` 的简写:原生目标上为全静态程序;openkal 目标上为一个静态 -`.oke` 加载体。 +`--mode static` 是 `--linkage static --mode self-contained` 的简写:原生目标上为全静态程序;openkal 目标上为每个架构 +一个静态 `.oke` 加运行器的树。把映像嵌入一个程序(载体携带映像,SPEC-013 §11 的 M3)只属于单文件(§5.2): +`--linkage static --pkg-format single-file`。 + +`[pack] default_mode` 接受规范名 `system`、`vendored`、`self-contained`、`static` 与别名 `bundle-project`、`bundle-all`。 ### 1.3 容器 @@ -65,16 +69,29 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` | `zip` | 一个 ZIP 归档,单一顶层目录;条目以 DEFLATE 压缩,`--compress none` 时存储(§5.1) | | `single-file` | 一个可执行文件(§5.2) | | 其他名字 | 依赖图中的提供者以 `pack-format` 声明的格式(§6) | +| 逗号分隔的列表 | 同一棵暂存树的各个容器,依次写出(§5.4) | 未给出 `--pkg-format` 与 `[pack] format` 时,Windows(PE)程序的容器为 `zip`,其余为 `tar`;显式给出的 `tar` -在任何目标上都写 tar。 +在任何目标上都写 tar。`[pack] format` 为字符串或列表。 + +### 1.4 构建腿与腿的合并 + +一次打包的腿为每个 `--target`,未给出时为 `[pack] targets`,都未给出时为构建的目标(`[build] target`,否则本机); +目标集合(`openkal`、`linux-musl`)按 SPEC-015 展开为目标表中所选平台的行。平台不放大腿:打包的腿只是命令所命名的。 +其余的腿逐条规划、一起构建(SPEC-015 §3),在暂存之前完成。多条腿按目标的种类与平台合并: + +| 目标的种类与平台 | 多条腿的结果 | +|---|---| +| `lib`、`shared` | 一个库包(docs/12) | +| 程序,平台声明 `runner`(openkal) | 一棵树(§4.2),包名 `--<平台名>` | +| 程序,Android 的行 | 一个 APK 树(`lib//`) | +| 其他程序 | 每条腿一个包,各以三元组命名;多于一个文件时 `-o` 只能是目录 | -### 1.4 构建腿 +单腿的包名为三元组,模式后缀的规则不变。工作空间多成员的打包接受目标集合,每个成员按上表。一个单腿的 openkal 打包在 +`Packed` 之后写出提示:只含该架构,`--target openkal` 或 `[pack] targets = ["openkal"]` 打平台的全部架构。 -`--target` 可以重复。每条腿是一次构建;各腿暂存进同一棵树(§4)。平台以 `[platform] universal = true` 声明其目标 -默认打**通用包**:对该平台任一目标的打包覆盖平台的全部 `targets`,其余的腿在打包过程中构建;`--no-universal` 或 -`[pack] universal = false` 只打所选目标;单文件不是通用包(§5.2)。一个产物为可执行文件的目标,只有在通用包中或在 -产物为共享对象的行(Android 的 `kind = "app"`)上接受多条腿。 +`-o` 以 `/` 结尾或为已存在的目录时命名一个目录:这次打包写出的每个文件以其缺省名写在其下(`--pkg-format dir` +时 `-o` 仍命名树本身)。 ### 1.5 组合的合法性 @@ -89,11 +106,16 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` | `--pkg-format single-file` 与 WebAssembly 目标(`wasm32-*`,不论目标来自 `--target`、`[build] target` 或其他来源) | 拒绝:单文件是一个系统的可执行文件,WebAssembly 模块由 JavaScript 引擎运行;出路 `--pkg-format tar`、`zip`、`dir` 或提供者的格式(页面为 `dist-web`)(2026.10.29.1) | | 需要宿主提供能力的程序(docs/04 §2.11)与 `--mode self-contained` 或 `static` | 拒绝:携带自己 C 库的包不能使用宿主提供的库;出路 `--mode vendored` | | `cxx_runtime = "toolchain-coupled"` 与 `--mode system` | 拒绝 | -| 产物为可执行文件、非通用包,且多条 `--target` | 拒绝:一个可执行文件属于一个目标 | -| `--systems` 而非 `--mode self-contained` 的通用包或映像的单文件 | 拒绝:只有携带加载器的包与映像的单文件按系统区分 | -| 映像的单文件,`--systems` 中某系统在映像架构上的行为本 mcpp 不构建的行(目标表标为 planned,如 `x86_64-macos`;或未列出,如 `aarch64-windows-gnu`) | 拒绝,写出该系统与行;出路为其余系统组成的 `--systems` | -| 映像的单文件,多个 `--systems` 与 `-o` | 拒绝:每个系统一个文件,`-o` 只能命名一个 | -| 平台未声明 `runner` 时打通用包 | 拒绝:启动器经平台的运行器启动映像;出路 `--no-universal` | +| `--systems` 而非 `self-contained` 的树或映像的单文件 | 拒绝:只有自带运行器的树与映像的单文件按系统区分;`[pack] systems` 不相容时写出提示后搁置 | +| `--systems` 的 `-` 不是任何腿的架构 | 拒绝 | +| 映像的单文件,`--systems` 中写出的系统在腿的架构上的行为本 mcpp 不构建的行(目标表标为 planned,如 `x86_64-macos`;或未列出,如 `aarch64-windows-gnu`) | 拒绝,写出该系统与行;出路为能构建的 `-` 组成的 `--systems`。`all` 只取能构建的行,其余写为未覆盖 | +| 结果多于一个文件(多条腿的原生程序、多个单文件)而 `-o` 命名一个文件 | 拒绝;出路 `-o <目录>/` | +| 平台未声明 `runner` 的映像,多条腿,且不是单文件 | 拒绝:多条映像腿合并为一个包只经平台的运行器;出路每腿一次打包,或 `--pkg-format single-file` | +| 腿不是所选平台的目标 | 拒绝 | +| 目标集合展开为空,或候选选到不同平台 | 拒绝(SPEC-015 §1) | +| `--pkg-format` 列表含 `dir` 或 `single-file`;列表与命名一个文件的 `-o`;`mcpp run --pkg-format` 的列表 | 拒绝(§5.4) | +| `self-contained` 的树而格式只有 `okb` | 拒绝:okb v1 只含映像,运行器会被丢弃;出路 `--pkg-format tar,okb` | +| 树的 `runs_on` 为空(`self` 与 `with` 都空,§4.7) | 拒绝;平台未声明 `runner.hosts` 而模式不是 `self-contained` 时写出提示,不拒绝 | | `--merge` 的输入缺少 `mcpp-package.json`,或名字、版本、平台、链接、模式不同 | 拒绝,写出不一致的字段 | | `--merge` 的两个输入在同一路径上有摘要不同的文件 | 拒绝,写出该路径、两个摘要与两个输入 | | 库目标(`lib`、`shared`)与提供者格式 | 拒绝:库包没有单一暂存树 | @@ -112,8 +134,12 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` `pack` 与 `run` 继续接受 `--format <格式>`,行为与 `--pkg-format` 相同,并以一条警告写出它是旧写法。 -`pack` 的其余选项:`--linkage static|shared`、`--compress <算法>[:<级别>]`、`--no-universal`、 -`--systems [,...]`、`--merge <包>`(可重复,也可逗号分隔)。 +`pack` 的其余选项:`--linkage static|shared`、`--compress <算法>[:<级别>]`、`--systems <词>[,<词>...]`(§4.4)、 +`--merge <包>`(可重复,也可逗号分隔)。`mcpp pack --help` 分两组:常用(`--target`、`--mode`、`--pkg-format`、 +`--systems`、`--linkage`、`-o`)在前,其余在后。 + +`--no-universal` 是已弃用的写法(docs/51 的 `no-universal-flag`):与一个目标同用时接受,写出布局的变化(映像的包为 +§4.2 的树;一个携带映像的本机程序为 `--pkg-format single-file`);与目标集合同用时拒绝。 ### 2.2 `[pack]` @@ -121,51 +147,56 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` |---|---| | `default_mode` | 未给 `--mode` 时的闭包模式 | | `linkage` | 未给 `--linkage` 时的链接(`static`、`shared`) | -| `format` | 未给 `--pkg-format` 时的容器 | +| `format` | 未给 `--pkg-format` 时的容器:字符串或列表 | | `compress` | 未给 `--compress` 时的压缩(§5.1) | +| `systems` | 未给 `--systems` 时的系统:字符串或列表;与这次打包不相容时写出提示后搁置 | +| `targets` | 未给 `--target` 时的腿:目标或目标集合,字符串或列表(SPEC-015) | | `sign` | 签名步骤的 argv,`{}` 为文件(§7) | -| `universal` | `false` 时等同 `--no-universal` | -| `include`、`exclude` | 工程中按 glob 选出的文件（相对工程根），放在暂存树中同样的相对路径下，一个包只放一份（通用包亦然）；`exclude` 去掉其中匹配的文件；`mcpp-package.json` 中角色为 `resource` | +| `universal` | 已弃用(docs/51 的 `pack-universal`):`true` 被拒绝,出路 `[pack] targets = ["openkal"]`;`false` 写出提示后忽略 | +| `include`、`exclude` | 工程中按 glob 选出的文件（相对工程根），放在暂存树中同样的相对路径下，一个包只放一份（树亦然）；`exclude` 去掉其中匹配的文件；`mcpp-package.json` 中角色为 `resource` | | `strip`、`debug_symbols`、`[pack.bundle-project]` | 既有键 | ## 3. 管线 一次打包依次经过: -1. **构建各腿**:按维度与腿构建,`release` 为缺省构建配置;通用包的其余腿在暂存之前构建。 +1. **构建各腿**:按维度与腿构建,`release` 为缺省构建配置;树与单文件的其余腿在暂存之前构建。 2. **收集闭包**:按 `--mode` 决定每条腿携带的库;闭包记录在暂存清单(docs/50)中,未解析的名字使 `dir` 与 `tar` 拒绝。 -3. **生成暂存树**:§4 的布局;通用包生成启动器;`self-contained` 加入加载器或载体。 +3. **生成暂存树**:§4 的布局;树生成启动器;`self-contained` 加入加载器或运行器。 4. **变换**:strip、重定位、平台的封装(SPEC-013 §9.3)、签名(§7),然后写出 `mcpp-package.json`(§4.5)。 -5. **写容器**:§5;提供者格式在第二遍准备中读取暂存树(§6)。 -6. **报告**:产物、每个文件产物的 sha256、`mcpp.pack` 信封(§9)。 +5. **写容器**:§5;提供者格式在第二遍准备中读取暂存树(§6);列表的其余容器读同一棵树(§5.4)。 +6. **报告**:产物、每个文件产物的 sha256、`Runs on` 与未覆盖的组合(§4.7)、`mcpp.pack` 信封(§9)。 `--merge` 是"读入若干包 → 合并 → 描述 → 写容器",不经第 1、2、3 步(§4.6)。 ## 4. 暂存树与包元数据 -### 4.1 单腿布局 +### 4.1 系统程序的布局 -单腿包的布局由目标的二进制格式决定(ELF 的 `bin/` 与 `lib/`,PE 的扁平目录,Mach-O 的 `bin/`,Android 的 `lib/`, -WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 +系统程序的包的布局由目标的二进制格式决定(ELF 的 `bin/` 与 `lib/`,PE 的扁平目录,Mach-O 的 `bin/`,Android 的 +`lib/`,WebAssembly 的同名文件族),规定见 docs/10。平台未声明 `runner` 的映像按 ELF 的单腿布局放在 `bin/`。 -### 4.2 通用包布局 +### 4.2 平台的树 + +声明 `runner` 的平台的映像只有一种布局,与腿数和 `--systems` 无关: ``` ---/ +--/ bin/ POSIX 启动器 - bin/.cmd Windows 启动器 - lib//.oke, *.okso 每个架构一组映像 + bin/.exe Windows 启动器(引擎的启动程序) + lib//.oke, *.okso 每条腿一组映像 libexec//-/... 仅 --mode self-contained mcpp-package.json ``` -包名带平台名(平台包名去掉 `-platform`),不带系统与架构;模式不是 `vendored` 时带模式的后缀(`-system`、`-static`、 -`self-contained` 为 `-bundle-all`),与原生包的命名相同。`[runtime] deploy` 放置的文件位于每条腿的映像旁边。 +多条腿的树以平台名(平台包名去掉 `-platform`)命名,不带系统与架构;一条腿的树以三元组命名;模式不是 `vendored` +时带模式的后缀(`-system`、`-static`、`self-contained` 为 `-bundle-all`),与原生包的命名相同。`[runtime] deploy` +放置的文件位于每条腿的映像旁边。不生成 `.cmd`。 ### 4.3 启动器 -启动器由引擎生成,内容只来自平台的 `runner` 与腿: +启动器由引擎生成,内容只来自平台的 `runner` 与腿,是派生文件:`--merge` 为合并后的腿重新生成(§4.6)。 - 包的根为启动器所在目录的上一级。 - 架构目录按 `uname -m`(POSIX)或 `%PROCESSOR_ARCHITEW6432%`、`%PROCESSOR_ARCHITECTURE%`(Windows)选择;映射为 @@ -174,14 +205,19 @@ WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 - 运行 `runner.cmd`,与 `mcpp run` 的展开相同:含 `{}` 的词中 `{}` 替换为所选架构目录中的入口映像,没有词含 `{}` 时映像追加在末尾;其余词原样传递,随后是启动器自己的参数;运行器先在 `libexec//-/` 中找, 再找 `PATH`。 -- POSIX 启动器把每个词写成单引号字符串,沿符号链接找到自身的位置(不依赖 `readlink -f`);`.cmd` 启动器中 `%` - 写成 `%%`。`.cmd` 的词不能含双引号与换行:含有它们的运行器在打包时拒绝。 +- POSIX 启动器把每个词写成单引号字符串,沿符号链接找到自身的位置(不依赖 `readlink -f`);`uname -s` 为 `MINGW*`、 + `MSYS*`、`CYGWIN*` 时交给 `bin/.exe`。 +- Windows 启动器是一个程序:源码随引擎提供,写明运行器的词与入口映像的名字,在打包时为 `x86_64-windows-gnu` + 构建(平台的托管安排服务该行时以它构建);它从自身路径(`GetModuleFileNameW`)找到包,按本机架构选 + `lib//`,包内没有该架构时写出包含的架构。它不含腿的列表,因而对任一组腿相同。本 mcpp 不构建该行时包中 + 没有它,写出原因,`runs_on` 不含 Windows。 ### 4.4 `--systems` -`--systems [,...]`(`linux`、`macos`、`windows`;缺省为本机系统)对 `self-contained` 的通用包,为每个系统 × -包的每个架构构建平台的载体(不携带任何内容,即运行器形式,SPEC-013 §11)并放入 -`libexec//-/`;对映像的单文件,为每个系统写一个文件(§5.2)。每个系统的程序在其行 +`--systems` 的每个词为 ``(`linux`、`macos`、`windows`,取腿的全部架构)、`-`(只取该架构的腿)或 +`all`(三个系统);可重复,也可逗号分隔;缺省为本机系统;未给出时取 `[pack] systems`。对 `self-contained` 的树,为 +每个系统 × 腿的每个架构构建平台的载体(不携带任何内容,即运行器形式,SPEC-013 §11)并放入 +`libexec//-/`;对映像的单文件,为每条腿与每个系统写一个文件(§5.2)。每个系统的程序在其行 (`-linux-musl`、`-macos`、`-windows-gnu`)上构建;平台的托管安排(SPEC-013 §5.4)服务该行时以它 构建,于是同一程序在任何打包的机器上相同,不需要该系统的 SDK 与 C 运行时;Linux 的程序为静态。托管安排不服务该行时 按惯常行构建,与本机系统和架构相同时取本机的行。本 mcpp 不构建程序的行(目标表标为 planned 或未列出)不放入包,并 @@ -215,11 +251,12 @@ WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 | `linkage` | `static`、`shared`,或 `declared`(每个库按自己的声明) | | `mode` | §1.2 的模式:`system`、`vendored`、`self-contained`;`--mode static` 的原生包为 `static`,映像为 `self-contained` | | `legs` | 腿的规范三元组 | -| `systems` | 包携带的加载器的 `-`:通用包为 `libexec//` 中的;单腿 `self-contained` 的 ok 映像包为其载体,即程序所属的系统与映像的架构(2026.10.29.1);无加载器时为空 | -| `launch` | 通用包为 `runner`(平台的 argv)与每个架构的 `entry`;单腿原生包为 `program`(程序的相对路径) | +| `systems` | 包携带的运行器的 `-`,即 `libexec//` 中的;无运行器时为空。它是输入的记录,`runs_on` 是答案 | +| `launch` | 树为 `runner`(平台的 argv)与每个架构的 `entry`;系统程序的包为 `program`(程序的相对路径) | +| `runs_on` | 包在哪里运行(§4.7):`self` 与 `with.`,各为 `-` 的列表 | | `files` | 树中每个文件(符号链接除外):`path`(相对于树根,`/` 分隔)、`role`、`leg`(所属的腿,共用文件为 `null`)、`sha256` | -`role` 取 `program`、`image`、`library`、`loader`、`launcher`、`resource`、`metadata` 之一:通用包中 `lib//` +`role` 取 `program`、`image`、`library`、`loader`、`launcher`、`resource`、`metadata` 之一:树中 `lib//` 下的文件为该腿的 `image`,生成的启动器为 `launcher`;其余按位置与名字判定 —— `libexec/` 下为 `loader`; `README*`、`LICENSE*`、`HOST-REQUIREMENTS`、`share/licenses/` 为 `metadata`;`.so` 与 `.so.<版本>`、`.dylib`、`.dll`、`.okso` 为 `library`;`.oke` 为 `image`;`bin/` 下的可执行文件为 `program`;`.cmd`、`.sh` 与树根的可执行文件为 `launcher`; @@ -229,16 +266,36 @@ WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。 `mcpp pack --merge --merge ...`(也可 `--merge ,`)读入至少两个本命令写出的包(目录,或 `.zip`、 `.tar`、`.tar.gz`、`.tgz`、`.tar.zst`、`.tar.xz` 归档),按 `mcpp-package.json` 合并为一棵树:名字、版本、平台、 -链接与模式**必须**相同;腿与系统取并集(同一映像在 Linux 与 macOS 上构建出相同字节;Windows 主机上 clang 在 -`__FILE__` 中保留它拼接包含路径时用的 `\`,其映像不同,合并时被拒绝,mcpp#850);每个文件在复制前计算摘要并与记录的摘要比较,不符时拒绝并写出文件、输入 -与两个摘要;同一路径的文件摘要**必须**相同,每个文件只取一份;合并后的树重新描述。`mcpp-package.json` 中的每个 +链接与模式**必须**相同;腿、系统与 `runs_on` 取并集(同一映像在 Linux 与 macOS 上构建出相同字节;Windows 主机上 +clang 在 `__FILE__` 中保留它拼接包含路径时用的 `\`,其映像不同,合并时被拒绝,mcpp#850);没有 `runs_on` 的输入由其 +`systems` 与映像的架构算出;每个文件在复制前计算摘要并与记录的摘要比较,不符时拒绝并写出文件、输入与两个摘要;同一 +路径的文件摘要**必须**相同,每个文件只取一份。启动器是派生文件,不参与同一路径的比较:POSIX 启动器为合并后的腿 +重新生成,Windows 启动器取一份(§4.3);合并后的树重新描述。`mcpp-package.json` 中的每个 路径**必须**是包内的规范相对路径。输出**必须**是新的路径,或一个已有的、不是输入之一的本命令写出的归档;目录输出 只写新目录。 容器取 `--pkg-format`,否则取 `[pack] format`,否则取 `-o` 的扩展名,否则取第一个输入的容器;`-o` 缺省为当前目录下与树同名的文件。 合并后的包的目录以输出命名,与 `mcpp pack -o` 相同:归档名去掉其扩展名(`-o app-1.0-openkal.zip` 含 -`app-1.0-openkal/`);不带 `-o` 时取第一个输入的目录名。 +`app-1.0-openkal/`);不带 `-o` 时取第一个输入的目录名,树的目录名中第一条腿的三元组换成平台名 +(`app-1.0-x86_64-openkal` → `app-1.0-openkal`)。 不进行任何构建。 +### 4.7 运行覆盖(`runs_on`) + +打包时计算一次,写入 `mcpp-package.json`(schema 1 的字段,不升级 schema): + +| 包 | `self` | `with.` | +|---|---|---| +| 树,`self-contained` | 腿的架构 × 构建出的运行器的 `-` | 腿的架构 × 平台的 `runner.hosts`,减去 `self` | +| 树,`vendored`、`system` | 空 | 腿的架构 × 平台的 `runner.hosts` | +| 系统程序的单腿包 | 三元组的 `-` | 空 | +| 单文件 | 每个文件各自的 `-` | 空 | +| WebAssembly、Android、iOS、裸机 | 空 | 空 | + +`runner.host = "any"` 时 `hosts` 列出运行器程序存在的组合,覆盖与腿的架构无关。树中没有 Windows 启动器时 `runs_on` +不含 Windows 的组合。`self` 与 `with` 都为空的树被拒绝(§1.5)。`Packed` 之后写出 `Runs on <组合>; with : +<组合>`;所请求而未覆盖的组合各写一行 `Not covered <组合> (<原因>)`(本 mcpp 不构建该行、没有 Windows 启动器、 +不在平台的 `runner.hosts` 中)。`mcpp emit xpkg`、`--merge`、`mcpp.pack` 的 `runsOn` 读这里,不再计算。 + ## 5. 容器 ### 5.1 压缩 @@ -279,14 +336,24 @@ liblzma)压缩,不调用 `tar` 等外部程序;产物扩展名为 `.tar.gz`(缺 5. 以原有的参数执行树中的程序(`bin/`;self-contained 的 Linux 树为根部的包装脚本;Windows 树为根部的 `.exe`),退出状态即程序的退出状态;启动器自身失败(载荷不可读、校验不符、无法执行)时写出原因并以 127 退出。 -openkal 目标的单文件:载体携带映像本身(映像不引用其他映像时),否则携带全部映像的一个 stored zip,各映像数据按 -64 KiB 对齐,`mcpp_carried_name` 为入口映像(SPEC-013 §11);`--systems` 的每个系统(缺省为本机系统)各写一个文件 -`target/dist/---`(Windows 上带 `.exe`)。运行时不写盘。 +openkal 目标的单文件:载体携带一条腿的映像本身(映像不引用其他映像时),否则携带该腿全部映像的一个 stored zip, +各映像数据按 64 KiB 对齐,`mcpp_carried_name` 为入口映像(SPEC-013 §11);每条腿 × `--systems` 在该腿架构上的每个 +系统(缺省为本机系统)各写一个文件 `---`(Windows 上带 `.exe`),即覆盖"腿 × 系统"的最少 +载体文件;多于一个文件时 `-o` 只能是目录。`--systems all` 略去本 mcpp 不构建的行,各写为未覆盖。运行时不写盘。 ### 5.3 一个文件在多个系统上运行 一个文件同时在多个系统与架构上运行(APE)不在本规范内(mcpp#849)。 +### 5.4 容器列表 + +`--pkg-format`(与 `[pack] format`)可以是逗号分隔的列表,如 `tar,okb`:一次构建、一棵暂存树,依次写出各容器。 + +- 列表只能含读同一棵暂存树的容器:`tar`、`zip` 与提供者的格式;`dir`、`single-file` 不进列表。 +- 列表时 `-o` 只能是目录(§1.4)。`--compress` 只作用于列表中的引擎容器;列表中没有引擎容器时拒绝。 +- `self-contained` 的树而列表只有 `okb` 时拒绝(okb v1 只含映像)。okb 含哪些架构由腿决定。 +- `mcpp run --pkg-format` 不接受列表。 + ## 6. 包格式提供者 一个包以构建程序的 `mcpp::provides_pack_format("")` 声明一个格式,以 `role = "artifact"` 的 action 生成产物 @@ -303,7 +370,7 @@ openkal 目标的单文件:载体携带映像本身(映像不引用其他映像 ## 7. 签名 `[pack] sign = ["", ..., "{}"]` 在写容器之前对暂存树中角色为 `program`、`image`、`library`、`loader` 的每个 -文件运行一次,也对每个单文件运行一次,`{}` 为文件路径;一个包格式提供者或构建插件**可以**代替它签名。任一次失败 +文件与 Windows 启动器(`bin/.exe`)运行一次,也对每个单文件运行一次,`{}` 为文件路径;一个包格式提供者或构建插件**可以**代替它签名。任一次失败 使打包失败,写出 argv 与输出。签名后 `mcpp-package.json` 重新计算摘要。 ## 8. 分发描述 @@ -312,22 +379,21 @@ openkal 目标的单文件:载体携带映像本身(映像不引用其他映像 生成 xim-pkgindex 条目(spec 2),资产地址为 `<前缀>/<文件名>`。每个归档各以一个 `--package` 给出;其后不带选项的 词(例如 shell 通配展开的第二个文件)被拒绝,而不是不加描述。 -- 通用包是一个资产,`linux`、`macosx`、`windows` 三项的 URL 与 sha256 相同;包不携带运行器(非 `self-contained`) - 时三项各以运行器为依赖(`deps = { "xim:" }`,`` 为平台运行器 argv 首项的程序名,即 xim-pkgindex - 中同名的包;openkal 为 `okrun`)。依赖带命名空间:不带命名空间的名字在家目录列出的第二个索引也有同名包时有歧义, - xim-pkgindex 拒绝它。 +- 树是一个资产,每个系统一项只写 `runs_on`(§4.7)覆盖到的架构,URL 与 sha256 相同;`runs_on` 为空时拒绝。包不携带 + 运行器(非 `self-contained`)时各项以运行器为依赖(`deps = { "xim:" }`,`` 为平台运行器 argv 首项的 + 程序名,即 xim-pkgindex 中同名的包;openkal 为 `okrun`)。依赖带命名空间:不带命名空间的名字在家目录列出的第二个 + 索引也有同名包时有歧义,xim-pkgindex 拒绝它。 - 各系统各架构一个的包:每个系统一项,一个以 `${arch}` 代替架构的 URL 与每个架构的 sha256;文件名除架构外不同时 拒绝。单腿包按其**程序**运行的系统与架构描述:腿为 Linux(Android 除外)、macOS 或 Windows 的三元组时,为该三元组的 - 系统(`linux`、`macosx`、`windows`)与架构;腿为 ok 映像而包携带其载体(`self-contained`,含 `--mode static`)时, - 程序是载体,为包在 `systems` 中记录的唯一 `-`(§4.5)。程序不是任何系统的程序时(WebAssembly 包、Android - 与 iOS 应用、裸机映像、不带载体的 ok 映像包),条目**必须**被拒绝,消息写出包、三元组与出路:WebAssembly 包以 - `dist-web` 发布为页面,不带载体的 ok 映像以平台的通用包(不带 `--no-universal` 的打包)或带载体的包描述。原因:条目的 + 系统(`linux`、`macosx`、`windows`)与架构。程序不是任何系统的程序时(WebAssembly 包、Android 与 iOS 应用、裸机 + 映像、平台未声明运行器的 ok 映像包),条目**必须**被拒绝,消息写出包、三元组与出路:WebAssembly 包以 `dist-web` + 发布为页面,ok 映像以平台的树(`--target openkal`)或单文件描述。原因:条目的 `xpm.<系统>` 安装该系统的程序,`archs` 为机器架构;WebAssembly 包没有启动器,其程序由 JavaScript 引擎运行, `wasm32` 不是任何系统的机器架构;一个 `.oke` 需要运行器(2026.10.29.1)。 - 安装把归档的唯一顶层目录移到安装目录。该目录以归档命名(`mcpp pack` 的写法),安装取下载文件的名字去掉扩展名, 对每个系统、架构与索引日后追加的版本都成立;打包后改名的归档与 xim 不解开的文件(非 `.tar.gz`、`.tar.zst`、 - `.tar.xz`、`.tar`、`.zip`)在写出条目时被拒绝;配置以 xvm 注册程序(通用包为 `bin/`,Windows 上为 - `bin/.cmd`;单腿原生包为程序所在的目录)。各系统的包把程序放在不同目录时(Windows 的程序在包根,POSIX + `.tar.xz`、`.tar`、`.zip`)在写出条目时被拒绝;配置以 xvm 注册程序(树为 `bin/`,Windows 上为 + `bin/.exe`;单腿原生包为程序所在的目录)。各系统的包把程序放在不同目录时(Windows 的程序在包根,POSIX 的程序在 `bin/`,包根为便利脚本),条目按系统写出各自的目录(`bindir[os.host()]`)。 不带 `--package` 时 `mcpp emit xpkg` 照旧生成源码包的描述。 @@ -338,27 +404,37 @@ openkal 目标的单文件:载体携带映像本身(映像不引用其他映像 | 字段 | 语义 | |---|---| -| `linkage`、`mode`、`pkgFormat` | 三个维度的取值 | -| `legs` | 腿的规范三元组 | -| `systems` | 加载器所属的 `-` | +| `linkage`、`mode`、`pkgFormat` | 三个维度的取值;`pkgFormat` 为第一个容器 | +| `pkgFormats` | 写出的全部容器,按命名的顺序(2026.10.30.1) | +| `legs` | 腿的规范三元组;每腿一个包时为全部包的腿,`artifacts[].targets` 为各自的腿 | +| `systems` | 运行器所属的 `-` | +| `runsOn` | `{ "self": [...], "with": { "": [...] } }`,同 §4.7(2026.10.30.1) | | `artifacts[].sha256` | 每个文件产物的 sha256;目录产物为 `null` | -`artifacts[].format` 保留,与 `pkgFormat` 相同。库目标的打包不带这些字段。 +`artifacts[].format` 保留,与 `pkgFormat` 相同。新字段只增加,不改变已有字段的含义。库目标的打包不带这些字段。 ## 10. 判据 1. 每个不合法的组合在任何编译之前被拒绝,消息写出冲突的选择与出路(e2e 954)。 2. `--format` 在 `pack` 与 `run` 上与 `--pkg-format` 等价,并写出警告(e2e 954)。 -3. 通用包在本机架构上经启动器运行;`self-contained` 的通用包在 `PATH` 上没有运行器时运行(e2e 955)。 -4. `--merge` 的结果含两个输入的全部腿、系统与文件,摘要一致;冲突与不同版本被拒绝(e2e 957)。 +3. `--target openkal` 是一棵树,在本机架构上经启动器运行;`self-contained` 的树在 `PATH` 上没有运行器时运行;一个 + 目标只打一条腿,布局相同(e2e 955、983)。 +4. `--merge` 的结果含两个输入的全部腿、系统与文件,摘要一致;冲突与不同版本被拒绝;两个单腿的树合并后启动器 + 覆盖两条腿(e2e 957、983)。 5. 单文件首次运行解压一次,再次运行不解压;静态单文件不带解释器;映像的单文件运行时缓存目录中无新文件(e2e 956)。 6. `mcpp-package.json` 的 `files` 覆盖树中每个文件,摘要与文件一致(e2e 954、957)。 -7. 通用包的 `mcpp emit xpkg` 条目在三个系统上为同一资产,并以运行器为依赖(e2e 957)。 +7. 树的 `mcpp emit xpkg` 条目为一个资产,只写 `runs_on` 覆盖的系统与架构,并以运行器为依赖;`runs_on` 为空时 + 拒绝(e2e 957、961)。 8. `mcpp emit xpkg` 拒绝 `wasm32-emscripten` 的包,写出包、三元组与 `dist-web`;`--pkg-format single-file` 在 WebAssembly 目标上于任何编译之前被拒绝(e2e 962)。 -9. 单腿的 `self-contained` ok 映像包的程序是映像架构的,`systems` 记录其载体的 `-`,`mcpp emit xpkg` 以该系统 - 与架构描述它,不带载体的单腿 ok 映像包被拒绝;`mcpp pack --target --list-formats` 列出该目标的平台提供的格式 - (`okb`),本机目标的列表不含它(e2e 961)。 +9. 映像的单文件是映像架构的程序;单腿 `self-contained` 的树的 `systems` 与 `runs_on.self` 为其运行器的 + `-`,`mcpp emit xpkg` 以该系统与架构描述它;`mcpp pack --target --list-formats` 列出该目标的平台 + 提供的格式(`okb`),本机目标的列表不含它(e2e 961)。 +10. 原生程序的目标集合每腿一个包,`-o` 命名一个文件时被拒绝(e2e 984)。 +11. `--pkg-format` 列表一次构建写出各容器;`dir`、`single-file` 在列表中、列表与一个文件的 `-o`、`mcpp run` 的列表、 + `self-contained` 只配 `okb` 被拒绝(e2e 985)。 +12. 映像的单文件覆盖腿 × 系统;`--systems all` 写出未覆盖的组合;多个文件与一个文件的 `-o` 被拒绝(e2e 986)。 +13. `--no-universal`、`[pack] universal`、`[platform] universal` 按 §2 迁移;`mcpp pack --help` 分组(e2e 987)。 ## 变更记录 @@ -368,3 +444,4 @@ openkal 目标的单文件:载体携带映像本身(映像不引用其他映像 | 1.1 | 2026-10-28 | 与实现对齐:zip 的条目以 DEFLATE 压缩;`[pack] universal`;压缩的拒绝与项目缺省值的搁置;`--systems` 也用于映像的单文件;启动器的根与退出状态;单文件的载荷格式、校验与执行;映像单文件的载体内容;`mcpp-package.json` 的 `linkage` 取值与角色判定;`--merge` 的输入、选项写法与容器;签名的范围;`mcpp emit xpkg --package`;判据对应 e2e 954–957。 | | 1.2 | 2026-10-28 | 包名的模式后缀;单文件启动器自身失败的退出状态 127;`--merge` 取 `[pack] format`,合并后的目录以输出命名;`--linkage static` 改为 musl 的行;`mcpp emit xpkg --package` 的每系统程序目录、带命名空间的运行器依赖、以归档名定位的顶层目录与多余词的拒绝。 | | 1.3 | 2026-10-29 | mcpp 2026.10.29.1:§0 ok 映像与 Web;§1.2 单腿 `self-contained` 包的载体为映像架构的程序;§1.5、§5.2 WebAssembly 目标上的单文件被拒绝;§4.5 单腿 `self-contained` ok 映像包的 `systems` 记录其载体;§6 `--list-formats` 规划所选目标;§8 单腿包按其程序运行的系统与架构描述(系统的三元组,或 ok 映像的载体),程序不是任何系统的程序的被拒绝;判据 8、9(e2e 961、962)。 | +| 2.0 | 2026-10-30 | mcpp 2026.10.30.1(设计 2026-10-29 §4、§13):§1.4 腿只是命令命名的目标(目标集合按 SPEC-015 展开),腿按目标的种类与平台合并,原生程序每腿一个包,`-o <目录>/`;删除通用包与 `[platform] universal`、`[pack] universal`、`--no-universal` 的作用(迁移见 §2、docs/51);§1.2、§4.2 声明运行器的平台的映像只有一种布局(树),M3 只属于单文件,`--mode static` 为静态树;§4.3 Windows 启动器为引擎的程序,不再生成 `.cmd`,启动器为派生文件;§4.4 `--systems` 接受 ``、`-`、`all` 与 `[pack] systems`;§4.6 合并重新生成启动器、合并 `runs_on`、树以平台名命名;§4.7 运行覆盖 `runs_on` 与 `Runs on`、未覆盖的摘要,空覆盖被拒绝;§5.2 单文件覆盖腿 × 系统;§5.4 容器列表;§7 签名含 Windows 启动器;§8 条目按 `runs_on` 描述;§9 `pkgFormats`、`runsOn`;判据 3、4、7、9 改写,判据 10–13(e2e 983–987)。 | diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index cbb4fa11c..6040c08af 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -4,10 +4,10 @@ |---|---| | 规范编号 | SPEC-013 | | 标题 | 平台:一类目标的可复用配置,其声明、选择、合并、对依赖图的补全、链接与封装、运行器与载体 | -| 状态 | 评审中 v1.2 | -| 版本 | 1.2 | -| 最后修改 | 2026-10-29 | -| 对应实现 | mcpp >= 2026.10.27.1;§10 的执行宿主与 §11 的载体架构 mcpp >= 2026.10.29.1;逐条标注 | +| 状态 | 评审中 v1.3 | +| 版本 | 1.3 | +| 最后修改 | 2026-10-30 | +| 对应实现 | mcpp >= 2026.10.27.1;§10 的执行宿主与 §11 的载体架构 mcpp >= 2026.10.29.1;`runner.hosts`、载体列表、`descriptor_keys` 与 `universal` 的弃用 mcpp >= 2026.10.30.1;逐条标注 | | 相关设计文档 | `.agents/docs/2026-10-28-platforms-design.md`、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md` | | 相关 issue | mcpp#846、mcpp#847、mcpp#848 | | 相关规范 | [SPEC-002](target-side.md)(目标侧的层)、[SPEC-004](manifest-semantics.md)(条件化形状)、[SPEC-006](toolchain-management.md)(工具链选择)、[SPEC-007](build-plugins.md)(构建程序与协议)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-014](packaging.md)(打包) | @@ -70,7 +70,7 @@ plugin = "build.mcpp" | `carrier` | 表或表的数组 | §11;每项的 `package`、`section`、`main` 三键都**必须**给出;`hosts` 为该项服务的 `-`(省略:服务每一对);`build` 为载体程序的构建模型,缺省 `"row"`(每个 `-` 一个程序,经平台的 hosted 安排构建),本 mcpp 不认识的模型的项被跳过并警告。**mcpp >= 2026.10.30.1**;更早的 mcpp 读到数组即拒绝,写数组的平台**必须**声明相应的 `mcpp` 下限 | | `descriptor_keys` | 非空字符串数组 | 平台目标读取描述符的哪些按平台块及其顺序(第一个存在的块被读取),取值为 `linux`、`macosx`、`windows`、`android`、`ios`、`emscripten`、`openkal`、`none`;未声明时为目标行自己的键(§14)。**mcpp >= 2026.10.30.1** | | `plugin` | 字符串 | 只能为 `"build.mcpp"`:平台包自身的构建程序(SPEC-007)在平台阶段运行(§12);其他值拒绝 | -| `universal` | 布尔 | `true` 时本平台任一目标的打包缺省为覆盖全部 `targets` 的通用包(SPEC-014 §1.4、§4.2) | +| `universal` | 布尔 | 已弃用,不起作用(docs/51 的 `platform-universal`):打包的腿只是命令命名的目标(SPEC-014 §1.4);读到时写出提示。**mcpp >= 2026.10.30.1** | | `layers` | 表 | §7.2;键只能是 §0 的四个层名 | | `tools` | 表 | §7.3 | | `build` | 表 | 四个标志列表,以及键形如 `cfg(...)` 的子表(同四个键) | @@ -307,7 +307,7 @@ plugin = "build.mcpp" 的运行器在产物架构的机器上运行,因此产物的架构不是本机架构、且给出了执行宿主(`run_under`,含 `MCPP_TARGET_<三元组>_RUN_UNDER`;SPEC-002 §6)而未给 `--no-runner` 时,首项为平台工具的运行器的程序**必须**是产物架构的: 平台的载体以运行器形式(§11,不携带任何内容)为本机系统与产物架构构建的程序,构建于 -`target/carrier-pack/<首项>--/`,与 `self-contained` 通用包放在 `libexec/<首项>/-/` 中的程序相同。 +`target/carrier-pack/<首项>--/`,与 `self-contained` 的树放在 `libexec/<首项>/-/` 中的程序相同。 平台未声明 `carrier`,或本 mcpp 不为该系统与架构构建程序(SPEC-014 §4.4)时,运行**必须**被拒绝并写出原因,**禁止**改用 本机的工具。`host = "any"` 的运行器从本机到达产物,其程序是本机的。`mcpp test` 为一次构建构建一次该程序,不运行测试时 (`--no-run`、已确定的拒绝)不构建。**已实现**(2026.10.29.1) @@ -326,10 +326,10 @@ run_under = ["qemu-aarch64"] ## 11. 载体(M3) -对一个依赖加载器的产物,`mcpp pack` 在三处生成一个系统程序的工程(`target/carrier-pack/--/`): -`--mode self-contained` 的单目标包(该程序携带产物)、`self-contained` 通用包的运行器(不携带任何内容,SPEC-014 -§4.4)、映像的单文件(SPEC-014 §5.2);`mcpp run` 与 `mcpp test` 为执行宿主构建另一架构的运行器(§10)时亦然。 -单目标包与单文件的程序的架构为产物的架构,系统为本机系统(单文件另按 `--systems`)。工程依赖 `carrier.package`; +对一个依赖加载器的产物,`mcpp pack` 在两处生成一个系统程序的工程(`target/carrier-pack/--/`): +`self-contained` 的树的运行器(不携带任何内容,SPEC-014 §4.4)与映像的单文件(该程序携带产物,SPEC-014 §5.2); +`mcpp run` 与 `mcpp test` 为执行宿主构建另一架构的运行器(§10)时亦然。携带映像的程序(M3)只属于单文件 +(2026.10.30.1)。单文件的程序的架构为该腿的架构,系统按 `--systems`(缺省为本机系统)。工程依赖 `carrier.package`; 入口是平台包中的 `carrier.main`;一个汇编文件以 `.incbin` 把携带的字节放进节 `carrier.section`(ELF 写作该名并带 `"a"` 标志,COFF 写作该名并带 `"dr"` 标志,Mach-O 写作 `__<大写名>,__carried`),给出符号 `mcpp_carried_begin`、`mcpp_carried_end`、`mcpp_carried_name` (入口的名字)与 `mcpp_carried_digest`(字符串)。 @@ -499,7 +499,7 @@ openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscri `--host-platform none` 时构建程序为本机程序(e2e 953)。 12. 给出执行宿主时,`mcpp run` 与 `mcpp test` 交给它的运行器程序是产物架构的 ELF;本机有该架构的 qemu 时映像经它运行、 测试通过;`mcpp self env` 报告拒绝的原因,给出执行宿主时报告另一架构的运行器(e2e 960)。 -13. `--mode self-contained` 的单目标包的载体是映像架构的程序(e2e 961)。 +13. 映像的单文件的载体是映像架构的程序;`--mode self-contained` 的单目标包是树,携带该架构的运行器(e2e 961)。 ## 变更记录 @@ -508,3 +508,4 @@ openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscri | 1.0 | 2026-10-28 | 首版(mcpp 2026.10.27.1,mcpp#847):`[platform]` 的七组键与 `versions`、`hosted`;三种写法与层次;目标图与宿主图的选择;`cfg(platform)`;合并;依赖图的补全、版本组与改名边界;工具链;链接、命名与封装;运行器的表形式;载体;平台阶段与协议 17;工作空间;目标表的平台列;锁、报告、机器输出、命令与拒绝码。 | | 1.1 | 2026-10-28 | `[platform] universal`;载体的三种携带形式(运行器形式、映像、映像组的 zip)、`mcpp_carried_digest`、Mach-O 节名 `__carried`、按系统的行;平台阶段可声明包格式与 artifact action;`self env --host` 报告运行器与工具;判据 10、11(e2e 952、953)。 | | 1.2 | 2026-10-29 | mcpp 2026.10.29.1:§10 给出执行宿主时 `same-arch` 运行器的程序为产物架构的载体运行器形式,`mcpp test` 写出完整的拒绝原因一次;§11 单目标包与单文件的程序为产物架构的,不构建的行在生成工程之前被拒绝;§14 ok 映像与 Web;§16 `self env` 报告运行器程序的架构、执行宿主与拒绝,平台包的 `mcpp test` 运行给出执行宿主的目标;判据 7、12、13(e2e 960、961)。 | +| 1.3 | 2026-10-30 | mcpp 2026.10.30.1(设计 2026-10-29 §4.3–§4.5、§13.3):§1.1 `universal` 弃用、不起作用;`runner.hosts`;`carrier` 接受列表,每项的 `hosts` 与 `build`;`descriptor_keys`;§10 执行宿主的运行器程序按 `hosts` 选择载体项;§11 携带映像的载体只属于单文件,树携带运行器形式;判据 13 改写(e2e 961)。 | diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index d5b236b8e..daf0e922d 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -57,8 +57,9 @@ glibc 专有行为（NSS 查询、`dlopen` 宿主插件）时的首选。 |---|---|---|---| | 链接 | `--linkage` | 库在程序里，还是在程序旁边 | `static`、`shared` | | 闭包 | `--mode` | 运行该包的机器提供什么 | `system`、`vendored`（默认）、`self-contained` | -| 容器 | `--pkg-format` | 交付成什么 | `tar`、`zip`、`dir`、`single-file`、某个包提供的格式；未给出时 Windows 程序为 `zip`，其余为 `tar` | -| 腿 | `--target`，可重复 | 一个包覆盖哪些目标 | 三元组 | +| 容器 | `--pkg-format` | 交付成什么 | `tar`、`zip`、`dir`、`single-file`、某个包提供的格式，或一棵树的容器列表（`tar,okb`）；未给出时 Windows 程序为 `zip`，其余为 `tar` | +| 腿 | `--target`，可重复 | 打包覆盖哪些目标 | 目标与目标集合（`x86_64-openkal`、`openkal`、`linux-musl`） | +| 系统 | `--systems` | 自带的运行器或单文件为哪些系统而做 | `linux`、`linux-aarch64`、`all` | - **链接**是程序的属性。`--linkage static|shared`（或 `[pack] linkage`）是这次打包的构建的依赖链接， 优先于 `[build] dependency_linkage` 与构建配置中的值；未给出时每个库按它自己的声明链接。Linux 上 @@ -67,8 +68,20 @@ glibc 专有行为（NSS 查询、`dlopen` 宿主插件）时的首选。 - **闭包**是包的属性：共享库闭包中有多少随程序携带。`--mode static` 是 `--linkage static --mode self-contained` 的简写。 - **容器**是交付的形状，见下文。 -- **腿**：每个 `--target` 是一次构建，暂存进同一棵树。声明了 `universal = true` 的平台 - 缺省覆盖它的全部目标。 +- **腿**是命令命名的目标（2026.10.30.1+）：每个 `--target`，未给出时为 `[pack] targets`，再未给出时为构建的 + 目标。目标集合（`openkal`、`linux-musl`）为目标表中该系统、所选平台服务的行。平台不增加腿。多条腿按目标的种类 + 合并： + + | 目标 | 多条腿 | + |---|---| + | `lib`、`shared` | 一个库包（[12](12-binary-distribution.md)） | + | 声明运行器的平台（openkal）的程序 | 一棵以平台命名的树，`--openkal` | + | Android 应用 | 一个 APK 树，每条腿一个 `lib//` | + | 其他程序 | 每条腿一个包，各以三元组命名 | + + 写出多于一个文件的打包要求 `-o` 为目录：以 `/` 结尾的路径或已存在的目录，每个文件以缺省名写在其下。一条 openkal + 腿的打包在 `Packed` 之后写出：`only x86_64; --target openkal or [pack] targets = ["openkal"] packs every + architecture`。 | Mode | 宿主必须提供 | 体积 | 使用场景 | |---|---|---|---| @@ -101,12 +114,15 @@ glibc 专有行为（NSS 查询、`dlopen` 宿主插件）时的首选。 | `--compress` 与 `--pkg-format dir` 或提供者的格式 | `--pkg-format tar`、`zip` 或 `single-file`：只有容器被压缩 | | `--compress zstd` 或 `xz` 与 `--pkg-format zip` | `--compress gzip`（DEFLATE）或 `none`，或 `--pkg-format tar` | | `--compress gzip` 或 `xz` 与 `--pkg-format single-file` | `--compress zstd[:]` 或 `none`：启动器解压 zstd | -| 通用包之外、对一个可执行文件给出多个 `--target` | 每个目标打一次 | -| 自带加载器的通用包与映像的单文件之外使用 `--systems` | 在声明 `universal = true` 的平台上用 `--mode self-contained`，或 `--pkg-format single-file` | -| 映像的单文件，`--systems` 中某系统的行本 mcpp 不构建程序（`x86_64-macos`、`aarch64-windows-gnu`） | 只列出会构建的系统的 `--systems` | -| 映像的单文件，多个 `--systems` 与 `-o` 同用 | 去掉 `-o`：每个系统一个文件 | -| 平台未声明 `runner` 时打通用包 | `--no-universal` | -| 通用包中给出平台 `targets` 之外的 `--target` | 平台服务的目标 | +| 自带运行器的映像树与映像的单文件之外使用 `--systems` | `--mode self-contained`，或 `--pkg-format single-file`；不相容的 `[pack] systems` 写出提示后搁置 | +| 映像的单文件，`--systems` 中写出的系统的行本 mcpp 不构建程序（`x86_64-macos`、`aarch64-windows-gnu`） | 只列出会构建的组合的 `--systems`；`--systems all` 取这些并写出其余 | +| 结果多于一个文件（每腿一个包、多个单文件）而 `-o` 命名一个文件 | `-o <目录>/` | +| 未声明 `runner` 的平台的多条映像腿，且不是单文件 | 每个目标打一次，或 `--pkg-format single-file` | +| 平台目标之外的 `--target` | 平台服务的目标 | +| 展开为空的目标集合，或其行选到不同平台 | 一个目标，或另一个集合 | +| `--pkg-format` 列表含 `dir` 或 `single-file`；列表与命名一个文件的 `-o`；`mcpp run` 的列表 | 一个格式，或 `-o <目录>/` | +| 映像的 `--mode self-contained` 而格式只有 `okb` | `--pkg-format tar,okb`：okb 只含映像，运行器会被丢弃 | +| 在任何地方都不能运行的树（`runs_on` 为空） | `--mode self-contained --systems <本 mcpp 构建的系统>` | | 对 `lib` 或 `shared` 目标使用被分派的格式 | 库的路线，[12](12-binary-distribution.md) | | 对 `lib` 或 `shared` 目标使用 `--pkg-format single-file` | `tar`、`zip` 或 `dir` | @@ -181,8 +197,10 @@ mcpp pack --pkg-format msi --features installer # activate root-package feature mcpp pack --linkage static # the libraries inside the program mcpp pack --pkg-format single-file # one executable file mcpp pack --pkg-format tar --compress zstd # .tar.zst -mcpp pack --target x86_64-openkal # a universal package: every architecture of the platform -mcpp pack --target x86_64-openkal --mode self-contained --systems linux,macos,windows +mcpp pack --target x86_64-openkal # one leg: the image's tree for x86_64 +mcpp pack --target openkal # every architecture of the platform, one tree +mcpp pack --target openkal --mode self-contained --systems all --pkg-format tar,okb -o dist/ +mcpp pack --target linux-musl -o dist/ # a static package per architecture mcpp pack --merge a.zip --merge b.zip -o all.zip # one package from packages built on several machines mcpp emit xpkg --package all.zip --url https://example.org/v1.0.0 # its xim-pkgindex entry ``` @@ -636,17 +654,14 @@ ok 映像经 `okrun` 在其架构的每个系统上运行（见 [24](24-openkal- | Mode | 包内内容 | 机器需提供 | |---|---|---| -| `system` | `bin/.oke` | `okrun` 与它所需的每个映像 | -| `vendored`（默认） | `bin/.oke` 与它所需的 `.okso` 映像 | `okrun` | -| `self-contained` | `bin/`：本机系统、映像架构的程序，以平台载体的段（ELF 与 COFF 上为 `.oke`，Mach-O 上为 `__OKE,__carried`）携带映像并经 libokrun 运行，旁边是所需的 `.okso` 映像 | 无 | -| `static`（`--linkage static --mode self-contained`） | `bin/`：本机系统、映像架构的程序，携带一个静态 `.oke` | 无 | - -以上是单个目标的包（`--no-universal`，或平台不声明通用包）。`--linkage static` 构建不带 `.okso` 的映像。 -`self-contained` 不重新编译映像：mcpp 把平台的载体（`[platform] carrier`，一个包与它的入口源码）生成为 -`target/carrier-pack/--/` 下的工程，把映像的字节放进载体的段，再按任何程序的方式打包该程序 —— -Linux 上为静态（musl），macOS 与 Windows 上只引用系统自带的库。该程序的架构为映像的架构（2026.10.29.1+）：在 -x86_64 Linux 上，`--target aarch64-openkal` 打出 aarch64 的 Linux 程序。本 mcpp 不为其构建程序的系统与架构（在 -Windows 机器上为 `aarch64-windows-gnu`）被拒绝，并写出该行。 +| `system` | 下文的树：`lib//.oke` 与启动器 | `okrun` 与它所需的每个映像 | +| `vendored`（默认） | 树，带映像所需的 `.okso` 映像 | `okrun` | +| `self-contained` | 树，带 `--systems` 的每个系统与每个架构的运行器，位于 `libexec/okrun/-/` | 无 | +| `static`（`--linkage static --mode self-contained`） | 每个架构一个静态 `.oke` 的 self-contained 树 | 无 | + +一条腿与多条腿的布局相同（2026.10.30.1+）。`--linkage static` 构建不带 `.okso` 的映像。以平台载体的段（ELF 与 +COFF 上为 `.oke`，Mach-O 上为 `__OKE,__carried`）携带映像的本机程序是单文件（`--pkg-format single-file`，见下文）， +且只是单文件。 ### Android：应用目标文件与它的闭包在 `lib/` 下 @@ -672,42 +687,60 @@ target/dist/myapp-0.1.0-x86_64-linux-android/ 闭包。`--mode` 不适用于这一行：设备恰好提供了平台库，因此既没有可以留给宿主 的库集合，也没有需要携带的 loader。 -### 通用包（`[platform] universal`） +### 平台的映像树 -声明了 `universal = true` 的平台（openkal-platform 即是）使它任一目标的打包覆盖其全部目标；`--no-universal` -只打所选的目标。一棵树服务该平台的每个系统与每个架构： +声明运行器的平台（openkal-platform 即是）的映像打成一棵树，腿数不论：`--target x86_64-openkal` 是一条腿的树 +`hello-0.1.0-x86_64-openkal/`；`--target openkal` 是两条腿的树，以平台命名。一棵树服务它所含的每个系统与每个架构： ``` target/dist/hello-0.1.0-openkal/ ├── bin/hello ← POSIX launcher -├── bin/hello.cmd ← Windows launcher -├── lib/x86_64/hello.oke ← one set of images per architecture +├── bin/hello.exe ← Windows launcher (a program) +├── lib/x86_64/hello.oke ← one set of images per leg ├── lib/aarch64/hello.oke ├── libexec/okrun/-/ ← only with --mode self-contained ├── share/… ← resources, once └── mcpp-package.json ``` -其余的腿在打包过程中构建（`Packed leg `）。启动器由平台的 `runner` 生成：以自身目录的上一级为包的 -根，按 `uname -m`（`x86_64`、`amd64` → `x86_64`；`aarch64`、`arm64` → `aarch64`）或 -`%PROCESSOR_ARCHITECTURE%` / `%PROCESSOR_ARCHITEW6432%` 选择架构目录，以该目录中的映像与自身的参数启动运行器， -先在 `libexec//-/` 中、再在 `PATH` 上查找运行器。包中没有本机架构时以状态 126 退出，写出两者。 -`[runtime] deploy` 放置的文件位于每条腿的映像旁边。`[pack] universal = false` 等同 `--no-universal`；平台未声明 -`runner` 时不能打通用包，打包被拒绝，出路为 `--no-universal`。 +其余的腿逐条规划、一起构建（`Packed leg `）。启动器由平台的 `runner` 生成：POSIX 脚本以自身目录的上一级为 +包的根，按 `uname -m`（`x86_64`、`amd64` → `x86_64`；`aarch64`、`arm64` → `aarch64`）选择架构目录，Git Bash、MSYS 与 +Cygwin 的 shell 交给 `bin/.exe`，以该目录中的映像与自身的参数启动运行器，先在 `libexec//-/` +中、再在 `PATH` 上查找运行器。Windows 启动器是为 `x86_64-windows-gnu` 构建的引擎的程序：从自身路径找到包，选本机 +架构的映像，以同样的方式启动运行器；不生成 `.cmd`。包中没有本机架构时以状态 126 退出，写出两者。 +`[runtime] deploy` 放置的文件位于每条腿的映像旁边。 + +`--mode self-contained` 时运行器随包携带：对 `--systems` 的每个系统（`linux`、`macos`、`windows` 取腿的全部架构； +`linux-aarch64` 取一个组合；`all`；缺省为本机系统），mcpp 构建不携带任何内容的平台载体 —— 一个像 `okrun` 一样运行 +其第一个参数所指映像的程序 —— 放入 `libexec//-/`。每个运行器在其行（`-linux-musl`、 +`-macos`、`-windows-gnu`）上以平台的托管安排构建，因此无论在哪台机器上打包结果都是同一个程序，不需要 +该系统的 SDK 与 C 运行时；Linux 的运行器为静态。本 mcpp 不构建程序的行（`x86_64-macos` 为 planned； +`aarch64-windows-gnu` 不在目标表中）不放入包并给出警告。在 Windows 主机上构建的映像与其他主机上的字节不同 +（#850）：树在 Linux 或 macOS 上打包。 + +**在哪里运行。** 打包把 `runs_on` 写进 `mcpp-package.json`，并在 `Packed` 之后写出：`self` 为凭包内携带的东西即可 +运行的 `-`，`with.` 为机器装有该运行器时可运行的组合（腿的架构 × 平台的 `runner.hosts`）。 -`--mode self-contained` 时运行器随包携带：对 `--systems linux,macos,windows`（缺省为本机系统）的每个系统与包的 -每个架构，mcpp 构建不携带任何内容的平台载体 —— 一个像 `okrun` 一样运行其第一个参数所指映像的程序 —— 放入 -`libexec//-/`。每个运行器在其行（`-linux-musl`、`-macos`、`-windows-gnu`）上 -以平台的托管安排构建，因此无论在哪台机器上打包结果都是同一个程序，不需要该系统的 SDK 与 C 运行时；Linux 的运行器 -为静态。本 mcpp 不构建程序的行（`x86_64-macos` 为 planned；`aarch64-windows-gnu` 不在目标表中）不放入包并给出警告， -启动器在那里从 `PATH` 查找运行器。在 Windows 主机上构建的映像与其他主机上的字节不同（#850）：通用包在 Linux 或 -macOS 上打包。 +``` + Packed target/dist/hello-0.1.0-openkal-bundle-all + Runs on linux-aarch64, linux-x86_64, macos-aarch64, windows-x86_64 + Not covered macos-x86_64 (no runner row: this mcpp builds no program for x86_64-macos) +``` + +在任何地方都不能运行的树（不携带运行器，平台也未写出运行器存在的组合）被拒绝；平台未声明 `runner.hosts` 时改为 +一条提示。 多台机器打出的包以 `mcpp pack --merge a.zip --merge b.zip`（也可 `--merge a.zip,b.zip`）合并。输入是本命令写出的 -包目录或归档（`.zip`、`.tar`、`.tar.gz`、`.tgz`、`.tar.zst`、`.tar.xz`），不进行任何构建。腿与系统取并集，每个 -文件只取一份，结果重新描述。名字、版本、平台、链接或模式不同的输入被拒绝；同一路径有两个摘要时也被拒绝， +包目录或归档（`.zip`、`.tar`、`.tar.gz`、`.tgz`、`.tar.zst`、`.tar.xz`），不进行任何构建。腿、系统与 `runs_on` +取并集，每个文件只取一份，结果重新描述。启动器是派生文件：为合并后的腿重新写出，因此两棵单腿的树合并为两条腿 +的树，不带 `-o` 时以平台命名（`hello-0.1.0-x86_64-openkal` → `hello-0.1.0-openkal`）。名字、版本、平台、链接或模式不同的输入被拒绝；同一路径有两个摘要时也被拒绝， 写出该路径、两个摘要与两个输入。容器取 `--pkg-format`，否则取 `-o` 的扩展名，否则取第一个输入的容器。 +**容器列表。** `--pkg-format tar,okb`（或 `[pack] format = ["tar", "okb"]`）构建一次、暂存一棵树，按命名的顺序 +从它写出各容器。列表只含同一棵树的容器 —— `tar`、`zip` 与提供者的格式 —— 不含 `dir` 与 `single-file`；列表要求 +`-o` 为目录；`--compress` 作用于其中的引擎容器。okb 只含映像，因此格式只有 okb 的 self-contained 树被拒绝 +（`--pkg-format tar,okb`）。`mcpp run --pkg-format` 只接受一个格式。 + ### 单文件包 `--pkg-format single-file` 交付一个可执行文件： @@ -728,8 +761,9 @@ macOS 上为 `~/Library/Caches`，Windows 上为 `%LOCALAPPDATA%`：先解到旁 不符的载荷从不解开。随后启动器成为程序 —— `bin/`、self-contained 的 Linux 树根部的包装脚本，或 Windows 树根部的 `.exe` —— 带着它的参数，退出状态即程序的退出状态。 -ok 映像的文件为 `target/dist/---`（Windows 上带 `.exe`），`--systems` 的每个系统 -（缺省为本机系统）各一个。映像不引用其他映像时载体只携带它，否则携带全部映像的一个 stored zip（每个映像对齐到 +ok 映像的文件为 `target/dist/---`（Windows 上带 `.exe`），每条腿与该腿架构上 +`--systems` 的每个系统（缺省为本机系统）各一个：`--target openkal --pkg-format single-file -o dist/` 每个架构写一个。 +`--systems all` 为本 mcpp 构建程序的每个行写出文件，并写出其余（`Not covered`）；多于一个文件时 `-o` 只能是目录。映像不引用其他映像时载体只携带它，否则携带全部映像的一个 stored zip（每个映像对齐到 64 KiB），载体的入口经 openkal-run 的 bundle 接口读取（openkal-platform 0.17.2 起）。 WebAssembly 目标没有单文件：`--pkg-format single-file` 与 `--target wasm32-emscripten` 在任何编译之前被拒绝 @@ -748,20 +782,19 @@ WebAssembly 目标没有单文件：`--pkg-format single-file` 与 `--target was ``` 每次打包都写出该文件，提供者的格式也不例外。每个文件的 `role` 为 `program`、`image`、`library`、`loader`、 -`launcher`、`resource` 或 `metadata`；通用包中 `lib//` 下的每个文件都是该腿的映像。符号链接不列出。 +`launcher`、`resource` 或 `metadata`；树中 `lib//` 下的每个文件都是该腿的映像。符号链接不列出。 `linkage` 为 `static`、`shared` 或 `declared`（每个库按它自己的声明）；`launch` 对单个原生腿为 `program`， -对通用包为 `runner` 加每个架构的 `entry`。 +对树为 `runner` 加每个架构的 `entry`；`runs_on` 为包在哪里运行（`{"self": [...], "with": {"okrun": [...]}}`）， +`systems` 为它携带的运行器。 `mcpp emit xpkg --package <归档> [--package …] --url <前缀> [-o <文件>]` 由本命令写出的包的 -`mcpp-package.json` 生成 xim-pkgindex 条目（spec 2）。通用包是 `linux`、`macosx`、`windows` 同样安装的一个资产， -包不携带运行器时以运行器为依赖（`deps = { "xim:okrun" }`）；各系统各架构一个的包，每个系统一项，一个带 `${arch}` 的 -URL 与每个架构的 sha256。安装把归档的顶层目录移到位，并向 xvm 注册程序（Windows 上为 `.cmd` 启动器）；各系统的包 +`mcpp-package.json` 生成 xim-pkgindex 条目（spec 2）。树是一个资产，每个系统一项只写 `runs_on` 覆盖到的架构， +包不携带运行器时以运行器为依赖（`deps = { "xim:okrun" }`），`runs_on` 为空的树被拒绝；各系统各架构一个的包，每个系统一项，一个带 `${arch}` 的 +URL 与每个架构的 sha256。安装把归档的顶层目录移到位，并向 xvm 注册程序（Windows 上为 `bin/.exe`）；各系统的包 把程序放在不同目录时（Windows 的程序在包根，POSIX 的程序在 `bin/`），按系统注册各自的目录。每个归档各以一个 -`--package` 给出。单腿包按其程序运行的系统与架构描述（2026.10.29.1+）：Linux、macOS 或 Windows 目标为三元组的 -系统与架构；带载体打包的 ok 映像（`--no-universal --mode self-contained`）为载体的系统与架构，包把它记录在 `systems` -中（在 Linux 上打包的 aarch64 映像为 `"systems": ["linux-aarch64"]`）。程序不是任何系统的程序的包被拒绝，并写出 -分发它的途径：`wasm32-emscripten` 的包发布为页面（`dist-web`），不带载体的 ok 映像以平台的通用包发布，Android、iOS -与裸机的包没有条目。 +`--package` 给出。Linux、macOS 或 Windows 目标的单腿包按三元组的系统与架构描述。程序不是任何系统的程序的包被 +拒绝，并写出分发它的途径：`wasm32-emscripten` 的包发布为页面（`dist-web`），ok 映像以平台的树（`--target openkal`） +或单文件发布，Android、iOS 与裸机的包没有条目。 ### 压缩与签名 @@ -770,7 +803,7 @@ URL 与每个架构的 sha256。安装把归档的顶层目录移到位，并向 `PATH` 上查找压缩程序，同一归档在每个宿主上字节相同。`zip` 的条目以 DEFLATE 压缩（`gzip[:level]`）或存储 （`none`）；单文件的载荷为 `zstd` 或 `none`；okb 中的映像以 stored 存放，使加载器原地映射。 -`[pack] sign = ["", …, "{}"]` 在写容器之前对暂存树中的每个程序、映像、库与加载器各运行一次，也对每个单文件 +`[pack] sign = ["", …, "{}"]` 在写容器之前对暂存树中的每个程序、映像、库、加载器与 Windows 启动器各运行一次，也对每个单文件 运行一次；`{}` 为该文件。失败使打包停止，写出 argv 与其输出；`mcpp-package.json` 写出签名后文件的摘要。 ## 配置项 @@ -781,10 +814,11 @@ URL 与每个架构的 sha256。安装把归档的顶层目录移到位，并向 [pack] default_mode = "static" # override the normal vendored default for bare `mcpp pack` linkage = "shared" # the default of --linkage -format = "tar" # the default of --pkg-format +format = "tar" # the default of --pkg-format; a list: ["tar", "okb"] +targets = ["openkal"] # the default of --target: targets and target sets +systems = ["linux", "macos"] # the default of --systems compress = "zstd" # the default of --compress sign = ["signtool", "sign", "{}"] # run on each program, image, library and loader before the container -universal = false # = --no-universal strip = true # default. false ships the artifacts as built debug_symbols = "dist/debug" # separate the debug info here instead of discarding it include = ["share/**", "config/*.toml"] # 工程文件，放在包中同样的相对路径下，只放一份 @@ -797,10 +831,12 @@ also_skip = ["libcustom.so"] # libraries assumed to exist on the target s force_bundle = ["libfoo.so"] # bundle even if matched by the PEP 600 list ``` -`[pack].default_mode` 当前接受既有的 manifest 拼写 `static`、 -`bundle-project` 与 `bundle-all`；`system` 这一 mode 只能通过 -`mcpp pack --mode system` 显式选择。CLI 输入同时接受上文的规范名称与兼容 -名称。 +`[pack].default_mode` 接受规范名 `system`、`vendored`、`self-contained`、`static`（2026.10.30.1+）与既有拼写 +`bundle-project`、`bundle-all`。 + +`[pack] universal`、`[platform] universal` 与 `--no-universal` 不再决定打包的内容（2026.10.30.1+； +[51](51-supported-versions.md)）：`[pack] universal = true` 被拒绝，出路为 `[pack] targets = ["openkal"]`；`false` 与 +`[platform] universal` 写出提示后忽略；`--no-universal` 与一个目标同用时写出提示，与目标集合同用时被拒绝。 `static` mode 还需要在 `[target.]` 下配置一套 musl 工具链；完整写法 参见 [`examples/03-pack-static`](../../examples/03-pack-static/) 中的 diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index 88969b52f..5b4ec9504 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -381,7 +381,7 @@ mcpp run --target x86_64-openkal # okrun bin/app.oke `okrun` 运行其自身架构的映像（`host = "same-arch"`）。在另一架构的机器上，`mcpp run` 与 `mcpp test` 拒绝该映像，并指出 `run_under` —— 陈述执行宿主的键。给出执行宿主后，runner 在那里运行，其程序是平台为映像 -架构准备的 runner：mcpp 为本机系统与该架构构建不携带任何内容的平台载体，即 self-contained 通用包放在 +架构准备的 runner：mcpp 为本机系统与该架构构建不携带任何内容的平台载体，即 self-contained 的映像树放在 `libexec/okrun/-/` 中的程序。用户态模拟器就是一种执行宿主；在装有 `qemu-aarch64` 的 x86_64 Linux 上： ```toml @@ -403,8 +403,8 @@ $ mcpp run --target aarch64-openkal `mcpp test --target aarch64-openkal` 以同样的方式运行测试。`MCPP_TARGET_AARCH64_OPENKAL_RUN_UNDER=qemu-aarch64` 为一次运行指定执行宿主，`~/.mcpp/config.toml` 中的 `[target.aarch64-openkal] run_under` 为一台机器指定。 `mcpp self env --target aarch64-openkal` 报告 runner 的程序与其架构、执行宿主的程序，或本机拒绝运行的原因。 -一个映像的 self-contained 包（`--no-universal --mode self-contained`）是本机系统、映像架构的程序 -（[10](10-pack-and-release.md)）。 +一个映像的单文件（`--pkg-format single-file`）是本机系统、映像架构的程序；它的 self-contained 包是带该运行器的 +映像树（[10](10-pack-and-release.md)）。 ### ok 映像与 Web diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index 282739ff7..a4ccd123e 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -60,8 +60,8 @@ mcpp run --target x86_64-openkal | 编译 | `[platform.build]`、`[platform.build.'cfg(…)']` | 目标每个翻译单元的标志 | | 链接 | `[platform.link]` | `driver`、`flags`、`start`、`script`、`inputs`、名字 `exe` 与 `shared`、封装步骤 `seal` | | 运行 | `runner`、`runners` | `mcpp run` 与 `mcpp test` 怎样启动产物,以及具名的设备动作 | -| 载体 | `carrier` | `mcpp pack` 为依赖加载器的产物按系统构建的程序:self-contained 的程序、通用包的运行器、单文件 | -| 包 | `universal` | 任一目标的打包覆盖全部目标,成为一个包([10](10-pack-and-release.md)) | +| 载体 | `carrier` | `mcpp pack` 为依赖加载器的产物按系统构建的程序:self-contained 树的运行器、单文件;一个表或表的列表,每项服务其 `hosts` 中的 `-` | +| 运行器存在之处 | `runner.hosts` | 运行器程序存在的 `-`:需要它的包在哪里运行(`runs_on.with`,[10](10-pack-and-release.md)) | | 代码 | `plugin` | 在平台阶段运行的构建程序,陈述随依赖图而变的部分 | | 版本组 | `[platform.versions]` | 一个家族中每个包的一个版本 | | 宿主部分 | `[platform.hosted]` | 宿主三元组选用该平台时适用的部分 | @@ -168,7 +168,6 @@ targets = ["x86_64-openkal", "aarch64-openkal"] toolchain = "llvm@23.1.3" runner = { cmd = ["okrun", "{}"], host = "same-arch" } carrier = { package = "openkal-run@0.1.3", section = ".oke", main = "carrier/main.cpp" } -universal = true plugin = "build.mcpp" [platform.layers] @@ -205,7 +204,7 @@ seal = [["okld", "note", "{}"], ["okld", "check", "{}", "--with", "{needed}"]] action,例如由暂存树生成 `okb`)。板级支持包的构建程序在构建阶段陈述同样的字段 ([34](34-authoring-a-bsp.md))。 - **载体的入口读取它携带的内容。** mcpp 把携带的字节放在 `mcpp_carried_begin` 与 `mcpp_carried_end` 之间 - (入口的名字在 `mcpp_carried_name`):程序为通用包的运行器时不携带任何内容,运行其第一个参数所指的映像; + (入口的名字在 `mcpp_carried_name`):程序为 self-contained 树的运行器时不携带任何内容,运行其第一个参数所指的映像; 或映像本身;或映像引用其他映像时的单文件所携带的全部映像的一个 stored zip(`PK\x03\x04`,每个映像对齐到 64 KiB)([SPEC-013](../specs/platforms.md) §11)。 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index fea4072c4..cda1dbd63 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -648,10 +648,12 @@ mcpp pack [target] [--pkg-format ] [--target ...] --message-format js | 字段 | | |---|---| -| `artifacts` | 每个产出的产物一条记录：`path`（绝对路径）、`type`（`file` 或 `directory`）、`format`（容器：`--pkg-format` 的取值；未给出时 Windows 程序为 `zip`，其余为 `tar`）、`targets`（进入该产物的每条腿的规范三元组），以及 `sha256`（文件的摘要；目录为 `null`；2026.10.27.1+）。被分派的格式报告本次请求引入的那些 action 的终端输出；一个多 `--target` 的 Android 打包报告一个产物，其 `targets` 列出每一条腿 | +| `artifacts` | 每个产出的产物一条记录：`path`（绝对路径）、`type`（`file` 或 `directory`）、`format`（容器：`--pkg-format` 的取值；未给出时 Windows 程序为 `zip`，其余为 `tar`）、`targets`（进入该产物的每条腿的规范三元组），以及 `sha256`（文件的摘要；目录为 `null`；2026.10.27.1+）。被分派的格式报告本次请求引入的那些 action 的终端输出；一个多 `--target` 的 Android 打包报告一个产物，其 `targets` 列出每一条腿；每腿一个包的打包（2026.10.30.1+）报告每个包及其自己的腿 | | `stage` | 该产物所来自的那棵树：`dir`、`manifest`（即下文的暂存清单）与 `closure`（`walked` 或 `not-walked`）；库包、没有暂存任何树的情形，以及对多个成员的打包，此字段为 `null` | | `linkage`、`mode`、`pkgFormat` *（2026.10.27.1+）* | 本次打包的三个选择（[10](10-pack-and-release.md)）：`linkage` 为 `static`、`shared` 或 `declared`；`pkgFormat` 为产物的 `format`（`tar`、`zip`、`dir`、`single-file` 或提供者的格式） | -| `legs`、`systems` *（2026.10.27.1+）* | 每条腿的规范三元组（通用包列出其平台的每个目标），以及自带加载器的通用包所携带的每个运行器的 `-`，或单个 ok 映像的 self-contained 包中作为程序的载体的 `-`（2026.10.29.1+） | +| `legs`、`systems` *（2026.10.27.1+）* | 命令命名的每条腿的规范三元组（目标集合已展开；2026.10.30.1+），以及自带运行器的映像树所携带的每个运行器的 `-` | +| `pkgFormats` *（2026.10.30.1+）* | 写出的全部容器，按 `--pkg-format` 命名的顺序；`pkgFormat` 为第一个 | +| `runsOn` *（2026.10.30.1+）* | 包在哪里运行，同其 `mcpp-package.json` 的 `runs_on`：`{"self": ["linux-x86_64", …], "with": {"okrun": […]}}`；单文件为各文件的组合 | | `stages` *（2026.10.1.1+）* | 只在对多个成员的打包（`--workspace`，或重复的 `-p`）中出现：每个成员一条记录，按 `[workspace] members` 的顺序，带 `member`（限定的包名），以及与 `stage` 相同的 `dir`、`manifest`、`closure` | 对多个成员的打包，`artifacts` 按同样的成员顺序列出每个成员的产物，每条记录多出 diff --git a/docs/zh/51-supported-versions.md b/docs/zh/51-supported-versions.md index 2ba476499..aefe888fb 100644 --- a/docs/zh/51-supported-versions.md +++ b/docs/zh/51-supported-versions.md @@ -67,6 +67,9 @@ schema，那是 [50 —— 机器可读输出](50-machine-output.md)。 | `abi-capability`：`abi:` 运行时能力（`abi:glibc`、`abi:libc=`、`abi:cxxstdlib=`），按目标侧某一层的需求读取 | 2026.10.26.1 | 2027.1 | `requires = ["mcpp:c-abi="]` 或 `["mcpp:c++-abi="]` | | `undeclared-cxx-runtime`：链接线点名、或链接线上的库需要的、本构建之外的另一套 C++ 运行时，且没有包声明 | 2026.10.26.1 | 2027.1 | 在需要它的包中写 `requires = [{ layer = "mcpp:c++-abi=", visibility = "private" }]` | | `plugin-imports-from-build-dependencies`：构建插件的库导入其自身 `[build-dependencies]` 中以 `host-module = true` 写出的包 | 2026.10.11.1 | 2027.1 | 插件 `[dependencies]` 中的该包 | +| `pack-universal`：`[pack] universal`——`true` 被拒绝，`false` 被忽略；打包的目标即其腿 | 2026.10.30.1 | 2027.1 | `[pack] targets = ["openkal"]`（全部架构），或一个 `--target` | +| `platform-universal`：平台的 `[platform] universal`，被忽略；打包的目标即其腿 | 2026.10.30.1 | 2027.1 | 平台中不写；由打包命名目标（`--target openkal`） | +| `no-universal-flag`：`mcpp pack --no-universal`，单目标打包不再需要它（与目标集合同用时拒绝） | 2026.10.30.1 | 2027.1 | `--target -`（一条腿） | ## 复现一次构建 diff --git a/modules/manifest/src/compat/registry.cppm b/modules/manifest/src/compat/registry.cppm index aaf0f31ef..177447b11 100644 --- a/modules/manifest/src/compat/registry.cppm +++ b/modules/manifest/src/compat/registry.cppm @@ -81,6 +81,25 @@ inline constexpr Deprecation kDeprecations[] = { "in the package that needs it", "manifest/deprecated", "docs/22-target-side.md", }, + { + "pack-universal", "2026.10.30.1", kRemovalFloor, + "`[pack] universal`: `true` refused, `false` ignored -- a pack's targets are its legs", + "[pack] targets = [\"openkal\"] (every architecture), or one --target", + "manifest/deprecated", "docs/10-pack-and-release.md", + }, + { + "platform-universal", "2026.10.30.1", kRemovalFloor, + "a platform's `[platform] universal`, ignored -- a pack's targets are its legs", + "nothing in the platform; the pack names its targets (`--target openkal`)", + "manifest/deprecated", "docs/35-platforms.md", + }, + { + "no-universal-flag", "2026.10.30.1", kRemovalFloor, + "`mcpp pack --no-universal`, which a pack of one target no longer needs (refused with a " + "target set)", + "--target - (one leg)", + "manifest/deprecated", "docs/10-pack-and-release.md", + }, }; // The row named `id`, or null. diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 0dc6a1ad9..715f4adc0 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -3799,10 +3799,11 @@ std::expected parse_document(const t::Document& documen // [pack] — `mcpp pack` configuration. See docs/10-pack-and-release.md. if (auto v = doc->get_string("pack.default_mode")) { const auto& s = *v; - if (s != "static" && s != "bundle-project" && s != "bundle-all") { + if (s != "system" && s != "vendored" && s != "self-contained" && s != "static" + && s != "bundle-project" && s != "bundle-all") { return std::unexpected(error(origin, std::format( - "[pack].default_mode = '{}' invalid; expected " - "'static' | 'bundle-project' | 'bundle-all'", s))); + "[pack].default_mode = '{}' invalid; expected 'system' | 'vendored' | " + "'self-contained' | 'static' (aliases: 'bundle-project', 'bundle-all')", s))); } m.packConfig.defaultMode = s; } @@ -3828,7 +3829,12 @@ std::expected parse_document(const t::Document& documen "[pack].linkage = '{}' invalid; expected 'static' | 'shared'", *v))); m.packConfig.linkage = *v; } - if (auto v = doc->get_string("pack.format")) m.packConfig.format = *v; + // `[pack] format` and `[pack] systems`: a string or a list (design + // 2026-10-29 §4.1, §4.6), checked with the flags that take the same words. + if (auto v = doc->get_string("pack.format")) m.packConfig.format = { *v }; + else if (auto a = doc->get_string_array("pack.format")) m.packConfig.format = *a; + if (auto v = doc->get_string("pack.systems")) m.packConfig.systems = { *v }; + else if (auto a = doc->get_string_array("pack.systems")) m.packConfig.systems = *a; if (auto v = doc->get_string("pack.compress")) m.packConfig.compress = *v; if (auto v = doc->get_string_array("pack.sign")) { if (v->empty()) diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index f2f523df7..b522834ca 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1838,11 +1838,12 @@ struct LibConfig { // `[pack]` — `mcpp pack` configuration. See docs/10-pack-and-release.md. // -// `default_mode` picks the bundling strategy when the user runs bare -// `mcpp pack` (no `--mode` flag): -// "static" — full musl static, no PT_INTERP / RUNPATH -// "bundle-project" — bundle only project's third-party .so (default) -// "bundle-all" — bundle every dynamic dep including libc / libstdc++ +// `default_mode` picks the mode when the user runs bare `mcpp pack` (no +// `--mode` flag), as `--mode` names it (SPEC-014 §1): +// "system" — the machine provides the libraries +// "vendored" (alias "bundle-project") — the project's libraries travel (default) +// "self-contained" (alias "bundle-all") — the machine provides nothing +// "static" — `--linkage static --mode self-contained" struct PackConfig { std::string defaultMode; // empty → "bundle-project" // THERE IS DELIBERATELY NO `[pack] profile`. Which profile `mcpp pack` @@ -1867,18 +1868,21 @@ struct PackConfig { std::vector alsoSkip; // libs to ALSO skip on top of PEP 600 std::vector forceBundle; // libs to bundle even if PEP 600 says skip // The three axes' project defaults (SPEC-014 §1): `linkage` ("static" | - // "shared"), `format` (the package format, as `--pkg-format` names it), - // `compress` (`[:]`, as `--compress` takes it). Empty: - // unstated. + // "shared"), `format` (the package formats, as `--pkg-format` names them: + // a string, or a list of containers of one staged tree), `compress` + // (`[:]`, as `--compress` takes it), `systems` (as + // `--systems` names them). Empty: unstated. std::string linkage; - std::string format; + std::vector format; std::string compress; + std::vector systems; // The signer (SPEC-014 §4): an argv run on each program, image and library // of the staged tree, and on a single file, before the container is // written; `{}` is the file. std::vector sign; - // `false`: a pack of a platform's target packs that target alone, where - // the platform states `universal = true` (SPEC-014 §3). + // `[pack] universal`, read only to answer it (COMPAT(pack-universal)): + // `true` is refused with `[pack] targets = ["openkal"]`, `false` is said + // and ignored (design 2026-10-29 §13.2). A pack's targets are its legs. std::optional universal; // The targets `mcpp pack` acts on when the command names none (SPEC-015 // §3.3): targets and target sets, as `--target` takes them (`openkal`, diff --git a/src/cli.cppm b/src/cli.cppm index 7eb1f13c4..4f919681b 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -112,6 +112,49 @@ void print_usage() { std::println("Docs: https://github.com/mcpp-community/mcpp/tree/main/docs"); } +// `mcpp pack --help`, IN TWO GROUPS (design 2026-10-29 §4.8): the options a +// pack is usually written with -- which targets, what the machine provides, +// the format, the runners or systems, the linkage, where -- and then the +// rest. cmdline lists options in one block in declaration order. +void print_pack_usage() { + std::println("Build + package: a program becomes a bundle, a library an interface + prebuilt binaries"); + std::println(""); + std::println("Usage: mcpp pack [target] [options]"); + std::println(""); + std::println("Options:"); + std::println(" --target A target (x86_64-linux-gnu) or a target set (openkal, linux-musl);"); + std::println(" repeatable: the legs (default: [pack] targets, else [build] target)"); + std::println(" --mode What the machine provides: system | vendored (default) |"); + std::println(" self-contained | static (= --linkage static --mode self-contained)"); + std::println(" --pkg-format tar | zip | dir | single-file | a format the resolved graph provides"); + std::println(" (okb, appimage, msi); a list of containers of one tree: tar,okb"); + std::println(" --systems The runners a self-contained package carries, or the systems single"); + std::println(" files are made for: linux | linux-aarch64 | all (comma-separated)"); + std::println(" --linkage static | shared: the program's libraries in it or beside it"); + std::println(" -o, --output The package's path; a directory (ending in /) when it writes several"); + std::println(""); + std::println("Advanced options:"); + std::println(" --compress The engine containers' compression: gzip | zstd | xz | none"); + std::println(" --merge Merge packages this command wrote (legs, systems) into one; repeatable"); + std::println(" --profile Build profile (default: [build] default-profile, else release)"); + std::println(" --release, --dev Shorthands for --profile release | dev"); + std::println(" --features Features of the root package, / of a dependency"); + std::println(" --no-strip Ship the artifacts as built (default: strip debug info)"); + std::println(" --debug-symbols Write the separated *.debug files here"); + std::println(" --toolchain Build with this toolchain for one invocation (llvm@22.1.8)"); + std::println(" --sysroot The target's base for this invocation (= MCPP_SYSROOT)"); + std::println(" --platform The target's platform for this invocation (= MCPP_PLATFORM)"); + std::println(" --host-platform The host graph's platform for this invocation"); + std::println(" -p, --package Pack the named workspace member; repeatable"); + std::println(" --workspace Pack every workspace member that has a program target"); + std::println(" --exclude With --workspace, leave the named member out"); + std::println(" --list-formats Print the formats --pkg-format accepts here (mcpp.pack.formats)"); + std::println(" --message-format human (default) | json (one mcpp.pack envelope on stdout)"); + std::println(" --no-universal Deprecated: a pack of one target packs that target alone"); + std::println(""); + std::println("Docs: docs/10-pack-and-release.md"); +} + // The ONE place this run's "has no UTF-8 spelling" records are reported: a // name that is not UTF-8 on POSIX, or one the process code page cannot spell // on a Windows host that ignores the UTF-8 code page mcpp.exe declares (#693). @@ -477,6 +520,14 @@ int run(int argc, char** argv) { if (argc >= 2) { std::string_view a = argv[1]; if (a == "--help" || a == "-h") { print_usage(); return 0; } + if (a == "pack") + for (int i = 2; i < argc; ++i) + if (std::string_view h = argv[i]; h == "--help" || h == "-h") { + print_pack_usage(); + return 0; + } else if (h == "--") { + break; + } if (a == "--version" || a == "-V") { std::println("mcpp {}", mcpp::toolchain::MCPP_VERSION); return 0; @@ -908,18 +959,18 @@ int run(int argc, char** argv) { .help("The program's libraries in it (static) or beside it (shared); rebuilds " "with [build] dependency_linkage (default: as the project builds)")) .option(cl::Option("target").takes_value().multiple() - .help("Triple, e.g. x86_64-linux-musl (repeatable: one leg per triple)")) + .help("A target (x86_64-linux-gnu) or a target set (openkal, linux-musl); " + "repeatable: the legs")) .option(cl::Option("no-universal") - .help("Pack the selected target alone where its platform packs all its targets " - "into one universal package")) - .option(cl::Option("systems").takes_value().value_name("LIST") - .help("Self-contained universal package: the systems whose loaders it carries " - "(linux,macos,windows; default: this machine's)")) + .help("Deprecated: a pack of one target packs that target alone")) + .option(cl::Option("systems").takes_value().multiple().value_name("LIST") + .help("The runners a self-contained package carries, or the systems single files " + "are made for: , -, all (default: this machine's)")) .option(cl::Option("compress").takes_value().value_name("ALG[:LEVEL]") .help("The container's compression: gzip (tar default) | zstd | xz | none; " "a zip deflates unless none")) .option(cl::Option("merge").takes_value().multiple().value_name("PACKAGE") - .help("Merge packages this command wrote (universal legs, systems) into one; " + .help("Merge packages this command wrote (legs, systems) into one; " "repeatable, nothing is built")) // "plus any format the resolved graph provides", because the // values are no longer a fixed list. A package declares one with @@ -928,12 +979,12 @@ int run(int argc, char** argv) { // constant, so the help text does not have to enumerate them. .option(cl::Option("pkg-format").takes_value().value_name("FORMAT") .help("The package format: tar | zip | dir | single-file | any format the " - "resolved graph provides (e.g. appimage, msi, okb); default: zip for a " - "Windows program, tar elsewhere")) + "resolved graph provides (e.g. appimage, msi, okb), or a list of containers " + "of one tree (tar,okb); default: zip for a Windows program, tar elsewhere")) .option(cl::Option("format").takes_value() .help("Earlier spelling of --pkg-format")) .option(cl::Option("output").short_name('o').takes_value() - .help("Override output path; with several members, the directory each archive or tree is written below")) + .help("Override output path; a directory (ending in /) when the pack writes several files")) // Packaging builds RELEASE by default — the artifact leaves this // machine. `[build] default-profile` still wins when it is set; // this only replaces the "dev" fallback every other command uses. diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 6a3ac1f32..be2f8a6f5 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -1135,6 +1135,12 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, mcpp::ui::warning("`mcpp run --format` is the earlier spelling of --pkg-format"); format = *f; } + // One format runs: a list is `mcpp pack`'s (design 2026-10-29 §4.6). + if (format.find(',') != std::string::npos) { + mcpp::ui::error(std::format("mcpp run --pkg-format names one format, and '{}' is a list.\n" + " use: --pkg-format {}", format, format.substr(0, format.find(',')))); + return 2; + } if (parsed.is_flag_set("list-runners")) return mcpp::build::list_runners(package_filter, cache_mode, no_cache, target_triple, features, profile, accel); diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index bea6b7144..4e1d4dba3 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -227,6 +227,30 @@ namespace { // of a reporting command everywhere else in mcpp (`self env --format json`), // and `mcpp pack`'s machine output is `--message-format`; pack's own // `--format` is accepted as the earlier spelling and says so. +// The words of a comma-separated value (`tar,okb`, `linux,macos-aarch64`), +// empty words dropped. +std::vector split_list(std::string_view v) { + std::vector out; + std::size_t at = 0; + while (at <= v.size()) { + auto comma = v.find(',', at); + if (comma == std::string_view::npos) comma = v.size(); + if (comma > at) out.emplace_back(v.substr(at, comma - at)); + at = comma + 1; + } + return out; +} + +// Where a pack of several packages runs: where any of them does. +void unite_runs_on(mcpp::pack::PackOutcome& into, const mcpp::pack::PackOutcome& from) { + auto add = [](std::vector& to, std::vector const& pairs) { + for (auto const& p : pairs) + if (std::ranges::find(to, p) == to.end()) to.push_back(p); + }; + add(into.runsOnSelf, from.runsOnSelf); + for (auto const& [runner, pairs] : from.runsOnWith) add(into.runsOnWith[runner], pairs); +} + std::optional pack_format_option(const mcpplibs::cmdline::ParsedArgs& parsed) { auto pkg = parsed.value("pkg-format"); auto old = parsed.value("format"); @@ -419,8 +443,9 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { }); } } else { - for (auto const& a : outcome.artifacts) - artifacts.push_back(artifact_json(a, outcome.format, outcome.targets)); + for (std::size_t i = 0; i < outcome.artifacts.size(); ++i) + artifacts.push_back(artifact_json(outcome.artifacts[i], outcome.format, + i < outcome.artifactTargets.size() ? outcome.artifactTargets[i] : outcome.targets)); if (!outcome.stageDir.empty()) stage = nlohmann::json{ {"dir", outcome.stageDir.lexically_normal().string()}, @@ -439,6 +464,13 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { env.data["pkgFormat"] = outcome.format; env.data["legs"] = outcome.targets; env.data["systems"] = outcome.systems; + // Every container, the first being `pkgFormat`, and where the package + // runs (design 2026-10-29 §3.5, §4.4): added fields, none redefined. + env.data["pkgFormats"] = outcome.formats.empty() ? std::vector{ outcome.format } + : outcome.formats; + nlohmann::json with = nlohmann::json::object(); + for (auto const& [runner, pairs] : outcome.runsOnWith) with[runner] = pairs; + env.data["runsOn"] = nlohmann::json{{"self", outcome.runsOnSelf}, {"with", std::move(with)}}; } mcpp::wire::emit(env); return 0; @@ -618,7 +650,7 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, for (auto const& i : inputs) req.inputs.emplace_back(i); if (auto v = parsed.value("output")) req.output = *v; if (auto f = pack_format_option(parsed)) req.format = *f; - else if (!packCfg.format.empty()) req.format = packCfg.format; + else if (!packCfg.format.empty()) req.format = packCfg.format.front(); // Only the flag's compression is the command's (a project's `[pack] // compress` describes that project's packs, not a merge of packages); // `merge` decides the container and refuses a compression that does @@ -653,14 +685,46 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, return 0; } - std::string formatText; - if (auto v = pack_format_option(parsed)) formatText = *v; - else formatText = packCfg.format; - if (auto v = pack_format_option(parsed); v && v->empty()) { - mcpp::ui::error("--pkg-format needs a value: tar | zip | dir | single-file | a format " - "the resolved graph provides"); - return 2; + // `--pkg-format` OR `[pack] format`: one format, or a list of the + // containers of one staged tree (design 2026-10-29 §4.6) -- `tar`, `zip` + // and a provider's format -- each written from the one build. + std::vector formatList; + if (auto v = pack_format_option(parsed)) { + if (v->empty()) { + mcpp::ui::error("--pkg-format needs a value: tar | zip | dir | single-file | a format " + "the resolved graph provides, or a list of containers (tar,okb)"); + return 2; + } + formatList = split_list(*v); + } else { + for (auto const& f : packCfg.format) + for (auto& w : split_list(f)) formatList.push_back(std::move(w)); + } + { + std::vector once; + for (auto& f : formatList) + if (std::ranges::find(once, f) == once.end()) once.push_back(std::move(f)); + formatList = std::move(once); } + if (formatList.size() > 1) { + for (auto const& f : formatList) + if (f == "dir" || f == "single-file") { + mcpp::ui::error(std::format( + "--pkg-format lists the containers of one staged tree (tar, zip, a provider's " + "format), and {} is not one: {}.\n use: --pkg-format {} alone", + f, f == "dir" ? "it is the staged tree itself" : "it carries the tree in a program", + f)); + return 2; + } + if (auto o = parsed.value("output"); o && !mcpp::pack::output_names_directory(*o)) { + mcpp::ui::error(std::format( + "--pkg-format names {} containers, and --output names one file.\n" + " use: -o / (each container under its own name)", formatList.size())); + return 2; + } + opts.formats = formatList; + } + std::string formatText = formatList.empty() ? std::string() : formatList.front(); if (!formatText.empty()) { opts.formatStated = true; if (formatText == "tar") opts.format = mcpp::pack::Format::Tar; @@ -715,13 +779,21 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, } using mcpp::pack::compress::Algorithm; const auto alg = opts.compression.algorithm; - const bool fits = - !(opts.format == mcpp::pack::Format::Dir || opts.format == mcpp::pack::Format::Dispatched) - && !(opts.format == mcpp::pack::Format::Zip && (alg == Algorithm::Zstd || alg == Algorithm::Xz)) + // A list's compression is its engine containers' (design 2026-10-29 + // §4.6): checked against those, the provider's formats aside. + const bool listsEngine = std::ranges::any_of(formatList, [](std::string const& f) { + return f == "tar" || f == "zip"; }); + const bool listsZip = std::ranges::find(formatList, "zip") != formatList.end(); + const bool writesNone = opts.formats.empty() + ? (opts.format == mcpp::pack::Format::Dir || opts.format == mcpp::pack::Format::Dispatched) + : !listsEngine; + const bool zipped = opts.format == mcpp::pack::Format::Zip || (!opts.formats.empty() && listsZip); + const bool fits = !writesNone + && !(zipped && (alg == Algorithm::Zstd || alg == Algorithm::Xz)) && !(opts.format == mcpp::pack::Format::SingleFile && (alg == Algorithm::Gzip || alg == Algorithm::Xz)); if (opts.compression.stated && !compressFlag && !fits) opts.compression = {}; if (opts.compression.stated) { - if (opts.format == mcpp::pack::Format::Dir || opts.format == mcpp::pack::Format::Dispatched) { + if (writesNone) { mcpp::ui::error(std::format( "--compress {} compresses a container, and --pkg-format {} writes none.\n" " use: --pkg-format tar | zip | single-file", @@ -729,7 +801,7 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, opts.format == mcpp::pack::Format::Dir ? std::string("dir") : opts.formatName)); return 2; } - if (opts.format == mcpp::pack::Format::Zip && (alg == Algorithm::Zstd || alg == Algorithm::Xz)) { + if (zipped && (alg == Algorithm::Zstd || alg == Algorithm::Xz)) { mcpp::ui::error(std::format( "a zip's members are deflated (gzip) or stored (none); --compress {} is a " "tar's.\n use: --pkg-format tar --compress {}, or --compress gzip | none", @@ -746,25 +818,26 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, } else if (opts.format == mcpp::pack::Format::SingleFile) { opts.compression.algorithm = Algorithm::Zstd; } - if (parsed.is_flag_set("no-universal")) opts.universal = false; - else opts.universal = packCfg.universal; - if (auto v = parsed.value("systems")) { - std::size_t at = 0; - while (at <= v->size()) { - auto comma = v->find(',', at); - if (comma == std::string::npos) comma = v->size(); - auto word = v->substr(at, comma - at); - if (!word.empty()) { - if (word != "linux" && word != "macos" && word != "windows") { - mcpp::ui::error(std::format( - "--systems names '{}'; the systems are linux, macos and windows", word)); - return 2; - } - if (std::ranges::find(opts.systems, word) == opts.systems.end()) - opts.systems.push_back(word); - } - at = comma + 1; + // `--systems` OR `[pack] systems` (design 2026-10-29 §4.1): `` + // (every architecture of the legs), `-`, or `all`. + std::vector systemWords; + if (auto o = parsed.option("systems")) + for (auto const& v : o->get().values) + for (auto& w : split_list(v)) systemWords.push_back(std::move(w)); + else { + systemWords = packCfg.systems; + opts.systemsFromManifest = !systemWords.empty(); + } + for (auto const& word : systemWords) { + if (word != "all" && word != "linux" && word != "macos" && word != "windows" + && !mcpp::manifest::platform::is_host_pair(word)) { + mcpp::ui::error(std::format( + "{} names '{}'; a system is linux, macos or windows, alone (every architecture " + "of the legs), as - (linux-aarch64), or all", + opts.systemsFromManifest ? "[pack] systems" : "--systems", word)); + return 2; } + if (std::ranges::find(opts.systems, word) == opts.systems.end()) opts.systems.push_back(word); } opts.sign = packCfg.sign; // A single file carries what it runs: the machine provides nothing. @@ -790,15 +863,57 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, if (auto v = parsed.value("debug-symbols")) opts.debugSymbols = *v; if (auto v = parsed.value("features")) opts.features = *v; - // `--target` is repeatable: one leg per triple, which is how a library - // package ships for several targets at once. The application path takes - // exactly one triple UNLESS the target is a `kind = "app"` whose form is - // a shared object on every requested row (#630 A9, below) — packing an - // executable for several triples would need several executables, and - // that refusal still stands for a `bin` target and for any row whose - // form is not a shared object. + // THE LEGS (design 2026-10-29 §4.2): every `--target`, else `[pack] + // targets`, a set (`openkal`, `linux-musl`) expanded to the target + // table's rows of the platform its members select; else the build's + // target. A platform is never a reason to pack more than was named. + std::vector requests; + if (auto o = parsed.option("target")) requests = o->get().values; + if (requests.empty()) requests = packCfg.targets; std::vector triples; - if (auto o = parsed.option("target")) triples = o->get().values; + bool fromSet = false; + if (!requests.empty()) { + mcpp::build::BuildOverrides selection; + selection.profile = opts.profile; + selection.features = opts.features; + auto legs = mcpp::toolchain::target_set::resolve(requests, + [&](const mcpp::toolchain::triple::Triple& t) { + return mcpp::build::platform_selection_for(t, selection); + }); + if (!legs) { mcpp::ui::error(legs.error()); return 2; } + for (auto const& l : *legs) { + triples.push_back(l.triple); + fromSet = fromSet || !l.fromSet.empty(); + } + } + // MIGRATION (design 2026-10-29 §13.2): a pack's targets are its legs. + // COMPAT(no-universal-flag) + if (parsed.is_flag_set("no-universal")) { + mcpp::pm::compat::record_use("no-universal-flag", "the command line", "--no-universal", true); + if (fromSet) { + mcpp::ui::error("--no-universal packs one target, and the target set names several: they " + "contradict.\n use: --target - for one leg, or leave " + "--no-universal out"); + return 2; + } + mcpp::ui::note(std::format( + "--no-universal is what a pack of one target does ({}); an image of a platform with a " + "runner is packed as its tree (bin/, bin/.exe, lib//), and one program " + "carrying the image is --pkg-format single-file", + mcpp::pm::compat::deprecation_clause(*mcpp::pm::compat::find_deprecation("no-universal-flag")))); + } + // COMPAT(pack-universal) + if (packCfg.universal) { + mcpp::pm::compat::record_use("pack-universal", "mcpp.toml", "[pack] universal", true); + if (*packCfg.universal) { + mcpp::ui::error("[pack] universal = true no longer decides what a pack holds: a pack's " + "targets are its legs.\n use: [pack] targets = [\"openkal\"]"); + return 2; + } + mcpp::ui::note(std::format( + "[pack] universal = false is what a pack of one target does; it is ignored ({})", + mcpp::pm::compat::deprecation_clause(*mcpp::pm::compat::find_deprecation("pack-universal")))); + } // `--linkage static` OF A LINUX PROGRAM IS THE MUSL ROW (SPEC-014 §1): a // program with no shared library at all, the C library's included, which // is what `--mode static` has always built. On another system the @@ -814,21 +929,14 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, t.replace(at, 10, "-linux-musl"); } } - if (!triples.empty()) opts.targetTriple = triples.back(); - // A program with no shared library is packed whole whatever the mode - // asked: there is nothing beside it to vendor, and nothing for the machine - // to provide. - if (opts.linkage == mcpp::pack::Linkage::Static - && opts.targetTriple.find("-linux-musl") != std::string::npos) { - opts.mode = mcpp::pack::Mode::Static; - modeFromUser = true; - } - // ─── Several members ───────────────────────────────────────────── + // ─── Which target, and therefore which kind of package ─────────── // - // Each is refused before anything is planned, let alone compiled: the - // command names one thing where several members are packed, and the - // members cannot share it. + // The positional is a target NAME. Its `kind` decides everything: a + // program becomes an application bundle (the four --mode depths), a + // library becomes a library package. Reading the answer out of the + // manifest is the whole reason there is no --lib flag. Several members + // pack programs. if (members.several) { if (const auto name = parsed.positional(0); !name.empty()) { mcpp::ui::error(std::format( @@ -838,50 +946,21 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, name, name)); return 2; } - if (triples.size() > 1) { - mcpp::ui::error( - "--target may be given once when several members are packed: each member's " - "program is built for one target,\n" - " and a pack for several targets stages one member's legs into one tree.\n" - " use: pack that member alone with -p "); - return 2; - } - opts.output.clear(); - if (typedOutput) { - std::error_code ec; - if (std::filesystem::exists(*typedOutput, ec) - && !std::filesystem::is_directory(*typedOutput, ec)) { - mcpp::ui::error(std::format( - "--output names a file, '{}', and several members are packed.\n" - " Each member's archive or tree is written below the directory --output " - "names, under the name\n" - " it has by default. use: --output ", typedOutput->string())); - return 2; - } - members.several->outputDir = *typedOutput; - } - auto out = mcpp::pack::build_and_pack_members(std::move(opts), modeFromUser, - *members.several); - if (report) *report = out; - return out.rc; } - - // ─── Which target, and therefore which kind of package ─────────── - // - // The positional is a target NAME. Its `kind` decides everything: a - // program becomes an application bundle (the four --mode depths), a - // library becomes a library package. Reading the answer out of the - // manifest is the whole reason there is no --lib flag. - auto route = mcpp::pack::route_pack_target(parsed.positional(0)); - if (!route) { mcpp::ui::error(route.error()); return 2; } - if (route->library) { + std::optional route; + if (!members.several) { + auto r = mcpp::pack::route_pack_target(parsed.positional(0)); + if (!r) { mcpp::ui::error(r.error()); return 2; } + route = std::move(*r); + } + if (route && route->library) { // A DISPATCHED FORMAT IS AN APPLICATION-BUNDLE OUTPUT, and a library // package has no bundle: `mcpp pack ` produces an interface plus // prebuilt binaries for one or more triples, and there is no single // staged tree for a member to turn into an installer. Refused rather // than ignored, because ignoring it would report `Packed` and hand back // a library package while the user asked for an installer. - if (opts.format == mcpp::pack::Format::Dispatched) { + if (opts.format == mcpp::pack::Format::Dispatched || !opts.formats.empty()) { mcpp::ui::error(std::format( "--pkg-format {} is a distributable produced from a program's staged " "bundle, and '{}' is a library target.\n" @@ -889,7 +968,7 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, "per triple; there is no\n" " single staged tree to hand a distribution format.\n" " use: --pkg-format tar | dir, or name a program target", - opts.formatName, route->targetName)); + opts.formats.empty() ? opts.formatName : formatText, route->targetName)); return 2; } if (modeFromUser) { @@ -898,60 +977,114 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, "library target '{}' yet; ignoring it", route->targetName)); } if (libraryRoute) *libraryRoute = true; + if (!triples.empty()) opts.targetTriple = triples.back(); return mcpp::pack::build_and_pack_library(route->targetName, triples, opts, libraryReport); } - // SEVERAL IMAGES' TARGETS ARE ONE UNIVERSAL PACKAGE'S LEGS (SPEC-014 - // §1.4): the pack builds the first and the others as its legs; whether the - // platform packs them into one package is decided once it is known. - if (triples.size() > 1 && std::ranges::all_of(triples, [](std::string const& t) { + + // THE PACKAGES (design 2026-10-29 §4.2): the images' legs are one tree + // (or one single file per leg and system), Android's rows one APK, and + // every other leg a package of its own, named by its triple. + std::vector> packs; + { + auto loaded = [](std::string const& t) { auto p = mcpp::toolchain::triple::parse(t); return p && p->loader_supplies_os(); - })) { - opts.legTargets = triples; - opts.targetTriple = triples.front(); - auto out = mcpp::pack::build_and_pack(std::move(opts), modeFromUser, route->targetName); - if (report) *report = out; - return out.rc; - } - if (triples.size() > 1) { - // #630 A9: THE ROUTE IS CHOSEN BY THE ARTIFACT'S FORM, NOT BY THE - // TARGET'S KIND. An `app` whose resolved link form is a shared - // object on EVERY requested row (Android) takes the several-triple - // path a library target already has: the other legs are built first - // (`build_extra_android_legs`, the same leg-loop shape - // `build_and_pack_library` uses) and staged as `lib//` beside - // the primary leg's own `lib//`, into ONE tree, behind ONE - // dispatch. A `bin` target, or any row whose form is an executable, - // keeps today's refusal — packing an executable for several triples - // would need several executables, and there is no "universal - // executable" mechanism the way there is a universal shared object - // (that is `lipo`, a different tool, and out of scope here). - if (!mcpp::pack::accepts_several_targets(*route, triples)) { - mcpp::ui::error( - "--target may be given once when packing a program: an application " - "bundle wraps one executable, and one executable has one target."); - return 2; + }; + std::vector images; + for (auto const& t : triples) if (loaded(t)) images.push_back(t); + const bool apk = route && triples.size() > 1 && mcpp::pack::accepts_several_targets(*route, triples); + if (apk) packs.push_back(triples); + else + for (auto const& t : triples) { + if (!loaded(t)) packs.push_back({ t }); + else if (t == images.front()) packs.push_back(images); + } + if (packs.empty()) packs.push_back({}); + } + if (packs.size() > 1 && opts.format == mcpp::pack::Format::Dir) { + mcpp::ui::error(std::format( + "this pack makes {} packages (one per leg), and --pkg-format dir names one tree.\n" + " use: --pkg-format tar | zip, or one --target", packs.size())); + return 2; + } + if (packs.size() > 1 && !opts.output.empty() && !mcpp::pack::output_names_directory(opts.output)) { + mcpp::ui::error(std::format( + "this pack makes {} packages (one per leg), and --output names one file.\n" + " use: -o / (each package under its own name)", packs.size())); + return 2; + } + + mcpp::pack::PackOutcome all; + for (auto const& legs : packs) { + auto one = opts; + bool oneFromUser = modeFromUser; + if (!legs.empty()) one.targetTriple = legs.front(); + // A program with no shared library is packed whole whatever the mode + // asked: there is nothing beside it to vendor, and nothing for the + // machine to provide. + if (one.linkage == mcpp::pack::Linkage::Static + && one.targetTriple.find("-linux-musl") != std::string::npos) { + one.mode = mcpp::pack::Mode::Static; + oneFromUser = true; } - auto extraLegs = mcpp::pack::build_extra_android_legs( - route->targetName, - std::span(triples).first(triples.size() - 1), - opts.profile, opts.features); - if (!extraLegs) return 1; // build_extra_android_legs already printed why - // #622 A10: `build_and_pack` now reports the artifact(s) it packed, for - // `mcpp run --format` to take as its operand -- `mcpp pack` itself only - // ever needed the exit code. - auto out = mcpp::pack::build_and_pack(std::move(opts), modeFromUser, - route->targetName, std::move(*extraLegs)); - if (report) *report = out; - return out.rc; - } - // #622 A10: `build_and_pack` now reports the artifact(s) it packed, for - // `mcpp run --format` to take as its operand -- `mcpp pack` itself only - // ever needed the exit code. - auto out = mcpp::pack::build_and_pack(std::move(opts), modeFromUser, route->targetName); - if (report) *report = out; - return out.rc; + mcpp::pack::PackOutcome out; + if (members.several) { + // Each member's package is written below the directory --output names. + one.output.clear(); + if (typedOutput) { + std::error_code ec; + if (std::filesystem::exists(*typedOutput, ec) + && !std::filesystem::is_directory(*typedOutput, ec)) { + mcpp::ui::error(std::format( + "--output names a file, '{}', and several members are packed.\n" + " Each member's archive or tree is written below the directory --output " + "names, under the name\n" + " it has by default. use: --output ", typedOutput->string())); + return 2; + } + members.several->outputDir = *typedOutput; + } + if (legs.size() > 1) one.legTargets = legs; + out = mcpp::pack::build_and_pack_members(std::move(one), oneFromUser, *members.several); + } else if (legs.size() > 1 && mcpp::pack::accepts_several_targets(*route, legs)) { + // #630 A9: THE ROUTE IS CHOSEN BY THE ARTIFACT'S FORM, NOT BY THE + // TARGET'S KIND. An `app` whose resolved link form is a shared + // object on EVERY requested row (Android): the other legs are built + // first and staged as `lib//` beside the primary leg's own, + // into ONE tree, behind ONE dispatch. + one.targetTriple = legs.back(); + auto extraLegs = mcpp::pack::build_extra_android_legs( + route->targetName, std::span(legs).first(legs.size() - 1), + one.profile, one.features); + if (!extraLegs) return 1; // build_extra_android_legs already printed why + out = mcpp::pack::build_and_pack(std::move(one), oneFromUser, route->targetName, + std::move(*extraLegs)); + } else { + if (legs.size() > 1) one.legTargets = legs; + out = mcpp::pack::build_and_pack(std::move(one), oneFromUser, route->targetName); + } + // Each package's artifacts with its own legs; the fields that name + // one tree are the first package's. + if (packs.size() > 1 && !members.several) + for (std::size_t i = 0; i < out.artifacts.size(); ++i) all.artifactTargets.push_back(out.targets); + if (&legs == &packs.front()) { + auto keep = std::move(all.artifactTargets); + all = out; + all.artifactTargets = std::move(keep); + } else { + all.rc = all.rc != 0 ? all.rc : out.rc; + all.ranBuildPrograms = all.ranBuildPrograms || out.ranBuildPrograms; + all.artifacts.insert(all.artifacts.end(), out.artifacts.begin(), out.artifacts.end()); + all.targets.insert(all.targets.end(), out.targets.begin(), out.targets.end()); + if (members.several) + all.members.insert(all.members.end(), out.members.begin(), out.members.end()); + unite_runs_on(all, out); + } + if (out.rc != 0) break; + } + if (report) *report = all; + return all.rc; } } // namespace diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index b79a5d438..89b77f817 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -1,11 +1,11 @@ // mcpp.pack.distribution — what a package is beyond its staged files // (SPEC-014 §4, §5): the containers, the package's metadata -// (`mcpp-package.json`), the launchers of a universal package, the payload and +// (`mcpp-package.json`), the launchers of a tree of images, the payload and // the launcher program of a single file, and `--merge`. // // THE ENGINE KNOWS NO PLATFORM HERE. A launcher is generated from the -// platform's `runner` and the legs; a loader of a self-contained universal -// package is the platform's carrier with nothing carried; a single file's +// platform's `runner` and the legs; a loader of a self-contained tree is the +// platform's carrier with nothing carried; a single file's // launcher is the engine's own program, written below as source and built for // the target at pack time like any program. @@ -102,7 +102,7 @@ struct Package { std::vector legs; // canonical triples std::vector systems; // `-` of the loaders under libexec/ std::vector runner; // empty for a native package - std::map entry; // arch -> path of the entry (universal) + std::map entry; // arch -> path of the entry (a tree of images) std::string program; // the program's path (one native leg) // WHERE THE PACKAGE RUNS (design 2026-10-29 §4.4), computed once when it // is packed: `self` the `-` pairs it runs on with what it @@ -145,7 +145,7 @@ std::string to_json(const Package& p); std::expected read(const std::filesystem::path& root); // The `-` directories of `root/libexec//`: the systems whose -// loaders a self-contained universal package carries, sorted. +// loaders a self-contained tree carries, sorted. std::vector loader_systems(const std::filesystem::path& root); // ─── a single file (SPEC-014 §5.2) ──────────────────────────────────────── @@ -189,7 +189,7 @@ struct Asset { }; // The xim-pkgindex entry (spec 2) of `assets`, fetched from `/`: a universal package is one asset every system installs, and its +// name>`: a tree of images is one asset every system it runs on installs, and its // runner is a dependency unless the package carries its loaders; packages of // one system and architecture each are that system's entry, the architecture // in the file name a `${arch}` of one URL. The install moves the archive's @@ -1068,8 +1068,12 @@ std::string exe_launcher_main(const Launch& l) { auto cxx = [](std::string_view w) { // a C++ raw string of `w` return std::format("R\"mcpp({})mcpp\"", w); }; - std::string entries, words; - for (auto const& [arch, file] : l.entry) entries += std::format(" {{ {}, {} }},\n", cxx(arch), cxx(file)); + // ONE PROGRAM FOR EVERY SET OF LEGS: the entry is the image of one name + // under each `lib//` (each leg's build of the one target), and the + // architectures are the directories the package holds -- so the program + // of a one-leg package is the program of the tree `--merge` makes of it. + std::string words; + const std::string entry = l.entry.empty() ? std::string() : l.entry.begin()->second; for (auto const& w : l.runner) words += std::format(" {},\n", cxx(w)); return std::format(R"MCPP(// The Windows launcher of {0} (design 2026-10-29 §4.3), written by mcpp pack. import std; @@ -1080,8 +1084,7 @@ namespace fs = std::filesystem; constexpr std::string_view kName = {1}; constexpr std::string_view kLoader = {2}; -constexpr std::pair kEntries[] = {{ -{3}}}; +constexpr std::string_view kEntry = {3}; constexpr std::string_view kRunner[] = {{ {4} std::string_view{{}}, }}; @@ -1096,16 +1099,16 @@ int main(int argc, char** argv) {{ for (auto& c : machine) c = static_cast(std::toupper(static_cast(c))); const std::string arch = machine == "AMD64" || machine == "X86_64" ? "x86_64" : machine == "ARM64" || machine == "AARCH64" ? "aarch64" : machine; - std::string_view entry; - for (auto const& [a, file] : kEntries) - if (a == arch) entry = file; - if (entry.empty()) {{ + std::error_code dec; + if (!fs::is_directory(top / "lib" / arch, dec)) {{ std::string all; - for (auto const& [a, file] : kEntries) all += (all.empty() ? "" : ", ") + std::string(a); - std::println(std::cerr, "{{}}: this machine is {{}}; the package holds {{}}", kName, arch, all); + for (auto const& d : fs::directory_iterator(top / "lib", dec)) + if (d.is_directory(dec)) all += (all.empty() ? "" : ", ") + d.path().filename().string(); + std::println(std::cerr, "{{}}: this machine is {{}}; the package holds {{}}", kName, arch, + all.empty() ? std::string("no architecture") : all); return 126; }} - const fs::path image = top / "lib" / arch / std::string(entry); + const fs::path image = top / "lib" / arch / std::string(kEntry); if (!fs::exists(image)) {{ std::println(std::cerr, "{{}}: the package has no {{}}", kName, image.string()); return 126; @@ -1142,7 +1145,7 @@ int main(int argc, char** argv) {{ std::println(std::cerr, "{{}}: cannot start {{}}", kName, program); return 127; }} -)MCPP", l.name, cxx(l.name), cxx(l.loader), entries, words); +)MCPP", l.name, cxx(l.name), cxx(l.loader), cxx(entry), words); } // ─── mcpp-package.json ──────────────────────────────────────────────────── @@ -1186,6 +1189,21 @@ std::optional host_pair_of(std::string_view triple) { return std::format("{}-{}", t->os, t->arch); } +// The pairs a package written before `runs_on` existed is taken to run on: +// every system, every architecture of its images. +std::vector every_pair(const Package& p) { + std::vector every; + for (auto system : { "linux", "macos", "windows" }) + for (auto const& [arch, _] : p.entry) every.push_back(std::format("{}-{}", system, arch)); + return every; +} + +// `from`'s pairs added to `to`, each once. +void add_pairs(std::vector& to, std::vector const& from) { + for (auto const& p : from) + if (std::ranges::find(to, p) == to.end()) to.push_back(p); +} + void compute_runs_on(Package& p, const std::vector& runnerHosts, bool runnerAnyArch) { p.runsOnSelf.clear(); p.runsOnWith.clear(); @@ -1583,9 +1601,9 @@ std::string distributed_instead(std::string_view triple) { if (t && t->is_wasm()) return "a page for the web, mcpp pack --pkg-format dist-web (mcpp-plugins)"; if (t && t->os == "openkal") - return "the platform's universal package (mcpp pack without --no-universal), which every " - "system installs, or a package that carries the image in its carrier (--mode " - "self-contained)"; + return "the tree of the platform's legs (mcpp pack --target openkal), which every system " + "its runner reaches installs, or a single file that carries the image in its carrier " + "(--pkg-format single-file)"; return "the package of a Linux, macOS or Windows target"; } @@ -1630,13 +1648,13 @@ std::expected xpkg_descriptor(const std::vector "package", assets.front().file.filename().string(), first.name, first.version, a.file.filename().string(), a.pkg.name, a.pkg.version)); - const bool universal = !first.runner.empty(); + const bool tree = !first.runner.empty(); std::vector archs; // system -> block text std::map blocks; - if (universal) { + if (tree) { if (assets.size() != 1) - return std::unexpected(std::string("a universal package is one asset; merge its legs first " + return std::unexpected(std::string("a tree of images is one asset; merge its legs first " "(mcpp pack --merge)")); auto const& a = assets.front(); // WHERE IT RUNS IS WHAT IT SAYS (design 2026-10-29 §4.4, #860): a @@ -1646,12 +1664,7 @@ std::expected xpkg_descriptor(const std::vector // first through `systems`, and the second not at all: every system, // every architecture of its images. Package pkg = first; - if (!pkg.runsOnSet) { - std::vector every; - for (auto system : { "linux", "macos", "windows" }) - for (auto const& [arch, _] : pkg.entry) every.push_back(std::format("{}-{}", system, arch)); - compute_runs_on(pkg, every, false); - } + if (!pkg.runsOnSet) compute_runs_on(pkg, every_pair(pkg), false); const bool carried = pkg.mode == "self-contained"; std::vector pairs = carried ? pkg.runsOnSelf : std::vector{}; if (!carried) @@ -1729,11 +1742,11 @@ std::expected xpkg_descriptor(const std::vector for (auto const& [system, text] : blocks) xpm += std::format(" {} = {{\n{} }},\n", system, text); // The command a person runs, and the entry's name: the launcher of a - // universal package (named after its images, `bin/`), else the + // tree of images (named after its images, `bin/`), else the // program -- not the package, which may be named otherwise // (`mcpp-language-server` runs as `mcppls`). std::string command; - if (universal && !first.entry.empty()) + if (tree && !first.entry.empty()) command = std::filesystem::path(first.entry.begin()->second).stem().string(); else if (!first.program.empty()) command = std::filesystem::path(first.program).stem().string(); @@ -1743,7 +1756,7 @@ std::expected xpkg_descriptor(const std::vector // under bin/ beside the root's convenience script); the entry then names // each system's directory. std::string bindir = "\"bin\"", alias, bindirs; - if (universal) { + if (tree) { alias = std::format("if os.host() == \"windows\" then opts.alias = \"{}.exe\" end\n ", lua_text(command)); } else { @@ -1823,7 +1836,7 @@ std::expected xpkg_descriptor(const std::vector " return true\n" "end\n", lua_text(command), lua_text(first.version), - universal ? std::format(", one package for every system ({})", lua_text(first.platform)) : std::string(), + tree ? std::format(", one package for every system ({})", lua_text(first.platform)) : std::string(), lua_list(archs), xpm, exts, bindir, alias, lua_text(first.name), bindirs); } @@ -1893,9 +1906,19 @@ std::expected merge(const MergeRequest& req) { if (r) return std::unexpected(*r); } // The merged package's directory: named after the output, as `mcpp pack - // -o` names it, else the first input's. - const fs::path wrapper = req.output.empty() ? trees.front().first.filename() - : fs::path(archive_directory(req.output)); + // -o` names it; else the first input's, where a tree of a platform's legs + // takes the platform's name in place of the first leg's triple (design + // 2026-10-29 §4.2: `hello-1.0-x86_64-openkal` -> `hello-1.0-openkal`). + fs::path wrapper = req.output.empty() ? trees.front().first.filename() + : fs::path(archive_directory(req.output)); + if (req.output.empty() && !first.runner.empty() && !first.platform.empty() && !first.legs.empty()) { + std::string platform = first.platform.substr(0, first.platform.find('@')); + platform = platform.substr(platform.find('.') + 1); + if (platform.ends_with("-platform")) platform.resize(platform.size() - 9); + std::string name = wrapper.string(); + if (auto at = name.find(first.legs.front()); at != std::string::npos && !platform.empty()) + wrapper = name.replace(at, first.legs.front().size(), platform); + } // The container: as asked, else as the output's name says, else as the // first input was written. @@ -1969,19 +1992,20 @@ std::expected merge(const MergeRequest& req) { merged.entry.clear(); Stated stated; std::map> seen; // path -> (digest, input) + std::string launcherName; for (std::size_t i = 0; i < trees.size(); ++i) { - auto const& [root, pkg] = trees[i]; + auto const& root = trees[i].first; + Package pkg = trees[i].second; + // A package written before `runs_on` says where it runs through + // `systems` and its images: every system of their architectures. + if (!pkg.runsOnSet) compute_runs_on(pkg, every_pair(pkg), false); for (auto const& l : pkg.legs) if (std::ranges::find(merged.legs, l) == merged.legs.end()) merged.legs.push_back(l); for (auto const& s : pkg.systems) if (std::ranges::find(merged.systems, s) == merged.systems.end()) merged.systems.push_back(s); // Where the merged package runs is where any of its inputs ran. - for (auto const& s : pkg.runsOnSelf) - if (std::ranges::find(merged.runsOnSelf, s) == merged.runsOnSelf.end()) merged.runsOnSelf.push_back(s); - for (auto const& [r, pairs] : pkg.runsOnWith) - for (auto const& h : pairs) - if (std::ranges::find(merged.runsOnWith[r], h) == merged.runsOnWith[r].end()) - merged.runsOnWith[r].push_back(h); + add_pairs(merged.runsOnSelf, pkg.runsOnSelf); + for (auto const& [r, pairs] : pkg.runsOnWith) add_pairs(merged.runsOnWith[r], pairs); merged.runsOnSet = merged.runsOnSet || pkg.runsOnSet; for (auto const& [arch, e] : pkg.entry) merged.entry[arch] = e; // Each file as it is, not as its package says: a file changed after the @@ -1989,6 +2013,15 @@ std::expected merge(const MergeRequest& req) { // its bytes do. for (auto const& f : pkg.files) { const auto digest = sha256_file(root / fs::path(f.path)); + // THE LAUNCHERS ARE DERIVED (design 2026-10-29 §4.3): written + // again below from the merged package, so two inputs' launchers + // never disagree. + if (f.role == "launcher" && !f.path.ends_with(".exe") && f.path.starts_with("bin/")) { + launcherName = f.path.substr(4); + stated.role[f.path] = f.role; + continue; + } + if (f.role == "launcher" && seen.contains(f.path)) continue; if (digest != f.sha256) return std::unexpected(std::format( "--merge: '{}' in {} has sha256 {}, and its {} states {}", f.path, @@ -2017,6 +2050,25 @@ std::expected merge(const MergeRequest& req) { std::ranges::sort(pairs); std::erase_if(pairs, [&](std::string const& h) { return std::ranges::find(merged.runsOnSelf, h) != merged.runsOnSelf.end(); }); } + std::erase_if(merged.runsOnWith, [](auto const& kv) { return kv.second.empty(); }); + merged.runsOnSet = true; + // The POSIX launcher of the merged legs; the Windows launcher is one + // program for every set of legs (`exe_launcher_main`). + if (!merged.runner.empty() && !launcherName.empty()) { + Launch launch; + launch.name = launcherName; + launch.runner = merged.runner; + // The package states each entry's path; the launcher names its file. + for (auto const& [arch, path] : merged.entry) launch.entry[arch] = fs::path(path).filename().string(); + if (merged.mode == "self-contained") launch.loader = fs::path(merged.runner.front()).stem().string(); + const auto script = out / "bin" / launcherName; + fs::create_directories(script.parent_path(), ec); + if (!spit(script, posix_launcher(launch))) + return std::unexpected(std::format("cannot write '{}'", script.string())); + fs::permissions(script, fs::perms::owner_all | fs::perms::group_read | fs::perms::group_exec + | fs::perms::others_read | fs::perms::others_exec, + fs::perm_options::replace, ec); + } if (auto d = describe(out, merged, stated); !d) return std::unexpected(d.error()); if (format == "dir") { diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 7829f27d3..93fe34f2f 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -98,8 +98,8 @@ enum class Linkage { Unstated, Static, Shared }; // single file, a dispatched format). enum class ArchiveKind { None, Tar, Zip }; -// One more architecture of a universal package (SPEC-014 §3): a leg of the -// platform's targets other than the one the pack built first, built by the +// One more architecture of a tree (design 2026-10-29 §4.2): a leg the +// command named other than the one the pack built first, built by the // pipeline before staging. `artifact` is that leg's program image; the images // it names are found beside it, as for the first leg. struct LoadedLeg { @@ -138,17 +138,20 @@ struct Options { bool formatStated = false; mcpp::pack::compress::Choice compression; bool compressionFromFlag = false; // --compress, not [pack] - // Universal packages (SPEC-014 §3). `universal`: nullopt follows the - // platform's `universal`; false packs the selected target alone. When the - // pipeline decides a universal package it sets `universalName` (the - // platform's name, which names the package instead of a triple) and - // `systems` (the systems whose loaders a self-contained package carries, - // ``, from `--systems`; empty: this machine's). - std::optional universal; + // THE TREE OF A PLATFORM WITH A RUNNER (design 2026-10-29 §4.2, §4.3): + // `tree` is set by the pipeline for an image of such a platform, whatever + // the number of legs -- `bin/`, `bin/.exe`, `lib//` -- + // and `universalName` (the platform's name, which names the package + // instead of a triple) when the tree holds more than one leg. `systems`: + // the runners a self-contained package carries, or the systems a single + // file is made for, each ``, `-` or `all` (`--systems`; + // empty: this machine's). + bool tree = false; std::string universalName; std::vector systems; - // Several `--target` naming a universal platform's targets: the legs, in - // place of the platform's whole `targets` (SPEC-014 §1.4). + bool systemsFromManifest = false; // `[pack] systems`, not --systems + // The legs a pack of several targets stages into one tree (design + // 2026-10-29 §4.2): the command's targets, a set expanded. std::vector legTargets; // `[pack] sign`: run on each program, image, library and loader of the // staged tree before the container is written, `{}` the file. @@ -164,7 +167,14 @@ struct Options { // programs have declared what they provide. It arrives before anything is // compiled, which is the earliest point at which it can be exact. std::string formatName; + // `--pkg-format` naming several containers (design 2026-10-29 §4.6): + // every format, in order, the first being `format`/`formatName`; each + // after the first is written from the same staged tree. Empty: one. + std::vector formats; std::filesystem::path output; // empty = derive from manifest + // `-o /` (design 2026-10-29 §4.2, §4.6): every file the pack + // writes is written below it under its default name. + std::filesystem::path outputDir; std::string targetTriple; // empty = host // Where a dependency NAME may be resolved to a file. // @@ -282,7 +292,7 @@ struct Plan { // `shared`, or `declared` (each library as it declares itself). Set by the // caller from the build. std::string linkageName = "declared"; - // The other legs of a universal package (SPEC-014 §3), set by the caller + // The other legs of a tree or a single file (SPEC-014 §3), set by the caller // after `make_plan` as `extraSharedLegs` is. std::vector loadedLegs; std::string packageName; @@ -540,14 +550,31 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg); std::expected, Error> single_file(const Plan& plan); // The row a program of `system` (linux | macos | windows) and `arch` is built -// on -- an image's single file, a universal package's runner -- when this mcpp +// on -- an image's single file, a self-contained tree's runner -- when this mcpp // builds no program for it: a row its table states as planned (`x86_64-macos`) // or does not state (`aarch64-windows-gnu`). Empty when it builds one. std::string unbuilt_row(std::string_view system, std::string_view arch); +// The systems `--systems` names for an architecture (design 2026-10-29 +// §4.1): an ``, an `-` of that architecture, or every system +// for `all`; this machine's system when it names none. +std::vector systems_for(const std::vector& stated, std::string_view arch); + +// The single files of an image (design 2026-10-29 §4.5): one per leg and +// system of the leg's architecture, the legs in the plan's order. `skipped` +// is why a row `--systems all` reaches is made nowhere (this mcpp builds no +// carrier for it); empty for a row that is made. +struct SingleFileRow { std::string system, arch, skipped; }; +std::vector single_file_rows(const Plan& plan); + +// `-o` names a directory (design 2026-10-29 §4.2, §4.5): it ends in a +// separator or is an existing directory. A pack writing more than one file +// writes each under it with its default name. +bool output_names_directory(const std::filesystem::path& output); + // The `-` of the loaders a package carries (`systems` of its // `mcpp-package.json` and of the `mcpp.pack` envelope, SPEC-014 §4.5): a -// self-contained universal package's under `libexec//`, and for a +// self-contained tree's under `libexec//`, and for a // self-contained package of one image its carrier, which is the program -- of // this machine's system and the image's architecture. `mcpp emit xpkg` reads // it as the system the carrier runs on (§8). @@ -555,7 +582,7 @@ std::vector carried_systems(const Plan& plan); // THE PLATFORM'S RUNNER FOR ANOTHER ARCHITECTURE (SPEC-013 §10, §11): the // carrier built with nothing carried, for this machine's system and `arch`, -// named `name` -- the program a self-contained universal package carries under +// named `name` -- the program a self-contained tree carries under // `libexec//-/`, and the one `mcpp run` and `mcpp test` hand an // execution host of that architecture. Built under // `/target/carrier-pack/--/`; returns the program. @@ -571,6 +598,11 @@ runner_program(const mcpp::platform::Resolved& platform, const std::filesystem:: // staged without its closure is finalized by the caller. std::expected finalize_staged(const Plan& plan); +// One more engine container (`tar` or `zip`) of a tree already staged and +// finalized, under its default name beside the plan's own (design 2026-10-29 +// §4.6): a `--pkg-format` list writes each container from one tree. +std::expected write_another_container(const Plan& plan, ArchiveKind kind); + // Helpers used by cli.cppm to render mode names + parse `--mode`. // Canonical name shown in `--help`/diagnostics (renamed for legibility). std::string_view mode_cli_name(Mode m); @@ -822,12 +854,12 @@ make_plan(const mcpp::manifest::Manifest& manifest, mcpp::pack::compress::name(opts.compression.algorithm))}); p.opts.compression = {}; } - // A universal package is named after its platform, not a triple: it is + // A tree of several legs is named after its platform, not a triple: it is // the same package on every system and architecture of the platform. const std::string nameTriple = opts.universalName.empty() ? p.triple : opts.universalName; auto distDir = projectRoot / "target" / "dist"; if (opts.output.empty()) { - p.archivePath = distDir / detail::default_archive_name( + p.archivePath = (opts.outputDir.empty() ? distDir : opts.outputDir) / detail::default_archive_name( p.packageName, p.packageVersion, nameTriple, opts.mode, p.archive == ArchiveKind::None ? ArchiveKind::Tar : p.archive, opts.compression.algorithm); @@ -1605,7 +1637,7 @@ std::expected finalize_tree(const Plan& plan) { for (auto const& l : plan.loadedLegs) pkg.legs.push_back(l.triple); for (auto const& l : plan.extraSharedLegs) pkg.legs.push_back(l.triple); d::Stated stated; - if (!plan.opts.universalName.empty()) { + if (plan.opts.tree) { pkg.runner = plan.platform.fields.runner.cmd; const auto stem = fs::path(plan.binaryName).stem().string(); stated.role["bin/" + stem] = "launcher"; @@ -1648,6 +1680,25 @@ std::expected finalize_tree(const Plan& plan) { auto windows = [](std::string const& p) { return p.starts_with("windows-"); }; std::erase_if(pkg.runsOnSelf, windows); for (auto& [r, pairs] : pkg.runsOnWith) std::erase_if(pairs, windows); + std::erase_if(pkg.runsOnWith, [](auto const& kv) { return kv.second.empty(); }); + } + // A TREE THAT RUNS NOWHERE IS NOT A PACKAGE (design 2026-10-29 §4.4): it + // carries no runner a system can start, and the platform names no + // system its runner exists for. A platform that states no + // `runner.hosts` says nothing either way, which is said, not refused. + if (plan.opts.tree && pkg.runsOnSelf.empty() && pkg.runsOnWith.empty()) { + auto const& runner = plan.platform.fields.runner; + if (runner.hostsSet || plan.opts.mode == Mode::BundleAll) + return std::unexpected(Error{std::format( + "{} runs on no system: it carries no runner built here{}, and {} names no " + "system its runner `{}` exists for (runner.hosts).\n use: --mode self-contained " + "--systems ", + plan.binaryName, plan.opts.mode == Mode::BundleAll ? "" : " (--mode vendored)", + plan.platform.display(), runner.cmd.empty() ? std::string() : runner.cmd.front())}); + mcpp::ui::note(std::format( + "{} states no runner.hosts, so the package names no system it runs on " + "(mcpp-package.json runs_on is empty); an index entry cannot be written for it", + plan.platform.display())); } // What `[pack] include` named is the package's resource, whatever its name. for (auto const& rel : *included) stated.role[rel] = "resource"; @@ -1655,7 +1706,9 @@ std::expected finalize_tree(const Plan& plan) { if (!described) return std::unexpected(Error{described.error()}); if (plan.opts.sign.empty()) return {}; for (auto const& f : described->files) { - if (f.role != "program" && f.role != "image" && f.role != "library" && f.role != "loader") + // Every program of `bin/` and `libexec/`, the Windows launcher with them. + if (f.role != "program" && f.role != "image" && f.role != "library" && f.role != "loader" + && !(f.role == "launcher" && f.path.ends_with(".exe"))) continue; std::vector argv; for (auto const& a : plan.opts.sign) @@ -2236,13 +2289,13 @@ run_wasm(const Plan& plan) // built by mcpp with that system's own conventions -- the // bootstrap boundary (§6.2), so no platform is selected for it. // -// A UNIVERSAL PACKAGE (SPEC-014 §3, §4.2) holds every target of a platform -// that states `universal = true`: one `lib//` per leg, launchers in -// `bin/` that pick the machine's architecture and start the platform's -// runner, and under `self-contained` the runner's program for each system of -// `--systems` and each architecture -- the carrier built with nothing carried, -// which runs the image its first argument names -- under -// `libexec//-/`. +// THE TREE (SPEC-014 §3, §4.2) is the one layout of an image of a platform +// that states a runner, whatever the number of legs: one `lib//` per +// leg, launchers in `bin/` that pick the machine's architecture and start the +// platform's runner, and under `self-contained` the runner's program for each +// system of `--systems` and each architecture of the legs -- the carrier built +// with nothing carried, which runs the image its first argument names -- +// under `libexec//-/`. // // `--linkage static` is a property of the image (no `.okso`), and `--mode // static` is `--linkage static --mode self-contained`. @@ -2577,8 +2630,9 @@ run_loaded(const Plan& plan) }; const bool bundles = plan.opts.mode != Mode::None; - if (plan.opts.universalName.empty()) { - // ONE LEG: the image in `bin/`, the images it names beside it, or + if (!plan.opts.tree) { + // NO TREE (a single file's staging, or a platform that states no + // runner): the image in `bin/`, the images it names beside it, or // under `self-contained` the carrier that carries it (M3). std::vector needed; if (bundles) { @@ -2610,13 +2664,12 @@ run_loaded(const Plan& plan) if (auto r = copy(n, plan.stagingRoot / "bin" / n.filename()); !r) return r; if (auto r = stage_runtime_files(plan, staged.parent_path()); !r) return r; } else { - // A UNIVERSAL PACKAGE: a `lib//` per leg, the launchers, and the - // runner's program per system and architecture under `self-contained`. + // THE TREE: a `lib//` per leg, the launchers, and the runner's + // program per system and architecture under `self-contained`. auto const& runner = plan.platform.fields.runner; if (!runner.set || runner.cmd.empty()) return std::unexpected(Error{std::format( - "a universal package of {} starts its images through the platform's runner, and " - "{} states none.\n use: --no-universal", + "the tree of {} starts its images through the platform's runner, and {} states none", plan.binaryName, plan.platform.display())}); struct Leg { std::string triple, arch; fs::path artifact; }; std::vector legs{ { plan.triple, arch_of(plan.triple), plan.builtBinary } }; @@ -2638,10 +2691,8 @@ run_loaded(const Plan& plan) if (auto r = stage_runtime_files(plan, dir, leg.artifact.parent_path()); !r) return r; } if (plan.opts.mode == Mode::BundleAll) { - std::vector systems = plan.opts.systems; - if (systems.empty()) systems.emplace_back(host_system()); - for (auto const& system : systems) - for (auto const& leg : legs) { + for (auto const& leg : legs) + for (auto const& system : mcpp::pack::systems_for(plan.opts.systems, leg.arch)) { // A row this mcpp builds no program for (a planned // target, `x86_64-macos`) carries no runner: said, and // the package's metadata lists the systems it carries. @@ -3320,10 +3371,67 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) } } +std::expected write_another_container(const Plan& plan, ArchiveKind kind) { + Plan q = plan; + q.archive = kind; + if (kind == ArchiveKind::Zip + && (q.opts.compression.algorithm == mcpp::pack::compress::Algorithm::Zstd + || q.opts.compression.algorithm == mcpp::pack::compress::Algorithm::Xz)) + q.opts.compression = {}; + q.archivePath = (q.opts.outputDir.empty() ? q.projectRoot / "target" / "dist" : q.opts.outputDir) + / detail::default_archive_name( + q.packageName, q.packageVersion, q.opts.universalName.empty() ? q.triple : q.opts.universalName, + q.opts.mode, kind, q.opts.compression.algorithm); + if (auto r = detail::write_container(q); !r) return std::unexpected(r.error()); + return q.archivePath; +} + std::expected finalize_staged(const Plan& plan) { return detail::finalize_tree(plan); } +std::vector systems_for(const std::vector& stated, std::string_view arch) { + if (stated.empty()) return { std::string(detail::host_system()) }; + std::vector out; + auto add = [&](std::string os) { + if (std::ranges::find(out, os) == out.end()) out.push_back(std::move(os)); + }; + for (auto const& e : stated) { + if (e == "all") { + for (auto os : { "linux", "macos", "windows" }) add(os); + continue; + } + const auto dash = e.find('-'); + if (dash == std::string::npos) add(e); + else if (std::string_view(e).substr(dash + 1) == arch) add(e.substr(0, dash)); + } + return out; +} + +std::vector single_file_rows(const Plan& plan) { + std::vector archs{ detail::arch_of(plan.triple) }; + for (auto const& l : plan.loadedLegs) archs.push_back(l.arch); + const bool all = std::ranges::find(plan.opts.systems, "all") != plan.opts.systems.end(); + std::vector rows; + for (auto const& arch : archs) + for (auto const& system : systems_for(plan.opts.systems, arch)) { + SingleFileRow r{ system, arch, {} }; + if (all) + if (auto row = unbuilt_row(system, arch); !row.empty()) + r.skipped = std::format("no carrier row: this mcpp builds no program for {}", row); + rows.push_back(std::move(r)); + } + return rows; +} + +bool output_names_directory(const std::filesystem::path& output) { + if (output.empty()) return false; + const auto text = output.native(); + if (text.back() == '/' || text.back() == '\\') return true; + std::error_code ec; + return std::filesystem::is_directory(output, ec); +} + std::string unbuilt_row(std::string_view system, std::string_view arch) { const auto row = detail::program_row(system, arch, true); return detail::row_is_built(row) ? std::string() : row.triple; @@ -3332,7 +3440,7 @@ std::string unbuilt_row(std::string_view system, std::string_view arch) { std::vector carried_systems(const Plan& plan) { // The one-leg tree of a single file holds the images, not the carrier // (`run_loaded`): the carrier is the file, made per system of `--systems`. - if (plan.opts.universalName.empty() && plan.targetIsLoaded && plan.opts.mode == Mode::BundleAll + if (!plan.opts.tree && plan.targetIsLoaded && plan.opts.mode == Mode::BundleAll && plan.opts.format != Format::SingleFile) return { std::format("{}-{}", detail::host_system(), detail::arch_of(plan.triple)) }; return mcpp::pack::distribution::loader_systems(plan.stagingRoot); @@ -3375,8 +3483,8 @@ std::expected, Error> single_file(const Plan& const auto distDir = plan.projectRoot / "target" / "dist"; auto output_for = [&](std::string_view qualifier, std::string_view suffix) { if (plan.opts.output.empty()) - return distDir / std::format("{}-{}-{}{}", plan.packageName, plan.packageVersion, - qualifier, suffix); + return (plan.opts.outputDir.empty() ? distDir : plan.opts.outputDir) + / std::format("{}-{}-{}{}", plan.packageName, plan.packageVersion, qualifier, suffix); if (!plan.opts.output.has_parent_path()) return distDir / plan.opts.output; return plan.opts.output; }; @@ -3385,42 +3493,65 @@ std::expected, Error> single_file(const Plan& // THE IMAGE, OR THE IMAGES AS ONE STORED ZIP: what the carrier's // entry reads between `mcpp_carried_begin` and `mcpp_carried_end` // (SPEC-013 §11): the image alone when it names no other, else every - // image of `bin/` as members of a zip whose data lies on 64 KiB - // boundaries, the entry named by `mcpp_carried_name`. - std::vector images; - for (auto const& e : fs::directory_iterator(plan.stagingRoot / "bin", ec)) - if (e.is_regular_file(ec)) images.push_back(e.path()); - std::ranges::sort(images); - fs::path carried = plan.stagingRoot / "bin" / plan.binaryName; - if (images.size() > 1) { - carried = plan.projectRoot / "target" / "carrier-pack" - / (fs::path(plan.binaryName).stem().string() + ".carried.zip"); + // image it needs as members of a zip whose data lies on 64 KiB + // boundaries, the entry named by `mcpp_carried_name`. One file per + // leg and system of the leg's architecture (design 2026-10-29 §4.5): + // the first leg's images are the staged `bin/`, another leg's are its + // artifact and the images that artifact names. + struct Leg { std::string arch; std::vector images; fs::path carried; }; + std::vector legs; + { + Leg first{ detail::arch_of(plan.triple), {}, {} }; + for (auto const& e : fs::directory_iterator(plan.stagingRoot / "bin", ec)) + if (e.is_regular_file(ec)) first.images.push_back(e.path()); + legs.push_back(std::move(first)); + } + for (auto const& l : plan.loadedLegs) { + auto n = detail::needed_images(l.artifact, plan.binaryName); + if (!n) return std::unexpected(n.error()); + Leg leg{ l.arch, { l.artifact }, {} }; + leg.images.insert(leg.images.end(), n->begin(), n->end()); + legs.push_back(std::move(leg)); + } + for (auto& leg : legs) { + std::ranges::sort(leg.images); + leg.carried = leg.images.empty() ? plan.stagingRoot / "bin" / plan.binaryName : leg.images.front(); + if (leg.images.size() < 2) continue; + leg.carried = plan.projectRoot / "target" / "carrier-pack" + / std::format("{}-{}.carried.zip", fs::path(plan.binaryName).stem().string(), leg.arch); + fs::create_directories(leg.carried.parent_path(), ec); std::vector entries; - for (auto const& i : images) entries.push_back({ i.filename().string(), i, false, {} }); + for (auto const& i : leg.images) entries.push_back({ i.filename().string(), i, false, {} }); mcpp::pack::zip::WriteOptions o; o.align = 65536; - if (auto r = mcpp::pack::zip::write(carried, entries, o); !r) + if (auto r = mcpp::pack::zip::write(leg.carried, entries, o); !r) return std::unexpected(Error{r.error()}); } - auto t = mcpp::toolchain::triple::parse(plan.triple); - const std::string arch = t ? t->arch : std::string(mcpp::toolchain::platform::host_arch); - std::vector systems = plan.opts.systems; - if (systems.empty()) systems.emplace_back(detail::host_system()); - if (systems.size() > 1 && !plan.opts.output.empty()) + struct File { std::string system, arch; fs::path carried; }; + std::vector files; + for (auto const& r : single_file_rows(plan)) { + if (!r.skipped.empty()) continue; + files.push_back({ r.system, r.arch, std::ranges::find(legs, r.arch, &Leg::arch)->carried }); + } + if (files.empty()) + return std::unexpected(Error{std::string( + "--systems names no system this mcpp makes a single file for, of the legs' " + "architectures")}); + if (files.size() > 1 && !plan.opts.output.empty()) return std::unexpected(Error{std::format( - "--output names one file, and --systems makes one single file per system ({}).\n" - " use: leave --output out; each file is named ---", - systems.size())}); - for (auto const& system : systems) { + "--output names one file, and this pack makes {} single files (one per leg and " + "system).\n use: -o / (each file is named ---)", + files.size())}); + for (auto const& f : files) { const auto stem = fs::path(plan.binaryName).stem().string(); - auto packed = detail::carrier_build(plan, { carried, plan.binaryName, stem, system, arch, true }); + auto packed = detail::carrier_build(plan, { f.carried, plan.binaryName, stem, f.system, f.arch, true }); if (!packed) return std::unexpected(packed.error()); - const auto suffix = system == "windows" ? std::string(".exe") : std::string(); + const auto suffix = f.system == "windows" ? std::string(".exe") : std::string(); const auto program = *packed / (stem + suffix); if (!fs::is_regular_file(program, ec)) return std::unexpected(Error{std::format("the carrier built for {}-{} has no {}", - system, arch, program.filename().string())}); - const auto out = output_for(std::format("{}-{}", system, arch), suffix); + f.system, f.arch, program.filename().string())}); + const auto out = output_for(std::format("{}-{}", f.system, f.arch), suffix); fs::create_directories(out.parent_path(), ec); fs::copy_file(program, out, fs::copy_options::overwrite_existing, ec); if (ec) return std::unexpected(Error{std::format("cannot write {}: {}", out.string(), ec.message())}); diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 6cec35820..a187a62f7 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -69,6 +69,15 @@ export struct PackOutcome { std::string linkage; std::string mode; std::vector systems; + // Every container written, in the order `--pkg-format` named them (`format` + // is the first), and where the package runs (design 2026-10-29 §3.5, §4.4): + // its `runs_on`, or a single file's own pairs. + std::vector formats; + std::vector runsOnSelf; + std::map> runsOnWith; + // A pack of a package per leg (design 2026-10-29 §4.2): each artifact's + // legs, beside `artifacts`; empty when every artifact holds `targets`. + std::vector> artifactTargets; // Whether a build program ran in this pack, for the envelope's `effects`. bool ranBuildPrograms = false; // What one packed member answered (member selection design 2026-09-30, @@ -88,6 +97,14 @@ export struct PackOutcome { std::string linkage; std::string mode; std::vector systems; + std::vector formats; + std::vector runsOnSelf; + std::map> runsOnWith; + // What the summary says after `Packed` (design 2026-10-29 §4.4, + // §4.8): the pairs a requested system or architecture is not covered + // on, each with its reason, and the hint of a one-leg tree. + std::vector notCovered; + std::string hint; }; std::vector members; }; @@ -417,19 +434,19 @@ bool member_carries(const mcpp::build::BuildContext& ctx, const MemberJob& m, // // Called inside the member's view (`with_member`): `ctx.manifest`, // `ctx.projectRoot` and the plan's link group are the member's. -// Whether a pack of `ctx`'s target is a universal package (SPEC-014 §3): an -// image of a platform that states `universal = true`, not refused by the -// command or `[pack] universal`, in a format other than a single file. -bool packs_universal(const mcpp::build::BuildContext& ctx, const Options& opts) { +// Whether a pack of `ctx`'s target stages the tree of a platform with a runner +// (design 2026-10-29 §4.2, §4.3): an image of a platform that states a +// runner, in a format other than a single file, whatever the number of legs. +bool packs_tree(const mcpp::build::BuildContext& ctx, const Options& opts) { auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); auto const& P = ctx.plan.platform; - return t && t->loader_supplies_os() && P.active && P.fields.universal.set - && P.fields.universal.value && opts.universal.value_or(true) + return t && t->loader_supplies_os() && P.active && P.fields.runner.set && !P.fields.runner.cmd.empty() && opts.format != mcpp::pack::Format::SingleFile; } // The combinations of a member's pack refused before anything is built -// (SPEC-014 §1.5): what its options and the plan's platform already decide. +// (SPEC-014 §1.5, design 2026-10-29 §4.7): what its options and the plan's +// platform already decide. std::optional refused_combination(const mcpp::build::BuildContext& ctx, const Options& opts) { // A single file carries what it runs, whichever source stated the mode. if (opts.format == mcpp::pack::Format::SingleFile && opts.mode == mcpp::pack::Mode::None) @@ -439,27 +456,32 @@ std::optional refused_combination(const mcpp::build::BuildContext& // the platform's carrier); a WebAssembly module is no program of a system, // and no launcher of `wasm32-linux-musl` (the row this would have tried) // exists. The web is reached through a provider's page. - if (auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); - opts.format == mcpp::pack::Format::SingleFile && t && t->is_wasm()) + auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); + if (opts.format == mcpp::pack::Format::SingleFile && t && t->is_wasm()) return std::format("--pkg-format single-file makes an executable of one system, and {} " "builds a WebAssembly module a JavaScript engine runs.\n" " use: --pkg-format tar | zip | dir, or a provider's format " "(dist-web of mcpp-plugins writes a page)", t->str()); - if (!opts.legTargets.empty() && !packs_universal(ctx, opts)) - return std::string("several --target name one package's legs only where the target's platform " - "packs them into one universal package (`[platform] universal = true`).\n" - " use: one pack per target"); - if (!packs_universal(ctx, opts)) return std::nullopt; auto const& P = ctx.plan.platform; - if (!P.fields.runner.set || P.fields.runner.cmd.empty()) - return std::format("a universal package starts its images through the platform's runner, and " - "{} states none.\n use: --no-universal", P.display()); - auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); + // Several images are one package's legs where the platform's runner + // starts them (the tree's launchers), or as single files. + if (opts.legTargets.size() > 1 && !packs_tree(ctx, opts) && opts.format != mcpp::pack::Format::SingleFile) + return std::format("several images are one package's legs where the platform starts them through " + "its runner, and {} states none.\n use: one pack per target, or " + "--pkg-format single-file", P.active ? P.display() : std::string("the target")); for (auto const& target : opts.legTargets) { auto lt = mcpp::toolchain::triple::parse(target); if (lt && t && lt->str() != t->str() && !mcpp::platform::serves(P.fields, P.package, *lt)) return std::format("--target {} is not a target of {}", lt->str(), P.display()); } + // okb holds the images alone (okb v1): a self-contained tree's runner + // would be dropped (design 2026-10-29 §4.6). + const bool selfContained = opts.mode == mcpp::pack::Mode::BundleAll || opts.mode == mcpp::pack::Mode::Static; + const auto& named = opts.formats.empty() ? std::vector{ opts.formatName } : opts.formats; + if (packs_tree(ctx, opts) && selfContained + && std::ranges::all_of(named, [](std::string const& f) { return f == "okb"; })) + return std::string("--mode self-contained carries the runner under libexec/, and okb holds the " + "images alone; the runner would be dropped.\n use: --pkg-format tar,okb"); return std::nullopt; } @@ -629,13 +651,14 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { } } - // ─── A loaded artifact: the static row, and the universal package ── + // ─── A loaded artifact: the static row, the tree and its legs ───── // // `--mode static` of an image is `--linkage static --mode self-contained` - // (SPEC-014 §1.2): one `.oke`, carried. A platform that states `universal - // = true` packs every one of its targets into one package unless the - // command or `[pack] universal` says otherwise (§3); a single file is one - // system's and one architecture's, so it is never universal. + // (SPEC-014 §1.2). An image of a platform that states a runner is packed + // as its tree whatever the number of legs (design 2026-10-29 §4.3); the + // legs are the targets the command named, a set expanded (§4.2), and a + // tree of several is named after the platform. A single file is made per + // leg and system. std::vector loadedLegs; { auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); @@ -643,15 +666,26 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { if (loaded && opts.mode == mcpp::pack::Mode::Static) opts.mode = mcpp::pack::Mode::BundleAll; auto const& P = ctx.plan.platform; if (auto why = refused_combination(ctx, opts)) return Refusal{2, about(run, m, *why)}; - if (packs_universal(ctx, opts)) { + // COMPAT(platform-universal): `[platform] universal` decides nothing. + static bool saidUniversal = false; + if (loaded && P.active && P.fields.universal.set && !std::exchange(saidUniversal, true)) { + mcpp::pm::compat::record_use("platform-universal", P.package, "[platform] universal", true); + mcpp::ui::note(std::format( + "{} states `universal`, which no longer decides what a pack holds: the legs are the " + "targets the command names ({}); `--target {}` packs every architecture", + P.display(), mcpp::pm::compat::deprecation_clause( + *mcpp::pm::compat::find_deprecation("platform-universal")), + mcpp::manifest::platform::platform_name(P.package.substr(P.package.find('.') + 1)))); + } + opts.tree = packs_tree(ctx, opts); + if (opts.tree && opts.legTargets.size() > 1) opts.universalName = mcpp::manifest::platform::platform_name( P.package.substr(P.package.find('.') + 1)); - // The legs: those the command named, else every target of the - // platform. The other legs are planned one after another and - // built at once (mcpp.build.legs); this leg is the pack's own. - auto const& legs = opts.legTargets.empty() ? P.fields.targets.value : opts.legTargets; + if (loaded && opts.legTargets.size() > 1) { + // The other legs are planned one after another and built at once + // (mcpp.build.legs); this leg is the pack's own. std::vector specs; - for (auto const& target : legs) { + for (auto const& target : opts.legTargets) { auto lt = mcpp::toolchain::triple::parse(target); if (!lt || lt->str() == t->str()) continue; mcpp::build::legs::Spec spec{g.ov}; @@ -693,7 +727,7 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { break; } if (artifact.empty()) return Refusal{1, about(run, m, std::format( - "the {} leg of the universal package built no '{}'", leg.triple, chosenLu->targetName))}; + "the {} leg built no '{}'", leg.triple, chosenLu->targetName))}; auto lt = mcpp::toolchain::triple::parse(leg.triple); mcpp::ui::status("Packed leg", leg.triple); loadedLegs.push_back({ leg.triple, lt ? lt->arch : std::string{}, artifact }); @@ -842,7 +876,6 @@ std::optional prepare_group(PackRun& run, GroupJob& g) { // refusals of a dispatched `--format`, before the build. std::optional check_format(PackRun& run, GroupJob& g) { auto const& ctx = *g.ctx; - const auto& formatName = run.opts.formatName; // What each member's options and the platform decide. for (auto const& m : g.members) { std::optional why; @@ -851,64 +884,94 @@ std::optional check_format(PackRun& run, GroupJob& g) { }); if (why) return Refusal{2, about(run, m, *why)}; } - // ─── `--systems` names the loaders a package carries ───────────── + // ─── `--systems` names the runners a package carries ───────────── // - // A self-contained universal package carries a loader per system, and a - // single file of an image is made per system; nothing else is made per - // system. Refused here, before the build (SPEC-014 §1.5). + // A self-contained tree carries a runner per system and architecture of + // its legs, and a single file of an image is made per leg and system; + // nothing else is made per system. Refused here, before the build + // (SPEC-014 §1.5, design 2026-10-29 §4.7); `[pack] systems` that does not + // fit the pack is said and left aside, as `[pack] compress` is. if (!run.opts.systems.empty()) { auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); const bool loaded = t && t->loader_supplies_os(); - const bool universal = packs_universal(ctx, run.opts); + const bool tree = packs_tree(ctx, run.opts); const bool selfContained = run.opts.mode == mcpp::pack::Mode::BundleAll || run.opts.mode == mcpp::pack::Mode::Static; - if (!(universal && selfContained) && !(loaded && run.opts.format == mcpp::pack::Format::SingleFile)) - return Refusal{2, universal - ? std::format("--systems names the systems whose loaders a self-contained package " - "carries, and --mode {} carries none.\n use: --mode self-contained", - mcpp::pack::mode_cli_name(run.opts.mode)) - : std::string("--systems names the systems whose loaders a self-contained universal " - "package carries, or for which a single file of an image is made; this " - "pack is neither.\n use: --mode self-contained over a platform that " - "states universal = true, or --pkg-format single-file")}; - // A single file is made per system, and `-o` names one file. - if (run.opts.format == mcpp::pack::Format::SingleFile && run.opts.systems.size() > 1 - && !run.opts.output.empty()) - return Refusal{2, std::format( - "--output names one file, and --systems makes one single file per system ({}).\n" - " use: leave --output out; each file is named ---", - run.opts.systems.size())}; - // A single file of an image is made for each system on this mcpp's - // row of it; a row it builds no program for is refused before the build. + const bool singleFile = loaded && run.opts.format == mcpp::pack::Format::SingleFile; + if (!(tree && selfContained) && !singleFile) { + if (run.opts.systemsFromManifest) { + mcpp::ui::note(std::format( + "[pack] systems names the runners a self-contained tree carries, or the systems " + "single files of an image are made for; this pack is neither (--mode {}), so it " + "is left aside", mcpp::pack::mode_cli_name(run.opts.mode))); + for (auto& m : g.members) m.request.systems.clear(); + run.opts.systems.clear(); + } else { + return Refusal{2, tree + ? std::format("--systems names the runners a self-contained package carries, and " + "--mode {} carries none.\n use: --mode self-contained", + mcpp::pack::mode_cli_name(run.opts.mode)) + : std::string("--systems names the runners a self-contained package of images " + "carries, or the systems single files of an image are made for; this " + "pack is neither.\n use: --mode self-contained over a platform that " + "states a runner, or --pkg-format single-file")}; + } + } + } + if (!run.opts.systems.empty()) { + auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); + const bool loaded = t && t->loader_supplies_os(); + std::vector archs; + for (auto const& l : run.opts.legTargets) + if (auto lt = mcpp::toolchain::triple::parse(l)) archs.push_back(lt->arch); + if (archs.empty() && t) archs.push_back(t->arch); + // An `-` of no leg's architecture names nothing the pack makes. + for (auto const& e : run.opts.systems) { + const auto dash = e.find('-'); + if (dash == std::string::npos) continue; + if (std::ranges::find(archs, e.substr(dash + 1)) == archs.end()) + return Refusal{2, std::format( + "--systems names {}, and no leg of this pack is of {}", e, e.substr(dash + 1))}; + } + // A single file of an image is made for each leg and system on this + // mcpp's row of it; a row it builds no program for, named, is refused + // before the build (`all` names the rows there are, and says the rest). if (loaded && run.opts.format == mcpp::pack::Format::SingleFile) { + std::size_t files = 0; std::vector unbuilt, built; - for (auto const& system : run.opts.systems) { - auto row = mcpp::pack::unbuilt_row(system, t->arch); - (row.empty() ? built : unbuilt).push_back(row.empty() ? system : std::format("{} ({})", system, row)); - } + const bool all = std::ranges::find(run.opts.systems, "all") != run.opts.systems.end(); + for (auto const& arch : archs) + for (auto const& system : mcpp::pack::systems_for(run.opts.systems, arch)) { + auto row = mcpp::pack::unbuilt_row(system, arch); + if (row.empty()) built.push_back(std::format("{}-{}", system, arch)); + else if (!all) unbuilt.push_back(std::format("{}-{} ({})", system, arch, row)); + if (row.empty() || !all) ++files; + } + if (files > 1 && !run.opts.output.empty()) + return Refusal{2, std::format( + "--output names one file, and this pack makes {} single files (one per leg and " + "system).\n use: -o / (each file is named ---)", + files)}; if (!unbuilt.empty()) { std::string say, use; for (auto const& u : unbuilt) say += (say.empty() ? "" : ", ") + u; for (auto const& b : built) use += (use.empty() ? "" : ",") + b; return Refusal{2, std::format( - "--systems: this mcpp builds no program for {}, so no single file of the {} " - "image is made there.\n use: {}", say, t->str(), + "--systems: this mcpp builds no program for {}, so no single file is made there.\n" + " use: {}", say, use.empty() ? std::string("--systems naming a system this mcpp builds for") : "--systems " + use)}; } } } - // ─── Is the requested format one anything provides? ────────────── - // - // BEFORE THE BUILD, because a refusal that arrives after a full compile is - // a worse refusal, and because this is the earliest point at which it can be - // exact: build programs have now run and declared what they provide. - // - // The set is read from a pass that asked for NOTHING. That is what the - // "declare unconditionally, submit conditionally" rule buys -- a member - // that declared only when asked would leave this list empty exactly when a - // user names a format, and the refusal would name nothing. - if (run.opts.format == mcpp::pack::Format::Dispatched) { + // Every format a `--pkg-format` list names past the engine's is checked + // here, before the one build (design 2026-10-29 §4.6). + std::vector dispatched; + if (run.opts.format == mcpp::pack::Format::Dispatched) dispatched.push_back(run.opts.formatName); + for (auto const& f : run.opts.formats) + if (!mcpp::pack::is_builtin_pack_format(f) && std::ranges::find(dispatched, f) == dispatched.end()) + dispatched.push_back(f); + for (auto const& formatName : dispatched) { auto const& provided = ctx.plan.providedPackFormats; if (std::ranges::find(provided, formatName) == provided.end()) { std::string avail; @@ -1141,6 +1204,53 @@ std::optional stage_member(PackRun& run, GroupJob& g, MemberJob& m) { for (auto const& leg : plan->extraSharedLegs) out.targets.push_back(leg.triple); for (auto const& leg : plan->loadedLegs) out.targets.push_back(leg.triple); out.systems = mcpp::pack::carried_systems(*plan); + out.formats = opts.formats.empty() ? std::vector{ out.format } : opts.formats; + // Where it runs (design 2026-10-29 §4.4): the package's `runs_on`, or a + // single file's pairs; and what a requested pair is not covered on. + if (opts.format == mcpp::pack::Format::SingleFile && plan->targetIsLoaded) { + for (auto const& r : mcpp::pack::single_file_rows(*plan)) { + const auto pair = std::format("{}-{}", r.system, r.arch); + if (r.skipped.empty()) out.runsOnSelf.push_back(pair); + else out.notCovered.push_back(std::format("{} ({})", pair, r.skipped)); + } + } else if (auto pkg = mcpp::pack::distribution::read(plan->stagingRoot)) { + out.runsOnSelf = pkg->runsOnSelf; + out.runsOnWith = pkg->runsOnWith; + } + if (opts.tree) { + auto covered = [&](std::string const& pair) { + if (std::ranges::find(out.runsOnSelf, pair) != out.runsOnSelf.end()) return true; + for (auto const& [r, pairs] : out.runsOnWith) + if (std::ranges::find(pairs, pair) != pairs.end()) return true; + return false; + }; + std::vector archs; + for (auto const& triple : out.targets) + if (auto lt = mcpp::toolchain::triple::parse(triple)) archs.push_back(lt->arch); + const bool carries = opts.mode == mcpp::pack::Mode::BundleAll; + const auto stem = std::filesystem::path(plan->binaryName).stem().string(); + std::error_code xec; + const bool windowsLauncher = std::filesystem::is_regular_file(plan->stagingRoot / "bin" / (stem + ".exe"), xec); + for (auto const& arch : archs) + for (auto const& system : carries ? mcpp::pack::systems_for(opts.systems, arch) + : std::vector{}) { + const auto pair = std::format("{}-{}", system, arch); + if (covered(pair)) continue; + const auto row = mcpp::pack::unbuilt_row(system, arch); + out.notCovered.push_back(std::format("{} ({})", pair, + !row.empty() ? std::format("no runner row: this mcpp builds no program for {}", row) + : system == "windows" && !windowsLauncher ? std::string("no Windows launcher") + : std::string("not in the platform's runner.hosts"))); + } + // ONE LEG OF A PLATFORM OF SEVERAL (design 2026-10-29 §4.8): said, with + // the spelling that packs them all. + auto const& P = ctx.plan.platform; + if (out.targets.size() == 1 && P.active && P.fields.targets.value.size() > 1 && !archs.empty()) { + const auto set = mcpp::manifest::platform::platform_name(P.package.substr(P.package.find('.') + 1)); + out.hint = std::format("only {}; `--target {}` or `[pack] targets = [\"{}\"]` packs every " + "architecture of {}", archs.front(), set, set, P.display()); + } + } if (m.stageFailure.empty()) { out.stageDir = plan->stagingRoot; out.stageManifest = mcpp::pack::stage_manifest_path(plan->stagingRoot); @@ -1433,11 +1543,25 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba reported.push_back(std::move(abs)); } if (!produced) continue; + // `-o /`: the distributable below it, under its own name. + if (!run.opts.outputDir.empty()) + for (auto& r : reported) { + const auto into = run.opts.outputDir / r.filename(); + std::filesystem::remove_all(into, ec); + std::filesystem::copy(r, into, std::filesystem::copy_options::recursive, ec); + if (ec) { + mcpp::ui::error(about(run, m, std::format("cannot write {}: {}", into.string(), ec.message()))); + m.result.rc = 1; + break; + } + r = into; + } + if (m.result.rc != 0) continue; report_packed(reported, m.pathCtx); // Every output consumed by another: a cycle a provider should not // write, reported as all outputs rather than as nothing. if (reported.empty()) reported = std::move(intermediate); - m.result.artifacts = std::move(reported); + m.result.artifacts.insert(m.result.artifacts.end(), reported.begin(), reported.end()); } return std::nullopt; } @@ -1477,6 +1601,36 @@ std::optional check_destinations(PackRun& run, std::vector& g return std::nullopt; } +// The dispatch pass of each group for the format `run.opts` names; a refusal +// fails the members of its group (P3), and the command when one member is +// packed: the exit code is returned then. +std::optional dispatch_each_group(PackRun& run, std::vector& groups, mcpp::build::Backend& be) { + for (auto& g : groups) { + if (g.rc != 0) continue; + auto refused = dispatch_group(run, g, be); + if (!refused) continue; + // A failed drive was reported where it failed. + if (!refused->message.empty()) mcpp::ui::error(refused->message); + for (auto& m : g.members) + if (m.result.rc == 0) m.result.rc = refused->rc; + if (!run.several) return refused->rc; + } + return std::nullopt; +} + +// Where a packed member runs, as the summary says it (design 2026-10-29 +// §4.4): `self`'s pairs, then each runner's. +std::string runs_on_text(const PackOutcome::Member& r) { + std::string runs; + for (auto const& p : r.runsOnSelf) runs += (runs.empty() ? "" : ", ") + p; + for (auto const& [runner, pairs] : r.runsOnWith) { + std::string with; + for (auto const& p : pairs) with += (with.empty() ? "" : ", ") + p; + runs += std::format("{}with {}: {}", runs.empty() ? "" : "; ", runner, with); + } + return runs; +} + // The pack: the plan of each configuration group and what is refused from it, // then for each group the build, each member's staging and the one dispatch // pass. `selection` is null for the package of the directory the command runs @@ -1618,6 +1772,46 @@ PackOutcome run_pack(PackRun run) { } } + // ─── The other containers of a `--pkg-format` list ─────────────── + // + // Written from the tree each member staged above, in the order named + // (design 2026-10-29 §4.6): an engine container beside the first, a + // provider's format by its dispatch pass. + std::vector staged; + for (auto& g : groups) + if (g.rc == 0) + for (auto& m : g.members) staged.push_back(&m); + for (std::size_t k = 1; k < run.opts.formats.size(); ++k) { + auto const& f = run.opts.formats[k]; + if (f != "tar" && f != "zip") { + run.opts.format = mcpp::pack::Format::Dispatched; + run.opts.formatName = f; + if (auto refused = dispatch_each_group(run, groups, *be)) return PackOutcome{*refused}; + continue; + } + for (auto* m : staged) { + if (m->result.rc != 0 || !m->plan) continue; + auto written = mcpp::pack::write_another_container( + *m->plan, f == "zip" ? mcpp::pack::ArchiveKind::Zip : mcpp::pack::ArchiveKind::Tar); + if (!written) { + mcpp::ui::error(about(run, *m, written.error().message)); + m->result.rc = 1; + if (!run.several) return PackOutcome{1}; + continue; + } + mcpp::ui::status("Packed", mcpp::ui::shorten_path(*written, m->pathCtx)); + m->result.artifacts.push_back(std::move(*written)); + } + } + + // ─── Where it runs (design 2026-10-29 §4.4, §4.8) ──────────────── + for (auto* m : staged) { + if (m->result.rc != 0) continue; + if (auto runs = runs_on_text(m->result); !runs.empty()) mcpp::ui::info("Runs on", about(run, *m, runs)); + for (auto const& n : m->result.notCovered) mcpp::ui::info("Not covered", about(run, *m, n)); + if (!m->result.hint.empty()) mcpp::ui::note(about(run, *m, m->result.hint)); + } + // ─── What was packed ───────────────────────────────────────────── // // A failing member is reported and the others continue (P3): the status is @@ -1657,6 +1851,9 @@ PackOutcome run_pack(PackRun run) { out.linkage = first.linkage; out.mode = first.mode; out.systems = first.systems; + out.formats = first.formats; + out.runsOnSelf = first.runsOnSelf; + out.runsOnWith = first.runsOnWith; return out; } @@ -1690,6 +1887,14 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, modeFromUser = true; // user-equivalent intent — block manifest override } PackRun run; + // `-o /` (design 2026-10-29 §4.2, §4.6): each file under its + // default name below it. A `dir` pack's `-o` names the tree itself. + if (opts.format != mcpp::pack::Format::Dir && mcpp::pack::output_names_directory(opts.output)) { + opts.outputDir = opts.output; + opts.output.clear(); + std::error_code dec; + std::filesystem::create_directories(opts.outputDir, dec); + } run.opts = std::move(opts); run.modeFromUser = modeFromUser; run.wantTarget = wantTarget; diff --git a/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh b/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh index 4625f9dd0..82aa3a87c 100755 --- a/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh +++ b/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh @@ -87,24 +87,25 @@ depcount=$(find "${staged2}bin" -name res.txt | wc -l) || fail "the deploy'd file was staged $depcount times, not once" pack2.log echo "the deploy'd file is staged once across both legs OK" -# ── 3. negative direction: an executable-form row refuses a second triple ─ +# ── 3. other rows: one package per leg (design 2026-10-29 §4.2) ────────── # # `x86_64-linux-gnu` is not Android: `application_form` answers `Executable` -# there, so `accepts_several_targets` must refuse this request exactly as it -# refused before #630 A9 existed. -out=$("$MCPP" pack myapp --target x86_64-linux-gnu --target x86_64-linux-gnu 2>&1) \ - && fail "packing an app for two executable-form triples must be refused" <(echo "$out") -expected="--target may be given once when packing a program: an application bundle wraps one executable, and one executable has one target." -grep -qF -- "$expected" <<<"$out" \ - || fail "the refusal message does not match today's wording" <(echo "$out") -echo "two executable-form triples are refused with today's message OK" +# there, so the rows are not one APK. A target named twice is one leg; an +# Android row and an executable row are a package each, named by its triple, +# and `-o` naming one file is refused. +"$MCPP" pack myapp --target x86_64-linux-gnu --target x86_64-linux-gnu --pkg-format dir > pack3.log 2>&1 \ + || fail "a target named twice is one leg" pack3.log +[ -d target/dist/myapp-0.1.0-x86_64-linux-gnu ] || fail "the one leg's package is not named by its triple" pack3.log +echo "a target named twice is one leg OK" -# Mixed direction: one Android (shared-object) row and one host (executable) -# row is not "every row is a shared object" either. -out=$("$MCPP" pack myapp --target aarch64-linux-android --target x86_64-linux-gnu 2>&1) \ - && fail "mixing a shared-object row with an executable row must be refused" <(echo "$out") -grep -qF -- "$expected" <<<"$out" \ - || fail "the mixed-row refusal message does not match today's wording" <(echo "$out") -echo "a shared-object row mixed with an executable row is refused OK" +"$MCPP" pack myapp --target aarch64-linux-android --target x86_64-linux-gnu > pack4.log 2>&1 \ + || fail "an Android row and an executable row are a package each" pack4.log +ls target/dist/myapp-0.1.0-x86_64-linux-gnu*.tar.gz > /dev/null 2>&1 \ + || fail "the executable row's package is not named by its triple" pack4.log +if "$MCPP" pack myapp --target aarch64-linux-android --target x86_64-linux-gnu -o one.tar.gz > pack5.log 2>&1; then + fail "-o naming one file was accepted for two packages" pack5.log +fi +grep -q "use: -o /" pack5.log || fail "the refusal names no directory" pack5.log +echo "an Android row and an executable row are a package each OK" echo "664: the universal APK is the library route applied to an app OK" diff --git a/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh b/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh index 04ae8ba5b..6acf8f5a2 100755 --- a/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh +++ b/tests/e2e/942_an_ok_image_is_built_once_and_run_by_okrun.sh @@ -24,9 +24,11 @@ # D. `import std;` with the platform's C++ run-time switched off is refused # before anything compiles, saying it was switched off and where. # E. A second build is up to date. -# F. `mcpp pack` stages the image (M2), and `--mode self-contained` stages a -# program of this machine that carries it and runs with nothing installed -# (M3), its image given the program's arguments and environment. +# F. `mcpp pack` stages the image in the tree of its platform (`lib//`, +# the launchers in `bin/`), and `--mode self-contained` adds the runner, +# so the launcher runs with nothing installed; `--pkg-format single-file` +# is a program of this machine that carries the image (M3). Each gives +# the image the arguments and the environment. # G. A workspace's root package is built as any root is: the image builds # and runs through okrun (the selected package is not the graph's first). set -e @@ -156,16 +158,20 @@ grep -q "Compiling" again.log && fail "E: the second build compiled again" again echo "ok: a second build is up to date" # F -"$MCPP" pack --target "$HOST_ARCH-openkal" --no-universal --pkg-format dir -o m2 > m2.log 2>&1 || fail "F: pack (M2)" m2.log -[ -f target/dist/m2/bin/app.oke ] || fail "F: the M2 package has no bin/app.oke" m2.log -"$MCPP" pack --target "$HOST_ARCH-openkal" --no-universal --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 \ - || fail "F: pack --mode self-contained (M3)" m3.log -m3=$(ls target/dist/m3/bin/app target/dist/m3/bin/app.exe 2>/dev/null | head -1) -[ -n "$m3" ] || fail "F: the M3 package has no program" m3.log -"$m3" one two > m3run.log 2>&1 || fail "F: the M3 program did not run" m3run.log -grep -q "image: three arguments" m3run.log || fail "F: the M3 program's image" m3run.log -grep -q "image: an environment" m3run.log || fail "F: the M3 program's image was given no environment" m3run.log -echo "ok: pack stages the image (M2) and a program that carries it (M3)" +"$MCPP" pack --target "$HOST_ARCH-openkal" --pkg-format dir -o m2 > m2.log 2>&1 || fail "F: pack" m2.log +[ -f "target/dist/m2/lib/$HOST_ARCH/app.oke" ] || fail "F: the package has no lib/$HOST_ARCH/app.oke" m2.log +[ -x target/dist/m2/bin/app ] || fail "F: the package has no launcher" m2.log +"$MCPP" pack --target "$HOST_ARCH-openkal" --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 \ + || fail "F: pack --mode self-contained" m3.log +env -i HOME="$HOME" PATH=/usr/bin:/bin target/dist/m3/bin/app one two > m3run.log 2>&1 \ + || fail "F: the self-contained launcher did not run" m3run.log +grep -q "image: three arguments" m3run.log || fail "F: the self-contained package's image" m3run.log +grep -q "image: an environment" m3run.log || fail "F: the self-contained package's image was given no environment" m3run.log +"$MCPP" pack --target "$HOST_ARCH-openkal" --pkg-format single-file -o m3-file > sf.log 2>&1 \ + || fail "F: pack --pkg-format single-file" sf.log +target/dist/m3-file one two > sfrun.log 2>&1 || fail "F: the single file did not run" sfrun.log +grep -q "image: three arguments" sfrun.log || fail "F: the single file's image" sfrun.log +echo "ok: pack stages the image in its tree, carries the runner, and a single file carries the image (M3)" # C before=$(cksum < "$(ls target/"$HOST_ARCH"-openkal/*/bin/app.oke | head -1)" ) diff --git a/tests/e2e/951_an_ok_image_needs_no_declaration.sh b/tests/e2e/951_an_ok_image_needs_no_declaration.sh index 29a8e2967..1927f2a76 100755 --- a/tests/e2e/951_an_ok_image_needs_no_declaration.sh +++ b/tests/e2e/951_an_ok_image_needs_no_declaration.sh @@ -18,8 +18,9 @@ # C. mcpp.lock records the packages the platform added, and those only they # reach, with `origin = "platform"`; a build for this machine leaves the # file as it is. -# D. `mcpp test` builds and runs a test image; `mcpp pack` stages the image -# and its loader-run program (M3). +# D. `mcpp test` builds and runs a test image; `mcpp pack --mode +# self-contained` stages the image's tree with its runner, and the +# launcher runs with nothing installed. # E. `mcpp self env --target … --format json` names the platform, its fields' # sources and the runner's program; `mcpp toolchain list --format json` # gives the row's platform. @@ -87,11 +88,10 @@ echo "ok: the lock says which packages the platform added, whichever configurati # D "$MCPP" test --target "$T" > test.log 2>&1 || fail "D: mcpp test" test.log grep -q "smoke ... ok" test.log || fail "D: the test image did not pass" test.log -"$MCPP" pack --target "$T" --no-universal --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 || fail "D: pack (M3)" m3.log -m3=$(ls target/dist/m3/bin/app target/dist/m3/bin/app.exe 2>/dev/null | head -1) -[ -n "$m3" ] || fail "D: the M3 package has no program" m3.log -"$m3" x > m3run.log 2>&1 || fail "D: the M3 program did not run" m3run.log -grep -q "image: caught (2 args)" m3run.log || fail "D: the M3 program's image" m3run.log +"$MCPP" pack --target "$T" --mode self-contained --pkg-format dir -o m3 > m3.log 2>&1 || fail "D: pack" m3.log +[ -x target/dist/m3/bin/app ] || fail "D: the package has no launcher" m3.log +env -i HOME="$HOME" PATH=/usr/bin:/bin target/dist/m3/bin/app x > m3run.log 2>&1 || fail "D: the launcher did not run" m3run.log +grep -q "image: caught (2 args)" m3run.log || fail "D: the package's image" m3run.log echo "ok: test and pack work with no declaration" # E diff --git a/tests/e2e/955_a_universal_package_holds_every_architecture.sh b/tests/e2e/955_the_openkal_set_is_one_tree_of_every_architecture.sh similarity index 79% rename from tests/e2e/955_a_universal_package_holds_every_architecture.sh rename to tests/e2e/955_the_openkal_set_is_one_tree_of_every_architecture.sh index 5569e8fb9..c751b1b62 100755 --- a/tests/e2e/955_a_universal_package_holds_every_architecture.sh +++ b/tests/e2e/955_the_openkal_set_is_one_tree_of_every_architecture.sh @@ -1,9 +1,10 @@ #!/usr/bin/env bash # requires: llvm python3 unix-shell -# 955 -- a universal package (SPEC-014 §3, §4.2): the platform of the ok-image -# targets states `universal = true`, so `mcpp pack` of either target packs both -# into one package, named after the platform, with a launcher that picks this -# machine's architecture and starts the platform's runner. +# 955 -- the `openkal` target set is one tree of every architecture (SPEC-014 +# §3, §4.2; design 2026-10-29 rev 2 §4.2, §4.3): `mcpp pack --target openkal` +# packs both ok-image targets into one package, named after the platform, with +# a launcher that picks this machine's architecture and starts the platform's +# runner. # # Criteria: # A. `--openkal/` holds `lib/x86_64/` and `lib/aarch64/` (each @@ -14,7 +15,7 @@ # C. `--mode self-contained` adds the runner's program for this system and # each architecture under `libexec//-/`, and the launcher # runs with no runner on PATH. -# D. `--no-universal` packs the selected target alone. +# D. `--target -openkal` packs that target alone, in the same layout. # E. (Linux) `--systems linux,macos,windows` carries the runner of each # system this mcpp builds one for, made here over the platform's hosted # arrangement: a Mach-O for macos-aarch64, a PE for windows-x86_64, and @@ -46,7 +47,7 @@ int main(int argc, char** argv) { CPP # A -"$MCPP" pack --target "$T" --pkg-format dir > a.log 2>&1 || fail "A: pack" a.log +"$MCPP" pack --target openkal --pkg-format dir > a.log 2>&1 || fail "A: pack" a.log P=target/dist/uni-0.1.0-openkal for f in "lib/x86_64/uni.oke" "lib/aarch64/uni.oke" "bin/uni" "bin/uni.exe" "mcpp-package.json"; do [ -f "$P/$f" ] || fail "A: the package has no $f" a.log @@ -77,7 +78,7 @@ grep -q "universal 3 args" b.log || fail "B: the image's output through the laun echo "ok: the launcher runs this machine's image through the runner" # C -"$MCPP" pack --target "$T" --mode self-contained --pkg-format dir -o sc > c.log 2>&1 || fail "C: pack --mode self-contained" c.log +"$MCPP" pack --target openkal --mode self-contained --pkg-format dir -o sc > c.log 2>&1 || fail "C: pack --mode self-contained" c.log # This machine's architecture's runner is always there; the other one is there # where this mcpp builds programs for it (not x86_64-macos, a planned row). [ -f "target/dist/sc/libexec/okrun/$OS-$HOST_ARCH/okrun" ] || fail "C: no runner for $OS-$HOST_ARCH" c.log @@ -92,17 +93,18 @@ d = json.load(open("target/dist/sc/mcpp-package.json")) carried = sorted(s for s in os.listdir("target/dist/sc/libexec/okrun")) assert d["mode"] == "self-contained" and d["systems"] == carried and "$OS-$HOST_ARCH" in carried, d PY -echo "ok: a self-contained universal package carries its runner and needs nothing installed" +echo "ok: a self-contained tree carries its runner and needs nothing installed" # D -"$MCPP" pack --target "$T" --no-universal --pkg-format dir -o one > d.log 2>&1 || fail "D: --no-universal" d.log -[ -f target/dist/one/bin/uni.oke ] || fail "D: --no-universal did not pack the target alone" d.log -[ -d target/dist/one/lib/$OTHER ] && fail "D: --no-universal packed the other architecture" d.log -echo "ok: --no-universal packs the selected target alone" +"$MCPP" pack --target "$T" --pkg-format dir -o one > d.log 2>&1 || fail "D: one target" d.log +[ -f "target/dist/one/lib/$HOST_ARCH/uni.oke" ] || fail "D: the target's leg is not in lib/$HOST_ARCH/" d.log +[ -f target/dist/one/bin/uni ] || fail "D: the one-leg package has no launcher" d.log +[ -d "target/dist/one/lib/$OTHER" ] && fail "D: one target packed the other architecture" d.log +echo "ok: one target is packed alone, in the same layout" # E if [ "$OS" = linux ]; then - "$MCPP" pack --target "$T" --mode self-contained --systems linux,macos,windows --pkg-format dir -o uni-0.1.0-all \ + "$MCPP" pack --target openkal --mode self-contained --systems linux,macos,windows --pkg-format dir -o uni-0.1.0-all \ > e.log 2>&1 || fail "E: --systems linux,macos,windows" e.log python3 - target/dist/uni-0.1.0-all <<'PY' || fail "E: the runners" e.log import os, sys @@ -117,4 +119,4 @@ PY echo "ok: the runners of other systems are made here, over the hosted arrangement" fi -echo "PASS: 955 a universal package holds every architecture" +echo "PASS: 955 the openkal set is one tree of every architecture" diff --git a/tests/e2e/956_a_single_file_runs_without_unpacking.sh b/tests/e2e/956_a_single_file_runs_without_unpacking.sh index d27706ad2..a4d879e90 100755 --- a/tests/e2e/956_a_single_file_runs_without_unpacking.sh +++ b/tests/e2e/956_a_single_file_runs_without_unpacking.sh @@ -13,9 +13,10 @@ # C. Refused before a build: `--compress xz` for a single file. # D. (llvm, an ok-image row) An image's single file runs, and writes nothing # into the cache. -# E. (an ok-image row) `-o` with several `--systems` is refused before a -# build: one file is made per system; so is a system whose row this mcpp -# builds no program for (`x86_64-macos`, a planned row). +# E. (an ok-image row) `-o` naming one file with several `--systems` is +# refused before a build: one file is made per leg and system; so is a +# system named whose row this mcpp builds no program for (`x86_64-macos`, +# a planned row). set -e TMP=$(mktemp -d) @@ -110,7 +111,7 @@ if [ -n "$T" ] && command -v python3 > /dev/null && "$MCPP" toolchain list --for if "$MCPP" pack --target "$T" --pkg-format single-file --systems linux,macos -o one-file > e.log 2>&1; then fail "E: -o with two systems was accepted" e.log fi - grep -q "one single file per system" e.log || fail "E: the refusal" e.log + grep -q "single files (one per leg and system)" e.log || fail "E: the refusal" e.log grep -q "Compiling" e.log && fail "E: refused after compiling" e.log if "$MCPP" pack --target x86_64-openkal --pkg-format single-file --systems linux,macos > e2.log 2>&1; then fail "E: a single file for x86_64-macos was accepted" e2.log diff --git a/tests/e2e/957_packages_merge_and_describe_themselves.sh b/tests/e2e/957_packages_merge_and_describe_themselves.sh index 75c3aa18c..bb9a78d33 100755 --- a/tests/e2e/957_packages_merge_and_describe_themselves.sh +++ b/tests/e2e/957_packages_merge_and_describe_themselves.sh @@ -3,7 +3,7 @@ # 957 -- packages written on several machines merge into one (SPEC-014 §4.6), # and a package describes its own distribution entry (§8). # -# A self-contained universal package carries the runner of the systems it was +# A self-contained package of `--target openkal` carries the runner of the systems it was # packed for. One packed on Linux and one packed on macOS hold the same images # and different runners; `--merge` makes the package for both. Here the second # is the first with its runner moved to another system's directory, which is @@ -15,10 +15,12 @@ # its directory is named after the output, as `mcpp pack -o` names one. # B. A file both packages hold with different digests is refused, naming the # file and both digests; packages of different versions are refused. -# C. `mcpp emit xpkg --package` of a universal package is one asset for -# linux, macosx and windows, with the runner as a dependency when the -# package does not carry it; an archive named without its own --package -# (a shell glob's second file) is refused, not left undescribed. +# C. `mcpp emit xpkg --package` of a package of images is one asset for +# each system its `runs_on` names, with the runner as a dependency when +# the package does not carry it; a package whose `runs_on` is empty (a +# platform that states no `runner.hosts`) is refused, saying so; an +# archive named without its own --package (a shell glob's second file) +# is refused, not left undescribed. # D. An archive merges as the directory it holds does. set -e @@ -36,7 +38,7 @@ mkdir -p "$TMP/app/src" cd "$TMP/app" printf '[package]\nname = "mrg"\nversion = "0.1.0"\n' > mcpp.toml printf 'import std;\nint main() { std::println("merged"); }\n' > src/main.cpp -"$MCPP" pack --target "$T" --mode self-contained --pkg-format dir -o here > pack.log 2>&1 || fail "pack" pack.log +"$MCPP" pack --target openkal --mode self-contained --pkg-format dir -o here > pack.log 2>&1 || fail "pack" pack.log # The same package as another system packs it: its runner in that system's # directory, and the metadata saying so. @@ -122,19 +124,31 @@ grep -q "version" b2.log || fail "B: the refusal does not name the version" b2.l echo "ok: --merge refuses what is not one package" # C -"$MCPP" pack --target "$T" --pkg-format zip -o mrg-0.1.0-openkal.zip > c.log 2>&1 || fail "C: pack" c.log -"$MCPP" emit xpkg --package target/dist/mrg-0.1.0-openkal.zip --url https://example.org/v0.1.0 > xpkg.lua 2> c.err \ - || fail "C: emit xpkg" c.err -python3 - xpkg.lua <<'PY' || fail "C: the entry" xpkg.lua -import hashlib, re, sys -text = open(sys.argv[1]).read() +"$MCPP" pack --target openkal --pkg-format zip -o mrg-0.1.0-openkal.zip > c.log 2>&1 || fail "C: pack" c.log +rc=0 +"$MCPP" emit xpkg --package target/dist/mrg-0.1.0-openkal.zip --url https://example.org/v0.1.0 > xpkg.lua 2> c.err || rc=$? +python3 - xpkg.lua "$rc" c.err <<'PY' || fail "C: the entry" xpkg.lua c.err +import hashlib, json, re, sys, zipfile +text, rc, err = open(sys.argv[1]).read(), int(sys.argv[2]), open(sys.argv[3]).read() +z = zipfile.ZipFile("target/dist/mrg-0.1.0-openkal.zip") +d = json.loads(z.read("mrg-0.1.0-openkal/mcpp-package.json")) +pairs = d["runs_on"]["with"].get("okrun", []) +if not pairs: + assert rc != 0 and "runs_on is empty" in err, (rc, err) + sys.exit(0) +assert rc == 0, err sha = hashlib.sha256(open("target/dist/mrg-0.1.0-openkal.zip", "rb").read()).hexdigest() -for system in ("linux", "macosx", "windows"): - block = re.search(system + r" = \{(.*?)\n \},", text, re.S) - assert block, system +for system, key in (("linux", "linux"), ("macos", "macosx"), ("windows", "windows")): + archs = sorted(p.split("-", 1)[1] for p in pairs if p.startswith(system + "-")) + block = re.search(key + r" = \{(.*?)\n \},", text, re.S) + if not archs: + assert not block, (system, text) + continue b = block.group(1) assert "https://example.org/v0.1.0/mrg-0.1.0-openkal.zip" in b and sha in b, (system, b) assert 'deps = { "xim:okrun" }' in b, (system, b) + for a in archs: + assert f'["{a}"]' in b, (system, a, b) assert 'xvm.add("mrg"' in text assert 'local top = pkginfo.install_file()' in text and '{ ".zip" }' in text, text PY @@ -143,6 +157,6 @@ if "$MCPP" emit xpkg --package target/dist/mrg-0.1.0-openkal.zip target/dist/mrg fail "C: an archive without its own --package was taken" c2.lua fi grep -q "each archive is its own --package" c2.err || fail "C: the refusal does not say how" c2.err -echo "ok: a universal package is one asset for every system" +echo "ok: a package of images is one asset for every system it runs on" echo "PASS: 957 packages merge and describe themselves" diff --git a/tests/e2e/961_a_one_target_package_carries_the_image_in_a_program_of_its_architecture.sh b/tests/e2e/961_a_single_file_carries_the_image_in_a_program_of_its_architecture.sh similarity index 52% rename from tests/e2e/961_a_one_target_package_carries_the_image_in_a_program_of_its_architecture.sh rename to tests/e2e/961_a_single_file_carries_the_image_in_a_program_of_its_architecture.sh index 11fc086a4..261905414 100755 --- a/tests/e2e/961_a_one_target_package_carries_the_image_in_a_program_of_its_architecture.sh +++ b/tests/e2e/961_a_single_file_carries_the_image_in_a_program_of_its_architecture.sh @@ -1,24 +1,31 @@ #!/usr/bin/env bash # requires: llvm python3 -# 961 -- a self-contained package of one ok-image target is a program of the -# image's architecture, and `--list-formats` lists what the target offers -# (SPEC-014 §1.2, §6; SPEC-013 §11; design 2026-10-29). +# 961 -- a single file of an ok-image target carries the image in a program +# of the image's architecture; a self-contained package of one target is the +# tree of its platform, described for the pair its runner covers; and +# `--list-formats` lists what the target offers (SPEC-014 §1.2, §6; SPEC-013 +# §11; design 2026-10-29 rev 2 §4.3, §4.4). # # Measured on 2026.10.28.1, on x86_64: `mcpp pack --target aarch64-openkal # --no-universal --mode self-contained` wrote an x86-64 program carrying an # aarch64 image, which runs on no machine; `mcpp pack --target aarch64-openkal # --list-formats` planned this machine and left out the platform's `okb`. +# Since design 2026-10-29 the image in a program (M3) is the single file's +# alone. # # Criteria: -# A. The package's `bin/app` is an ELF of the image's architecture; where -# this machine has that architecture's qemu, it runs the image. +# A. `--pkg-format single-file` of the other architecture's target is an +# ELF of that architecture; where this machine has its qemu, it runs the +# image. # B. `mcpp pack --target -openkal --list-formats` lists `okb`, provided by # the platform; `mcpp pack --list-formats` for this machine does not. -# C. The package records its carrier as the loader it carries (`systems`: -# this system and the image's architecture), and `mcpp emit xpkg` -# describes it under this system for that architecture; a one-target -# package of the image without its carrier (`--mode vendored`) is no -# program of a system and is refused, naming the universal package. +# C. `--mode self-contained` of that target is the tree: `lib//`, the +# launchers, the runner of this system and that architecture; `systems` +# and `runs_on.self` are that pair, and `mcpp emit xpkg` describes it +# under this system for that architecture alone. The tree without its +# runner (`--mode vendored`) runs where the platform's runner exists: +# described with the runner as a dependency, or refused, saying its +# `runs_on` is empty, where the platform states no `runner.hosts`. set -e TMP=$(mktemp -d) @@ -50,10 +57,10 @@ int main(int argc, char** argv) { CPP # A -"$MCPP" pack --target "$OT" --no-universal --mode self-contained --pkg-format dir -o m3 > a.log 2>&1 \ - || fail "A: pack" a.log -[ -f target/dist/m3/bin/app ] || fail "A: the package has no program" a.log -python3 - target/dist/m3/bin/app "$OTHER" <<'PY' || fail "A: the program's architecture" a.log +"$MCPP" pack --target "$OT" --pkg-format single-file -o m3-file > a.log 2>&1 || fail "A: pack" a.log +F=target/dist/m3-file +[ -f "$F" ] || fail "A: no single file" a.log +python3 - "$F" "$OTHER" <<'PY' || fail "A: the program's architecture" a.log import struct, sys d = open(sys.argv[1], "rb").read(20) assert d[:4] == b"\x7fELF", "not an ELF file" @@ -63,11 +70,11 @@ PY grep -q "into a program of linux-$OTHER" a.log || fail "A: the Packing line names another architecture" a.log QEMU=$(command -v "qemu-$OTHER" || command -v "qemu-$OTHER-static" || true) if [ -n "$QEMU" ]; then - "$QEMU" target/dist/m3/bin/app x > a-run.log 2>&1 || fail "A: the program under $QEMU" a-run.log + "$QEMU" "$F" x > a-run.log 2>&1 || fail "A: the program under $QEMU" a-run.log grep -q "image: caught (2 args)" a-run.log || fail "A: the program's image under $QEMU" a-run.log echo " ok: $QEMU runs it" fi -echo "ok: a one-target self-contained package is a program of the image's architecture" +echo "ok: a single file of one target is a program of the image's architecture" # B "$MCPP" pack --target "$OT" --list-formats > b.json 2> b.err || fail "B: --list-formats --target $OT" b.err b.json @@ -83,24 +90,37 @@ PY echo "ok: --list-formats lists the target's formats" # C -python3 - target/dist/m3/mcpp-package.json "$SYSTEM-$OTHER" <<'PY' || fail "C: the package's systems" target/dist/m3/mcpp-package.json +"$MCPP" pack --target "$OT" --mode self-contained --pkg-format dir -o m3 > c0.log 2>&1 || fail "C: pack" c0.log +for f in "lib/$OTHER/app.oke" "bin/app" "libexec/okrun/$SYSTEM-$OTHER/okrun"; do + [ -f "target/dist/m3/$f" ] || fail "C: the tree has no $f" c0.log +done +python3 - target/dist/m3/mcpp-package.json "$SYSTEM-$OTHER" "$OTHER" <<'PY' || fail "C: the package's systems" target/dist/m3/mcpp-package.json import json, sys d = json.load(open(sys.argv[1])) assert d["systems"] == [sys.argv[2]], d["systems"] -assert d["launch"] == {"program": "bin/app"}, d["launch"] +assert d["runs_on"]["self"] == [sys.argv[2]], d["runs_on"] +assert d["launch"]["entry"] == {sys.argv[3]: f"lib/{sys.argv[3]}/app.oke"}, d["launch"] PY tar czf m3.tar.gz -C target/dist m3 "$MCPP" emit xpkg --package m3.tar.gz --url https://example.org/v0.1.0 > c.lua 2> c.err \ - || fail "C: emit xpkg of the carrier package" c.err + || fail "C: emit xpkg of the self-contained tree" c.err grep -q "^ $XSYSTEM = {" c.lua || fail "C: the entry is not under $XSYSTEM" c.lua grep -qF "archs = {\"$OTHER\"}" c.lua || fail "C: the entry's architecture is not $OTHER" c.lua [ "$(grep -cE '^ (linux|macosx|windows) = [{]' c.lua)" = 1 ] || fail "C: the entry names more than one system" c.lua -"$MCPP" pack --target "$OT" --no-universal -o bare.tar.gz > c2.log 2>&1 || fail "C: the image's vendored package" c2.log +"$MCPP" pack --target "$OT" -o bare.tar.gz > c2.log 2>&1 || fail "C: the image's vendored package" c2.log rc=0 "$MCPP" emit xpkg --package target/dist/bare.tar.gz --url https://example.org/v0.1.0 > c2.lua 2> c2.err || rc=$? -[ "$rc" = 2 ] || fail "C: emit xpkg of an image without its carrier exited $rc, not 2" c2.err c2.lua -grep -q "is a package of $OT, which is no program of Linux, macOS or Windows" c2.err || fail "C: the refusal" c2.err -grep -q "universal package" c2.err || fail "C: the refusal names no route" c2.err -echo "ok: the carrier package is described for its system and architecture; the bare image is refused" +python3 - target/dist/bare.tar.gz "$rc" c2.err c2.lua <<'PY' || fail "C: the vendored tree's entry" c2.err c2.lua +import json, sys, tarfile +with tarfile.open(sys.argv[1]) as t: + d = json.load(t.extractfile("bare/mcpp-package.json")) +rc, err, text = int(sys.argv[2]), open(sys.argv[3]).read(), open(sys.argv[4]).read() +assert d["runs_on"]["self"] == [], d["runs_on"] +if not d["runs_on"]["with"].get("okrun"): + assert rc != 0 and "runs_on is empty" in err, (rc, err) +else: + assert rc == 0 and 'deps = { "xim:okrun" }' in text, (rc, err, text) +PY +echo "ok: the self-contained tree is described for its pair; the vendored tree as its runner's coverage says" -echo "PASS: 961 a one-target package carries the image in a program of its architecture" +echo "PASS: 961 a single file carries the image in a program of its architecture" diff --git a/tests/e2e/983_a_pack_holds_the_legs_it_names.sh b/tests/e2e/983_a_pack_holds_the_legs_it_names.sh new file mode 100644 index 000000000..9e16636b7 --- /dev/null +++ b/tests/e2e/983_a_pack_holds_the_legs_it_names.sh @@ -0,0 +1,123 @@ +#!/usr/bin/env bash +# requires: llvm python3 unix-shell +# 983 -- a pack holds the legs it names (SPEC-014 §1.4, §3; design 2026-10-29 +# rev 2 §4.2-§4.4, §4.8). +# +# A platform no longer packs more than the command named: `--target +# x86_64-openkal` is one leg, `--target openkal` (a target set) is every +# architecture of the platform, and an image of a platform that states a +# runner is packed as one layout whatever the number of legs -- launchers in +# `bin/`, the images in `lib//`, under `self-contained` the runner per +# system and architecture in `libexec/`. The package says where it runs. On +# 2026.10.29.1 `--target x86_64-openkal` packed both architectures, and +# `--target openkal` was an unknown target. +# +# Criteria: +# A. `--target -openkal` writes `---openkal/` with +# `lib//` alone, `bin/` and `bin/.exe`, its metadata +# naming the one leg; the summary says it holds one architecture and how +# to pack them all. +# B. `--target openkal` writes `--openkal/` with both legs. +# C. `--mode self-contained --systems -`: the runner of that pair +# alone; `Runs on` names it, and `mcpp-package.json` `runs_on.self` is it. +# D. `--message-format json`: `legs`, `pkgFormats` and `runsOn` describe the +# package. +# E. `--merge` of two one-leg packages writes the tree of both, named after +# the platform, with launchers written for both legs (the two inputs' +# launchers differ and are not a conflict); it runs here. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -m)" in + x86_64|amd64) HOST_ARCH=x86_64; OTHER=aarch64 ;; + aarch64|arm64) HOST_ARCH=aarch64; OTHER=x86_64 ;; + *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; +esac +case "$(uname -s)" in Linux) OS=linux ;; Darwin) OS=macos ;; *) echo "SKIP: a POSIX launcher"; exit 0 ;; esac +HT="$HOST_ARCH-openkal" + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "legs"\nversion = "0.1.0"\n' > mcpp.toml +cat > src/main.cpp <<'CPP' +import std; +int main(int argc, char**) { std::println("legs {} args", argc); } +CPP + +# A +"$MCPP" pack --target "$HT" --pkg-format dir > a.log 2>&1 || fail "A: pack" a.log +P="target/dist/legs-0.1.0-$HT" +for f in "lib/$HOST_ARCH/legs.oke" "bin/legs" "bin/legs.exe" "mcpp-package.json"; do + [ -f "$P/$f" ] || fail "A: the package has no $f" a.log +done +[ -d "$P/lib/$OTHER" ] && fail "A: one leg named, and the package holds $OTHER" a.log +[ -d target/dist/legs-0.1.0-openkal ] && fail "A: one leg named, and a tree of the platform was written" a.log +python3 - "$P/mcpp-package.json" "$HT" <<'PY' || fail "A: the metadata" "$P/mcpp-package.json" +import json, sys +d = json.load(open(sys.argv[1])) +assert d["legs"] == [sys.argv[2]], d["legs"] +assert d["launch"]["runner"][0].startswith("okrun"), d["launch"] +PY +grep -q "only $HOST_ARCH; \`--target openkal\` or \`\[pack\] targets = \[\"openkal\"\]\` packs every architecture" a.log \ + || fail "A: the summary does not say how to pack every architecture" a.log +echo "ok: one leg named is one leg packed, in the tree layout" + +# B +"$MCPP" pack --target openkal --pkg-format dir > b.log 2>&1 || fail "B: --target openkal" b.log +B=target/dist/legs-0.1.0-openkal +for f in "lib/x86_64/legs.oke" "lib/aarch64/legs.oke" "bin/legs" "bin/legs.exe"; do + [ -f "$B/$f" ] || fail "B: the package has no $f" b.log +done +grep -q "only $HOST_ARCH" b.log && fail "B: the summary of both legs says one" b.log +okrun=$("$MCPP" self env --target "$HT" --format json 2>/dev/null \ + | python3 -c 'import json,sys; print(json.load(sys.stdin)["data"]["runner"]["program"])') +PATH="$(dirname "$okrun"):$PATH" "$B/bin/legs" x y > b-run.log 2>&1 || fail "B: the launcher" b-run.log +grep -q "legs 3 args" b-run.log || fail "B: the launcher's output" b-run.log +echo "ok: the target set is one tree of every architecture" + +# C +"$MCPP" pack --target openkal --mode self-contained --systems "$OS-$HOST_ARCH" --pkg-format dir -o one-runner \ + > c.log 2>&1 || fail "C: --systems $OS-$HOST_ARCH" c.log +C=target/dist/one-runner +[ -d "$C/libexec/okrun/$OS-$HOST_ARCH" ] || fail "C: no runner for $OS-$HOST_ARCH" c.log +[ "$(ls "$C/libexec/okrun")" = "$OS-$HOST_ARCH" ] || fail "C: runners other than $OS-$HOST_ARCH" c.log +grep -q "Runs on $OS-$HOST_ARCH" c.log || fail "C: the summary does not say where it runs" c.log +python3 - "$C/mcpp-package.json" "$OS-$HOST_ARCH" <<'PY' || fail "C: runs_on" "$C/mcpp-package.json" +import json, sys +d = json.load(open(sys.argv[1])) +assert d["runs_on"]["self"] == [sys.argv[2]], d["runs_on"] +assert d["systems"] == [sys.argv[2]], d["systems"] +PY +env -i HOME="$HOME" PATH=/usr/bin:/bin "$C/bin/legs" z > c-run.log 2>&1 || fail "C: the self-contained launcher" c-run.log +grep -q "legs 2 args" c-run.log || fail "C: the self-contained package's output" c-run.log +echo "ok: --systems - carries that runner alone, and the package says where it runs" + +# D +"$MCPP" pack --target openkal --mode self-contained --systems "$OS" --message-format json > d.json 2> d.err \ + || fail "D: json" d.err d.json +python3 - d.json "$OS-$HOST_ARCH" <<'PY' || fail "D: the envelope" d.json +import json, sys +d = json.load(open(sys.argv[1]))["data"] +assert d["legs"] == ["x86_64-openkal", "aarch64-openkal"], d["legs"] +assert d["pkgFormat"] == "tar" and d["pkgFormats"] == ["tar"], d +assert sys.argv[2] in d["runsOn"]["self"] and isinstance(d["runsOn"]["with"], dict), d["runsOn"] +PY +echo "ok: the envelope names the legs, the formats and where the package runs" + +# E +"$MCPP" pack --target "$HT" --mode self-contained --pkg-format dir > e1.log 2>&1 || fail "E: pack $HT" e1.log +"$MCPP" pack --target "$OTHER-openkal" --mode self-contained --pkg-format dir > e2.log 2>&1 || fail "E: pack $OTHER-openkal" e2.log +"$MCPP" pack --merge "target/dist/legs-0.1.0-$HT-bundle-all" --merge "target/dist/legs-0.1.0-$OTHER-openkal-bundle-all" \ + --pkg-format dir > e.log 2>&1 || fail "E: --merge of two one-leg packages" e.log +M=legs-0.1.0-openkal-bundle-all +[ -d "$M" ] || fail "E: the merged tree is not named after the platform" e.log +grep -q "'x86_64') entry='legs.oke'" "$M/bin/legs" && grep -q "'aarch64') entry='legs.oke'" "$M/bin/legs" \ + || fail "E: the merged launcher does not name both legs" "$M/bin/legs" +env -i HOME="$HOME" PATH=/usr/bin:/bin "$M/bin/legs" q > e-run.log 2>&1 || fail "E: the merged launcher" e-run.log +grep -q "legs 2 args" e-run.log || fail "E: the merged package's output" e-run.log +echo "ok: --merge writes the launchers of the merged legs" + +echo "PASS: 983 a pack holds the legs it names" diff --git a/tests/e2e/984_a_set_of_programs_is_a_package_per_leg.sh b/tests/e2e/984_a_set_of_programs_is_a_package_per_leg.sh new file mode 100644 index 000000000..6cad66582 --- /dev/null +++ b/tests/e2e/984_a_set_of_programs_is_a_package_per_leg.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# requires: gcc python3 +# 984 -- a target set of programs of a system is a package per leg (design +# 2026-10-29 rev 2 §4.2). +# +# Only a platform with a runner merges legs into one tree, and only Android's +# rows are one APK; any other program is packed per leg, each package named by +# its triple. A pack that writes more than one file takes a directory for +# `-o`. On 2026.10.29.1 `--target linux-musl` was an unknown target. +# +# Criteria: +# A. `mcpp pack --target linux-musl -o out/` writes +# `--x86_64-linux-musl-static.tar.gz` and the aarch64 one +# below `out/`, each holding the program of its architecture. +# B. `-o` naming one file is refused before anything is built, saying a +# directory is wanted. +# C. `--message-format json` lists both packages and both legs. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +[ "$(uname -s)" = Linux ] || { echo "SKIP: the linux-musl rows are built on Linux"; exit 0; } + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "per"\nversion = "0.1.0"\n' > mcpp.toml +printf 'import std;\nint main() { std::println("per leg"); }\n' > src/main.cpp + +# A +if ! "$MCPP" pack --target linux-musl -o out/ > a.log 2>&1; then + if grep -qE "cannot be built on this host|install_packages failed|not found in the synced index" a.log; then + echo "SKIP: a linux-musl row cannot be built here"; exit 0 + fi + fail "A: pack --target linux-musl" a.log +fi +for arch in x86_64 aarch64; do + f="out/per-0.1.0-$arch-linux-musl-static.tar.gz" + [ -f "$f" ] || fail "A: no $f" a.log + python3 - "$f" "$arch" <<'PY' || fail "A: the $arch package's program" a.log +import struct, sys, tarfile +want = {"x86_64": 62, "aarch64": 183}[sys.argv[2]] +with tarfile.open(sys.argv[1]) as t: + m = next(m for m in t.getmembers() if m.name.endswith("/bin/per")) + head = t.extractfile(m).read(20) +assert head[:4] == b"\x7fELF" and struct.unpack_from(" b.log 2>&1; then fail "B: -o one file was accepted" b.log; fi +grep -q "use: -o /" b.log || fail "B: the refusal does not name a directory" b.log +grep -q "Compiling" b.log && fail "B: refused after compiling" b.log +echo "ok: -o naming one file is refused for two packages" + +# C +"$MCPP" pack --target linux-musl --message-format json > c.json 2> c.err || fail "C: json" c.err c.json +python3 - c.json <<'PY' || fail "C: the envelope" c.json +import json, sys +d = json.load(open(sys.argv[1]))["data"] +assert sorted(d["legs"]) == ["aarch64-linux-musl", "x86_64-linux-musl"], d["legs"] +assert len(d["artifacts"]) == 2, d["artifacts"] +PY +echo "ok: the envelope lists both packages" + +echo "PASS: 984 a set of programs is a package per leg" diff --git a/tests/e2e/985_a_format_list_writes_each_container_of_one_tree.sh b/tests/e2e/985_a_format_list_writes_each_container_of_one_tree.sh new file mode 100644 index 000000000..f16de60eb --- /dev/null +++ b/tests/e2e/985_a_format_list_writes_each_container_of_one_tree.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# requires: llvm python3 unix-shell +# 985 -- a `--pkg-format` list writes each container of one staged tree +# (design 2026-10-29 rev 2 §4.6). +# +# A list names the containers of one tree: the engine's `tar` and `zip` and a +# provider's format (`okb` of openkal-platform). The pack builds once, stages +# once and writes each container from that tree; a list takes a directory for +# `-o`, and refuses what is no container of a tree. On 2026.10.29.1 +# `--pkg-format tar,zip` was an unknown format. +# +# Criteria: +# A. `--pkg-format tar,zip -o out/` writes both containers of one tree below +# `out/`, holding the same files; the envelope's `pkgFormat` is `tar` and +# `pkgFormats` is `["tar", "zip"]`. +# B. Refused before a build: `dir` or `single-file` in a list; `-o` naming +# one file with a list; `mcpp run --pkg-format tar,zip`. +# C. `--target openkal --mode self-contained --pkg-format tar,okb` writes the +# tree's tar and the platform's okb; `--pkg-format okb` alone is refused +# under `self-contained`, saying the runner would be dropped. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "lst"\nversion = "0.1.0"\n' > mcpp.toml +printf 'import std;\nint main() { std::println("list"); }\n' > src/main.cpp + +# A +"$MCPP" pack --pkg-format tar,zip -o out/ --message-format json > a.json 2> a.log || fail "A: tar,zip" a.log a.json +python3 - a.json <<'PY' || fail "A: the containers" a.json a.log +import json, os, sys, tarfile, zipfile +d = json.load(open(sys.argv[1]))["data"] +assert d["pkgFormat"] == "tar" and d["pkgFormats"] == ["tar", "zip"], d +paths = [a["path"] for a in d["artifacts"]] +assert len(paths) == 2 and all(os.path.dirname(p).endswith("/out") for p in paths), paths +tgz = next(p for p in paths if p.endswith(".tar.gz")) +zp = next(p for p in paths if p.endswith(".zip")) +with tarfile.open(tgz) as t: + in_tar = sorted(m.name for m in t.getmembers() if m.isfile()) +with zipfile.ZipFile(zp) as z: + in_zip = sorted(n for n in z.namelist() if not n.endswith("/")) +assert in_tar == in_zip, (in_tar, in_zip) +PY +[ "$(grep -c "Compiling lst" a.log)" -le 1 ] || fail "A: the program was compiled more than once" a.log +echo "ok: a list writes each container of one tree" + +# B +refused() { + local label=$1; shift + if "$@" > b.log 2>&1; then fail "B: $label was accepted" b.log; fi + grep -q "Compiling" b.log && fail "B: $label was refused after compiling" b.log + return 0 +} +refused "dir in a list" "$MCPP" pack --pkg-format tar,dir +grep -q "lists the containers of one staged tree" b.log || fail "B: the refusal of dir" b.log +refused "single-file in a list" "$MCPP" pack --pkg-format single-file,tar +refused "-o one file with a list" "$MCPP" pack --pkg-format tar,zip -o one.tar.gz +grep -q "use: -o /" b.log || fail "B: the refusal of -o" b.log +refused "mcpp run with a list" "$MCPP" run --pkg-format tar,zip +grep -q "names one format" b.log || fail "B: the refusal of run" b.log +echo "ok: what is no container of a tree, and -o naming one file, are refused" + +# C +case "$(uname -m)" in x86_64|amd64|aarch64|arm64) ;; *) echo "PASS: 985 (no ok-image row here)"; exit 0 ;; esac +"$MCPP" pack --target openkal --mode self-contained --pkg-format tar,okb -o kit/ > c.log 2>&1 || fail "C: tar,okb" c.log +ls kit/lst-0.1.0-openkal-bundle-all.tar.gz > /dev/null 2>&1 || fail "C: no tar" c.log +ls kit/*.okb > /dev/null 2>&1 || fail "C: no okb" c.log +if "$MCPP" pack --target openkal --mode self-contained --pkg-format okb > c2.log 2>&1; then + fail "C: okb alone was accepted under self-contained" c2.log +fi +grep -q "the runner would be dropped" c2.log || fail "C: the refusal" c2.log +grep -q "use: --pkg-format tar,okb" c2.log || fail "C: the refusal names no remedy" c2.log +echo "ok: tar,okb writes both; okb alone does not drop a self-contained package's runner" + +echo "PASS: 985 a format list writes each container of one tree" diff --git a/tests/e2e/986_single_files_cover_the_legs_and_systems.sh b/tests/e2e/986_single_files_cover_the_legs_and_systems.sh new file mode 100644 index 000000000..0571bfee9 --- /dev/null +++ b/tests/e2e/986_single_files_cover_the_legs_and_systems.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# requires: llvm python3 unix-shell +# 986 -- single files cover the legs and the systems (design 2026-10-29 rev 2 +# §4.1, §4.5). +# +# `--pkg-format single-file` of an image makes one program per leg and system +# of the leg's architecture: the platform's carrier with the image in it. +# `--systems` names ``, `-` or `all`; `all` makes a file for +# every row this mcpp builds a program for and says the rest. More than one +# file takes a directory for `-o`. On 2026.10.29.1 `--target openkal` was an +# unknown target and `--systems` took only ``. +# +# Criteria: +# A. `--target openkal --pkg-format single-file -o files/` writes a file for +# each architecture on this system, each a program of its architecture; +# this machine's runs. +# B. `--systems -` with both legs makes one file. +# C. (Linux) `--systems all` makes the files of every row this mcpp builds a +# program for and says each other pair, `Not covered`, with its reason. +# D. `-o` naming one file with two files is refused before a build. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -m)" in + x86_64|amd64) HOST_ARCH=x86_64 ;; + aarch64|arm64) HOST_ARCH=aarch64 ;; + *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; +esac +case "$(uname -s)" in Linux) OS=linux ;; Darwin) OS=macos ;; *) echo "SKIP: a POSIX host"; exit 0 ;; esac + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "sf"\nversion = "0.1.0"\n' > mcpp.toml +printf 'import std;\nint main(int argc, char**) { std::println("single {} args", argc); }\n' > src/main.cpp + +# A +"$MCPP" pack --target openkal --pkg-format single-file -o files/ > a.log 2>&1 || fail "A: pack" a.log +for arch in x86_64 aarch64; do + [ -f "files/sf-0.1.0-$OS-$arch" ] || fail "A: no file for $OS-$arch" a.log +done +"files/sf-0.1.0-$OS-$HOST_ARCH" x > a-run.log 2>&1 || fail "A: this machine's file" a-run.log +grep -q "single 2 args" a-run.log || fail "A: the file's output" a-run.log +echo "ok: a single file per leg" + +# B +"$MCPP" pack --target openkal --pkg-format single-file --systems "$OS-$HOST_ARCH" -o one/ > b.log 2>&1 \ + || fail "B: --systems $OS-$HOST_ARCH" b.log +[ "$(ls one)" = "sf-0.1.0-$OS-$HOST_ARCH" ] || fail "B: the files are not the one pair" b.log +echo "ok: --systems - makes that pair's file" + +# C +if [ "$OS" = linux ]; then + "$MCPP" pack --target openkal --pkg-format single-file --systems all -o all/ > c.log 2>&1 || fail "C: --systems all" c.log + for pair in linux-x86_64 linux-aarch64 windows-x86_64 macos-aarch64; do + ls all/sf-0.1.0-$pair* > /dev/null 2>&1 || fail "C: no file for $pair" c.log + done + grep -q "Not covered macos-x86_64 (no carrier row" c.log || fail "C: the pair without a row is not said" c.log + echo "ok: --systems all makes every file this mcpp can and says the rest" +fi + +# D +if "$MCPP" pack --target openkal --pkg-format single-file -o one-file > d.log 2>&1; then + fail "D: -o one file with two files was accepted" d.log +fi +grep -q "use: -o /" d.log || fail "D: the refusal" d.log +grep -q "Compiling" d.log && fail "D: refused after compiling" d.log +echo "ok: -o naming one file is refused for two files" + +echo "PASS: 986 single files cover the legs and systems" diff --git a/tests/e2e/987_the_universal_spellings_are_migrated.sh b/tests/e2e/987_the_universal_spellings_are_migrated.sh new file mode 100644 index 000000000..355f5f54c --- /dev/null +++ b/tests/e2e/987_the_universal_spellings_are_migrated.sh @@ -0,0 +1,84 @@ +#!/usr/bin/env bash +# requires: llvm python3 +# 987 -- the spellings of the universal package are migrated (design +# 2026-10-29 rev 2 §13.2, §4.8; docs/51). +# +# A pack's targets are its legs, so nothing turns a platform's targets into +# legs any more. `--no-universal` with one target is what the pack does and +# is said; with a target set it contradicts the set. `[pack] universal = +# true` asked for every target and is refused with the spelling that does +# it; `false` is said and ignored. Each is in `mcpp why compat`. On +# 2026.10.29.1 `[pack] universal = true` was accepted. +# +# Criteria: +# A. `--no-universal --target -openkal` packs the one leg and notes the +# layout (`--pkg-format single-file` for one program). +# B. `--no-universal --target openkal` is refused: the two contradict. +# C. `[pack] universal = true` is refused naming `[pack] targets = +# ["openkal"]`; `false` is noted and the pack proceeds. +# D. `[pack] targets = ["openkal"]` packs both legs with no `--target`. +# E. `mcpp pack --help` puts the common options first, then the rest. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -m)" in + x86_64|amd64) HOST_ARCH=x86_64 ;; + aarch64|arm64) HOST_ARCH=aarch64 ;; + *) echo "SKIP: no ok-image row for $(uname -m)"; exit 0 ;; +esac +HT="$HOST_ARCH-openkal" + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "mig"\nversion = "0.1.0"\n' > mcpp.toml +printf 'import std;\nint main() { std::println("mig"); }\n' > src/main.cpp + +# A +"$MCPP" pack --no-universal --target "$HT" --pkg-format dir > a.log 2>&1 || fail "A: --no-universal" a.log +[ -f "target/dist/mig-0.1.0-$HT/lib/$HOST_ARCH/mig.oke" ] || fail "A: the one leg's tree" a.log +grep -q "note: --no-universal is what a pack of one target does" a.log || fail "A: the note" a.log +grep -q -- "--pkg-format single-file" a.log || fail "A: the note names no single file" a.log +echo "ok: --no-universal with one target is noted" + +# B +if "$MCPP" pack --no-universal --target openkal > b.log 2>&1; then fail "B: --no-universal with a set was accepted" b.log; fi +grep -q "contradict" b.log || fail "B: the refusal" b.log +echo "ok: --no-universal with a set is refused" + +# C +printf '\n[pack]\nuniversal = true\n' >> mcpp.toml +if "$MCPP" pack --target "$HT" > c.log 2>&1; then fail "C: [pack] universal = true was accepted" c.log; fi +grep -q 'use: \[pack\] targets = \["openkal"\]' c.log || fail "C: the refusal names no remedy" c.log +sed -i.bak 's/^universal = true$/universal = false/' mcpp.toml +"$MCPP" pack --target "$HT" --pkg-format dir > c2.log 2>&1 || fail "C: [pack] universal = false" c2.log +grep -q "note: \[pack\] universal = false is what a pack of one target does; it is ignored" c2.log \ + || fail "C: the note" c2.log +echo "ok: [pack] universal = true is refused, false is noted" + +# D +sed -i.bak 's/^universal = false$/targets = ["openkal"]/' mcpp.toml +"$MCPP" pack --pkg-format dir > d.log 2>&1 || fail "D: [pack] targets" d.log +for arch in x86_64 aarch64; do + [ -f "target/dist/mig-0.1.0-openkal/lib/$arch/mig.oke" ] || fail "D: no $arch leg" d.log +done +echo "ok: [pack] targets names the legs of a pack" + +# E +"$MCPP" pack --help > e.log 2>&1 || fail "E: pack --help" e.log +python3 - e.log <<'PY' || fail "E: the help's groups" e.log +import sys +text = open(sys.argv[1]).read() +common, advanced = text.index("Options:"), text.index("Advanced options:") +assert common < advanced, "the groups are not in order" +for flag in ("--target", "--mode", "--pkg-format", "--systems", "--linkage", "--output"): + at = text.index(flag) + assert common < at < advanced, flag +for flag in ("--compress", "--merge", "--no-universal"): + assert text.index(flag) > advanced, flag +PY +echo "ok: pack --help puts the common options first" + +echo "PASS: 987 the universal spellings are migrated" diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 34aba42dd..b6fb25cb8 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -7093,6 +7093,33 @@ targets = "linux-musl" EXPECT_EQ(one->packConfig.targets, (std::vector{"linux-musl"})); } +// `[pack] format` and `[pack] systems` are a string or a list, and +// `default_mode` takes the canonical names (design 2026-10-29 §4.1, §4.6). +TEST(Manifest, PackFormatAndSystemsAreAStringOrAList) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "p" +version = "0.1.0" +[pack] +default_mode = "self-contained" +format = ["tar", "okb"] +systems = "linux-aarch64" +)"); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_EQ(m->packConfig.defaultMode, "self-contained"); + EXPECT_EQ(m->packConfig.format, (std::vector{"tar", "okb"})); + EXPECT_EQ(m->packConfig.systems, (std::vector{"linux-aarch64"})); + auto one = mcpp::manifest::parse_string(R"( +[package] +name = "p" +version = "0.1.0" +[pack] +format = "zip" +)"); + ASSERT_TRUE(one.has_value()) << one.error().format(); + EXPECT_EQ(one->packConfig.format, (std::vector{"zip"})); +} + TEST(Manifest, PackTargetsRefuseWhatIsNeitherATargetNorASet) { auto m = mcpp::manifest::parse_string(R"( [package] diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index 36db61850..141c4c1d4 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -218,7 +218,11 @@ TEST(PackDistribution, TheWindowsLauncherIsAProgram) { EXPECT_NE(src.find("os::env::image()"), std::string::npos) << src; EXPECT_NE(src.find("PROCESSOR_ARCHITEW6432"), std::string::npos) << src; EXPECT_NE(src.find("R\"mcpp(--say=\"50%\")mcpp\""), std::string::npos) << src; - EXPECT_NE(src.find("R\"mcpp(aarch64)mcpp\""), std::string::npos) << src; + // One program for every set of legs: the entry's name, the architectures + // the directories under `lib/`. + EXPECT_NE(src.find("kEntry = R\"mcpp(hello.oke)mcpp\""), std::string::npos) << src; + EXPECT_EQ(src.find("R\"mcpp(aarch64)mcpp\""), std::string::npos) << src; + EXPECT_NE(src.find("fs::directory_iterator(top / \"lib\""), std::string::npos) << src; EXPECT_NE(src.find("(\"windows-\" + arch)"), std::string::npos) << src; EXPECT_NE(src.find("os::process::exec"), std::string::npos) << src; const auto manifest = d::exe_launcher_manifest("hello", "1.0.0"); @@ -541,6 +545,73 @@ TEST(PackMerge, AFileThatIsNotItsDigestIsRefused) { EXPECT_FALSE(fs::exists(t.path / "out")); } +// THE LAUNCHERS ARE DERIVED (design 2026-10-29 §4.3): two one-leg trees whose +// POSIX launchers differ merge into the tree of both, its launcher written for +// both legs; the Windows launcher, one program for every set of legs, is taken +// once. A package written before `runs_on` existed is given its pairs. +TEST(PackMerge, TheLaunchersAreWrittenForTheMergedLegs) { + Tmp t; + auto a = leg_package(t.path / "a", "x86_64-openkal", "x86_64"); + auto b = leg_package(t.path / "b", "aarch64-openkal", "aarch64"); + for (auto const& [root, arch] : { std::pair{ a, "x86_64" }, std::pair{ b, "aarch64" } }) { + write(root / "bin/app", std::format("#!/bin/sh\n# the launcher of {}\n", arch)); + write(root / "bin/app.exe", std::format("MZ the program of {}", arch)); + auto pkg = d::read(root); + ASSERT_TRUE(pkg) << pkg.error(); + d::Stated stated; + stated.role["bin/app"] = "launcher"; + stated.role["bin/app.exe"] = "launcher"; + ASSERT_TRUE(d::describe(root, *pkg, stated)); + } + d::MergeRequest req; + req.inputs = { a, b }; + req.format = "dir"; + req.output = t.path / "out"; + auto r = d::merge(req); + ASSERT_TRUE(r) << r.error(); + const auto script = slurp(*r / "bin/app"); + EXPECT_NE(script.find("'x86_64') entry='app.oke'"), std::string::npos) << script; + EXPECT_NE(script.find("'aarch64') entry='app.oke'"), std::string::npos) << script; + EXPECT_EQ(slurp(*r / "bin/app.exe"), "MZ the program of x86_64"); + auto merged = d::read(*r); + ASSERT_TRUE(merged) << merged.error(); + ASSERT_TRUE(merged->runsOnSet); + ASSERT_TRUE(merged->runsOnWith.contains("okrun")); + EXPECT_NE(std::ranges::find(merged->runsOnWith["okrun"], "macos-aarch64"), merged->runsOnWith["okrun"].end()); + for (auto const& f : merged->files) + EXPECT_EQ(d::sha256_file(*r / f.path), f.sha256) << f.path; +} + +// Merged without `-o`, a tree of a platform's legs is named after the platform +// (design 2026-10-29 §4.2). +TEST(PackMerge, AMergedTreeIsNamedAfterThePlatform) { + Tmp t; + std::vector inputs; + for (auto [leg, arch] : { std::pair{ "x86_64-openkal", "x86_64" }, std::pair{ "aarch64-openkal", "aarch64" } }) { + const auto root = t.path / arch / std::format("app-1.0-{}", leg); + write(root / "lib" / arch / "app.oke", std::format("image of {}", arch)); + d::Package p; + p.name = "app"; + p.version = "1.0"; + p.platform = "mcpplibs.openkal-platform@0.17.3"; + p.legs = { leg }; + p.runner = { "okrun", "{}" }; + p.entry = { { arch, std::format("lib/{}/app.oke", arch) } }; + ASSERT_TRUE(d::describe(root, p, {})); + inputs.push_back(root); + } + std::error_code ec; + const auto cwd = fs::current_path(); + fs::current_path(t.path, ec); + d::MergeRequest req; + req.inputs = inputs; + req.format = "dir"; + auto r = d::merge(req); + fs::current_path(cwd, ec); + ASSERT_TRUE(r) << r.error(); + EXPECT_EQ(r->filename(), "app-1.0-openkal"); +} + // The legs merge into a new directory; an output that exists, or is an input, // is refused, and an input written with a trailing separator is its directory. TEST(PackMerge, TheOutputIsNewOrAPackage) { @@ -617,7 +688,7 @@ TEST(PackMerge, TheCompressionFitsTheContainer) { ASSERT_TRUE(fits) << fits.error(); } -// #860 (design 2026-10-29 §4.4): a self-contained universal package runs where +// #860 (design 2026-10-29 §4.4): a self-contained tree of images runs where // it carries a runner, and its entry names exactly those pairs -- macOS aarch64 // only, Windows x86_64 only -- with no runner to install. A package that needs // the machine's runner names the pairs the platform's runner exists for. @@ -801,7 +872,7 @@ TEST(PackDistribution, XpkgDescriptorDescribesProgramsOfASystem) { for (auto [triple, says] : { Row{ "wasm32-emscripten", "dist-web" }, Row{ "aarch64-linux-android", "Linux, macOS or Windows target" }, Row{ "riscv64-none-elf", "Linux, macOS or Windows target" }, - Row{ "x86_64-openkal", "universal package" } }) { + Row{ "x86_64-openkal", "--target openkal" } }) { auto r = d::xpkg_descriptor({ one(triple, std::format("web-0.1.0-{}", triple), "bin/web.js") }, "https://example.org/0.1.0"); ASSERT_FALSE(r) << triple << ": " << *r; diff --git a/tests/unit/test_pack_modes.cpp b/tests/unit/test_pack_modes.cpp index d63d72f79..49b09aec7 100644 --- a/tests/unit/test_pack_modes.cpp +++ b/tests/unit/test_pack_modes.cpp @@ -52,3 +52,23 @@ TEST(PackClosureUnavailableOutcome, DispatchedFormatStagesWithoutIt) { EXPECT_EQ(closure_unavailable_outcome(Format::Dispatched), ClosureUnavailableOutcome::StageWithoutClosure); } + +// `--systems` (design 2026-10-29 §4.1): `` for every architecture of the +// legs, `-` for one, `all` for the three systems; unstated, this +// machine's. +TEST(PackSystems, TheWordsNameSystemsForAnArchitecture) { + using V = std::vector; + EXPECT_EQ(mcpp::pack::systems_for({ "linux", "macos-aarch64" }, "x86_64"), V{ "linux" }); + EXPECT_EQ(mcpp::pack::systems_for({ "linux", "macos-aarch64" }, "aarch64"), (V{ "linux", "macos" })); + EXPECT_EQ(mcpp::pack::systems_for({ "all", "linux" }, "x86_64"), (V{ "linux", "macos", "windows" })); + EXPECT_EQ(mcpp::pack::systems_for({}, "x86_64").size(), 1u); +} + +// `-o` names a directory when it ends in a separator or is one (design +// 2026-10-29 §4.2). +TEST(PackSystems, AnOutputEndingInASeparatorIsADirectory) { + EXPECT_TRUE(mcpp::pack::output_names_directory("dist/")); + EXPECT_FALSE(mcpp::pack::output_names_directory("dist/app.tar.gz")); + EXPECT_FALSE(mcpp::pack::output_names_directory("")); + EXPECT_TRUE(mcpp::pack::output_names_directory(std::filesystem::temp_directory_path())); +} From 025b6f4e7e67a5b0c5428cf4283eedb8b33e0788 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 05:21:59 +0900 Subject: [PATCH 08/20] a tree that runs nowhere is refused: with runner hosts unstated read as every system, an empty runs_on is the platform's statement Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/10-pack-and-release.md | 5 +++-- docs/specs/packaging.md | 5 +++-- docs/zh/10-pack-and-release.md | 4 ++-- src/pack/pack.cppm | 20 +++++++------------- 4 files changed, 15 insertions(+), 19 deletions(-) diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 1ed89f6a7..0b13cd241 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -844,8 +844,9 @@ has the runner (the legs' architectures × the platform's `runner.hosts`). Not covered macos-x86_64 (no runner row: this mcpp builds no program for x86_64-macos) ``` -A tree that runs nowhere — no runner carried and none the platform names — is -refused; over a platform that states no `runner.hosts` this is a note instead. +A platform that states no `runner.hosts` is read as every system of each +architecture. A tree that runs nowhere — no runner carried and none of the +platform's runner pairs of its architectures — is refused. Packages built on several machines are joined with `mcpp pack --merge a.zip --merge b.zip` (also `--merge a.zip,b.zip`). The inputs diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 3c2fc8c1f..98a89ea7c 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -115,7 +115,7 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` | 目标集合展开为空,或候选选到不同平台 | 拒绝(SPEC-015 §1) | | `--pkg-format` 列表含 `dir` 或 `single-file`;列表与命名一个文件的 `-o`;`mcpp run --pkg-format` 的列表 | 拒绝(§5.4) | | `self-contained` 的树而格式只有 `okb` | 拒绝:okb v1 只含映像,运行器会被丢弃;出路 `--pkg-format tar,okb` | -| 树的 `runs_on` 为空(`self` 与 `with` 都空,§4.7) | 拒绝;平台未声明 `runner.hosts` 而模式不是 `self-contained` 时写出提示,不拒绝 | +| 树的 `runs_on` 为空(`self` 与 `with` 都空,§4.7) | 拒绝 | | `--merge` 的输入缺少 `mcpp-package.json`,或名字、版本、平台、链接、模式不同 | 拒绝,写出不一致的字段 | | `--merge` 的两个输入在同一路径上有摘要不同的文件 | 拒绝,写出该路径、两个摘要与两个输入 | | 库目标(`lib`、`shared`)与提供者格式 | 拒绝:库包没有单一暂存树 | @@ -291,7 +291,8 @@ clang 在 `__FILE__` 中保留它拼接包含路径时用的 `\`,其映像不同 | 单文件 | 每个文件各自的 `-` | 空 | | WebAssembly、Android、iOS、裸机 | 空 | 空 | -`runner.host = "any"` 时 `hosts` 列出运行器程序存在的组合,覆盖与腿的架构无关。树中没有 Windows 启动器时 `runs_on` +`runner.host = "any"` 时 `hosts` 列出运行器程序存在的组合,覆盖与腿的架构无关;平台未声明 `hosts` 时按每个系统 × +腿的每个架构读取(未知不等于无处)。树中没有 Windows 启动器时 `runs_on` 不含 Windows 的组合。`self` 与 `with` 都为空的树被拒绝(§1.5)。`Packed` 之后写出 `Runs on <组合>; with : <组合>`;所请求而未覆盖的组合各写一行 `Not covered <组合> (<原因>)`(本 mcpp 不构建该行、没有 Windows 启动器、 不在平台的 `runner.hosts` 中)。`mcpp emit xpkg`、`--merge`、`mcpp.pack` 的 `runsOn` 读这里,不再计算。 diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index daf0e922d..1a57d2a5f 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -727,8 +727,8 @@ Cygwin 的 shell 交给 `bin/.exe`，以该目录中的映像与自身的 Not covered macos-x86_64 (no runner row: this mcpp builds no program for x86_64-macos) ``` -在任何地方都不能运行的树（不携带运行器，平台也未写出运行器存在的组合）被拒绝；平台未声明 `runner.hosts` 时改为 -一条提示。 +平台未声明 `runner.hosts` 时按每个系统 × 每个架构读取。在任何地方都不能运行的树（不携带运行器，平台的运行器组合 +也不含它的架构）被拒绝。 多台机器打出的包以 `mcpp pack --merge a.zip --merge b.zip`（也可 `--merge a.zip,b.zip`）合并。输入是本命令写出的 包目录或归档（`.zip`、`.tar`、`.tar.gz`、`.tgz`、`.tar.zst`、`.tar.xz`），不进行任何构建。腿、系统与 `runs_on` diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 93fe34f2f..6ae472d0b 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -1684,21 +1684,15 @@ std::expected finalize_tree(const Plan& plan) { } // A TREE THAT RUNS NOWHERE IS NOT A PACKAGE (design 2026-10-29 §4.4): it // carries no runner a system can start, and the platform names no - // system its runner exists for. A platform that states no - // `runner.hosts` says nothing either way, which is said, not refused. + // system of its legs' architectures its runner exists for. if (plan.opts.tree && pkg.runsOnSelf.empty() && pkg.runsOnWith.empty()) { auto const& runner = plan.platform.fields.runner; - if (runner.hostsSet || plan.opts.mode == Mode::BundleAll) - return std::unexpected(Error{std::format( - "{} runs on no system: it carries no runner built here{}, and {} names no " - "system its runner `{}` exists for (runner.hosts).\n use: --mode self-contained " - "--systems ", - plan.binaryName, plan.opts.mode == Mode::BundleAll ? "" : " (--mode vendored)", - plan.platform.display(), runner.cmd.empty() ? std::string() : runner.cmd.front())}); - mcpp::ui::note(std::format( - "{} states no runner.hosts, so the package names no system it runs on " - "(mcpp-package.json runs_on is empty); an index entry cannot be written for it", - plan.platform.display())); + return std::unexpected(Error{std::format( + "{} runs on no system: it carries no runner built here{}, and {} names no system of " + "its architectures its runner `{}` exists for (runner.hosts).\n use: --mode " + "self-contained --systems ", + plan.binaryName, plan.opts.mode == Mode::BundleAll ? "" : " (--mode vendored)", + plan.platform.display(), runner.cmd.empty() ? std::string() : runner.cmd.front())}); } // What `[pack] include` named is the package's resource, whatever its name. for (auto const& rel : *included) stated.role[rel] = "resource"; From 772a82da99383a492df36025fc25ec2b1aa36cc0 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 05:15:12 +0900 Subject: [PATCH 09/20] design: the clangd probe's result -- clangd stays native per system inside the one mcppls package Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...arget-sets-packaging-and-editors-design.md | 33 ++++++++++++------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md b/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md index 93f2dc134..b7e04dc2b 100644 --- a/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md +++ b/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md @@ -363,7 +363,7 @@ app 的 SELinux 域不允许**读** `/`。openkal-linux 以只读方式打开预 |---|---|---| | 服务器 | `mcppls.okb`（已统一） | 不变 | | okrun | 本系统的一个 | `runner.hosts` 的全部 | -| clangd | 每系统原生（Ubuntu 20.04 + Clang 12、macOS 14、Windows + MSVC） | `clangd.okb`：两个架构的映像（§8 构建；§9.4 的 probe 决定） | +| clangd | 每系统原生（Ubuntu 20.04 + Clang 12、macOS 14、Windows + MSVC） | 每系统原生，放进同一个包（§9.4：本轮映像不可用） | | kit | 按宿主选一个 | 包内 openkal kit；目标 kit 按需（§9.2） | ### 9.2 kit 按目标（S4 修订） @@ -392,17 +392,28 @@ app 的 SELinux 域不允许**读** `/`。openkal-linux 以只读方式打开预 | xlings | 每系统一项 | 一个资产三系统，可装范围取自 `runs_on` | | 构建 | 4 种主机与工具链 | 一台 Linux；测试在四个系统上（§3.4 的 `--archive`） | -- 覆盖随 okrun 扩大：musl 系统、Alpine、Termux（#857 之后）可用，clangd 不再要求 glibc 2.31；Intel Mac、Windows arm64 在平台加入 - 对应运行器行后可用。 -- 体积（待实测）：vsix 约 50–60 MB（0.0.19 每平台 30–40 MB），前提是 clangd 映像的压缩率与原生相近；okb 以 stored 存放，安装后 - 含另一架构的 clangd（约 75 MB）。 +- 覆盖：服务器随 okrun 扩大；clangd 是每系统原生（§9.4），所以整个包的覆盖是"原生 clangd 的系统 ∩ 运行器的宿主"，与今天的四个 + 平台相同。 +- 体积（待实测）：一个 vsix 含四个原生 clangd，约为今天四个平台包之和减去重复的服务器与 kit。 -### 9.4 clangd 映像的 probe（进行中） +### 9.4 clangd 映像的 probe（结论） -mcppls-clangd 的 draft PR（分支 `probe/clangd-openkal-image`），LLVM 源码经 xlings-res 的 GitCode 镜像获取。测量：编译错误清单、 -四个系统上 `clangd --check`、Windows 的路径风格（openkal 在 Windows 上给出原生名字；LLVM 的 `Style::native` 在编译期决定，需要 -补丁改为运行时读 `kal_env_system`）、与原生 clangd 的耗时与内存（musl 分配器的风险，必要时链接 mimalloc 或 scudo）、映像与 okb -大小。结论写入本节。 +mcppls-clangd 的 draft PR #11（分支 `probe/clangd-openkal-image`，run 40908632199）。两个架构的 2848 个目标文件编译通过，映像 +链接并封装成功。`clangd --check` 的结果（3 次取中位数）： + +| 系统 | `--version` | 无头文件 | `import m` | 系统 C++ 标准库 | +|---|---|---|---|---| +| Linux x86_64 | 通过 | 2.49 s / 232 MiB（原生 2.23 s / 122 MiB） | 0.40 s / 141 MiB（原生 0.13 s / 50 MiB） | SIGSEGV | +| Linux aarch64 | 通过 | 2.00 s / 231 MiB（原生 1.88 s / 122 MiB） | 0.33 s / 145 MiB（原生 0.14 s / 49 MiB） | SIGSEGV | +| macOS arm64 | 通过 | 1.56 s / 245 MiB（原生 1.10 s / 153 MiB） | 0.46 s / 165 MiB（原生 0.25 s / 40 MiB） | 1.43 s / 258 MiB（原生 1.11 s / 171 MiB） | +| Windows x64 | 通过 | 失败 | 失败 | 失败 | + +- Linux 上解析系统标准库头文件时崩溃，原因未定位。Windows 上 LLVM 的路径风格在编译期为 POSIX，`C:/…` 被当作相对路径，读文件 + 返回 `EINVAL`。常驻内存比原生多约 90–125 MiB，与映像大小相当（文件 `mmap` 在 openkal 上返回 `ENOSYS`，映像页不共享）。 +- **结论：本轮 mcppls 的 clangd 保持每系统原生，四个原生 clangd 与 `mcppls.okb`、全部运行器放进同一个包**（§9.1、§9.3 的"一个 + 包"不变，体积以原生 clangd 计）。clangd 映像在上面四项解决后替换包内的原生 clangd，包的形状不变。 +- probe 发现的两个栈内缺陷在本轮修复：`__cxa_thread_atexit` 在 ok 映像上未导出（openkal-llvm-runtime 0.16.9）；`pread`、`pwrite` + 返回 `ENOSYS`（openkal-musl 0.22.9）。文件映射（规范层面）、Windows 的 `EINVAL`、Linux 的崩溃另立 issue。 ## 10. 编辑器 @@ -475,7 +486,7 @@ mcppls-clangd 的 draft PR（分支 `probe/clangd-openkal-image`），LLVM 源 | 7 | mcpplibs.os | 钉新版本组 | | 8 | mcpp | SPEC-015、SPEC-014 rev、§5、§6、§8.1；同时接受平台的新旧两种写法；`triple.cppm:909` 钉 openkal-platform 新版 | | 9 | mcpp-plugins | `deps-cmake` 读契约；`dep_dir` 的歧义失败 | -| 10 | mcppls-clangd | clangd 映像（§9.4 决定）、路径风格补丁、宿主 tblgen | +| 10 | mcppls-clangd | 无改动（§9.4：本轮 clangd 保持原生）；probe PR #11 关闭 | | 11 | mcppls | 单包、S4 修订、kit 流程、CI 改用 `--archive` | | 12 | mcpp-ide | VS Code（§10.1）、组件库、CLion API、CI 迁移（§13.1） | | 13 | xim-pkgindex | mcpp-ide、mcppls、okrun 的条目重新生成 | From bab910d267ceb43b280f7d50b76a721ae201ac07 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 05:21:58 +0900 Subject: [PATCH 10/20] SPEC-015, the target sets and test archives in docs/21, docs/08 and docs/50, and the release's notes for target sets and the pack model Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 32 +++++++++ docs/08-testing.md | 18 +++++ docs/21-the-target-triple.md | 24 +++++++ docs/50-machine-output.md | 10 +++ docs/README.md | 3 +- docs/specs/README.md | 1 + docs/specs/packaging.md | 2 +- docs/specs/target-sets.md | 120 ++++++++++++++++++++++++++++++++ docs/zh/08-testing.md | 15 ++++ docs/zh/21-the-target-triple.md | 20 ++++++ docs/zh/50-machine-output.md | 7 ++ docs/zh/README.md | 3 +- 12 files changed, 252 insertions(+), 3 deletions(-) create mode 100644 docs/specs/target-sets.md diff --git a/CHANGELOG.md b/CHANGELOG.md index af8111cb3..f8c849ccd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,18 @@ ### Added +- **Target sets (SPEC-015).** A `--target` whose first segment is a system + word names every row of the target table with that system and environment: + `openkal` is `x86_64-openkal` and `aarch64-openkal`, `linux-musl` the musl + rows this mcpp builds. Each member is a leg. `build` builds every leg (planned + one after another, built at once); `run` runs the first leg this machine + reaches, its own architecture first; `test` runs the legs it reaches, reports + the others built and not run, and fails when no test ran. The `mcpp.build` + envelope adds `legs`, the events their `triple`; `self env` adds `legs` + (e2e 963). `[build] target` takes one target. +- **Tests built once, run elsewhere.** `mcpp test --no-run --archive ` + writes every leg's tests with `mcpp-tests.json`; `mcpp test --archive ` + in a checkout runs what this machine reaches and builds nothing (e2e 964). - **The build's contract, `${mcpp.contract}`.** Every configuration writes `mcpp-contract.json` beside its compile database: the compiler of each language with the words every unit is compiled with (target, sysroot, @@ -28,6 +40,26 @@ ### Changed +- **A pack holds the legs it names (SPEC-014 2.0).** `--target + x86_64-openkal` packs that architecture; `--target openkal` packs both into + one tree. The platform no longer widens a pack: `[platform] universal`, + `[pack] universal` and `--no-universal` lose their effect, with a message + that names the new spelling (`[pack] targets = ["openkal"]`; docs/51). + Images of a platform with a runner have one layout whatever the number of + legs: `bin/`, `bin/.exe`, `lib//`, and with + `--mode self-contained` `libexec//-/`. The image embedded + in a carrier belongs to `--pkg-format single-file` only. A set of native + programs is a package per leg (e2e 955, 983, 984). +- **The Windows launcher is a program.** `bin/.exe` replaces + `bin/.cmd`: no "Terminate batch job" prompt, and the process is named + after the package. The POSIX launcher hands over to it under Git Bash, MSYS + and Cygwin. +- **`--systems` takes ``, `-` and `all`, and `[pack] systems`.** + `--pkg-format` takes a list of containers of one staged tree (`tar,okb`), + built once (e2e 985). Single files cover legs × systems (e2e 986). +- **The pack summary names where the package runs.** `Runs on` and one `Not + covered` line per pair the package does not reach; `mcpp.pack` adds + `pkgFormats` and `runsOn`. A tree that runs nowhere is refused. - **A target reads the descriptor blocks its row names, never the build machine's (#851).** A target that is not Linux, macOS or Windows read the block of the system mcpp ran on, so a Web build took a dependency's `linux` diff --git a/docs/08-testing.md b/docs/08-testing.md index ffbd62f2c..57d07ca62 100644 --- a/docs/08-testing.md +++ b/docs/08-testing.md @@ -197,6 +197,24 @@ otherwise, so it needs no `libc++_shared.so` on the device. The runner itself, named runners, and what a board declares are [41 — Reaching a Device](41-devices.md). +### Built once, run elsewhere (mcpp 2026.10.30.1+) + +An ok image is the same bytes on every system, and a test built for +`x86_64-windows-gnu` on Linux runs on Windows. `--archive` separates the two +halves: + +```bash +mcpp test --target openkal --no-run --archive tests.tar.gz # build every leg's tests, write them +mcpp test --archive tests.tar.gz # in a checkout: run what this machine reaches +``` + +The archive holds each leg's test programs and `mcpp-tests.json`, which names +each test's leg, arguments and working directory. Reading it builds nothing; +the tests run in the checkout, through this machine's runner or system, and the +report is `mcpp test`'s. A failed test exits 1, and so does an archive of which +no test ran. Legs this machine does not reach are reported as not run +([SPEC-015 §4](specs/target-sets.md)). + ## Tests that cannot run beside each other `mcpp test` runs test programs on a worker pool. One board on one probe, one diff --git a/docs/21-the-target-triple.md b/docs/21-the-target-triple.md index 7c9a7f85c..f98c89485 100644 --- a/docs/21-the-target-triple.md +++ b/docs/21-the-target-triple.md @@ -196,6 +196,30 @@ different artefacts. Writing the segment is how that choice is made. default fill, and `msvc` is a different object ABI rather than a different C library, so the segment is meaningful there in both systems. +## Target Sets (mcpp 2026.10.30.1+) + +A `--target` whose first segment is a system word — `linux`, `macos`, +`windows`, `openkal`, `emscripten`, `ios`, `android`, `none` — names a **target +set**: every row of the target table with that system and environment that +this mcpp builds. The environment is completed as above (`linux` is +`linux-gnu`, `windows` is `windows-gnu`); `none` has no default and is refused. + +| Set | Members (2026.10.30.1, openkal-platform 0.18.0) | +|---|---| +| `openkal` | `x86_64-openkal`, `aarch64-openkal` | +| `linux-musl` | `x86_64-linux-musl`, `aarch64-linux-musl` | +| `linux` | `x86_64-linux-gnu`, `aarch64-linux-gnu` | + +Every member must select the same platform; a set whose members select +different platforms is refused, and the message lists each member's. A command +acts once per member, each member a **leg**: `build` builds every leg, `run` +runs the first leg this machine reaches (its own architecture first), `test` +runs the legs it reaches and fails when none ran, and `pack` packs the legs +as [10](10-pack-and-release.md) describes. Reports and machine output name the +members, never the set. `[build] target` takes one target; a set belongs on the +command line and in `[pack] targets`. The rules are +[SPEC-015](specs/target-sets.md). + ## The Build Report The report heads with the target as written and resolves it to the compiler's diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index b3c22eda4..76228de6c 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -949,6 +949,16 @@ A run ended by SIGINT, SIGTERM or SIGHUP leaves `cancelled` in both files "data":{"result":"/…/target/x86_64-linux-gnu/c0641c458b2bde43/result.json","status":"ok","finishedAt":1792964065954}} ``` +**Several legs** *(2026.10.30.1+)*. A command over a target set, or over more +than one `--target`, builds a leg per target ([SPEC-015](specs/target-sets.md)). +Every leg writes its own `result.json`. The `mcpp.build` envelope's `data` is the +first leg's record, its `status` is the legs' together, and `legs` holds every +leg's record; each `artifact` and `diagnostic` event carries the `triple` of its +leg. `.last-result.json` names the first leg's record and adds `legs`, one +`{triple, result, status}` per leg. With one leg the envelope, the events and +both files are as before. `self env --target ` adds `legs` to `mcpp.env`, +one report per leg, the top-level fields being the first leg's. + ### `mcpp.launch` — the way `mcpp run` starts the program ``` diff --git a/docs/README.md b/docs/README.md index 54396ef5c..0d442e937 100644 --- a/docs/README.md +++ b/docs/README.md @@ -176,4 +176,5 @@ downstream tooling. - [SPEC-011 — Concurrent runs: the build directory lock, whole outputs, optimistic writes, cancellation](specs/concurrency.md) - [SPEC-012 — Output for people: two streams, two media, the kinds of line, colour](specs/human-output.md) - [SPEC-013 — Platforms: the configuration of a class of targets, its selection, merge, graph completion, link, seal, runner and carrier](specs/platforms.md) - - [SPEC-014 — Packaging and distribution: link, closure and container, legs, the staged tree, universal and single-file packages](specs/packaging.md) + - [SPEC-014 — Packaging and distribution: link, closure and container, legs, the staged tree, the platform's tree, run coverage and single-file packages](specs/packaging.md) + - [SPEC-015 — Target sets and legs: spelling and expansion, commands over several legs, test archives](specs/target-sets.md) diff --git a/docs/specs/README.md b/docs/specs/README.md index bf45eeef3..7efdf98a2 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -44,6 +44,7 @@ | [SPEC-012](human-output.md) | 面向人的输出：两条流、两种介质、行的种类、颜色 | 评审中 v1.0 | 2026-10-26 | mcpp >= 2026.10.26.1 | | [SPEC-013](platforms.md) | 平台:`[platform]` 的七组键、版本组与 `hosted`,三种写法与层次,目标图与宿主图的选择,`cfg(platform)`,依赖图的补全,链接、命名与封装,运行器与载体,平台阶段与协议 17 | 评审中 v1.2 | 2026-10-29 | mcpp >= 2026.10.27.1;§10 的执行宿主与 §11 的载体架构 mcpp >= 2026.10.29.1 | | [SPEC-014](packaging.md) | 打包与分发:链接、闭包、容器三个维度与构建腿,腿的合并,暂存树与 `mcpp-package.json`,平台的树、运行覆盖、单文件、容器列表、签名、压缩、合并与分发描述 | 评审中 v2.0 | 2026-10-30 | mcpp >= 2026.10.30.1;逐条标注 | +| [SPEC-015](target-sets.md) | 目标集合与腿:集合的写法与展开,命令对多条腿的语义,串行规划与一起构建,测试归档,机器输出 | 评审中 v1.0 | 2026-10-30 | mcpp >= 2026.10.30.1 | ## 文档约定 diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 98a89ea7c..c982ac663 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -10,7 +10,7 @@ | 对应实现 | mcpp >= 2026.10.30.1 | | 相关设计文档 | `.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md`(§4、§13)、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`(§2–§7、§14)、`.agents/docs/2026-10-28-platforms-design.md`(§4.7)、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md`(§2.2、§2.6) | | 相关 issue | mcpp#847、mcpp#849、mcpp#860 | -| 相关规范 | [SPEC-007](build-plugins.md)(包格式提供者、artifact action)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-013](platforms.md)(平台、运行器、载体)、SPEC-015(目标集合与腿) | +| 相关规范 | [SPEC-007](build-plugins.md)(包格式提供者、artifact action)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-013](platforms.md)(平台、运行器、载体)、[SPEC-015](target-sets.md)(目标集合与腿) | | 使用文档 | [10 —— 发布打包](../zh/10-pack-and-release.md) | 用语按 RFC 2119:**必须 / 禁止**为强制,**应当**为建议,**可以**为可选。 diff --git a/docs/specs/target-sets.md b/docs/specs/target-sets.md new file mode 100644 index 000000000..6b0f34880 --- /dev/null +++ b/docs/specs/target-sets.md @@ -0,0 +1,120 @@ +# SPEC-015:目标集合与腿 + +| 项 | 值 | +|---|---| +| 规范编号 | SPEC-015 | +| 标题 | 目标集合与腿:集合的写法与展开,命令对集合的语义,腿的规划与构建,测试归档,机器输出 | +| 状态 | 评审中 v1.0 | +| 版本 | 1.0 | +| 最后修改 | 2026-10-30 | +| 对应实现 | mcpp >= 2026.10.30.1 | +| 相关设计文档 | `.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md`(§3) | +| 相关 issue | mcpp#847 | +| 相关规范 | [SPEC-011](concurrency.md)(锁与原子发布)、[SPEC-013](platforms.md)(平台的选择、运行器、执行宿主)、[SPEC-014](packaging.md)(打包的腿与合并) | +| 使用文档 | [21 —— 目标三元组](../zh/21-the-target-triple.md)、[08 —— 测试](../zh/08-testing.md)、[50 —— 机器输出](../zh/50-machine-output.md) | + +用语按 RFC 2119:**必须 / 禁止**为强制,**应当**为建议,**可以**为可选。 + +## 0. 范围 + +本规范规定 `--target` 的一种写法:**目标集合**,即一个系统(及至多一个环境)所含的、本 mcpp 构建的全部目标;以及 +`build`、`run`、`test`、`pack`、`self env` 对一组目标(**腿**)的行为。腿的打包与合并属于 SPEC-014 §1.4。 + +## 1. 写法与展开 + +### 1.1 集合的识别 + +- **精确目标**是三元组,含义不变(docs/21)。 +- **目标集合**是首段属于系统词表的写法。词表:`linux`、`macos`、`windows`、`openkal`、`emscripten`、`ios`、`android`、 + `none`。识别只看首段:`triple::parse` 不校验首段(`unknown-linux-gnu` 解析为架构 `unknown`),所以集合不以解析失败识别。 +- 集合由系统与至多一个环境组成:`openkal`、`linux-musl`、`windows-gnu`、`none-eabihf`。多于两段的写法被拒绝。 + +### 1.2 环境的补全 + +| 写法 | 系统 | 环境 | +|---|---|---| +| `linux` | linux | `gnu` | +| `linux-` | linux | `` | +| `windows` | windows | `gnu`(不含 `msvc`) | +| `android` | linux | `android`(不接受更多的段) | +| `none` | — | 拒绝:裸机没有缺省环境,写出 `none-elf`、`none-eabi`、`none-eabihf` | +| `none-`、`ios`、`ios-` | 同名 | 按写法 | +| `openkal`、`macos`、`emscripten` | 同名 | 无;写出环境段时被拒绝 | + +### 1.3 展开 + +1. 候选为内置目标表中系统与环境都相同、且等级不是 `planned` 的行。 +2. 每个候选按 SPEC-013 的规则选择平台(目标行的平台列、`[target.] platform`、`MCPP__PLATFORM`)。全部候选 + **必须**选到同一个平台;否则命令被拒绝,消息逐行写出每个候选及其平台。 +3. 没有候选时命令被拒绝;表中有 `planned` 的行时,消息写出这些行。 +4. 结果为规范三元组,按目标表的顺序。 + +| 写法 | 展开(mcpp 2026.10.30.1 的目标表与 openkal-platform 0.18.0) | +|---|---| +| `openkal` | `x86_64-openkal`、`aarch64-openkal` | +| `linux-musl` | `x86_64-linux-musl`、`aarch64-linux-musl`(`planned` 的行不在内) | +| `linux` | `x86_64-linux-gnu`、`aarch64-linux-gnu` | + +### 1.4 一条命令的腿 + +- 一条命令的腿是全部 `--target` 的并集:集合展开为其成员,精确目标保持原写法(单独规划时的补全与报告不变)。 +- 同一目标只算一次(以规范形式比较)。顺序为目标表的顺序;表中没有的目标排在其后,保持书写的顺序。 +- 机器输出与摘要只写展开后的目标;集合的写法不进入记录。 +- `[build] target` **必须**是精确目标;写集合时清单被拒绝。集合可以写在命令行与 `[pack] targets`(SPEC-014 §2.2)。 + +## 2. 命令的语义 + +| 命令 | 一条腿 | 多条腿 | +|---|---|---| +| `build` | 构建这条腿 | 构建全部腿,每条腿在 `target//` 下;一个 `Finished` | +| `run` | 本机架构经运行器;另一架构经执行宿主(SPEC-013 §10),否则拒绝 | 选第一条能到达的腿,本机架构优先;只构建这一条;写出所选的腿;都不能到达时拒绝 | +| `test` | 同 `run` 的到达规则;到达不了即拒绝 | 构建全部腿,运行能到达的腿,其余逐条报告"已构建、未运行"及原因;**一个测试都没有运行时失败**(退出码 1) | +| `pack` | 一个包 | SPEC-014 §1.4 | +| `self env` | 不变 | `data.legs[]` 每项是一份精确目标的报告;顶层取第一条腿 | + +平台包中不带 `--target` 的 `mcpp test` 遍历平台的 `targets`,即多条腿的一个特例,行为同上表。 + +## 3. 规划与构建 + +- 腿**逐条串行规划**:一次规划一条腿,`mcpp.lock` 只在规划阶段写入,避免 SPEC-011 §4 的乐观写入冲突。 +- 规划完成后各腿**一起构建**:`[build] jobs` 按腿静态分摊;构建目录锁按目标名排序获取。 +- 一条腿失败时,其余腿完成构建;命令的状态为第一条失败的腿的状态。 +- 单腿时规划、构建、`.last-result.json` 与记录的 `configuration` 与 2026.10.29.1 相同。多腿时 `.last-result.json` 记录每条腿, + 根目录的编译数据库为第一条腿的。 + +## 4. 测试归档 + +- **写**:`mcpp test [--target ...] --no-run --archive ` 构建各腿的测试程序,与清单 `mcpp-tests.json`(每个测试的腿、 + 参数、工作目录)一起写为一个 tar 归档。归档是引擎的格式,与平台的容器无关。 +- **读**:`mcpp test --archive ` 在同一项目的检出中运行归档内本机能到达的测试,不构建;工作目录为项目检出;报告与 + `mcpp test` 相同。 +- 读取时:有测试失败则退出码 1;一个测试都没有运行则退出码 1 并写明;本机到达不了的腿报告为"未运行"。 +- 原生交叉同样适用:在 Linux 上构建的 `x86_64-windows-gnu` 测试可以在 Windows 上运行。 + +## 5. 机器输出 + +- `mcpp.build`:单腿时逐字节不变。多腿时 `data` 的顶层字段取第一条腿,`status` 汇总全部腿,`legs[]` 每项为一份完整的结果 + 记录;`artifact` 与 `diagnostic` 事件带 `triple`。 +- `mcpp.env`(`self env --format json`):多腿时增加 `legs[]`。 +- `mcpp.pack`:`legs`、`pkgFormats`、`runsOn`(SPEC-014 §9)。 +- 新字段只增加,不改变已有字段的含义(docs/50)。 + +## 6. 判据 + +1. `mcpp build --target openkal` 构建两条腿,各在 `target//` 下,一个 `Finished`;信封的 `data` 为第一条腿的记录, + `legs` 含两条,每个 `artifact` 事件写出其腿(e2e 963)。 +2. `mcpp run --target openkal` 运行本机的腿并写出所选的腿;`mcpp test --target openkal` 运行本机的腿,另一条报告为 + 已构建、未运行,退出码 0(e2e 963)。 +3. 各腿都到达不了时 `mcpp test` 不运行任何测试,退出码 1 并写明(e2e 963)。 +4. `mcpp self env --target openkal --format json` 报告两条腿(e2e 963)。 +5. `none`(无环境)、`linux-msvc`(无成员)与 `[build] target` 中的集合被拒绝(e2e 963)。 +6. 集合的识别、补全、展开、去重与顺序,`planned` 的行不在内,候选选到不同平台被拒绝(unit `TargetSet`)。 +7. 腿的规划串行、构建一起、作业分摊与第一条失败的腿(unit `BuildLegs`)。 +8. `--no-run --archive` 写出的归档含两条腿与清单;在没有 `target/` 的检出中 `--archive` 运行本机的腿、不构建;失败的 + 测试使退出码为 1;没有一条腿能到达的归档不运行任何测试且退出码为 1(e2e 964)。 + +## 变更记录 + +| 版本 | 日期 | 变更 | +|---|---|---| +| 1.0 | 2026-10-30 | 首版(mcpp 2026.10.30.1,设计 2026-10-29 §3):集合的识别、补全与展开,一条命令的腿,`build`、`run`、`test`、`self env` 对多条腿的语义,串行规划与一起构建,测试归档,机器输出;判据 1–8(e2e 963、964,unit `TargetSet`、`BuildLegs`)。 | diff --git a/docs/zh/08-testing.md b/docs/zh/08-testing.md index 1ea4a9e18..e06a686ff 100644 --- a/docs/zh/08-testing.md +++ b/docs/zh/08-testing.md @@ -173,6 +173,21 @@ test result ok. 0 passed; 0 failed; 2 built, not run runner 本身、具名 runner，以及一块板子声明了什么，见 [41 —— 抵达一台设备](41-devices.md)。 +### 构建一次，各处运行（mcpp 2026.10.30.1+） + +ok 映像在每个系统上是同样的字节；在 Linux 上为 `x86_64-windows-gnu` 构建的测试 +在 Windows 上运行。`--archive` 把两半分开： + +```bash +mcpp test --target openkal --no-run --archive tests.tar.gz # 构建各腿的测试并写出 +mcpp test --archive tests.tar.gz # 在检出中运行本机能到达的部分 +``` + +归档含各腿的测试程序与 `mcpp-tests.json`（每个测试的腿、参数与工作目录）。读取 +时不构建；测试在检出中经本机的运行器或系统运行，报告与 `mcpp test` 相同。有测试 +失败时退出码为 1，一个测试都没运行的归档也是 1。本机到达不了的腿报告为未运行 +（[SPEC-015 §4](../specs/target-sets.md)）。 + ## 不能彼此并排运行的测试 `mcpp test` 在一个工作池上运行测试程序。一块接了一个探针的板子、一块 GPU、一个串口， diff --git a/docs/zh/21-the-target-triple.md b/docs/zh/21-the-target-triple.md index 891b687f7..1c2d8e0c1 100644 --- a/docs/zh/21-the-target-triple.md +++ b/docs/zh/21-the-target-triple.md @@ -180,6 +180,26 @@ LLVM 23.1.3 与 glibc。`aarch64-linux-musl` 仍是静态 musl 产物的显式 而 `msvc` 是不同的对象 ABI 而非不同的 C 库，因此那一段在两套体系下都 有意义。 +## 目标集合（mcpp 2026.10.30.1+） + +首段是系统词的 `--target`（`linux`、`macos`、`windows`、`openkal`、`emscripten`、 +`ios`、`android`、`none`）是一个**目标集合**：目标表中系统与环境相同、且本 +mcpp 构建的全部行。环境按上一节补全（`linux` 即 `linux-gnu`，`windows` 即 +`windows-gnu`）；`none` 没有缺省环境，被拒绝。 + +| 集合 | 成员（2026.10.30.1，openkal-platform 0.18.0） | +|---|---| +| `openkal` | `x86_64-openkal`、`aarch64-openkal` | +| `linux-musl` | `x86_64-linux-musl`、`aarch64-linux-musl` | +| `linux` | `x86_64-linux-gnu`、`aarch64-linux-gnu` | + +全部成员必须选到同一个平台；成员选到不同平台时命令被拒绝，消息写出每个成员 +的平台。命令对每个成员各执行一次，每个成员是一条**腿**：`build` 构建全部腿， +`run` 运行本机能到达的第一条腿（本机架构优先），`test` 运行能到达的腿、一个都 +没运行时失败，`pack` 按 [10](10-pack-and-release.md) 打包。报告与机器输出写成员， +不写集合。`[build] target` 只接受一个目标；集合写在命令行与 `[pack] targets`。 +规则见 [SPEC-015](../specs/target-sets.md)。 + ## 构建报告 报告以写下的目标为标题，并把它解析成编译器自己的拼法： diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index cda1dbd63..d432cfb5b 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -848,6 +848,13 @@ CI 按 `launch` 启动程序，与 `mcpp run` 的启动一致。 "data":{"result":"/…/target/x86_64-linux-gnu/c0641c458b2bde43/result.json","status":"ok","finishedAt":1792964065954}} ``` +**多条腿** *（2026.10.30.1+）*。对目标集合或多个 `--target` 的命令每个目标构建一条腿 +（[SPEC-015](../specs/target-sets.md)），每条腿写自己的 `result.json`。`mcpp.build` 信封的 `data` +为第一条腿的记录，`status` 汇总全部腿，`legs` 含每条腿的记录；每个 `artifact` 与 `diagnostic` +事件带其腿的 `triple`。`.last-result.json` 指向第一条腿的记录，并增加 `legs`，每条腿一项 +`{triple, result, status}`。只有一条腿时信封、事件与两个文件与此前相同。`self env --target <集合>` +在 `mcpp.env` 中增加 `legs`，每条腿一份报告，顶层字段取第一条腿。 + ### `mcpp.launch` —— `mcpp run` 启动程序的方式 ``` diff --git a/docs/zh/README.md b/docs/zh/README.md index 6ebd79aa2..6f41ba2e8 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -171,4 +171,5 @@ - [SPEC-011 —— 并发运行：构建目录锁、整体发布、乐观写入与取消](../specs/concurrency.md) - [SPEC-012 —— 面向人的输出：两条流、两种介质、行的种类与颜色](../specs/human-output.md) - [SPEC-013 —— 平台：一类目标的配置，其选择、合并、对依赖图的补全、链接、封装、运行器与载体](../specs/platforms.md) - - [SPEC-014 —— 打包与分发：链接、闭包与容器，构建腿，暂存树，通用包与单文件包](../specs/packaging.md) + - [SPEC-014 —— 打包与分发：链接、闭包与容器，构建腿，暂存树，平台的树、运行覆盖与单文件包](../specs/packaging.md) + - [SPEC-015 —— 目标集合与腿：写法与展开，命令对多条腿的语义，测试归档](../specs/target-sets.md) From 9d7d057477409dfa82cb865c4c7a8bd9924465c0 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 05:28:55 +0900 Subject: [PATCH 11/20] ask.log is no file of this repository: a shell redirect's leftover Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ask.log | 1 - 1 file changed, 1 deletion(-) delete mode 100644 ask.log diff --git a/ask.log b/ask.log deleted file mode 100644 index ad3f15e50..000000000 --- a/ask.log +++ /dev/null @@ -1 +0,0 @@ -/bin/bash: line 32: : command not found From 3074fa442e4886bb4565a252b84374b242843da3 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 05:46:46 +0900 Subject: [PATCH 12/20] e2e 868: two --target over several members are two legs of each, no longer refused Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ..._members_is_refused_before_anything_is_compiled.sh | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/tests/e2e/868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh b/tests/e2e/868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh index a9873e63c..d94b404a0 100644 --- a/tests/e2e/868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh +++ b/tests/e2e/868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh @@ -12,7 +12,8 @@ # E. A selected member without the requested `--format` is refused by name. # `plain` provides nothing; `cli` provides `zap`. # F. A target name is refused with several members: it names a target of one -# package. `--target` given twice is refused too. +# package. (`--target` given twice is two legs of every member since +# 2026.10.30.1, SPEC-014 §1.4, and is no longer refused.) # G. `--output` with several members must be a directory, and a path that exists # as a file is refused. # H. A member with no program target is refused by name when `-p` names it, and @@ -127,15 +128,11 @@ echo "ok: E, a member without the format is refused by name, before the build" set +e "$MCPP" pack --mode system -p cli -p plain cli > f1.log 2>&1 rc1=$? -"$MCPP" pack --mode system -p cli -p plain --target x86_64-linux-gnu --target aarch64-linux-gnu > f2.log 2>&1 -rc2=$? set -e [ "$rc1" -ne 0 ] && grep -q "target name" f1.log && grep -q "'cli'" f1.log \ || fail "F: a target name with several members was not refused, naming it" f1.log -[ "$rc2" -ne 0 ] && grep -q -- "--target may be given once" f2.log \ - || fail "F: two --target with several members were not refused" f2.log -compiled_nothing . || fail "F: a refusal arrived after a compile" f1.log f2.log -echo "ok: F, a target name and two --target are refused with several members" +compiled_nothing . || fail "F: a refusal arrived after a compile" f1.log +echo "ok: F, a target name is refused with several members" # ── G ──────────────────────────────────────────────────────────────────────── : > a-file From 2ac9ef6eeb128cda45f2e000dea1a790958c479f Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 05:59:00 +0900 Subject: [PATCH 13/20] =?UTF-8?q?design=20=C2=A710.4:=20one=20logo=20for?= =?UTF-8?q?=20every=20editor=20integration;=20the=20CLion=20plugin=20and?= =?UTF-8?q?=20tool=20window=20icons=20are=20the=20logo,=20not=20text?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...arget-sets-packaging-and-editors-design.md | 22 ++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md b/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md index b7e04dc2b..5aa29e6dd 100644 --- a/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md +++ b/.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md @@ -452,7 +452,23 @@ mcppls-clangd 的 draft PR #11（分支 `probe/clangd-openkal-image`，run 40908 jar 核对后定。 - `plugin.xml`：261 上 test runner 属于平台核心，optional 依赖只在其存在的版本声明，注释中的"2025.2"改正。 -### 10.4 其他 +### 10.4 图标：一个 logo + +mcpp 的编辑器集成只有一个图标：mcpp 的 logo（MC++ 图，`mcpp-vscode/media/logo.png`，与 mcppls 的 +`docs/imgs/mcpp-logo.png`、`editors/vscode/icon.png` 字节相同）。文字图标不是 mcpp 的图标。 + +| 位置 | 今天 | 本设计 | +|---|---|---| +| mcpp-ide VS Code 扩展 `icon`、活动栏 | `media/logo.png`、`media/activity-bar.png`，与 0.9.0 字节相同 | 不变 | +| mcpp-ide CLion 插件 `META-INF/pluginIcon.svg` | 蓝底白字 `m++` | logo，按 mcppls 的做法展平（一个裁剪的嵌入图，无 `symbol`/`use`、无 DOCTYPE，IntelliJ 的每个 SVG 渲染器都显示），与 mcppls 的 `pluginIcon.svg` 字节相同；不另设 `pluginIcon_dark.svg`（logo 自带深色底） | +| mcpp-ide CLion 工具窗口 `icons/mcpp.svg`、`mcpp_dark.svg` | 蓝底白字 `m+` | logo 的 16×16 版本，同样展平 | +| mcppls CLion 插件、VS Code 扩展 | logo | 不变 | + +- mcpp-ide 的 CI 检查：VS Code 的 `logo.png` 与 0.9.0 字节相同；CLion 的 `pluginIcon.svg` 与工具窗口图标嵌入的图解码后来自 + 同一 logo，且不含 ``。 +- JetBrains Marketplace 与 VS Code Marketplace 的列表图由上面的文件给出，发布后两处显示同一 logo。 + +### 10.5 其他 - 目标选择器显示集合的一行（§3.2）。 - Open VSX 发布 1.0.x 之后的版本；mcpp-vscode 仓库在 Open VSX 同步后归档。 @@ -468,7 +484,7 @@ mcppls-clangd 的 draft PR #11（分支 `probe/clangd-openkal-image`，run 40908 - 不回落：描述符键表的每一行；wasm 构建在三种宿主上读同一段；openkal 映像在 Linux 与 macOS 宿主上字节相同；#852 的两个 rules。 - #857：probe 的逐步记录；Termux 应用沙箱 job 恢复通过。 - 外部构建：一个 CMake 静态库经 `deps-cmake` 与契约链接成 ok 映像并在三系统运行；宿主工具的 CMake 项目。 -- 编辑器：§10.1 的三道门。 +- 编辑器：§10.1 的三道门；§10.4 的图标检查。 ## 12. 交付（一轮）与级联 @@ -488,7 +504,7 @@ mcppls-clangd 的 draft PR #11（分支 `probe/clangd-openkal-image`，run 40908 | 9 | mcpp-plugins | `deps-cmake` 读契约；`dep_dir` 的歧义失败 | | 10 | mcppls-clangd | 无改动（§9.4：本轮 clangd 保持原生）；probe PR #11 关闭 | | 11 | mcppls | 单包、S4 修订、kit 流程、CI 改用 `--archive` | -| 12 | mcpp-ide | VS Code（§10.1）、组件库、CLion API、CI 迁移（§13.1） | +| 12 | mcpp-ide | VS Code（§10.1）、组件库、CLion API、图标（§10.4）、CI 迁移（§13.1） | | 13 | xim-pkgindex | mcpp-ide、mcppls、okrun 的条目重新生成 | | 14 | mcpp-vscode | Open VSX 同步后归档 | From 79e5ce10024f2a44f7d7e6ec5057e2ebbecffe9c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 06:32:12 +0900 Subject: [PATCH 14/20] the Windows launcher reads its own path with / between the names: over the hosted arrangement a \-named path has no parent, and the package held no architecture (e2e 988) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/distribution.cppm | 9 +++- ...r_finds_its_package_from_a_windows_path.sh | 49 +++++++++++++++++++ 2 files changed, 56 insertions(+), 2 deletions(-) create mode 100755 tests/e2e/988_the_windows_launcher_finds_its_package_from_a_windows_path.sh diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 89b77f817..50acb9762 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -1091,9 +1091,14 @@ constexpr std::string_view kRunner[] = {{ int main(int argc, char** argv) {{ // The package is the parent of the directory this program is in, found - // from the program's own path and not from argv[0]. + // from the program's own path and not from argv[0]. The path is taken with + // `/` between its names: the program is built over the platform's hosted + // arrangement, whose paths are POSIX's, and Windows names a program with + // `\`, which such a path does not split (its parent would be empty). const auto self = os::env::image(); - const fs::path top = self ? self->parent_path().parent_path() : fs::current_path(); + std::string me = self ? self->string() : std::string(); + std::ranges::replace(me, '\\', '/'); + const fs::path top = self ? fs::path(me).parent_path().parent_path() : fs::current_path(); std::string machine = os::env::get("PROCESSOR_ARCHITEW6432").value_or( os::env::get("PROCESSOR_ARCHITECTURE").value_or("")); for (auto& c : machine) c = static_cast(std::toupper(static_cast(c))); diff --git a/tests/e2e/988_the_windows_launcher_finds_its_package_from_a_windows_path.sh b/tests/e2e/988_the_windows_launcher_finds_its_package_from_a_windows_path.sh new file mode 100755 index 000000000..e6512cccc --- /dev/null +++ b/tests/e2e/988_the_windows_launcher_finds_its_package_from_a_windows_path.sh @@ -0,0 +1,49 @@ +#!/usr/bin/env bash +# requires: llvm python3 wine +# 988 -- the Windows launcher finds its package from a Windows path (SPEC-014 +# §4.3; design 2026-10-29 rev 2 §4.3). +# +# `bin/.exe` is built over the platform's hosted arrangement, whose paths +# are POSIX's. Windows names the program with `\` (`D:\a\…\bin\app.exe`), which +# such a path does not split, so the launcher took an empty parent as the +# package and said "the package holds no architecture" with `lib/x86_64/` in +# place (openkal-platform CI on windows-2022, 2026-10-30). The launcher now +# reads its own path with `/` between the names. +# +# Criteria: +# A. `mcpp pack --target x86_64-openkal --mode self-contained --systems +# linux,windows` writes `bin/.exe` and the Windows runner. +# B. Under wine, which names the program `Z:\…\bin\.exe`, the launcher +# runs the image through the carried runner with its arguments. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +case "$(uname -s)-$(uname -m)" in + Linux-x86_64|Linux-amd64) ;; + *) echo "SKIP: the Windows runner is x86_64 and wine runs it on Linux x86_64"; exit 0 ;; +esac + +mkdir -p "$TMP/app/src" +cd "$TMP/app" +printf '[package]\nname = "wl"\nversion = "0.1.0"\n' > mcpp.toml +cat > src/main.cpp <<'CPP' +import std; +int main(int argc, char**) { std::println("wl ok: {} args", argc); } +CPP + +# A +"$MCPP" pack --target x86_64-openkal --mode self-contained --systems linux,windows \ + --pkg-format dir -o t > a.log 2>&1 || fail "A: pack" a.log +[ -f target/dist/t/bin/wl.exe ] || fail "A: no bin/wl.exe" a.log +[ -f target/dist/t/libexec/okrun/windows-x86_64/okrun.exe ] || fail "A: no Windows runner" a.log +echo "ok: the tree holds the Windows launcher and runner" + +# B +WINEDEBUG=-all timeout 120 wine target/dist/t/bin/wl.exe one two 2>&1 | tr -d '\r' > b.log || true +grep -q "wl ok: 3 args" b.log || fail "B: the launcher under wine" b.log +echo "ok: the launcher finds its package from a Windows path" + +echo "PASS: 988 the Windows launcher finds its package from a Windows path" From b5ac3738bf93fc739740c3d1eb8cd55e63a7ae51 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 05:32:12 +0900 Subject: [PATCH 15/20] openkal-platform 0.18.0: the built-in pin, the examples and the e2e that names it Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- docs/04-mcpp-toml.md | 2 +- docs/10-pack-and-release.md | 2 +- docs/24-openkal-cross.md | 4 ++-- docs/35-platforms.md | 20 +++++++++---------- docs/50-machine-output.md | 4 ++-- docs/specs/packaging.md | 2 +- docs/specs/platforms.md | 4 ++-- docs/zh/04-mcpp-toml.md | 2 +- docs/zh/10-pack-and-release.md | 2 +- docs/zh/24-openkal-cross.md | 4 ++-- docs/zh/35-platforms.md | 20 +++++++++---------- docs/zh/50-machine-output.md | 4 ++-- modules/toolchain-model/src/triple.cppm | 4 ++-- ...ost_graph_runs_build_programs_as_images.sh | 2 +- tests/unit/test_pack_distribution.cpp | 2 +- 15 files changed, 39 insertions(+), 39 deletions(-) diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 7fed0511b..cb6f32cb1 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1496,7 +1496,7 @@ the platform's own key names ([35 — Platforms](35-platforms.md)): ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.17.3" # or { path = "…" }, or "none" +platform = "openkal-platform@0.18.0" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 0b13cd241..cf144ad8a 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -918,7 +918,7 @@ and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it ([SPEC-014](specs/packaging.md) §4.5): ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.3", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.18.0", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index 56941c93e..04872c803 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -111,7 +111,7 @@ practice. ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.3" +platform = "openkal-platform@0.18.0" ``` One table (2026.10.27.1+). It selects the openkal platform for the host triples, @@ -454,7 +454,7 @@ run_under = ["qemu-aarch64"] ``` $ mcpp run --target aarch64-openkal - Packing okrun for linux-aarch64 (openkal-platform 0.17.3's carrier, the runner on the execution host) + Packing okrun for linux-aarch64 (openkal-platform 0.18.0's carrier, the runner on the execution host) Running `okrun … target/aarch64-openkal/…/bin/app.oke` Via qemu-aarch64 [/…/app/mcpp.toml:6] ``` diff --git a/docs/35-platforms.md b/docs/35-platforms.md index e0415a21e..a29981ea1 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -40,12 +40,12 @@ mcpp run --target x86_64-openkal ``` Resolved llvm@23.1.3 → x86_64-openkal → …/xim-x-llvm/23.1.3/bin/clang++ - the platform's (openkal-platform 0.17.3), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` + the platform's (openkal-platform 0.18.0), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` Target x86_64-openkal → x86_64-unknown-none-elf compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.6, platform) c-abi musl (openkal-musl@0.22.6, platform) c++-abi libc++ (openkal-llvm-runtime@0.16.6, platform) - Platform openkal-platform 0.17.3 (built-in) + Platform openkal-platform 0.18.0 (built-in) tools okld, okrun (openkal-tools@0.1.3, platform) runner okrun (platform) link lld, start ok_main (platform) @@ -113,9 +113,9 @@ The first of these that answers selects the target's platform: | Source | Spelling | |---|---| -| the command line | `--platform openkal-platform@0.17.3`, `--platform path:../openkal-platform`, `--platform none` | +| the command line | `--platform openkal-platform@0.18.0`, `--platform path:../openkal-platform`, `--platform none` | | the environment | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| the project | `[target.] platform = "openkal-platform@0.17.3"`, or `{ path = "…" }` | +| the project | `[target.] platform = "openkal-platform@0.18.0"`, or `{ path = "…" }` | | this machine | the same key in `~/.mcpp/config.toml` | | the target table | the platform column (`mcpp toolchain list`) | @@ -129,11 +129,11 @@ its own conventions. `[host] platform` selects a platform for them instead: ```toml [host] -platform = "openkal-platform@0.17.3" +platform = "openkal-platform@0.18.0" ``` ``` - Host platform openkal-platform 0.17.3 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun + Host platform openkal-platform 0.18.0 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun ``` Every build program then compiles as an ok image of this machine's @@ -151,7 +151,7 @@ implementation: ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.3" +platform = "openkal-platform@0.18.0" ``` On Windows the arrangement is the GNU ABI's row (`--target x86_64-windows-gnu`); @@ -165,7 +165,7 @@ packages pin one another exactly. A graph that holds another version of one of them is refused before anything compiles: ``` -error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.3 is a version group that names openkal-musl 0.22.6. +error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.18.0 is a version group that names openkal-musl 0.22.9. requested by: own The family's packages pin one another exactly, so two versions of one of them cannot be linked into one program. Either select the platform release @@ -184,8 +184,8 @@ abridged: [package] namespace = "mcpplibs" name = "openkal-platform" -version = "0.17.3" -mcpp = ">=2026.10.27.1" +version = "0.18.0" +mcpp = ">=2026.10.30.1" [platform] targets = ["x86_64-openkal", "aarch64-openkal"] diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 76228de6c..69ee1cb82 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -349,7 +349,7 @@ select as they do for a build ([35](35-platforms.md)). "triple": "x86_64-openkal", "graph": "target", // or "host" "toolchain": {"compiler": "/…/xim-x-llvm/23.1.3/bin/clang++", "origin": "platform"}, - "platform": {"name": "mcpplibs.openkal-platform", "version": "0.17.3", "source": "built-in", + "platform": {"name": "mcpplibs.openkal-platform", "version": "0.18.0", "source": "built-in", "hosted": false, "fields": {"runner": {"value": ["okrun", "{}"], "source": "platform"}}}, "tools": {"okld": "/…/okld", "okrun": "/…/okrun"}, "runner": {"argv": ["okrun", "{}"], "host": "same-arch", "exclusive": false, @@ -495,7 +495,7 @@ installation, which is located on the machine rather than installed by mcpp. A target row is `{target, note, toolchain, pin, status, default}`, and `status` is one of `installed` / `available` / `via dependency graph` / `planned`. A target row also carries `platform` (2026.10.27.1+): the platform package the -target table selects for it (`mcpplibs.openkal-platform@0.17.3`), or `null`. +target table selects for it (`mcpplibs.openkal-platform@0.18.0`), or `null`. **`toolchain` and `pin` are not the same field twice.** `toolchain` is what the row is associated with — the installed payload on an installed row, the diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index c982ac663..7b5d135d4 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -233,7 +233,7 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` "schema": 1, "name": "hello", "version": "0.1.0", - "platform": "openkal-platform@0.17.3", + "platform": "openkal-platform@0.18.0", "linkage": "declared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 6040c08af..d10028c01 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -430,7 +430,7 @@ openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscri 目标报告中,平台加入的层的来源写作 `platform`,图已提供的写作 `graph`。层之后为平台块: ``` - Platform openkal-platform 0.17.3 (built-in) + Platform openkal-platform 0.18.0 (built-in) tools okld, okrun (openkal-tools@0.1.3, platform) runner okrun (platform) link lld, start ok_main (platform) @@ -445,7 +445,7 @@ openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscri `null`;对象形式: ```json -{"name": "mcpplibs.openkal-platform", "version": "0.17.3", "source": "built-in", "hosted": false, +{"name": "mcpplibs.openkal-platform", "version": "0.18.0", "source": "built-in", "hosted": false, "fields": {"toolchain": {"value": "llvm@23.1.3", "source": "platform"}, "layers.c-abi": {"value": "openkal-musl@0.22.6", "source": "platform"}}} ``` diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index fc18d6cbb..3dc863887 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1380,7 +1380,7 @@ runner 与 `run_under`。 ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.17.3" # or { path = "…" }, or "none" +platform = "openkal-platform@0.18.0" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 1a57d2a5f..b71a511cd 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -775,7 +775,7 @@ WebAssembly 目标没有单文件：`--pkg-format single-file` 与 `--target was 提供的格式、`--merge` 与 `mcpp emit xpkg` 读取它（[SPEC-014](../specs/packaging.md) §4.5）： ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.17.3", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.18.0", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index 5b4ec9504..0f007ec0b 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -93,7 +93,7 @@ SDK，或者区分 `linux`/`windows`/`macos`。那些是 C 环境层或平台层 ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.3" +platform = "openkal-platform@0.18.0" ``` 一张表（2026.10.27.1+）。它为宿主三元组选择 openkal 平台，平台的宿主部分命名编译器与目标侧三个层的包； @@ -395,7 +395,7 @@ run_under = ["qemu-aarch64"] ``` $ mcpp run --target aarch64-openkal - Packing okrun for linux-aarch64 (openkal-platform 0.17.3's carrier, the runner on the execution host) + Packing okrun for linux-aarch64 (openkal-platform 0.18.0's carrier, the runner on the execution host) Running `okrun … target/aarch64-openkal/…/bin/app.oke` Via qemu-aarch64 [/…/app/mcpp.toml:6] ``` diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index a4ccd123e..b8c0754f9 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -32,12 +32,12 @@ mcpp run --target x86_64-openkal ``` Resolved llvm@23.1.3 → x86_64-openkal → …/xim-x-llvm/23.1.3/bin/clang++ - the platform's (openkal-platform 0.17.3), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` + the platform's (openkal-platform 0.18.0), replacing your gcc@16.1.0 — override with `[target.x86_64-openkal] toolchain` Target x86_64-openkal → x86_64-unknown-none-elf compiler-runtime compiler-rt (openkal-llvm-runtime@0.16.6, platform) c-abi musl (openkal-musl@0.22.6, platform) c++-abi libc++ (openkal-llvm-runtime@0.16.6, platform) - Platform openkal-platform 0.17.3 (built-in) + Platform openkal-platform 0.18.0 (built-in) tools okld, okrun (openkal-tools@0.1.3, platform) runner okrun (platform) link lld, start ok_main (platform) @@ -98,9 +98,9 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 | 来源 | 写法 | |---|---| -| 命令行 | `--platform openkal-platform@0.17.3`、`--platform path:../openkal-platform`、`--platform none` | +| 命令行 | `--platform openkal-platform@0.18.0`、`--platform path:../openkal-platform`、`--platform none` | | 环境变量 | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| 工程 | `[target.] platform = "openkal-platform@0.17.3"`,或 `{ path = "…" }` | +| 工程 | `[target.] platform = "openkal-platform@0.18.0"`,或 `{ path = "…" }` | | 本机 | `~/.mcpp/config.toml` 中的同一个键 | | 目标表 | 平台列(`mcpp toolchain list`) | @@ -112,11 +112,11 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 ```toml [host] -platform = "openkal-platform@0.17.3" +platform = "openkal-platform@0.18.0" ``` ``` - Host platform openkal-platform 0.17.3 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun + Host platform openkal-platform 0.18.0 (mcpp.toml:2): build programs are x86_64-openkal images run by okrun ``` 此后每个构建程序都编译为本机架构的 ok 映像,在 Linux、macOS 与 Windows 上使用同一个工具链,经平台的运行器 @@ -130,7 +130,7 @@ platform = "openkal-platform@0.17.3" ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.17.3" +platform = "openkal-platform@0.18.0" ``` Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[platform.hosted] targets` 之外的 @@ -142,7 +142,7 @@ Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[pl 另一个版本时,在任何编译之前拒绝: ``` -error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.17.3 is a version group that names openkal-musl 0.22.6. +error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0.18.0 is a version group that names openkal-musl 0.22.9. requested by: own The family's packages pin one another exactly, so two versions of one of them cannot be linked into one program. Either select the platform release @@ -160,8 +160,8 @@ error: openkal-musl 0.22.2 is in this build, and the platform openkal-platform 0 [package] namespace = "mcpplibs" name = "openkal-platform" -version = "0.17.3" -mcpp = ">=2026.10.27.1" +version = "0.18.0" +mcpp = ">=2026.10.30.1" [platform] targets = ["x86_64-openkal", "aarch64-openkal"] diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index d432cfb5b..ba519c690 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -307,7 +307,7 @@ mcpp self env --format json "triple": "x86_64-openkal", "graph": "target", // or "host" "toolchain": {"compiler": "/…/xim-x-llvm/23.1.3/bin/clang++", "origin": "platform"}, - "platform": {"name": "mcpplibs.openkal-platform", "version": "0.17.3", "source": "built-in", + "platform": {"name": "mcpplibs.openkal-platform", "version": "0.18.0", "source": "built-in", "hosted": false, "fields": {"runner": {"value": ["okrun", "{}"], "source": "platform"}}}, "tools": {"okld": "/…/okld", "okrun": "/…/okrun"}, "runner": {"argv": ["okrun", "{}"], "host": "same-arch", "exclusive": false, @@ -438,7 +438,7 @@ mcpp toolchain list --format json 的形状是 `{target, note, toolchain, pin, status, default}`，`status` 取 `installed` / `available` / `via dependency graph` / `planned` 之一。 目标行另有 `platform`（2026.10.27.1+）：目标表为该目标选择的平台包 -（`mcpplibs.openkal-platform@0.17.3`），或 `null`。 +（`mcpplibs.openkal-platform@0.18.0`），或 `null`。 **`toolchain` 与 `pin` 不是把同一个字段写了两遍。** `toolchain` 表示这一行 关联到什么 —— 已装的行是装好的载荷，词表行是它自己的约定。`pin` 只承载目标 diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index da981d5a9..c0eb467b1 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -906,9 +906,9 @@ inline constexpr TargetInfo kKnownTargets[] = { // `openkal-platform` (SPEC-013), pinned here. The two architectures the // loader relocates. { "x86_64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.17.3" }, + "mcpplibs.openkal-platform@0.18.0" }, { "aarch64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.17.3" }, + "mcpplibs.openkal-platform@0.18.0" }, }; inline std::span known_targets() { return kKnownTargets; } diff --git a/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh b/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh index 48e64ae2d..ee59d3934 100755 --- a/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh +++ b/tests/e2e/953_the_host_graph_runs_build_programs_as_images.sh @@ -34,7 +34,7 @@ name = "hostgraph" version = "0.1.0" [host] -platform = "openkal-platform@0.17.3" +platform = "openkal-platform@0.18.0" [build-dependencies] helper = { path = "helper" } diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index 141c4c1d4..6b2d1fa6f 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -593,7 +593,7 @@ TEST(PackMerge, AMergedTreeIsNamedAfterThePlatform) { d::Package p; p.name = "app"; p.version = "1.0"; - p.platform = "mcpplibs.openkal-platform@0.17.3"; + p.platform = "mcpplibs.openkal-platform@0.18.0"; p.legs = { leg }; p.runner = { "okrun", "{}" }; p.entry = { { arch, std::format("lib/{}/app.oke", arch) } }; From 87505c71544cf1fb220c9ae6ddf0c6540b1876cf Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 08:12:00 +0900 Subject: [PATCH 16/20] mcpplibs.os 0.2.3: mcpp starts programs in a directory it may search but not read (openkal-linux 0.18.3, #857) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 3 ++- mcpp.toml | 2 +- modules/invocation/mcpp.toml | 2 +- modules/manifest/mcpp.toml | 2 +- modules/toolchain-model/mcpp.toml | 2 +- 5 files changed, 6 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f8c849ccd..9aa988c4e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -97,7 +97,8 @@ since 2026.10.26.2. A static strace showed one failing call, before any process was created: the root directory opened for reading, which the domain may search but not read. openkal-linux 0.18.3 opens every directory - handle with `O_PATH`. + handle with `O_PATH`; mcpp is built on mcpplibs.os 0.2.3, which pins it, and + openkal-platform 0.18.0 carries it to every ok image and hosted program. ## [2026.10.29.1] - 2026-10-29 diff --git a/mcpp.toml b/mcpp.toml index 7c8fe1d25..e9116a5da 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -91,7 +91,7 @@ mcpplibs.cmdline = "0.0.1" # The operating system is mcpplibs.os's (design 2026-10-26 §25): mcpp keeps no # layer of its own. [dependencies.mcpplibs] -os = "0.2.0" +os = "0.2.3" # The codecs of a package's container (SPEC-014 §5): gzip and raw DEFLATE, # zstd, xz. The reference libraries, so an archive is written the same way on diff --git a/modules/invocation/mcpp.toml b/modules/invocation/mcpp.toml index 12bbfb6db..84fa40b7b 100644 --- a/modules/invocation/mcpp.toml +++ b/modules/invocation/mcpp.toml @@ -13,4 +13,4 @@ description = "The environment of one invocation of mcpp: statements over the st license = "Apache-2.0" [dependencies.mcpplibs] -os = "0.2.0" +os = "0.2.3" diff --git a/modules/manifest/mcpp.toml b/modules/manifest/mcpp.toml index efaf50086..6376dc060 100644 --- a/modules/manifest/mcpp.toml +++ b/modules/manifest/mcpp.toml @@ -28,7 +28,7 @@ versioning = { path = "../versioning" } # state its contract in isolation; the root `tests/unit/` states the facts two # layers must agree on. [dependencies.mcpplibs] -os = "0.2.0" +os = "0.2.3" [dev-dependencies.compat] gtest = "1.15.2" diff --git a/modules/toolchain-model/mcpp.toml b/modules/toolchain-model/mcpp.toml index 6db08da8a..ff0598951 100644 --- a/modules/toolchain-model/mcpp.toml +++ b/modules/toolchain-model/mcpp.toml @@ -27,7 +27,7 @@ versioning = { path = "../versioning" } # state its contract in isolation; the root `tests/unit/` states the facts two # layers must agree on. [dependencies.mcpplibs] -os = "0.2.0" +os = "0.2.3" [dev-dependencies.compat] gtest = "1.15.2" From 75831844092a6c017a5354e07f702b23219b1920 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 08:44:16 +0900 Subject: [PATCH 17/20] a container already at the -o directory is kept: it was removed as the destination of its own copy (e2e 985 D) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/pack/pipeline.cppm | 6 +++++- ...85_a_format_list_writes_each_container_of_one_tree.sh | 9 +++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index a187a62f7..a687366bf 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -1546,7 +1546,11 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba // `-o /`: the distributable below it, under its own name. if (!run.opts.outputDir.empty()) for (auto& r : reported) { - const auto into = run.opts.outputDir / r.filename(); + const auto into = std::filesystem::absolute(run.opts.outputDir / r.filename(), ec).lexically_normal(); + // A provider may have written it there already (the okb is + // written beside the staged tree, in target/dist/): removing + // the destination would remove the product. + if (into == r.lexically_normal()) continue; std::filesystem::remove_all(into, ec); std::filesystem::copy(r, into, std::filesystem::copy_options::recursive, ec); if (ec) { diff --git a/tests/e2e/985_a_format_list_writes_each_container_of_one_tree.sh b/tests/e2e/985_a_format_list_writes_each_container_of_one_tree.sh index f16de60eb..e03e42288 100644 --- a/tests/e2e/985_a_format_list_writes_each_container_of_one_tree.sh +++ b/tests/e2e/985_a_format_list_writes_each_container_of_one_tree.sh @@ -18,6 +18,8 @@ # C. `--target openkal --mode self-contained --pkg-format tar,okb` writes the # tree's tar and the platform's okb; `--pkg-format okb` alone is refused # under `self-contained`, saying the runner would be dropped. +# D. `-o target/dist/`, the directory the containers are written in anyway, +# keeps them: the okb is not removed as the destination of its own copy. set -e TMP=$(mktemp -d) @@ -76,4 +78,11 @@ grep -q "the runner would be dropped" c2.log || fail "C: the refusal" c2.log grep -q "use: --pkg-format tar,okb" c2.log || fail "C: the refusal names no remedy" c2.log echo "ok: tar,okb writes both; okb alone does not drop a self-contained package's runner" +# D +"$MCPP" pack --target openkal --mode self-contained --systems all --pkg-format tar,okb -o target/dist/ \ + > d.log 2>&1 || fail "D: -o target/dist/" d.log +ls target/dist/lst-0.1.0-openkal-bundle-all.tar.gz > /dev/null 2>&1 || fail "D: no tar in target/dist/" d.log +ls target/dist/lst-0.1.0.okb > /dev/null 2>&1 || fail "D: no okb in target/dist/" d.log +echo "ok: -o target/dist/ keeps what is already written there" + echo "PASS: 985 a format list writes each container of one tree" From b2102cf6b2dbac04bf684769576f7c6707f1caa3 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 09:35:29 +0900 Subject: [PATCH 18/20] CI: six Linux e2e shards with the new tests' measured durations; e2e 981 runs over the emsdk the scan job installs, e2e 988 in the wine job; e2e 730 reports the shared home's state when the rung is not taken Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .github/workflows/ci-linux-e2e.yml | 12 +++++++----- .github/workflows/ci-target-matrix.yml | 8 ++++++++ .github/workflows/cross-build-test.yml | 12 ++++++++++++ ...compat_dependency_is_not_a_refresh_miss.sh | 19 +++++++++++++++++-- tests/e2e/timings/linux.tsv | 11 +++++++++++ 5 files changed, 55 insertions(+), 7 deletions(-) diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 6ce1284cd..8c14dbc46 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -1,8 +1,10 @@ name: ci-linux-e2e # The e2e suite (tests/e2e/run_all.sh) on Linux, called by ci.yml beside -# ci-linux.yml after the Linux build. Four shards, assigned by measured -# duration (tests/e2e/timings/linux.tsv, rule R4 of the 2026-10-02 CI record), +# ci-linux.yml after the Linux build. Six shards, assigned by measured +# duration (tests/e2e/timings/linux.tsv, rule R4 of the 2026-10-02 CI record): +# the table holds about 72 minutes of tests (2026-10-30, with the target-set +# and packaging tests of 2026.10.30.1), about 12 a shard, # each running the one binary build.yml produced (use-built-mcpp, rule R1). # Each shard writes a per-test report that the e2e-coverage job of ci.yml reads # (rule R5). @@ -15,15 +17,15 @@ on: jobs: e2e: - name: e2e ${{ matrix.shard }}/4 (linux x86_64, self-host) + name: e2e ${{ matrix.shard }}/6 (linux x86_64, self-host) runs-on: ubuntu-24.04 timeout-minutes: 45 strategy: fail-fast: false matrix: - shard: [1, 2, 3, 4] + shard: [1, 2, 3, 4, 5, 6] env: - E2E_SHARD: ${{ matrix.shard }}/4 + E2E_SHARD: ${{ matrix.shard }}/6 E2E_TIMINGS: tests/e2e/timings/linux.tsv E2E_REPORT: ${{ github.workspace }}/e2e-report-linux-${{ matrix.shard }}.tsv MCPP_HOME: /home/runner/.mcpp diff --git a/.github/workflows/ci-target-matrix.yml b/.github/workflows/ci-target-matrix.yml index 3fd792256..1a8bbd891 100644 --- a/.github/workflows/ci-target-matrix.yml +++ b/.github/workflows/ci-target-matrix.yml @@ -420,6 +420,14 @@ jobs: exit 1 fi echo "ok: wasm32-emscripten ran through the payload's runner; the decoy node was never run" + # #851 (e2e 981) needs the emsdk payload this step has just installed; + # the shards have none and skip it, so it runs here, in that home. + MCPP="$MCPP_UNDER_TEST" bash tests/e2e/981_a_web_link_has_the_profiles_level_and_the_wasm_exception_model.sh 2>&1 \ + | tee "$RUNNER_TEMP/981.log" + if grep -q '^SKIP' "$RUNNER_TEMP/981.log" || ! grep -qx 'PASS: 981' "$RUNNER_TEMP/981.log"; then + echo "::error::e2e 981 did not run to its PASS line over the installed emsdk" + exit 1 + fi coverage: # THE DENOMINATOR. Every check above is per host, and no per-host check diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index ea9f100c1..0d7e39d58 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -431,6 +431,18 @@ jobs: grep -qxF "$line" "$log" || { echo "::error::797 did not print: $line"; exit 1; } done + # The Windows launcher of an ok-image tree (`bin/.exe`, built over + # the platform's hosted arrangement) finds its package from a Windows + # path. The shards have no wine; this job does. + - name: "e2e: the Windows launcher finds its package from a Windows path (988)" + run: | + set -o pipefail + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + log="$RUNNER_TEMP/988.log" + bash tests/e2e/988_the_windows_launcher_finds_its_package_from_a_windows_path.sh 2>&1 | tee "$log" + grep -qxF "PASS: 988 the Windows launcher finds its package from a Windows path" "$log" \ + || { echo "::error::988 did not print its PASS line"; exit 1; } + # 257 needs a Linux-hosted MinGW compiler AND wine, which only this job # has; on the shards it skipped, and it had run on no runner (finding F9 # of the 2026-10-02 CI record). diff --git a/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh b/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh index 8d70040a6..920e647a4 100755 --- a/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh +++ b/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh @@ -44,8 +44,23 @@ echo 'int main() { return 0; }' > "$TMP/app/src/main.cpp" cd "$TMP/app" MCPP_OFFLINE=1 "$MCPP" emit build-database --format json -v > out.json 2> err.txt || true -grep -q "resolved to 'compat.$DESC' through the deprecated bare-name search" err.txt \ - || fail "A: the resolver did not take the bare-name rung for $DESC" err.txt +# What the shared home held when the rung was not taken: the test passes on a +# fresh home and failed only after other tests of its shard, so the state of +# the home is the evidence. +home_state() { + local h="${MCPP_HOME:-$HOME/.mcpp}" + echo "--- home: $h ---" + ls -la "$h/indices" 2>&1 + for d in "$h"/indices/*/; do + echo "index $d: $(git -C "$d" log -1 --format='%h %cI' 2>/dev/null || echo 'no git')" + ls "$d/pkgs/c" 2>/dev/null | grep -i "$DESC" | sed 's/^/ /' + done + echo "--- $h/config.toml ---"; cat "$h/config.toml" 2>&1 +} +if ! grep -q "resolved to 'compat.$DESC' through the deprecated bare-name search" err.txt; then + home_state + fail "A: the resolver did not take the bare-name rung for $DESC" err.txt +fi if grep -a "index: $DESC@$VER: offline mode" err.txt; then fail "B: the refresh decision asked for a network refresh of a dependency the resolver found" err.txt fi diff --git a/tests/e2e/timings/linux.tsv b/tests/e2e/timings/linux.tsv index 2b513fb91..1b881be97 100644 --- a/tests/e2e/timings/linux.tsv +++ b/tests/e2e/timings/linux.tsv @@ -589,5 +589,16 @@ 960_an_image_of_another_architecture_runs_through_its_execution_host.sh 150000 961_a_one_target_package_carries_the_image_in_a_program_of_its_architecture.sh 120000 962_a_webassembly_package_is_neither_an_xlings_entry_nor_a_single_file.sh 1000 +963_a_target_set_builds_every_member_and_runs_where_it_is_reached.sh 58687 +964_tests_built_once_run_from_an_archive.sh 89194 96_msvc_unavailable_nonwindows.sh 38 +980_an_ambiguous_dependency_name_is_no_packages.sh 2257 +981_a_web_link_has_the_profiles_level_and_the_wasm_exception_model.sh 5 +982_an_action_reads_the_builds_contract.sh 815 +983_a_pack_holds_the_legs_it_names.sh 155041 +984_a_set_of_programs_is_a_package_per_leg.sh 16305 +985_a_format_list_writes_each_container_of_one_tree.sh 240000 +986_single_files_cover_the_legs_and_systems.sh 240000 +987_the_universal_spellings_are_migrated.sh 107535 +988_the_windows_launcher_finds_its_package_from_a_windows_path.sh 5 98_reflection_import_std.sh 7170 From 32cb9fa4b605e1748b3281b2fcd5a630b04b73ee Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 11:05:07 +0900 Subject: [PATCH 19/20] a test archive is compressed as its name states, gzip passes zlib in pieces, and a program that cannot start is reported with the system's account - tar_algorithm(name), the inverse of tar_extension, is the one rule by which unpack, --merge and the test archive read a name; `mcpp test --archive` refuses a name that states no tar archive before the build (exit 2). The archive was written gzip whatever its name and read by its name (e2e 964 E). - zlib counts in 32 bits: input and output pass in 1 GiB pieces, positions kept outside zlib, so a .tar.gz over 4 GiB is one member (unit PackCompress). - errno_of classifies invalid, no memory, directory, not supported and too large; RunResult::spawn_detail carries os's context and native value into the reports of `mcpp test` and `mcpp test --archive`, which said only "error 5". Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 14 +++- docs/08-testing.md | 4 +- docs/specs/target-sets.md | 6 +- docs/zh/08-testing.md | 3 +- src/build/execute.cppm | 25 ++++--- src/build/runner_lookup.cppm | 9 ++- src/cli/cmd_build.cppm | 9 +++ src/pack/compress.cppm | 12 ++++ src/pack/compress_codecs.cpp | 66 ++++++++++++------- src/pack/distribution.cppm | 11 +--- src/run.cppm | 7 +- src/run/run.cpp | 38 ++++++++--- ...64_tests_built_once_run_from_an_archive.sh | 18 +++++ tests/unit/test_pack_distribution.cpp | 28 ++++++++ 14 files changed, 190 insertions(+), 60 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9aa988c4e..1b66899a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,9 @@ (e2e 963). `[build] target` takes one target. - **Tests built once, run elsewhere.** `mcpp test --no-run --archive ` writes every leg's tests with `mcpp-tests.json`; `mcpp test --archive ` - in a checkout runs what this machine reaches and builds nothing (e2e 964). + in a checkout runs what this machine reaches and builds nothing. The archive + is a tar compressed as its name states (`.tar.gz`, `.tgz`, `.tar.zst`, + `.tar.xz`, `.tar`); another name is refused before the build (e2e 964). - **The build's contract, `${mcpp.contract}`.** Every configuration writes `mcpp-contract.json` beside its compile database: the compiler of each language with the words every unit is compiled with (target, sysroot, @@ -99,6 +101,16 @@ domain may search but not read. openkal-linux 0.18.3 opens every directory handle with `O_PATH`; mcpp is built on mcpplibs.os 0.2.3, which pins it, and openkal-platform 0.18.0 carries it to every ok image and hosted program. +- **A gzip stream over 4 GiB.** zlib counts in 32 bits, and mcpp handed it the + whole tree at once: a `.tar.gz` of more than 4 GiB was refused as "an input + over 4 GiB", by `mcpp pack` and by a test archive alike. Input and output + now pass in pieces of 1 GiB, so such a tree is one gzip member as any other. +- **A program that cannot be started is reported with the system's account.** + Every condition beyond not found, permission and not a program was reported + as "error 5" (EIO), the system's own message dropped. The classification now + covers invalid, no memory, directory, not supported and too large, and the + report of `mcpp test` and `mcpp test --archive` carries what was being done + and the kernel layer's value. ## [2026.10.29.1] - 2026-10-29 diff --git a/docs/08-testing.md b/docs/08-testing.md index 57d07ca62..4e7f23ffa 100644 --- a/docs/08-testing.md +++ b/docs/08-testing.md @@ -213,7 +213,9 @@ each test's leg, arguments and working directory. Reading it builds nothing; the tests run in the checkout, through this machine's runner or system, and the report is `mcpp test`'s. A failed test exits 1, and so does an archive of which no test ran. Legs this machine does not reach are reported as not run -([SPEC-015 §4](specs/target-sets.md)). +([SPEC-015 §4](specs/target-sets.md)). The archive is a tar compressed as its +name states: `.tar.gz` (or `.tgz`), `.tar.zst`, `.tar.xz` or `.tar`; another +name is refused before anything is built. ## Tests that cannot run beside each other diff --git a/docs/specs/target-sets.md b/docs/specs/target-sets.md index 6b0f34880..d4fb5fef7 100644 --- a/docs/specs/target-sets.md +++ b/docs/specs/target-sets.md @@ -85,7 +85,8 @@ ## 4. 测试归档 - **写**:`mcpp test [--target ...] --no-run --archive ` 构建各腿的测试程序,与清单 `mcpp-tests.json`(每个测试的腿、 - 参数、工作目录)一起写为一个 tar 归档。归档是引擎的格式,与平台的容器无关。 + 参数、工作目录)一起写为一个 tar 归档。归档是引擎的格式,与平台的容器无关。压缩方式由文件名决定 + (`.tar.gz`/`.tgz`、`.tar.zst`、`.tar.xz`、`.tar`),读写同一规则;其他名称在构建前拒绝(退出码 2)。 - **读**:`mcpp test --archive ` 在同一项目的检出中运行归档内本机能到达的测试,不构建;工作目录为项目检出;报告与 `mcpp test` 相同。 - 读取时:有测试失败则退出码 1;一个测试都没有运行则退出码 1 并写明;本机到达不了的腿报告为"未运行"。 @@ -111,7 +112,8 @@ 6. 集合的识别、补全、展开、去重与顺序,`planned` 的行不在内,候选选到不同平台被拒绝(unit `TargetSet`)。 7. 腿的规划串行、构建一起、作业分摊与第一条失败的腿(unit `BuildLegs`)。 8. `--no-run --archive` 写出的归档含两条腿与清单;在没有 `target/` 的检出中 `--archive` 运行本机的腿、不构建;失败的 - 测试使退出码为 1;没有一条腿能到达的归档不运行任何测试且退出码为 1(e2e 964)。 + 测试使退出码为 1;没有一条腿能到达的归档不运行任何测试且退出码为 1;`.tar.zst` 写为 zstd 帧并同样运行,不表明 tar + 归档的名称在构建前以退出码 2 拒绝(e2e 964);文件名与压缩方式互逆(unit `PackCompress`)。 ## 变更记录 diff --git a/docs/zh/08-testing.md b/docs/zh/08-testing.md index e06a686ff..1207571e7 100644 --- a/docs/zh/08-testing.md +++ b/docs/zh/08-testing.md @@ -186,7 +186,8 @@ mcpp test --archive tests.tar.gz # 在检出中运 归档含各腿的测试程序与 `mcpp-tests.json`（每个测试的腿、参数与工作目录）。读取 时不构建；测试在检出中经本机的运行器或系统运行，报告与 `mcpp test` 相同。有测试 失败时退出码为 1，一个测试都没运行的归档也是 1。本机到达不了的腿报告为未运行 -（[SPEC-015 §4](../specs/target-sets.md)）。 +（[SPEC-015 §4](../specs/target-sets.md)）。归档是 tar，压缩方式由文件名决定：`.tar.gz`（或 +`.tgz`）、`.tar.zst`、`.tar.xz` 或 `.tar`；其他名称在构建前即被拒绝。 ## 不能彼此并排运行的测试 diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 7187a1e78..cb3241755 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -128,7 +128,8 @@ std::string test_dest(std::string_view triple, std::string_view test, std::strin std::string program_dest(std::string_view triple, std::string_view file); // Copies `copies` into a fresh tree under `stage`, writes the manifest beside -// them and the tree as `archive` (gzip-compressed tar). +// them and the tree as `archive`: a tar compressed as its name states +// (`.tar.gz`/`.tgz`, `.tar.zst`, `.tar.xz`, `.tar`); another name is refused. std::expected write(const Manifest& m, const std::vector& copies, const std::filesystem::path& stage, const std::filesystem::path& archive); @@ -215,8 +216,13 @@ std::expected write(const Manifest& m, const std::vector write(const Manifest& m, const std::vector open(const fs::path& archive, const fs::path& into) { @@ -4291,12 +4297,14 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, bool timedOut = false; int exitCode = 0; int spawnErr = 0; + std::string spawnDetail; std::string runOutput; if (capture) { auto rr = mcpp::run::capture_exec_deadline( r.argv, r.env, deadline, &timedOut, {}, &spawnErr); exitCode = rr.exit_code; runOutput = std::move(rr.output); + spawnDetail = std::move(rr.spawn_detail); } else { mcpp::ui::status("Running", std::format("bin/{}", r.name)); exitCode = mcpp::run::run_exec_deadline( @@ -4315,7 +4323,7 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, using namespace mcpp::build::runner_lookup; std::string reason; if (!runnerTmpl.empty()) - reason = spawn_failed_message(r.argv.front(), spawnErr); + reason = spawn_failed_message(r.argv.front(), spawnErr, spawnDetail); else if (classify(spawnErr) == SpawnClass::Unloadable) reason = std::format( "this host cannot execute {} artifacts: {} (error {}); " @@ -4324,7 +4332,7 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, runnerChoice.tripleKey, errno_text(spawnErr), spawnErr, runnerChoice.tripleKey); else - reason = spawn_failed_message(r.argv.front(), spawnErr); + reason = spawn_failed_message(r.argv.front(), spawnErr, spawnDetail); if (!hostCannotRun.exchange(true)) { hostCannotRunReason = reason; if (!json) mcpp::ui::warning(reason); @@ -4765,7 +4773,8 @@ export int run_test_archive(std::span passthrough, BuildOverr if (spawnErr != 0) { ++notRun; reasons.push_back(std::format("{}: {}", leg.triple, - mcpp::build::runner_lookup::spawn_failed_message(argv.front(), spawnErr))); + mcpp::build::runner_lookup::spawn_failed_message(argv.front(), spawnErr, + rr.spawn_detail))); } else if (ok) { ++passed; } else { diff --git a/src/build/runner_lookup.cppm b/src/build/runner_lookup.cppm index f0bb3ac39..6f12a56b6 100644 --- a/src/build/runner_lookup.cppm +++ b/src/build/runner_lookup.cppm @@ -144,9 +144,12 @@ inline std::string not_found_message(std::string_view triple, std::string_view a argv0, triple, dirs); } -inline std::string spawn_failed_message(std::string_view program, int e) { - return std::format("'{}' could not be started: {} (error {})", - program, errno_text(e), e); +// `detail` is the system's own account (`RunResult::spawn_detail`), which +// names what the classification `e` does not. +inline std::string spawn_failed_message(std::string_view program, int e, + std::string_view detail = {}) { + return std::format("'{}' could not be started: {} (error {}){}{}", + program, errno_text(e), e, detail.empty() ? "" : "; ", detail); } // The hosted sibling of mcpp::freestanding::no_runner_message. It reports what diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index be2f8a6f5..08b6d9a73 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -48,6 +48,7 @@ import mcpp.home.files; import mcpp.ui.terminal; import mcpp.project; import mcpp.manifest; +import mcpp.pack.compress; // the compression a test archive's name states import mcpp.toolchain.fingerprint; import mcpp.ui; import mcpp.wire; @@ -1226,6 +1227,14 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, return 2; } } + // The archive is a tar compressed as its name states, written and read so: + // a name that states none is refused before anything is built. + if (!to.archive.empty() + && !mcpp::pack::compress::tar_algorithm(to.archive.filename().string())) { + mcpp::ui::error(std::format("--archive {}: the name states no tar archive (.tar, .tar.gz, .tgz, " + ".tar.zst, .tar.xz)", to.archive.string())); + return 2; + } if (runArchive) { if (!testLegs.empty() || !ov.target_triple.empty()) { mcpp::ui::error("mcpp test --archive runs the legs the archive holds; --target " diff --git a/src/pack/compress.cppm b/src/pack/compress.cppm index 3e01ace24..cb03a9ec3 100644 --- a/src/pack/compress.cppm +++ b/src/pack/compress.cppm @@ -39,6 +39,11 @@ std::string_view name(Algorithm a); // `.tar`. std::string_view tar_extension(Algorithm a); +// The compression a tar archive's file name states, the inverse of +// `tar_extension` (`.tgz` is `.tar.gz`); none for a name that is no tar +// archive's. Writers and readers of an archive by name take it from here. +std::optional tar_algorithm(std::string_view filename); + // The whole of `in`, compressed to `out` under `c` (a gzip member, a zstd // frame, an xz stream; a copy for None). std::expected file(const std::filesystem::path& in, @@ -98,6 +103,13 @@ std::string_view tar_extension(Algorithm a) { return ".tar.gz"; } +std::optional tar_algorithm(std::string_view filename) { + if (filename.ends_with(".tgz")) return Algorithm::Gzip; + for (auto a : {Algorithm::Gzip, Algorithm::Zstd, Algorithm::Xz, Algorithm::None}) + if (filename.ends_with(tar_extension(a))) return a; + return std::nullopt; +} + std::expected parse(std::string_view text) { Choice c; c.stated = true; diff --git a/src/pack/compress_codecs.cpp b/src/pack/compress_codecs.cpp index 6de8bac42..cc9ec359c 100644 --- a/src/pack/compress_codecs.cpp +++ b/src/pack/compress_codecs.cpp @@ -14,6 +14,11 @@ namespace mcpp::pack::compress::codec { namespace { +// zlib's counters are `uInt` (32 bits) and `uLong` (32 bits on Windows): the +// input is handed over and the output taken in pieces of at most `kZlibPiece`, +// positions are kept here, so a tree over 4 GiB is one gzip member as any other. +constexpr std::size_t kZlibPiece = std::size_t{1} << 30; + // zlib's deflate with `windowBits`: 15 + 16 is a gzip member, -15 is raw // DEFLATE. A gzip member's header is stated rather than left to zlib, whose own // header names the system zlib was built for (3 on Unix, 10 or 11 on Windows): @@ -32,15 +37,28 @@ std::expected zlib_deflate(std::string_view in, int le return std::unexpected(std::string("zlib: deflateSetHeader failed")); } std::string out; - out.resize(deflateBound(&s, static_cast(in.size())) + 64); - s.next_in = reinterpret_cast(const_cast(in.data())); - s.avail_in = static_cast(in.size()); - s.next_out = reinterpret_cast(out.data()); - s.avail_out = static_cast(out.size()); - const int rc = ::deflate(&s, Z_FINISH); - const auto written = s.total_out; + out.resize(std::min(in.size(), kZlibPiece) + in.size() / 1000 + 1024); + std::size_t fed = 0, written = 0; + int rc = Z_OK; + while (rc != Z_STREAM_END) { + if (s.avail_in == 0 && fed < in.size()) { + const auto piece = std::min(in.size() - fed, kZlibPiece); + s.next_in = reinterpret_cast(const_cast(in.data() + fed)); + s.avail_in = static_cast(piece); + fed += piece; + } + if (written == out.size()) out.resize(out.size() + std::max(out.size() / 2, std::size_t{1024})); + const auto room = std::min(out.size() - written, kZlibPiece); + s.next_out = reinterpret_cast(out.data() + written); + s.avail_out = static_cast(room); + rc = ::deflate(&s, fed == in.size() ? Z_FINISH : Z_NO_FLUSH); + written += room - s.avail_out; + if (rc != Z_OK && rc != Z_STREAM_END && rc != Z_BUF_ERROR) { + deflateEnd(&s); + return std::unexpected(std::format("zlib: deflate returned {}", rc)); + } + } deflateEnd(&s); - if (rc != Z_STREAM_END) return std::unexpected(std::format("zlib: deflate returned {}", rc)); out.resize(written); return out; } @@ -48,14 +66,10 @@ std::expected zlib_deflate(std::string_view in, int le } // namespace std::expected gzip(std::string_view in, int level) { - if (in.size() > std::numeric_limits::max()) - return std::unexpected(std::string("gzip: an input over 4 GiB")); return zlib_deflate(in, level, 15 + 16); } std::expected deflate(std::string_view in, int level) { - if (in.size() > std::numeric_limits::max()) - return std::unexpected(std::string("deflate: an input over 4 GiB")); return zlib_deflate(in, level, -15); } @@ -104,11 +118,10 @@ namespace { // zlib's inflate until the stream ends: a gzip member (windowBits 15 + 32 // detects the header), its output growing, or raw DEFLATE (-15) of a stated -// `size`, its output never more than that. +// `size`, its output never more than that. Input and output pass in pieces +// (`kZlibPiece`), as for deflate. std::expected zlib_inflate(std::string_view in, int windowBits, std::optional size) { - if (in.size() > std::numeric_limits::max()) - return std::unexpected(std::string("zlib: an input over 4 GiB")); // DEFLATE expands at most 1032 times (a run of one byte): a stated size // beyond that is a lie, refused before anything is allocated, and a stream // that grows past it is refused as it grows. @@ -122,11 +135,18 @@ std::expected zlib_inflate(std::string_view in, int wi // One byte past a stated size: a stream that fills it holds more than it // said. out.resize(size ? *size + 1 : in.size() * 4 + 1024); - s.next_in = reinterpret_cast(const_cast(in.data())); - s.avail_in = static_cast(in.size()); + std::size_t fed = 0, written = 0; int rc = Z_OK; - while (rc == Z_OK) { - if (s.total_out >= out.size()) { + while (rc == Z_OK || rc == Z_BUF_ERROR) { + if (s.avail_in == 0 && fed < in.size()) { + const auto piece = std::min(in.size() - fed, kZlibPiece); + s.next_in = reinterpret_cast(const_cast(in.data() + fed)); + s.avail_in = static_cast(piece); + fed += piece; + } else if (rc == Z_BUF_ERROR && s.avail_in == 0 && written < out.size()) { + break; // the input ended before the stream did + } + if (written >= out.size()) { if (size) { inflateEnd(&s); return std::unexpected(std::format("zlib: the stream holds more than its {} bytes", @@ -138,12 +158,12 @@ std::expected zlib_inflate(std::string_view in, int wi } out.resize(std::min(out.size() * 2, most + 1)); } - s.next_out = reinterpret_cast(out.data() + s.total_out); - s.avail_out = static_cast(std::min(out.size() - s.total_out, - std::numeric_limits::max())); + const auto room = std::min(out.size() - written, kZlibPiece); + s.next_out = reinterpret_cast(out.data() + written); + s.avail_out = static_cast(room); rc = ::inflate(&s, Z_NO_FLUSH); + written += room - s.avail_out; } - const auto written = s.total_out; inflateEnd(&s); if (rc != Z_STREAM_END) return std::unexpected(std::format("zlib: inflate returned {}", rc)); out.resize(written); diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 50acb9762..c742cf4f4 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -901,11 +901,7 @@ std::expected unpack(const fs::path& package, const fs::p const auto name = package.filename().string(); using mcpp::pack::compress::Algorithm; const bool zip = name.ends_with(".zip"); - const std::optional a = name.ends_with(".tar.gz") || name.ends_with(".tgz") ? Algorithm::Gzip - : name.ends_with(".tar.zst") ? Algorithm::Zstd - : name.ends_with(".tar.xz") ? Algorithm::Xz - : name.ends_with(".tar") ? std::optional(Algorithm::None) - : std::nullopt; + const std::optional a = mcpp::pack::compress::tar_algorithm(name); if (!zip && !a) return std::unexpected(std::format( "'{}' is not a package this command writes (.zip, .tar, .tar.gz, .tar.zst, .tar.xz, " @@ -1942,10 +1938,7 @@ std::expected merge(const MergeRequest& req) { auto compression = req.compression; if (!compression.stated && format == "tar") { using mcpp::pack::compress::Algorithm; - compression.algorithm = firstName.ends_with(".tar.zst") ? Algorithm::Zstd - : firstName.ends_with(".tar.xz") ? Algorithm::Xz - : firstName.ends_with(".tar") ? Algorithm::None - : Algorithm::Gzip; + compression.algorithm = mcpp::pack::compress::tar_algorithm(firstName).value_or(Algorithm::Gzip); } // The compression fits the container and the name the output is given. { diff --git a/src/run.cppm b/src/run.cppm index b979f7ee6..64ec6e5a6 100644 --- a/src/run.cppm +++ b/src/run.cppm @@ -30,6 +30,7 @@ struct RunResult { std::string error_output; // Streams::output_error_kept: its standard error bool timed_out = false; // a bound or `should_stop` ended it int spawn_error = 0; // errno-style when it could not be started, else 0 + std::string spawn_detail; // then the system's own account, beside the classification }; // ─── This process ──────────────────────────────────────────────────────── @@ -71,8 +72,10 @@ void end_for_closed_output(); // // A SPAWN FAILURE IS REPORTED EXACTLY ONCE AND NEVER DROPPED (#544). When the // program could not be started a launcher returns 127 and either stores an -// errno-style code in `*spawn_error` (ENOENT, EACCES, ENOEXEC, EIO) and prints -// nothing, because a caller that asked owns the report, or, with +// errno-style code in `*spawn_error` (ENOENT, EACCES, ENOEXEC, EINVAL, ENOMEM, +// EISDIR, ENOTDIR, ENOTSUP, E2BIG, else EIO) -- the system's own account in a +// captured result's `spawn_detail` -- and prints nothing, because a caller that +// asked owns the report, or, with // `spawn_error == nullptr`, reports it itself -- on stderr for the launchers // that inherit it, into `output` for the capturing ones. `*spawn_error` is 0 // whenever the program was started, whatever it then did. diff --git a/src/run/run.cpp b/src/run/run.cpp index e34ecaa4c..9f603f882 100644 --- a/src/run/run.cpp +++ b/src/run/run.cpp @@ -113,21 +113,35 @@ struct Outcome { bool timed_out = false; int spawn_error = 0; // errno-style; 0 when started std::string spawn_message; + std::string spawn_detail; // the system's own account: what was done, and its code }; // The errno a caller classifies a refused start by (ENOENT, EACCES, ENOEXEC). int errno_of(os::errc c) { switch (c) { - case os::errc::not_found: return 2; - case os::errc::permission: return 13; - case os::errc::not_program: return 8; - default: return 5; + case os::errc::not_found: return static_cast(std::errc::no_such_file_or_directory); + case os::errc::permission: return static_cast(std::errc::permission_denied); + case os::errc::not_program: return static_cast(std::errc::executable_format_error); + case os::errc::invalid: return static_cast(std::errc::invalid_argument); + case os::errc::no_memory: return static_cast(std::errc::not_enough_memory); + case os::errc::is_directory: return static_cast(std::errc::is_a_directory); + case os::errc::not_directory: return static_cast(std::errc::not_a_directory); + case os::errc::not_supported: return static_cast(std::errc::not_supported); + case os::errc::too_large: return static_cast(std::errc::argument_list_too_long); + default: return static_cast(std::errc::io_error); } } -std::string spawn_failure(std::string_view program, int error) { - return std::format("cannot start '{}' (error {}): {}\n", program, error, - std::generic_category().message(error)); +// What the system said, beside the classification: the operation and the +// object (`context`) and the kernel layer's own value, which a classification +// of "error 5" (EIO) would otherwise hide. +std::string detail_of(const os::error& e) { + return e.native != 0 ? std::format("{} (native {})", e.message(), e.native) : e.message(); +} + +std::string spawn_failure(std::string_view program, int error, std::string_view detail = {}) { + return std::format("cannot start '{}' (error {}): {}{}{}\n", program, error, + std::generic_category().message(error), detail.empty() ? "" : "; ", detail); } // The environment a started program receives: this invocation's statements, @@ -217,7 +231,8 @@ Outcome launch(const Spec& s) { auto p = Process::spawn(o); if (!p) { out.spawn_error = errno_of(p.error().code); - out.spawn_message = spawn_failure(s.argv.front(), out.spawn_error); + out.spawn_detail = detail_of(p.error()); + out.spawn_message = spawn_failure(s.argv.front(), out.spawn_error, out.spawn_detail); return out; } Registered guard(&*p); @@ -304,6 +319,7 @@ RunResult result_of(Outcome&& o, int* spawn_error, bool messageIntoOutput) { r.output = std::move(o.output); if (o.spawn_error != 0) { r.exit_code = 127; + r.spawn_detail = std::move(o.spawn_detail); if (spawn_error) *spawn_error = o.spawn_error; else if (messageIntoOutput) r.output = o.spawn_message; } @@ -397,6 +413,7 @@ int run_foreground(const std::vector& argv, const Env& extraEnv, in if (argv.empty()) return 127; const auto program = program_of(argv.front()); int error = 2; + std::string detail; if (program) { os::process::spawn_options o; o.program = *program; @@ -407,10 +424,11 @@ int run_foreground(const std::vector& argv, const Env& extraEnv, in std::fflush(stdout); std::fflush(stderr); auto r = os::process::exec(o); // returns only when the program could not be started - error = r ? 5 : errno_of(r.error().code); + error = r ? static_cast(std::errc::io_error) : errno_of(r.error().code); + if (!r) detail = detail_of(r.error()); } if (spawn_error) *spawn_error = error; - else std::fputs(spawn_failure(argv.front(), error).c_str(), stderr); + else std::fputs(spawn_failure(argv.front(), error, detail).c_str(), stderr); return 127; } diff --git a/tests/e2e/964_tests_built_once_run_from_an_archive.sh b/tests/e2e/964_tests_built_once_run_from_an_archive.sh index 59febe17e..f61b31679 100755 --- a/tests/e2e/964_tests_built_once_run_from_an_archive.sh +++ b/tests/e2e/964_tests_built_once_run_from_an_archive.sh @@ -20,6 +20,10 @@ # C. A test that fails in the archive fails the run (exit 1). # D. An archive none of whose legs this machine reaches runs nothing and # exits 1, saying so. +# E. The archive is compressed as its name states (`.tar.zst` is a zstd +# frame and runs as `.tar.gz` does); a name that states no tar archive is +# refused before anything is built (exit 2). 2026.10.30.1's first build +# wrote gzip whatever the name, and read by the name. set -e TMP=$(mktemp -d) @@ -89,4 +93,18 @@ rc=0; "$MCPP" test --archive "$TMP/d.tar.gz" > d2.log 2>&1 || rc=$? grep -q "no test of .* ran on this machine" d2.log || fail "D: the failure does not say so" d2.log echo "ok: an archive run that runs nothing fails" +# E +cd "$TMP/app" +"$MCPP" test --target "$HOST_ARCH-openkal" --no-run --archive "$TMP/e.tar.zst" > e1.log 2>&1 \ + || fail "E: --archive e.tar.zst" e1.log +[ "$(head -c4 "$TMP/e.tar.zst" | od -An -tx1 | tr -d ' \n')" = "28b52ffd" ] || fail "E: e.tar.zst is no zstd frame" e1.log +cd "$TMP/co" +"$MCPP" test --archive "$TMP/e.tar.zst" > e2.log 2>&1 || fail "E: mcpp test --archive e.tar.zst" e2.log +cd "$TMP/app" && rm -rf target +rc=0; "$MCPP" test --no-run --archive "$TMP/e.zip" > e3.log 2>&1 || rc=$? +[ "$rc" = 2 ] || fail "E: --archive e.zip exited $rc, not 2" e3.log +grep -q "states no tar archive" e3.log || fail "E: the refusal does not say why" e3.log +[ ! -d target ] || fail "E: something was built before the name was refused" e3.log +echo "ok: the archive is compressed as its name states" + echo "PASS: 964 tests built once run from an archive" diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index 6b2d1fa6f..fea136ebd 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -116,6 +116,34 @@ TEST(PackCompress, EveryCodecRoundTrips) { EXPECT_EQ(*back, text); } +// The name of a tar archive states its compression, the inverse of +// tar_extension; a test archive is written and read by it. +TEST(PackCompress, ATarNameStatesItsCompression) { + for (auto a : { c::Algorithm::Gzip, c::Algorithm::Zstd, c::Algorithm::Xz, c::Algorithm::None }) + EXPECT_EQ(c::tar_algorithm(std::format("t{}", c::tar_extension(a))), a) << c::name(a); + EXPECT_EQ(c::tar_algorithm("t.tgz"), c::Algorithm::Gzip); + EXPECT_FALSE(c::tar_algorithm("t.zip")); + EXPECT_FALSE(c::tar_algorithm("t.gz")); + EXPECT_FALSE(c::tar_algorithm("tests")); +} + +// zlib counts in 32 bits; the input is handed over in pieces, so one gzip +// member holds more than one piece (1 GiB) and comes back whole. A member cut +// short is refused, not waited on. +TEST(PackCompress, AGzipMemberLargerThanOnePieceRoundTrips) { + std::string big((std::size_t{1} << 30) + 4096, '\0'); + big.replace(big.size() - 7, 7, "the end"); + auto packed = c::bytes(big, *c::parse("gzip:1")); + ASSERT_TRUE(packed) << packed.error(); + auto back = c::decompress(*packed, c::Algorithm::Gzip); + ASSERT_TRUE(back) << back.error(); + EXPECT_EQ(back->size(), big.size()); + EXPECT_TRUE(back->ends_with("the end")); + big = {}; + back = {}; + EXPECT_FALSE(c::decompress(std::string_view(*packed).substr(0, packed->size() / 2), c::Algorithm::Gzip)); +} + TEST(PackDistribution, Sha256OfKnownVectors) { EXPECT_EQ(d::sha256(""), "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"); EXPECT_EQ(d::sha256("abc"), "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"); From de6e3218abe69eb2c55a48b7ced5b741a6701348 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Fri, 30 Oct 2026 11:56:13 +0900 Subject: [PATCH 20/20] a macOS leg's tests start from an archive: the loader variable is stated only where the system has one (e2e 989) runtime_library_path_key() is empty on macOS, whose programs keep their rpaths; `mcpp test --archive` stated it regardless, and openkal-macos refuses an environment entry of an empty name, so every test program of an aarch64-macos leg was "invalid argument" -- reported as "error 5" before the spawn detail. e2e 989 archives the host leg's tests and runs them from a checkout on every system, with no llvm; 964's openkal legs run on Linux only. Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 5 +- docs/specs/target-sets.md | 3 +- src/build/execute.cppm | 10 +-- ...a_native_legs_tests_run_from_an_archive.sh | 65 +++++++++++++++++++ 4 files changed, 77 insertions(+), 6 deletions(-) create mode 100755 tests/e2e/989_a_native_legs_tests_run_from_an_archive.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 1b66899a0..69dec8814 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,10 @@ writes every leg's tests with `mcpp-tests.json`; `mcpp test --archive ` in a checkout runs what this machine reaches and builds nothing. The archive is a tar compressed as its name states (`.tar.gz`, `.tgz`, `.tar.zst`, - `.tar.xz`, `.tar`); another name is refused before the build (e2e 964). + `.tar.xz`, `.tar`); another name is refused before the build (e2e 964). A + leg of the host's own system runs from an archive as an openkal leg does + (e2e 989, on every system); on macOS the first build of this release started + none of them, giving each an environment entry of an empty name. - **The build's contract, `${mcpp.contract}`.** Every configuration writes `mcpp-contract.json` beside its compile database: the compiler of each language with the words every unit is compiled with (target, sysroot, diff --git a/docs/specs/target-sets.md b/docs/specs/target-sets.md index d4fb5fef7..eb1b31e78 100644 --- a/docs/specs/target-sets.md +++ b/docs/specs/target-sets.md @@ -113,7 +113,8 @@ 7. 腿的规划串行、构建一起、作业分摊与第一条失败的腿(unit `BuildLegs`)。 8. `--no-run --archive` 写出的归档含两条腿与清单;在没有 `target/` 的检出中 `--archive` 运行本机的腿、不构建;失败的 测试使退出码为 1;没有一条腿能到达的归档不运行任何测试且退出码为 1;`.tar.zst` 写为 zstd 帧并同样运行,不表明 tar - 归档的名称在构建前以退出码 2 拒绝(e2e 964);文件名与压缩方式互逆(unit `PackCompress`)。 + 归档的名称在构建前以退出码 2 拒绝(e2e 964);本机系统的腿(由系统直接启动的测试程序)同样从归档运行,不需要 + llvm(e2e 989);文件名与压缩方式互逆(unit `PackCompress`)。 ## 变更记录 diff --git a/src/build/execute.cppm b/src/build/execute.cppm index cb3241755..cb6b1bc0c 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -4749,11 +4749,13 @@ export int run_test_archive(std::span passthrough, BuildOverr argv.insert(argv.begin(), tb.runUnderPrefix.begin(), tb.runUnderPrefix.end()); auto env = legEnv; // The files beside the program are its own: the loader searches its - // directory, and a runner reads the list it is handed. + // directory, and a runner reads the list it is handed. macOS names + // no such variable (its programs keep their rpaths); an entry of an + // empty name is refused by the system, and every program of a macOS + // leg was "invalid argument" (e2e 989). std::array dir{exe.parent_path()}; - env.emplace_back(mcpp::runtime::launch_env::runtime_library_path_key(), - mcpp::runtime::launch_env::prepend_path_list( - mcpp::runtime::launch_env::runtime_library_path_key(), dir)); + if (const auto key = mcpp::runtime::launch_env::runtime_library_path_key(); !key.empty()) + env.emplace_back(key, mcpp::runtime::launch_env::prepend_path_list(key, dir)); { const auto list = exe.parent_path() / ".mcpp-runtime-files.tsv"; std::ofstream os(list, std::ios::binary | std::ios::trunc); diff --git a/tests/e2e/989_a_native_legs_tests_run_from_an_archive.sh b/tests/e2e/989_a_native_legs_tests_run_from_an_archive.sh new file mode 100755 index 000000000..de88a089a --- /dev/null +++ b/tests/e2e/989_a_native_legs_tests_run_from_an_archive.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +# 989 -- a native leg's tests run from an archive (SPEC-015 §4; design +# 2026-10-29 rev 2 §3.4). +# +# e2e 964 archives the legs of the openkal set, which run through the +# platform's runner and need llvm; a leg of the host's own system, whose test +# programs the system starts directly, was archived and run on no runner. On +# macOS such a program, unpacked from an archive written on Linux, was not +# started ("invalid argument", mcppls CI 2026-10-30). This test needs nothing +# but the host's toolchain, so every system's shards run it. +# +# Criteria: +# A. `mcpp test --no-run --archive t.tar.gz` writes the host leg's tests. +# B. In a copy of the project without target/, `mcpp test --archive` starts +# them, passes, and builds nothing. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +# What the system says about an archived program it would not start. +evidence() { + local program + program=$(find "$TMP/co/target/.test-archive" -type f -name 'smoke*' ! -name '*.tsv' 2>/dev/null | head -1) + [ -n "$program" ] || { echo "--- no unpacked program ---"; return; } + echo "--- $program ---" + file "$program" 2>&1 || true + ls -l "$program" 2>&1 || true + if [ "$(uname -s)" = Darwin ]; then + ls -l@ "$program" 2>&1 || true + xattr -l "$program" 2>&1 || true + codesign -dvvv "$program" 2>&1 | head -20 || true + fi + echo "--- started directly ---" + "$program"; echo "exit $?" + cp "$program" "$TMP/copy" && "$TMP/copy"; echo "a copy: exit $?" +} + +mkdir -p "$TMP/app/src" "$TMP/app/tests" +cd "$TMP/app" +printf '[package]\nname = "app"\nversion = "0.1.0"\n' > mcpp.toml +printf 'import std;\nint main() { std::println("app"); }\n' > src/main.cpp +cat > tests/smoke.cpp <<'CPP' +import std; +int main() { std::vector v{2, 1}; std::ranges::sort(v); return v.front() == 1 ? 0 : 1; } +CPP + +# A +"$MCPP" test --no-run --archive "$TMP/t.tar.gz" > a.log 2>&1 || fail "A: --no-run --archive" a.log +[ -s "$TMP/t.tar.gz" ] || fail "A: no archive written" a.log +echo "ok: the host leg's tests are archived" + +# B: a checkout with nothing built. +mkdir -p "$TMP/co" && cp -R "$TMP/app/src" "$TMP/app/tests" "$TMP/app/mcpp.toml" "$TMP/co/" +cd "$TMP/co" +if ! "$MCPP" test --archive "$TMP/t.tar.gz" > b.log 2>&1 || ! grep -q "smoke ... ok" b.log; then + evidence > ev.log 2>&1 + fail "B: the host leg's archived test did not pass" b.log ev.log +fi +[ -z "$(find target -path '*/.test-archive' -prune -o -name 'smoke*' -print 2>/dev/null)" ] \ + || fail "B: the archive's tests were built again" b.log +echo "ok: the host leg's tests run from the archive" + +echo "PASS: 989 a native leg's tests run from an archive"