Repository navigation
[WIP] Fix runner exit code shell - #329
Closed
shahmishal wants to merge 23 commits into
Closed
shahmishal wants to merge 23 commits into
shahmishal wants to merge 23 commits into
Conversation
Bring together the best aspects of the swiftlang GitHub Actions workflows in this repository and the SwiftNIO ones at apple/swift-nio, through a new reusable workflow that sits beside the existing one and a hierarchy of supporting infrastructure. Key design features: 1) GitHub-release-based versioning 2) No skipped jobs 3) Scripts are cloned not curled 4) Custom matrix builds 5) Minimum Swift version detection `package_test.yml` runs a range of common test and build configurations on a variety of platforms against multiple Swift versions. It sits atop a matrix generation layer which takes inputs and produces a canonical work definition in YAML or JSON; that definition is then expanded into one job per entry, each executing one slice of the work. Benchmarking is a further consumer of the same layer rather than part of the recommended test workflow. The design is layered, so an adopter may use the whole stack or customize portions beyond the level the workflow allows: * Use `package_test.yml` for the full suite of conveniences. * Use a custom workflow which calls `toolchain_matrix.yml` for the matrix and `execute_matrix.yml` to run its own command across it. * Generate the work definition by some other means, or hard-code it into a workflow, and pass it to `execute_matrix.yml`. The matrix generation layer is a Swift script, `generate-matrix.swift`, run by the matrix job on `ubuntu-latest`, which comes with the Swift, jq and yq it needs. It reads the workflow's inputs from the environment, applies the defaults and the minimum version filter, and writes the work definition to standard output. In SwiftNIO it was a bash script, which executes faster, but the unified version is much more complex, so Swift seemed like the more maintainable option. A test package under `tests/MatrixGeneratorValidator` covers the inputs, their defaults and failure modes. Execution times are reduced by defaulting to running natively on the runner rather than inside a container. The toolchain is installed with swiftly, which was previously used only for macOS toolchains, so using it on Linux is new to both upstreams. Containers remain available for a package that needs a particular distribution or system dependencies, through `linux_use_docker` or by naming a distribution in `linux_os`. The existing workflows are untouched, so a repository migrates when it suits it. The most visible change on migrating is that the `*_exclude_swift_versions` inputs are gone, replaced by explicit version lists and minimum-version detection which drops anything the package manifest cannot support. A repository that was only excluding old versions can delete those inputs entirely. Migrating from the SwiftNIO workflows is a change in spelling rather than a reduction in flexibility: per-version inputs become one map, which removes the churn of adding an input per Swift release. Functionality new to this repository, some inspired by SwiftNIO and some new to both: * Simulator testing. iOS was build-only; xcodebuild now builds and tests on simulators, with tvOS, watchOS and visionOS alongside it. Mac Catalyst is new to both upstreams. * Benchmarks across the matrix. `benchmarks.yml` checks committed thresholds under `Thresholds/<swift-version>/` on every platform and Swift version, failing the job and printing a diff when one moves. This differs from the existing `performance_test.yml`, which compares a pull request against its merge base in one container and posts the table as a comment: that reports what a change did to performance, this holds a package to a standard it has written down. * A C++ interoperability check. A release-configuration build needs no dedicated input: it is a second labeled command, so `Release-Build Linux Swift 6.3` runs alongside `Debug-Test Linux Swift 6.3` from one call. * A semantic version pull request label check, `pull_request_label.yml`. * An importable Swift version matrix, `toolchain_matrix.yml`, which gives a custom workflow the current supported matrix as plain YAML, so it can be filtered or extended with yq before it is executed. * Per-version overrides, so one Swift version can take extra arguments or a different command without splitting the matrix by hand. Known limitation: a pinned tag still does not pin the scripts. This is inherited behavior. `swift_package_test.yml` and `soundness.yml` check this repository out with no `ref:`, so they resolve to its default branch, and one variant hardcodes `ref: main`. A caller on `@0.0.14` has therefore always run that tag's YAML against the default branch's scripts, and the unified workflows inherit that default. Resolving it is possible but would require stamping versions as part of the release process.
The workflows' version-list inputs now default to empty, which the generator reads as its own list, so a new release is added in one place.
* Add a reference for the matrix YAML to architecture.md * Rename Aims to Goals and drop the progenitor paragraph * Explain why only macOS checks out linked PRs in execute_matrix.yml
An SDK build takes its own `*_sdk_version_overrides`, and falls back to `linux_version_overrides` when it has none.
Jobs run their setup command from the repository root, then enter `package_path` for the command, and minimum Swift version detection reads that directory's manifest.
Forks do not run Actions until the owner enables them, so the input guarded only the case of a fork that had. Its generator check and the inputs that forwarded it are gone.
The singular input duplicated the plural, which covers one path with a one-element list and defaults to the current directory when empty.
The generator emits `timeout_minutes` on Windows and FreeBSD entries from `windows_job_timeout` and `freebsd_job_timeout`, and `execute_matrix.yml` reads it from the entry, falling back to `job_timeout`. The executor no longer has a timeout input per platform, and a hand-written matrix can set one per entry.
Job names are required status checks, and a new Swift release renamed them. The default version window also lived in the generator, which a pinned workflow version does not pin, so a release moved the versions under every adopter. `current`, `current-1` and `current-2` are now version labels, resolved by `SwiftReleases`, and a job is named for the label the caller wrote. The releases they count back from come from new `swift_current_release` and `swift_nightly_release` inputs whose defaults live in each workflow, so the generator holds none of its own and a pinned workflow keeps the versions it was released with. The `macos_swiftly_toolchains` default moved back into the workflow for the same reason, and a test checks that the three workflows declare the same releases.
`SwiftReleases` parsed a label from its string shape and re-derived facts `SwiftVersion` already held, which needed `components` widened to `fileprivate`. `VersionLabel` is now the only place that rejects a malformed relative label, `SwiftVersion` exposes `major`, `minor` and `init(majorMinor:)`, and `SwiftReleases.ordered` resolves and orders a label in one call, so `components` is private again. The releases-agree test now compares the other workflows against the first.
An override that applies to every version had to be written out once per version, so a change of release meant editing a block of identical lines. A `default` key in a `*_version_overrides` input now sets the arguments for every version whose own key does not name some, and a version key replaces it for the fields it names. Arguments written alone, with no key, are the same as `default`. A command under `default` is an error, since it would give every command label the same one.
An unreadable directory was read as having no `Package.swift`, which raises no floor, so minimum-version detection could let a too-old Swift version through. `MinimumVersion.detected` now stops with the directory and the error, as reading a manifest already does. The test support's cleanup and scalar decoding no longer swallow errors silently: the cleanup prints a warning, and the decode says why a failed shape is not an error.
A version label, the version it resolves to, and the release a relative label counts back from were all `version`, `current` or `releaseToken`, and the test runner shared the generator's type name. `Combination.version` is `versionLabel`, as are the parameters of the override lookups, `Variant.swiftVersions` is `versionLabels`, `SwiftReleases.version(for:)` is `resolvedVersion(for:)`, and `SwiftReleases.current` is `currentRelease`. `Toolchain` and `Configuration` both name the nightly branch `nightlyBranch`, and the script's global is `configuration`. In the test support, `Generator` is `GeneratorRunner`, `Generated` is `GeneratorRun`, and the entry's FreeBSD configuration is `FreeBSDBuild`. The JSON keys are unchanged.
Whether a name is a nightly was decided by a string prefix at five sites, the Android build was three optional fields that had to be set together, and an unset release was an empty string checked where it was used. `VersionLabel.isNightly` is the one definition of a nightly. `SwiftBuildJobs.android` holds the triples, NDK releases and emulator flag of the Android build as one value, so a half-configured one cannot be represented. `SwiftReleases` holds the releases as optionals, with the empty input turned into `nil` where it is read, and the messages for an unset release are unchanged.
Wrapped two long messages, reflowed two comments, put a blank line after multi-line blocks and between the cases of switches with a long case, replaced guard-not with if, and dropped redundant comparisons on Bool.
The checks every swiftlang package is expected to pass (macOS, macOS with swiftly, the Static Linux, Wasm, Embedded Wasm and Android SDK builds, the Android emulator tests, and the Cxx interoperability check) were opt-in, so a package adopting `package_test.yml` ran none of them unless it asked. `package_test.yml` and the generator now enable each of them by default, and a package opts out with `enable_x: false`. FreeBSD stays opt-in. `benchmarks.yml` and `toolchain_matrix.yml` set the groups they do not use to false so their matrices are unchanged, and the self-tests that were about Linux and Windows name what they leave out. The test runner disables every job group before applying a test's own environment.
A step's shell: key does not accept expressions, so the workflow file was rejected before any job ran. The job-level default does accept matrix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.