Skip to content

fix(evaluations): send criterion errors as errorMessage and skip criteria for rows that did not generate - #228

Merged
aknight-ld merged 1 commit into
mainfrom
fix/evaluations-criterion-events
Oct 28, 2026
Merged

fix(evaluations): send criterion errors as errorMessage and skip criteria for rows that did not generate#228
aknight-ld merged 1 commit into
mainfrom
fix/evaluations-criterion-events

Conversation

@aknight-ld

@aknight-ld aknight-ld commented Oct 27, 2026 •
edited by cursor Bot
Loading

Copy link
Copy Markdown
Contributor

What

Criterion ERROR events now report failures the way ingest reads them, and rows whose generation failed are no longer scored.

Behavior change

  • Error fields. A criterion ERROR event carries a top-level errorMessage, which falls back to "Unknown error".
    • The nested error: {code, message} object is no longer sent. Its code was a string cause name that ingest ignores.
    • errorCode stays omitted until integer codes are assigned.
    • Cause names remain internal, and still appear as error.type on the evaluate span.
  • Empty messages. An error with an empty message now sends "Unknown error". Before, it sent the repr of the internal error mapping.
  • Rows that did not generate. A row whose generation did not complete gets no judge call, no scorer call and no criterion event. Its generation ERROR event already records the row as failed, so a criterion event for it was never accounted for. These rows no longer take a concurrency slot. The generation_incomplete cause is removed.

The events module isn't part of the public API report, so there is no API change.

Tests

  • Error cases: judge and scorer errors (invalid_judge_output, scorer_raised, invalid_score) assert errorMessage and the absence of error and errorCode. They tell causes apart through the internal results.
  • Fallback: a unit test covers the "Unknown error" fallback.
  • Failed generation: a two-row run, where one row raises during generation, asserts the following:
    • judges and scorers run only for the generated row;
    • no criterion event is tracked for the failed row;
    • the run still flushes and polls.
  • Mutation check: all 8 new or changed cases fail against the previous source.

make lint format-check typecheck api-report-check and uv run pytest pass: 3636 passed, 18 skipped.

🤖 Generated with Claude Code

Note

Overview
Criterion ERROR events now match what ingest expects: only a top-level errorMessage is sent (with "Unknown error" when the message is empty). The nested error object is dropped from CriterionEventPayload and tracking payloads; string cause codes stay on internal results and error.type on evaluate spans.

Failed generation rows are no longer scored. _run_criteria_for_results runs judges and scorers only for rows with status == "COMPLETE", so errored rows get no criterion events and no concurrency slots—the generation ERROR event alone accounts for the row. The generation_incomplete criterion path is removed from scorer/judge helpers and docs.

Tests assert wire shape (no error / errorCode), cover scorer/judge failures via internal results, and replace the old “generation incomplete criterion event” case with a two-row test that proves only the successful row is judged/scored.

Reviewed by Cursor Bugbot for commit cc12093. Bugbot is set up for automated code reviews on this repo. Configure here.

@aknight-ld @claude
fix(evaluations): send criterion errors as errorMessage and skip crit…
cc12093 
…eria for rows that did not generate

A criterion ERROR event now carries only a top-level errorMessage, with
"Unknown error" as the fallback. The nested error object, whose code was a
string cause name ingest never reads, is no longer sent, and errorCode stays
omitted until integer codes are assigned. The cause names remain on the
internal result, where the evaluate span still reports them as error.type.
An error whose message is empty now falls back to "Unknown error" instead of
sending the repr of the internal error mapping.

A row whose generation did not complete is no longer scored: no judge or
scorer runs for it, it takes no concurrency slot, and no criterion event is
tracked for it. Its generation ERROR event already settles the row as
failed, so a criterion event for it would be unaccounted for. The
generation_incomplete cause is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aknight-ld aknight-ld added the api-compatible An owner judged a flagged public API change compatible; passes the API compatibility check label Oct 27, 2026
@aknight-ld
aknight-ld marked this pull request as ready for review October 27, 2026 20:07
@donei003 donei003 mentioned this pull request Oct 28, 2026
Merged
3 of 4 tasks
@aknight-ld
aknight-ld merged commit 8b05b56 into main Oct 28, 2026
11 of 12 checks passed
@aknight-ld
aknight-ld deleted the fix/evaluations-criterion-events branch October 28, 2026 17:15
@github-actions github-actions Bot mentioned this pull request Oct 28, 2026
Merged
donei003 added a commit that referenced this pull request Oct 29, 2026
@donei003 @claude
feat(evaluations)!: promote Evaluations from Code to core (#235)
0bc43a5 
Promotes Evaluations from Code from
`launchdarkly_ai_server.experimental.evaluations` to the package root.
This follows the Agent Skills promotion in #210.

> **Draft.** This PR releases together with launchdarkly/js-ai-sdk#181.
A Jira ticket is not linked yet.

## Migration

Import Evaluations from Code names from `launchdarkly_ai_server` instead
of `launchdarkly_ai_server.experimental.evaluations`. The names,
signatures and behavior do not change.

```python
# before
from launchdarkly_ai_server.experimental.evaluations import EvalTool, init_evaluations
# after
from launchdarkly_ai_server import EvalTool, init_evaluations
```

This is a breaking change (`!`). It bumps the minor version while the
package is 0.x.

## What changed

- **Root exports.** The 14 names that `experimental/evaluations.py`
exported are now in `launchdarkly_ai_server.__all__`, with the same
spelling. No name collides with a current root export.
`launchdarkly-ai-python` gets the names through its existing `import *`.
- **Experimental module removed.** `experimental/evaluations.py` is
deleted. `experimental/__init__.py` stays as a docstring only, so a
future experimental feature has a place to go.
- **Usage ID.** `init_evaluations` now reports `client.initEvaluations`,
not `experimental.initEvaluations`. Queries that count the old ID must
also count the new one.
- **API reports.** Regenerated with `make api-report`. The names are in
`launchdarkly_ai_server.txt`, and
`launchdarkly_ai_server.experimental.evaluations.txt` is deleted.
- **Changelog.** The three `evaluations` rows are removed from
`changelog-sections`, so `feat(evaluations):` lands under Features. The
`AGENTS.md` scope examples now use a placeholder scope.
- **Docs.** The client README, the `launchdarkly-ai-python` README,
`packages/client/agents.md` and `AGENTS.md` no longer call Evaluations
from Code experimental. The README heading anchor changed from
`#run-an-evaluation-from-code-experimental` to
`#run-an-evaluation-from-code`.

## Tests

- `TestPackageExports` checks that every evaluations name is on the root
and that `experimental.evaluations` no longer imports.
- `test_public_surface.py` adds the 14 names to the root snapshot.
- `test_reexports.py` checks that the barrel re-exports the names.
- `test_core_does_not_import_experimental_code` now checks only
`launchdarkly_ai_server.experimental`, because the root loads
`evaluations`. A new test checks that `lifecycle` imports no evaluations
module.
- `test_sdk_usage.py` expects `client.initEvaluations`.

## Import time

`import launchdarkly_ai_server` now loads the `evaluations` package.
Measured with `python -X importtime`, it adds about 33 ms to a root
import of about 130 ms. The largest part is `evaluations.api`.

## Checks

- [x] `make lint`, `make format-check`, `make typecheck`, `make
api-report-check`: pass
- [x] `python -m scripts.api_breaking pr --base origin/main`: no
breaking changes to the public API
- [x] `make test`: 3640 passed, 18 skipped
- [ ] Run one evaluation end to end against staging with the root import

## Conflicts

The open evaluations PRs #228, #227, #143, #135 and #92 edit the same
test files. Merge them first, or rebase them after this PR.

## Related PRs

Releases together with:
- launchdarkly/js-ai-sdk#181

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> **Evaluations from Code** moves from
`launchdarkly_ai_server.experimental.evaluations` to the
**`launchdarkly_ai_server` package root** (and through
**`launchdarkly-ai-python`** re-exports). The experimental shim is
**removed**; callers should import `init_evaluations`, `EvalTool`,
`Judge`, `Scorer`, and the rest from the root instead.
> 
> The **public API contract** shifts accordingly: generated
**`launchdarkly_ai_server.txt`** now lists the evaluations surface, and
**`launchdarkly_ai_server.experimental.evaluations.txt`** is deleted.
**`init_evaluations`** usage telemetry changes from
**`experimental.initEvaluations`** to **`client.initEvaluations`**.
> 
> **Release/changelog policy** is updated: **`evaluations` is no longer
an Experimental scope** in `release-please-config.json`, and
**`AGENTS.md`** / READMEs describe Evaluations (and Skills) as core,
with docs and examples using root imports and a non-experimental README
anchor.
> 
> Tests and snapshots (**`test_public_surface`**,
**`TestPackageExports`**, **`test_reexports`**, lifecycle import guards)
enforce the new export layout and that **`experimental.evaluations`**
cannot be imported.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
3ee3612. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
jeffdupont pushed a commit that referenced this pull request Oct 30, 2026
@github-actions
chore: release main (#231)
7aacbba 
🤖 I have created a release *beep* *boop*
---


<details><summary>launchdarkly-ai-server: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-server-0.5.0...launchdarkly-ai-server-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **client:** init_client raises when the LaunchDarkly client did not
initialize
([#254](#254))
* **evaluations:** route generation and judges to a scoped wildcard
handler only for its named providers, and accept two scoped wildcards of
one mode when their lists differ
* **AIC-3439:** let wildcard handlers accept only named providers
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **evaluations:** promote Evaluations from Code to core
([#235](#235))
* **client:** capture conversation content by default
([#213](#213))
* **evaluations:** remove default pass rate threshold
([#238](#238))
* **evaluations:** set default pass rate threshold
([#226](#226))
* **client:** spans carry launchdarkly.project_id instead of
highlight.project_id
* **AIC-3687:** rename create_openai_agent_handler to
create_openai_agents_handler
([#225](#225))
* **skills:** rename FDv2SkillStore to LaunchDarklySkillStore
([#223](#223))

### Features

* **AIC-3439:** let wildcard handlers accept only named providers
([454a4de](454a4de))
* **AIC-3687:** rename create_openai_agent_handler to
create_openai_agents_handler
([#225](#225))
([a7cdbc0](a7cdbc0))
* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))
* **client:** point judge spans at the run they graded
([#253](#253))
([863d423](863d423))
* **evaluations:** promote Evaluations from Code to core
([#235](#235))
([0bc43a5](0bc43a5))
* **evaluations:** remove default pass rate threshold
([#238](#238))
([882d099](882d099))
* **evaluations:** route generation and judges to a scoped wildcard
handler only for its named providers, and accept two scoped wildcards of
one mode when their lists differ
([454a4de](454a4de))
* **evaluations:** set default pass rate threshold
([#226](#226))
([7ee368c](7ee368c))
* **skills:** rename FDv2SkillStore to LaunchDarklySkillStore
([#223](#223))
([11f2c0b](11f2c0b))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
* **AIC-3601:** stop graph routing text inviting a router to answer for
its target
([#230](#230))
([27b8d09](27b8d09))
* **client:** init_client raises when the LaunchDarkly client did not
initialize
([#254](#254))
([6e66c00](6e66c00))
* **client:** stamp launchdarkly.project_id on the telemetry resource
([#229](#229))
([5347558](5347558))
* **evaluations:** copy inline tool schemas and name both sources on a
key collision
([#227](#227))
([bfbdd4c](bfbdd4c))
* **evaluations:** send criterion errors as errorMessage and skip
criteria for rows that did not generate
([#228](#228))
([8b05b56](8b05b56))
* **graph:** report a node's own judge scores on its node_done stream
event ([#236](#236))
([2f3f9ee](2f3f9ee))


### Documentation

* **AIC-3692:** say which project id attribute the telemetry resource
carries
([#239](#239))
([64eff43](64eff43))
* state that Agent Skills and Evaluations from Code need 1.0.0
([#191](#191))
([fe3ef08](fe3ef08))
</details>

<details><summary>launchdarkly-ai-python: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-python-0.4.0...launchdarkly-ai-python-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **evaluations:** promote Evaluations from Code to core
([#235](#235))

### Features

* **evaluations:** promote Evaluations from Code to core
([#235](#235))
([0bc43a5](0bc43a5))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))


### Documentation

* state that Agent Skills and Evaluations from Code need 1.0.0
([#191](#191))
([fe3ef08](fe3ef08))
</details>

<details><summary>launchdarkly-ai-claude-agents: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-claude-agents-0.3.0...launchdarkly-ai-claude-agents-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **claude-agents:** honour the config's max_tokens and disable thinking
unless the config asks
([#237](#237))
* **client:** capture conversation content by default
([#213](#213))
* **AIC-3685:** make the internal provider helpers private
([#224](#224))
* **skills:** rename FDv2SkillStore to LaunchDarklySkillStore
([#223](#223))

### Features

* **AIC-3685:** make the internal provider helpers private
([#224](#224))
([9afdb94](9afdb94))
* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))
* **skills:** rename FDv2SkillStore to LaunchDarklySkillStore
([#223](#223))
([11f2c0b](11f2c0b))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
* **claude-agents:** honour the config's max_tokens and disable thinking
unless the config asks
([#237](#237))
([a7ded1b](a7ded1b))
</details>

<details><summary>launchdarkly-ai-claude-messages: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-claude-messages-0.3.0...launchdarkly-ai-claude-messages-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **client:** capture conversation content by default
([#213](#213))

### Features

* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
* **AIC-3735:** send Claude Messages sampling parameters through
extra_body
([#256](#256))
([c6d0376](c6d0376))
</details>

<details><summary>launchdarkly-ai-openai-agents: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-openai-agents-0.3.0...launchdarkly-ai-openai-agents-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **client:** capture conversation content by default
([#213](#213))
* **AIC-3685:** make the internal provider helpers private
([#224](#224))
* **AIC-3687:** rename create_openai_agent_handler to
create_openai_agents_handler
([#225](#225))

### Features

* **AIC-3685:** make the internal provider helpers private
([#224](#224))
([9afdb94](9afdb94))
* **AIC-3687:** rename create_openai_agent_handler to
create_openai_agents_handler
([#225](#225))
([a7cdbc0](a7cdbc0))
* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
* **AIC-3716:** build OpenAI native graph node tools with FunctionTool
([#251](#251))
([b859b67](b859b67))
* **AIC-3726:** realign three usage IDs with the 1.0 factory names
([#252](#252))
([1c4d904](1c4d904))
</details>

<details><summary>launchdarkly-ai-vercel-agents: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-vercel-agents-0.2.0...launchdarkly-ai-vercel-agents-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))

### Features

* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
</details>

<details><summary>launchdarkly-ai-litellm-agents: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-litellm-agents-0.2.0...launchdarkly-ai-litellm-agents-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **client:** capture conversation content by default
([#213](#213))

### Features

* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
</details>

<details><summary>launchdarkly-ai-openai-messages: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-openai-messages-0.3.0...launchdarkly-ai-openai-messages-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **client:** capture conversation content by default
([#213](#213))

### Features

* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
* **AIC-3726:** realign three usage IDs with the 1.0 factory names
([#252](#252))
([1c4d904](1c4d904))
</details>

<details><summary>launchdarkly-ai-vercel-messages: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-vercel-messages-0.2.0...launchdarkly-ai-vercel-messages-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))

### Features

* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
</details>

<details><summary>launchdarkly-ai-litellm-messages: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-litellm-messages-0.2.0...launchdarkly-ai-litellm-messages-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **client:** capture conversation content by default
([#213](#213))

### Features

* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
</details>

<details><summary>launchdarkly-ai-langchain-agents: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-langchain-agents-0.3.0...launchdarkly-ai-langchain-agents-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **evaluations:** route generation and judges to a scoped wildcard
handler only for its named providers, and accept two scoped wildcards of
one mode when their lists differ
* **AIC-3439:** let wildcard handlers accept only named providers
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **client:** capture conversation content by default
([#213](#213))

### Features

* **AIC-3439:** let wildcard handlers accept only named providers
([454a4de](454a4de))
* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))
* **evaluations:** route generation and judges to a scoped wildcard
handler only for its named providers, and accept two scoped wildcards of
one mode when their lists differ
([454a4de](454a4de))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3590:** build Python langchain-agents with
langchain.agents.create_agent
([#175](#175))
([8218626](8218626))
* **AIC-3592:** make native LangGraph hand-offs complete
([#246](#246))
([bfb738f](bfb738f))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
</details>

<details><summary>launchdarkly-ai-langchain-messages: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-langchain-messages-0.3.0...launchdarkly-ai-langchain-messages-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **evaluations:** route generation and judges to a scoped wildcard
handler only for its named providers, and accept two scoped wildcards of
one mode when their lists differ
* **AIC-3439:** let wildcard handlers accept only named providers
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **client:** capture conversation content by default
([#213](#213))

### Features

* **AIC-3439:** let wildcard handlers accept only named providers
([454a4de](454a4de))
* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))
* **evaluations:** route generation and judges to a scoped wildcard
handler only for its named providers, and accept two scoped wildcards of
one mode when their lists differ
([454a4de](454a4de))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3573:** never send a NativeTool to the model as a function tool
([#248](#248))
([73caeae](73caeae))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
* **AIC-3726:** realign three usage IDs with the 1.0 factory names
([#252](#252))
([1c4d904](1c4d904))
</details>

<details><summary>launchdarkly-ai-opencode-agents: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-opencode-agents-0.3.0...launchdarkly-ai-opencode-agents-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))

### Features

* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
</details>

<details><summary>launchdarkly-ai-typesafe: 1.0.0</summary>

##
[1.0.0](launchdarkly-ai-typesafe-0.2.0...launchdarkly-ai-typesafe-1.0.0)
(2026-10-30)


### ⚠ BREAKING CHANGES

* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
* **client:** capture conversation content by default
([#213](#213))

### Features

* **client:** add result variant and value to the feature_flag span
event ([#240](#240))
([040d487](040d487))
* **client:** capture conversation content by default
([#213](#213))
([77a7cf5](77a7cf5))


### Bug Fixes

* **AIC-3568:** give the barrel core's __all__, and drop private-spec
references
([#234](#234))
([88e57e6](88e57e6))
* **AIC-3598:** make Python packaging and public types GA-ready
([#218](#218))
([5860b40](5860b40))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> This is a **Release Please** cut that moves every monorepo package to
**1.0.0** and records the GA release in changelogs. It updates
`.release-please-manifest.json`, each package’s `pyproject.toml` /
`__version__`, and prepends **1.0.0** notes to every `CHANGELOG.md`.
> 
> Handler and barrel packages now require
**`launchdarkly-ai-server>=1.0.0,<2`**, so installs must upgrade the
core client in lockstep with adapters.
> 
> There is **no new application logic** in this diff—only release
metadata. The changelog text still matters for reviewers: it summarizes
breaking work already on `main`, including **Evaluations from Code
promoted to core**, **conversation content captured by default**,
**`init_client` failing when LD init fails**, telemetry
**`launchdarkly.project_id`**, renames like **`LaunchDarklySkillStore`**
and **`create_openai_agents_handler`**, tightened **wildcard handler
routing**, and **GA packaging / public `__all__`** cleanup.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
2e904f7. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeffdupont jeffdupont jeffdupont approved these changes
@donei003 donei003 Awaiting requested review from donei003
@andrewklatzke andrewklatzke Awaiting requested review from andrewklatzke

Assignees

No one assigned

Labels

api-compatible An owner judged a flagged public API change compatible; passes the API compatibility check

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@aknight-ld @jeffdupont