perf: Map identity overrides, evaluate environment flags once per document refresh by khvn26 · Pull Request #262 · Flagsmith/flagsmith-python-client

khvn26 · 2026-10-16T17:40:26Z

Contributes to #198.

In this PR, we resolve two performance regressions in local evaluation mode, both dating from the move to the evaluation context engine (4.0.0/5.0.0).

Identity overrides are mapped to segments, one per unique set of overrides, and every one of them was matched on every identity evaluation. They're now indexed by identifier once per environment document refresh, and each evaluation only includes the requested identity's own segment. This decouples local evaluation runtime from the number of identity overrides.
Environment flags were evaluated by the engine on every get_environment_flags() call, although they only depend on the environment document. They're now evaluated once per environment document refresh.

Evaluation state now lives in a new LocalEvaluation object, rebuilt on every environment document refresh.

Environment flags and single identity flags are now well ahead of 3.8.0.

An all_flags() call for an identity is still 7–18% slower than 3.8.0; the remaining gap will be closed engine-side.

Benchmark: gist

Identity overrides segments are indexed by identifier, and each identity is evaluated with its own only, narrowed to its identifier. Evaluation cost no longer grows with the number of identity overrides. Environment flags are evaluated once per environment document. Refs #198

for more information, see https://pre-commit.ci

`SegmentOverridesIndex` is now the read-only mapping consumers take, so `SegmentOverridesMapping` goes. Indexes are built as dicts of a new `SegmentContexts` alias, which also shortens repeated annotations.

One type for every index, built or read: `dict` and the per-identity `ChainMap` are both mutable mappings.

emyller

Looks solid in its logic. Left a few comments that aim to contribute to interface and sticking to testing patterns.

I'd also advise against using base to name anything. It does not add meaning.

emyller · 2026-10-20T14:13:54Z

+    any other segment."""
+    if (segment_context.get("metadata") or {}).get("source") != "identity_overrides":
+        return None
+    # As built by `map_identifiers_to_identity_overrides_rules`.


nit: Maybe this comment hints at a missing type? 🤔

Corrected in ed17ceb, and then gone in 2276774.

emyller · 2026-10-20T14:15:48Z

+def map_identifiers_to_identity_overrides_rules(
+    identifiers: typing.List[str],
+) -> typing.List[SegmentRule]:


Suggested change

def map_identifiers_to_identity_overrides_rules(

identifiers: typing.List[str],

) -> typing.List[SegmentRule]:

def map_identifiers_to_identity_overrides_rule(

identifiers: typing.List[str],

) -> SegmentRule: # or `IdentityOverrideSegmentRule` (see [other thread](https://github.com/Flagsmith/flagsmith-python-client/pull/262/changes#r4325989902))

Yup, good call. 25617c1.

emyller · 2026-10-20T14:21:28Z

+# Segment contexts by their key in the evaluation context.
+SegmentContexts = typing.Dict[str, SegmentContext[SegmentMetadata, FeatureMetadata]]
+# Feature name -> segment contexts overriding that feature.
+SegmentOverridesIndex = typing.MutableMapping[str, SegmentContexts]


nit

Suggested change

# Segment contexts by their key in the evaluation context.

SegmentContexts = typing.Dict[str, SegmentContext[SegmentMetadata, FeatureMetadata]]

# Feature name -> segment contexts overriding that feature.

SegmentOverridesIndex = typing.MutableMapping[str, SegmentContexts]

# SegmentKey = FeatureName = str

SegmentContexts = typing.Dict[SegmentKey, SegmentContext[SegmentMetadata, FeatureMetadata]]

SegmentOverridesIndex = typing.MutableMapping[FeatureName, SegmentContexts]

emyller · 2026-10-20T14:22:44Z

+    # When
+    # the environment is refreshed
+    local_eval_flagsmith._evaluation_context = evaluation_context
+    local_eval_flagsmith.get_environment_flags()
+
+    # Then
+    assert spy.call_count == 2


note: Minor pattern break makes me wonder if this portion is worth a new test case.

Agreed, split out in 029fcae.

emyller · 2026-10-20T14:23:27Z

+    assert local_evaluation.overrides_index == {}
+
+
+def test_local_evaluation__get_identity_context__no_identity_overrides__returns_base(


Suggested change

def test_local_evaluation__get_identity_context__no_identity_overrides__returns_base(

def test_local_evaluation__get_identity_context_no_identity_overrides__returns_base(

ffd2d3e + addressed the naming concern re: base.

emyller · 2026-10-20T14:25:47Z

+    assert len(identity_segments) == 1
+    [segment] = identity_segments
+    assert segment["rules"][0]["conditions"] == [
+        {
+            "property": "$.identity.identifier",
+            "operator": "IN",
+            "value": [identifier],
+        }
+    ]


Suggested change

assert len(identity_segments) == 1

[segment] = identity_segments

assert segment["rules"][0]["conditions"] == [

{

"property": "$.identity.identifier",

"operator": "IN",

"value": [identifier],

}

]

assert identity_segments == [{...}]

Since 2276774 the segment key is deterministic, so the whole segment is now asserted literally.

The client kept the evaluation context next to the `LocalEvaluation` built from it. It now keeps the `LocalEvaluation` only, whose `context` is the context without identity overrides segments, formerly `base_context`. Identity overrides segments are kept apart, by key. Also drops name mangling, as `__evaluation_context` goes.

`map_identifiers_to_identity_overrides_rules` built a single rule, so it is now `map_identifiers_to_identity_overrides_rule`, returning that rule.

Identity overrides conditions are typed as `InOperatorSegmentCondition` where they are built, and identifiers are read back as such, rather than cast to a list of strings with a comment.

Replaces comments explaining what the keys are.

Also drops `base` from test and variable names.

Identity overrides were mapped to segments, one per unique set of overrides, merged into the evaluation context, then picked back out of it by `LocalEvaluation` and narrowed to an identity on every evaluation. They are now mapped straight from the environment document to a segment per identifier, kept out of the evaluation context. `LocalEvaluation` indexes each one's overrides once per environment document, so an identity evaluation only adds its own segment.

Drops comments, shortens the `LocalEvaluation` docstring, and specs the `LocalEvaluation` mocks in streaming manager tests.

khvn26 · 2026-10-20T16:07:43Z

Updated the logic to map env doc's identity_overrides straight into the new cache.

khvn26 requested a review from a team as a code owner October 16, 2026 17:40

khvn26 requested review from emyller and removed request for a team October 16, 2026 17:40

[pre-commit.ci] auto fixes from pre-commit.com hooks

32392d1

for more information, see https://pre-commit.ci

khvn26 marked this pull request as draft October 16, 2026 17:41

This was referenced Oct 16, 2026

chore(main): release 6.3.1 #253

Open

feat: Support $.flags context values in segment conditions #261

Open

khvn26 added 2 commits October 19, 2026 12:29

refactor: Use one segment overrides index type

7b72b4c

`SegmentOverridesIndex` is now the read-only mapping consumers take, so `SegmentOverridesMapping` goes. Indexes are built as dicts of a new `SegmentContexts` alias, which also shortens repeated annotations.

refactor: Type segment overrides indexes as mutable mappings

e6afb07

One type for every index, built or read: `dict` and the per-identity `ChainMap` are both mutable mappings.

khvn26 marked this pull request as ready for review October 19, 2026 11:06

khvn26 changed the title ~~perf: Fix local evaluation regressions~~ perf: Map identity overrides, evaluate environment flags once per document refresh Oct 19, 2026

khvn26 assigned emyller Oct 19, 2026

style: Use plain comments

620a1dc

emyller requested changes Oct 20, 2026

View reviewed changes

khvn26 added 7 commits October 20, 2026 16:35

refactor: Map identifiers to a single identity overrides rule

25617c1

`map_identifiers_to_identity_overrides_rules` built a single rule, so it is now `map_identifiers_to_identity_overrides_rule`, returning that rule.

refactor: Type identity overrides conditions

ed17ceb

Identity overrides conditions are typed as `InOperatorSegmentCondition` where they are built, and identifiers are read back as such, rather than cast to a list of strings with a comment.

refactor: Name segment overrides index keys with type aliases

2900a6e

Replaces comments explaining what the keys are.

test: Split re-evaluating environment flags on refresh into its own test

029fcae

test: Name local evaluation tests with three parts

ffd2d3e

Also drops `base` from test and variable names.

test: Assert identity overrides segments whole

8fd7a17

khvn26 force-pushed the perf/local-eval-regression branch from 3e37352 to 8fd7a17 Compare October 20, 2026 14:39

khvn26 added 2 commits October 20, 2026 17:12

chore: Apply review suggestions

e80d1c4

Drops comments, shortens the `LocalEvaluation` docstring, and specs the `LocalEvaluation` mocks in streaming manager tests.

khvn26 requested a review from emyller October 20, 2026 16:07

		assert local_evaluation.overrides_index == {}


		def test_local_evaluation__get_identity_context__no_identity_overrides__returns_base(

Conversation

khvn26 commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emyller left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

khvn26 commented Oct 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

khvn26 commented Oct 16, 2026 •

edited

Loading