Don't dirty display: none nodes because of their children

[mozzius]

Summary:

YogaLayoutableShadowNode::updateYogaChildren marks a cloned node dirty if any of its children is dirty. For a display: none node that's never needed: Yoga lays it out with zero size whatever its children are.

It's also actively costly. When Yoga lays out a hidden node's parent, zeroOutLayoutRecursively resets the hidden subtree and the hidden node is marked clean, but its descendants keep their dirty flags. Fabric's layout() doesn't recurse into display: none nodes either. So a child that was created or changed while its parent was hidden stays dirty until the parent is shown. Every later commit that clones the hidden node sees that child, dirties the hidden node and every ancestor up to the root, and RootShadowNode::layoutIfNeeded re-runs layoutTree, including roundLayoutResultsToPixelGrid over every node in the surface.

React commits lay out from the root anyway, so this costs the commits that don't go through React: setNativeProps, JS-driven Animated (which uses setNativeProps on Fabric), native state updates and animation libraries that commit to the ShadowTree. The clearest case in core is a JS-driven Animated.loop on a view inside a display: 'none' container, which re-lays out the whole surface on every frame. Content hidden by <Activity mode="hidden"> ends up in the same state, since the renderer hides it by cloning the top-level host nodes with display: 'none'.

This change skips the children comparison for display: none nodes. It keeps the node's own dirty check and the child count check. Showing the node again changes its style, which dirties it in updateYogaProps, and the children that changed while it was hidden are laid out then.

Fixes #59112

Changelog:

[GENERAL] [FIXED] - Commits under a display: 'none' view no longer re-run layout over the whole surface when something under it was created or changed while it was hidden

Test Plan:

Unit tests: two new YogaDirtyFlagTest cases in ViewTest.cpp:

• dirtyChildOfHiddenNodeMustNotDirtyIt: hides the inner node, changes a child's width while it's hidden, then changes the child's opacity. Neither commit may need layout.
• showingHiddenNodeMustDirtyItAndLayOutItsChildren: hides the inner node, changes a child's width while hidden, then shows the node again. That commit must need layout, and the child must come out 42 wide.

There's no OSS runner for these, so I compiled the YogaDirtyFlagTest cases from this branch into the repro app, against RN 0.87.1 built from source, with a minimal stand-in for the gtest macros. With this change, all 10 cases pass. Without it, dirtyChildOfHiddenNodeMustNotDirtyIt fails both of its EXPECT_FALSE checks and the other 9 pass. I'm still relying on CI to run them properly.

End to end: with the repro from #59112 (https://github.com/mozzius/hidden-subtree-relayout-repro). Its fixed branch applies this exact change to RN 0.87.1 built from source. iOS 26.5 simulator, Release, 1000 rows on screen; per-commit time on the JS thread, median / p90 over 9 runs of 120 commits:

	stock	with this change
setNativeProps on a view created inside a display: 'none' parent	1.06 / 2.02 ms	0.15 / 0.21 ms
… after React re-rendered a sibling while the parent was hidden	1.01 / 2.06 ms	0.16 / 0.20 ms
… inside a hidden <Activity>, after a re-render while hidden	1.18 / 2.03 ms	0.16 / 0.21 ms
JS-driven Animated.loop on a view inside a display: 'none' parent	1.09 / 1.95 ms	0.13 / 0.17 ms
control: setNativeProps on a view in a visible parent	0.19 / 0.23 ms	0.19 / 0.24 ms

Counting with lldb, stock ran YogaLayoutableShadowNode::layoutTree for every commit in the first four rows, and this change runs it for none. A yoga::Node::setDirty breakpoint on stock shows each of those commits starting with the hidden node going from clean to dirty in updateYogaChildren ← ShadowNode::cloneTree ← UIManager::setNativeProps_DEPRECATED.

Showing hidden views again: the repro's -scenario unhide hides three containers, re-renders a label in each and changes each child 30 times with setNativeProps while hidden. It then shows them again with setNativeProps({style: {display: 'flex'}}), with a React state change, and by switching an <Activity> back to visible. All three lay out the same as a never-hidden reference, with and without this change.

Context: we found this in the Bluesky app, where a Reanimated useAnimatedStyle on a hidden spinner in the profile header committed on every scroll frame, and each commit re-ran layout for the whole screen. We worked around it in app code (bluesky-social/social-app#12000). It's a sibling of #59100, fixed in #59101: the same symptom with a different cause, and the two changes are independent.

🤖 Generated with Claude Code

[meta-codesync]

@javache has imported this pull request. If you are a Meta employee, you can view this in D125454304.