Repository navigation
Conversation
…tions in one query getTransformations with extraParams=True ran one query on AdditionalParameters per returned transformation. They are now read with one query per 1000 transformations, which getAdditionalParameters also uses.
chrisburr
force-pushed
the
perf/bulk-transformation-extra-params
branch
from
October 27, 2026 12:23
021502f to
82766c0
Compare
chrisburr
changed the title
Contributor
|
I am investigating the failing tests, this is due to one of my private PR I think |
Contributor
|
At least some should be fixed with: #8858 |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TransformationDB.getTransformations(..., extraParams=True)ran one query onAdditionalParametersper returned transformation (theTODO: optimize by getting all the extra params at once). They are now read with a singleIN (...)query per 1000 transformations and matched back to each transformation, andgetAdditionalParametersgoes through the same code. The result is unchanged, including the conversion ofIntType/LongTypevalues.This makes it cheap to read the additional parameters of a list of transformations in one service call, e.g.
getTransformations({"TransformationID": transIDs}, columns=["TransformationID"], extraParams=True), which LHCbDIRAC's MCSimulationTestingAgent now does every cycle.BEGINRELEASENOTES
*TransformationSystem
CHANGE: TransformationDB.getTransformations reads the additional parameters of all the transformations at once when extraParams is set
ENDRELEASENOTES