Repository navigation
CLDSRV-1020: Improve KMS upfront decrypt with many parts - #6341
Conversation
Provide an env var to override or skip.
Instead of synchronous callback that trigger stack overflow with many parts in async 2.5 (branch 7.X and 8.X). Also should be more optimized.
If only 1 of multiple parallel queries fails the objectGet, cleanup early the decipher stream of other successful KMS queries
Those streams for cryptography can hold native C++ memory for OpenSSL. It's good to release them early without waiting for GC.
Hello bourgoismickael,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Incorrect fix versionThe
Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:
Please check the |
Request integration branchesWaiting for integration branch creation to be requested by the user. To request integration branches, please comment on this pull request with the following command: Alternatively, the |
|
If it changes what user sees, we should document the cons (may be release notes or something else) |
|
/create_integration_branches |
The latest is 26 with breaks with tests needing python 3.9
cfd1363 to
8503658
Compare
History mismatchMerge commit #cfd136373489893f4123b1cc55d12e4c5542f5ea on the integration branch It is likely due to a rebase of the branch Please use the The following options are set: create_integration_branches |
|
@bert-e reset |
Reset completeI have successfully deleted this pull request's integration branches. The following options are set: create_integration_branches |
ConflictA conflict has been raised during the creation of I have not created the integration branch. Here are the steps to resolve this conflict: git fetch
git checkout -B w/7.70/improvement/CLDSRV-1020-kms-upfront origin/development/7.70
git merge origin/improvement/CLDSRV-1020-kms-upfront
# <intense conflict resolution>
git commit
git push -u origin w/7.70/improvement/CLDSRV-1020-kms-upfrontThe following options are set: create_integration_branches |
ConflictA conflict has been raised during the creation of I have not created the integration branch. Here are the steps to resolve this conflict: git fetch
git checkout -B w/8.8/improvement/CLDSRV-1020-kms-upfront origin/development/8.8
git merge origin/w/7.70/improvement/CLDSRV-1020-kms-upfront
# <intense conflict resolution>
git commit
git push -u origin w/8.8/improvement/CLDSRV-1020-kms-upfrontThe following options are set: create_integration_branches |
ConflictA conflict has been raised during the creation of I have not created the integration branch. Here are the steps to resolve this conflict: git fetch
git checkout -B w/9.0/improvement/CLDSRV-1020-kms-upfront origin/development/9.0
git merge origin/w/8.8/improvement/CLDSRV-1020-kms-upfront
# <intense conflict resolution>
git commit
git push -u origin w/9.0/improvement/CLDSRV-1020-kms-upfrontThe following options are set: create_integration_branches |
ConflictA conflict has been raised during the creation of I have not created the integration branch. Here are the steps to resolve this conflict: git fetch
git checkout -B w/9.2/improvement/CLDSRV-1020-kms-upfront origin/development/9.2
git merge origin/w/9.1/improvement/CLDSRV-1020-kms-upfront
# <intense conflict resolution>
git commit
git push -u origin w/9.2/improvement/CLDSRV-1020-kms-upfrontThe following options are set: create_integration_branches |
ConflictA conflict has been raised during the creation of I have not created the integration branch. Here are the steps to resolve this conflict: git fetch
git checkout -B w/9.3/improvement/CLDSRV-1020-kms-upfront origin/development/9.3
git merge origin/w/9.2/improvement/CLDSRV-1020-kms-upfront
# <intense conflict resolution>
git commit
git push -u origin w/9.3/improvement/CLDSRV-1020-kms-upfrontThe following options are set: create_integration_branches |
ConflictA conflict has been raised during the creation of I have not created the integration branch. Here are the steps to resolve this conflict: git fetch
git checkout -B w/9.4/improvement/CLDSRV-1020-kms-upfront origin/development/9.4
git merge origin/w/9.3/improvement/CLDSRV-1020-kms-upfront
# <intense conflict resolution>
git commit
git push -u origin w/9.4/improvement/CLDSRV-1020-kms-upfrontThe following options are set: create_integration_branches |
ConflictA conflict has been raised during the creation of I have not created the integration branch. Here are the steps to resolve this conflict: git fetch
git checkout -B w/9.5/improvement/CLDSRV-1020-kms-upfront origin/development/9.5
git merge origin/w/9.4/improvement/CLDSRV-1020-kms-upfront
# <intense conflict resolution>
git commit
git push -u origin w/9.5/improvement/CLDSRV-1020-kms-upfrontThe following options are set: create_integration_branches |
Integration data createdI have created the integration data for the additional destination branches.
The following branches will NOT be impacted:
You can set option The following options are set: create_integration_branches |
Waiting for approvalThe following approvals are needed before I can proceed with the merge:
The following options are set: create_integration_branches |
|
/approve |
In the queueThe changeset has received all authorizations and has been added to the The changeset will be merged in:
The following branches will NOT be impacted:
This pull request does not target the following hotfix branch(es) so they
There is no action required on your side. You will be notified here once IMPORTANT Please do not attempt to modify this pull request.
If you need this pull request to be removed from the queue, please contact a The following options are set: approve, create_integration_branches |
Queue build failedThe corresponding build for the queue failed:
Remove the pull request from the queue
|
|
ping |
|
I have successfully merged the changeset of this pull request
The following branches have NOT changed:
Please check the status of the associated issue CLDSRV-1020. Goodbye bourgoismickael. |
Describe the trade-off of upfront parallel KMS Decrypt before streaming parts done by:
- parallel queries instead of in series between each parts for faster overall streaming
- can delay time to first byte
- can flood the KMS, parallel should be limited
- can hold native memory, should be cleaned asap if not consumed (on error) without waiting for GC
Defaults to 100, with environment variable
GET_OBJECT_MAX_PARALLEL_KMS_DECRYPTto override or skip upfront KMS decryptThis fixes comments from this backport #6322