Add backup file extraction feature by JoaoJandre · Pull Request #13935 · apache/cloudstack

JoaoJandre · 2026-08-20T13:45:46Z

Description

This PR adds backup file extraction feature. Currently only supported by KBOSS.

This feature must be enabled in the backup offering through the allowextractfile parameter. If it is set to true, it will be possible to extract files from backups created with that offering.

The listBackupFilesystems API was added to allow listing file systems inside backup volumes:

Parameter	Description	Required?
`backupId`	ID of the backup for which the file systems should be listed	Yes
`volumeId`	ID of the volume belonging to the backup for which the file systems should be listed. If not provided, the file systems of all backup volumes will be listed	No

Example execution of the listBackupFilesystems API

(labinterno) 🐱 > list backupfilesystems backupid=ecb15182-d6a8-4044-8251-6d437b27b971 
{
  "count": 1,
  "datastoreobject": [
    {
      "format": "ext4",
      "isdirectory": false,
      "isfilesystem": true,
      "name": "/dev/sda1",
      "size": 0,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    }
  ]
}

The listBackupFiles API was added to allow listing files inside backup volumes:

Parameter	Description	Required?
`backupId`	ID of the backup for which the files should be listed	Yes
`volumeId`	ID of the volume belonging to the backup for which the files should be listed	Yes
`filesystem`	Name of the file system that should be mounted in order to list the files	Yes
`path`	Path to be listed inside the mounted file system	Yes
`issymlink`	If the directory is a symbolic link to another directory. If not informed, ACS will check it	No

The filesystem and path parameters expect that the files do not contain special characters such as (, |, or >, for example. Both are expected to receive values in the format /path/to/something.

Example execution of the listBackupFiles API

(labinterno) 🐱 > list backupfiles backupid=ecb15182-d6a8-4044-8251-6d437b27b971 volumeid=4af9cee2-6014-44fa-ac86-09a649b67363 filesystem=/dev/sda1 path=/tmp
{
  "count": 8,
  "datastoreobject": [
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".ICE-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".Test-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".X11-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".XIM-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".font-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:45+0000",
      "name": "netplan_a2jbieem",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:42+0000",
      "name": "systemd-private-5f39ee6fdaa3434caa2eb42ad39db34f-systemd-resolved.service-jLhWCz",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": "systemd-private-5f39ee6fdaa3434caa2eb42ad39db34f-systemd-timesyncd.service-3aOZJF",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    }
  ]
}

To download files, the downloadBackupFile API was added:

Parameter	Description	Required?
`backupId`	ID of the backup from which the file should be downloaded	Yes
`volumeId`	ID of the volume belonging to the backup from which the file should be downloaded	Yes
`filesystem`	Name of the file system that should be mounted in order to download the file	Yes
`path`	Path to be downloaded inside the mounted filesystem. If it is a file, it will be compressed using `gzip`; if it is a directory, it will be added to a `tar` archive and compressed using `gzip`	Yes

The filesystem and path parameters expect that the files do not contain special characters such as (, |, or >, for example. Both are expected to receive values in the format /path/to/something.

Example execution of the downloadBackupFile API

(labinterno) 🐱 > download backupfile backupid=ecb15182-d6a8-4044-8251-6d437b27b971 volumeid=4af9cee2-6014-44fa-ac86-09a649b67363 filesystem=/dev/sda1 path=/tmp/systemd-private-5f39ee6fdaa3434caa2eb42ad39db34f-systemd-resolved.service-jLhWCz
{
  "downloadbackupfileresponse": {
    "name": "systemd-private-5f39ee6fdaa3434caa2eb42ad39db34f-systemd-resolved.service-jLhWCz",
    "state": "DOWNLOAD_URL_CREATED",
    "url": "http://192.168.123.142/userdata/215a45db-70e1-4649-93cb-258abe572573/5967473b-29b1-4057-85a3-cd924a458033.gz"
  }
}

The list and download file workflow was also added to the GUI.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Test descriptions

Test	Result
List filesystems for a volume containing filesystems	Filesystems returned
List filesystems for a volume without filesystems	Empty list returned
List filesystems for a backup without specifying the volume	List of all filesystems across all backup volumes returned
List files in a standard directory	List of files returned, with correct file types and sizes
List files in a symlink to a directory, specifying the `isSymlink=true` flag	List of files returned, with correct file types and sizes
List files in a symlink to a directory, without specifying the `isSymlink` flag	List of files returned, with correct file types and sizes
List files in an empty directory	Empty list returned
List files in a directory containing symlinks	List of files returned, including the symlink file, with its canonical path information
List files in a directory containing a symlink pointing to a non-existent path	List of files returned, excluding the symlink file
List files in a non-existent directory	Error (expected)
List files in a symlink pointing to another symlink	Error (expected)
Download a standard file	File downloaded successfully
Download a standard directory	File downloaded successfully
Download a symlink pointing to a file	File downloaded successfully
Download a symlink pointing to a directory	File downloaded successfully
Download a symlink pointing to another symlink	Error (expected)

All the tests above were executed for both Linux ROOT volumes (ext4) and Windows ROOT volumes (NTFS).

codecov · 2026-08-20T13:55:08Z

Codecov Report

❌ Patch coverage is 23.78121% with 641 lines in your changes missing coverage. Please review.
✅ Project coverage is 20.67%. Comparing base (b535ae8) to head (d3c4702).

Files with missing lines	Patch %	Lines
.../apache/cloudstack/backup/KbossBackupProvider.java	0.00%	104 Missing ⚠️
...rg/apache/cloudstack/backup/BackupManagerImpl.java	0.00%	73 Missing and 1 partial ⚠️
.../apache/cloudstack/utils/qemu/GuestfishClient.java	73.45%	52 Missing and 8 partials ⚠️
.../apache/cloudstack/backup/to/BackupFileObject.java	0.00%	50 Missing ⚠️
...he/cloudstack/backup/ExtractBackupFileCommand.java	0.00%	43 Missing ⚠️
...api/command/user/backup/DownloadBackupFileCmd.java	0.00%	34 Missing ⚠️
...org/apache/cloudstack/backup/ListFilesCommand.java	0.00%	34 Missing ⚠️
...ck/api/command/user/backup/ListBackupFilesCmd.java	0.00%	30 Missing ⚠️
...apper/LibvirtDownloadBackupFileCommandWrapper.java	3.70%	26 Missing ⚠️
.../command/user/backup/ListBackupFilesystemsCmd.java	0.00%	21 Missing ⚠️
... and 13 more

Additional details and impacted files

@@            Coverage Diff             @@
##               main   #13935    +/-   ##
==========================================
  Coverage     20.66%   20.67%            
- Complexity    21720    21753    +33     
==========================================
  Files          6492     6507    +15     
  Lines        590728   591649   +921     
  Branches      72448    72534    +86     
==========================================
+ Hits         122101   122304   +203     
- Misses       455355   456057   +702     
- Partials      13272    13288    +16

Flag	Coverage Δ
uitests	`3.87% <ø> (-0.01%)`	⬇️
unittests	`21.97% <23.78%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

JoaoJandre · 2026-08-20T20:03:20Z

It would be nice if someone could test this using RPM, since I have not tested installing the RPM packages.

github-actions · 2026-08-21T09:48:43Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

GaOrtiga

LGTM

…tion

JoaoJandre · 2026-09-16T13:41:47Z

@blueorangutan package

DaanHoogland · 2026-10-01T14:35:59Z

@blueorangutan test

DaanHoogland · 2026-10-02T10:52:54Z

@blueorangutan shutup

blueorangutan · 2026-10-02T11:58:47Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19379

DaanHoogland · 2026-10-02T14:38:34Z

@blueorangutan test

blueorangutan · 2026-10-02T14:40:10Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-10-02T15:21:56Z

[SF] Trillian Build Failed (tid-17068)

github-actions · 2026-10-25T16:15:11Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

…tion

DaanHoogland · 2026-10-28T07:55:38Z

@blueorangutan package

blueorangutan · 2026-10-28T07:56:04Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

DaanHoogland · 2026-10-28T07:57:18Z

It would be nice if someone could test this using RPM, since I have not tested installing the RPM packages.

I can give it a quick look, @JoaoJandre . Do you need more then just a monkey test, i.e. specific functional tests? Or do you have a scripted version of the tests to perform?

blueorangutan · 2026-10-28T09:33:05Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19791

JoaoJandre · 2026-10-28T11:24:30Z

It would be nice if someone could test this using RPM, since I have not tested installing the RPM packages.

I can give it a quick look, @JoaoJandre . Do you need more then just a monkey test, i.e. specific functional tests? Or do you have a scripted version of the tests to perform?

@DaanHoogland
A simple test would work:

Install packages
Set the backup provider to KBOSS
Create a backup offering with Allow file extraction
Take a backup using such offering
Check if it is possible to browse/download a file

Honestly my main concern is just if the guestfish is being correctly installed when the RPMs are installed. The rest should work.

DaanHoogland · 2026-10-28T13:00:43Z

@blueorangutan test keepEnv

blueorangutan · 2026-10-28T13:02:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-10-28T13:08:14Z

[SF] Trillian Build Failed (tid-17312)

blueorangutan · 2026-10-28T14:10:55Z

[SF] Trillian Build Failed (tid-17315)

blueorangutan · 2026-10-28T16:07:32Z

[SF] Trillian Build Failed (tid-17317)

Feature to extract files from KBOSS backups

554e36c

JoaoJandre requested review from GaOrtiga and bernardodemarco August 20, 2026 13:45

boring-cyborg Bot added component:api component:backup component:kvm component:packaging component:secondary-storage component:ssvm component:UI distro:ubuntu labels Aug 20, 2026

remove leftover variable

e00fb60

weizhouapache added this to the 4.24.0 milestone Aug 20, 2026

fix test

4f86194

JoaoJandre added status:needs-testing status:needs-review labels Aug 20, 2026

github-actions Bot added the status:has-conflicts label Aug 21, 2026

apache deleted a comment from blueorangutan Aug 23, 2026

GaOrtiga approved these changes Aug 24, 2026

View reviewed changes

DaanHoogland added this to CloudStack Testing Aug 25, 2026

github-project-automation Bot moved this to Backlog in CloudStack Testing Aug 25, 2026

DaanHoogland moved this from Backlog to conflict/waiting for author in CloudStack Testing Aug 31, 2026

JoaoJandre added 2 commits September 10, 2026 09:43

Merge remote-tracking branch 'origin/main' into add-kboss-file-extrac…

0257ffb

…tion

address reviews

d8cf1d0

github-actions Bot removed the status:has-conflicts label Sep 10, 2026

github-actions Bot added the status:has-conflicts label Oct 25, 2026

Merge remote-tracking branch 'origin/main' into add-kboss-file-extrac…

d3c4702

…tion

github-actions Bot removed the status:has-conflicts label Oct 27, 2026

DaanHoogland self-assigned this Oct 28, 2026

Conversation

JoaoJandre commented Aug 20, 2026

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Test descriptions

Uh oh!

codecov Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

JoaoJandre commented Aug 20, 2026

Uh oh!

github-actions Bot commented Aug 21, 2026

Uh oh!

GaOrtiga left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JoaoJandre commented Sep 16, 2026

Uh oh!

DaanHoogland commented Oct 1, 2026

Uh oh!

DaanHoogland commented Oct 2, 2026

Uh oh!

blueorangutan commented Oct 2, 2026

Uh oh!

DaanHoogland commented Oct 2, 2026

Uh oh!

blueorangutan commented Oct 2, 2026

Uh oh!

blueorangutan commented Oct 2, 2026

Uh oh!

github-actions Bot commented Oct 25, 2026

Uh oh!

DaanHoogland commented Oct 28, 2026

Uh oh!

blueorangutan commented Oct 28, 2026

Uh oh!

DaanHoogland commented Oct 28, 2026

Uh oh!

blueorangutan commented Oct 28, 2026

Uh oh!

JoaoJandre commented Oct 28, 2026

Uh oh!

DaanHoogland commented Oct 28, 2026

Uh oh!

blueorangutan commented Oct 28, 2026

Uh oh!

blueorangutan commented Oct 28, 2026

Uh oh!

blueorangutan commented Oct 28, 2026

Uh oh!

blueorangutan commented Oct 28, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov Bot commented Aug 20, 2026 •

edited

Loading