Disaster recovery in cloudstack

[TheKunalSen]

So I have built a cloudstack setup where I have a nfs server, 3 hosts and 1 management server in one zone (172.16.17.0/24)

And I have created a system gateway vm which is connected to a cloudflare tunnel and serving all my websites (routing the traffic into the internal 10.x.x.0/16 vms) through reverse proxy.

For in-zone ha if the host goes down the vms migrate to another host this is okay

But if I want to build a disaster recovery system where if this site goes down totally i still want my websites to be alive!

There are a couple methods I have thought
Which are

1. I build a site b setup almost same as this one and copy all the vms and all to that setup and setup a cloudflare tunnel load balancer which will load balance the traffic between two sites (note that the site b would be a new zone in that same management server which will be in a different public ip network 192.168.1.0/24)
   But in this case the problem is suppose I have 100 vms and I copied all of them manually to site b-zone-b and it's working, but if I create another vm on site-a-zone-a and I want that vm to replicate to site-b-zone-b, how is this possible cause the nfs storage would be different, the host ids would be different, the only thing which will be connected are the management servers.

2. If I try to add a site-b host which is on a different network on this zone then connect both the network on site to site vpn then the hosts gets added but the system vms on that host doesn't work or the agent states stays blank, even if i create another pod on that site -b network inside that site-a-zone-a the hosts gets added but the vms doesn't work properly, if this thing happens then host from different sites can migrate the vms between themselves.

What do you think what would be the best strategy for disaster recovery in cloudstack for me!

[Flpvoigt]

I would not put the two sites in the same CloudStack zone and expect CloudStack HA to replicate or migrate running VMs across the WAN. CloudStack HA is primarily for host/VM failure within a zone; it is not a complete-site DR mechanism.

For this topology, I would use:

1. Separate sites as separate zones (or separate regions if they have independent Management Server clusters), rather than adding site-B hosts to site-A's zone.
2. Replicate the VM data outside the hypervisor layer: use the storage platform's replication or a supported CloudStack Backup & Recovery provider. In CloudStack 4.22+, a cross-zone backup repository can be used to create a new instance in another zone; the repository must be reachable from the destination zone.
3. Replicate/copy the CloudStack database and secondary storage as part of the DR plan. Templates and snapshots alone are not enough if you need the whole environment and its metadata.
4. Replicate the application data as well (database replication, object storage, etc.). Cloudflare Load Balancing can direct traffic to site B, but it does not replicate VM state or application data.

A practical runbook is: take scheduled backups/snapshots in site A, copy them to a repository available in site B, keep equivalent networks/offerings/templates in site B, and restore/create the instances there during a site-A outage. Test the restore and DNS/Cloudflare failover regularly. I would avoid live migration across sites unless the hypervisor/storage vendor explicitly supports it with the required latency and shared storage guarantees.

References:

• https://docs.cloudstack.apache.org/en/latest/adminguide/backup_and_recovery.html
• https://docs.cloudstack.apache.org/en/latest/conceptsandterminology/choosing_deployment_architecture.html

[DRDAVIDBANNER]

Hi Kunal, I’m Pete, CTO at Sendense. This is a use case we handle with Sendense: agentless VM replication and disaster recovery between CloudStack sites.

Building on the suggestion above, I’d set this up as two separately operable sites, with site B having its own CloudStack management, compute, storage and networking. Sendense then keeps recovery copies of your workloads ready at the second site and provides the replication, testing and failover workflow.

The source and destination storage can also be different. You can keep your existing NFS storage at site A and use a different storage type at site B—for example, NFS at one site and block storage at the other. Each site can use the storage that suits its requirements and budget. Sendense handles protection and recovery across those environments, with no NFS workaround needed.

The different host IDs and storage locations you mentioned are handled through the destination configuration. Each site uses its own CloudStack resources, and the replica is prepared for the destination’s compute, storage and networks. You don’t need to reproduce the source host IDs or manually reconcile copied disk files.

For your setup, the workflow would look like this:

1. Configure replication to site B. Select the VMs you want to protect, their destination placement and the replication schedule.
2. Bring new VMs into the same workflow. As you add workloads at site A, add them to protection through Sendense, which manages their replication to the recovery site.
3. Test recovery while production stays running. Use an isolated test failover to boot the recovered VMs and check that the websites and their dependencies work.
4. Activate the recovery workloads when needed. Bring the services up at site B, then use your Cloudflare routing to direct traffic to the recovered environment.

For your websites, I’d include the database VMs and gateway/reverse proxy in the recovery plan, then test the complete path from Cloudflare through to the application. That gives you a practical way to verify the service will work from the second site.

There’s more on the workflow here: [Sendense disaster recovery](https://sendense.com/solutions/dr-at-scale).

Happy to walk through your two-site layout and show how this would work with your existing NFS setup and the storage you’re considering for site B.