I have a remote backup setup at my parents, which consists of a rpi4 and an external hdd (old NAS HDD). The rpi also has a wireguard interface running. If I need to back stuff up (via a restic-rest container) I open the vpn from my side, start the container, mount/decrypt the hdd and trigger the restic update. Afterwards everything gets disconnected again.
The rpi needs to be always on and you need to be able to forward a port to the rpi.
The data should be secure though, as the disk is encrypted and not mounted. (You can even store the decryption key on your side of the network, so you need the vpn to be able to mount the hdd)
On the one running on my parents side I run raspberryOS. I do have an rpi on my side which is triggering the script and vpn (and also backs up their NAS) That one is running NixOS, but raspberryOS works fine.