So far all my setups have had root on SSD mirror with separate hard disk storage pool for all the data. Years ago I used to keep the app config, databases and docker files on the root filesystem, while the app data resided on the storage pool. That was cumbersome for backups and storage size. Eventually I moved all app data to the storage pool. Essentially the apps can be started on any machine with a Linux OS that has docker installed. Database access is slower but it’s a decent compromise for having trivial all-in-one snapshots and backup. Now I’m setting up a new NAS for a friend and I’m wondering whether it’s worth keeping the root filesystem separate from the storage pool. If I put it on the disks, I’d get trivial full system snapshots and backups. I’d have the same hardware reliability as the storage pool. There wouldn’t be issues with root filling up. The caveat is that the OS would be slower. Has anyone reasoned and/or tried this? Should I go for it?
E: I recently put my laptop’s root on ZFS and the ability to do full backups while the system is running is pretty great. The full system can be pretty trivialy restored to a new drive with zfs send / recv during setup.
Right, the flexibility angle makes sense if using a typical root fs like Ext4 with or without LVM. That’s a reason I’ve always kept the OS separate. But with ZFS there’s unlimited flexibility. Separate datasets or volumes within the same storage pool are trivial. I could do root on ZFS on separate SSDs and get those benefits but it’s more complicated that slapping it all in a single pool. Then maybe use the SSDs for cache. :D