I prefer using separate subnets for different network adapters. In any case, create vswitch and vmkernel for each network port and configure IPs.

VMware vSAN cluster, ceph cluster, building HA for different services, containers, k8s cluster. The list goes on.

TrueNAS is an option. However, I am not sure if you can configure active-active cluster with it. As for VMware vSAN, you can place a Witness on a small device (e.g. Raspberry PI). It is a nice option for a homelab. https://williamlam.com/2020/10/vsan-witness-using-raspberry-pi-4-esxi-arm-fling.html

Since you already have Starwinds licenses, I would go for it. 2 node cluster on top of ESXi works great. We have customers using it with no issues.

I haven’t heard about Cosmo. I need to test it. Thanks for sharing.