I had an old MicroVAX... micro...yeah... must have weighed in at about 80 lbs. or so. Anyway, it ran Ultrix. Ultrix was a DEC Unix. Don't know if you can get a hold of a copy of it, but if you want a *nix OS on a VAX, Ultrix will run.
I always did like VMS and then later, OpenVMS. I had work accounts on several VAX's and then later an AlphaVAX. VMS was a nice OS once you got used to the syntax and all. Having to enclose directory names in square brackets always seemed a bit crude to me.. Setting permissions and a few other things took longer command lines than *nix OS's chmod commands too. But it was rock solid and stable and once you got used to it it was good. Oh, and the text editors... they had nothing as powerful as vi. EDIT and EEVE were OK, but a lot slower than vi. Also, not being case-sensitive limited things a bit. But compared to DOS it was killer. I still like the VMS MAIL utility better than sendmail.
But *nix is really the way to go and if you can get a copy of Ultrix it should load on any VAX box nicely.