The only reason we still have this plugin paradigm is because that’s how N64 emulator culture evolved over time. That’s deliberate though, there’s no “mess” here.
I can easily download a cycle-accurate N64 emulator that depends on zero plugins. However, being the N64, it obviously takes a lot of power to emulate in such an accurate way.
Even hypervisors can have software bugs - running GBA games on the ARM9 core in the DSi is possible and even closer to “actual hardware” than a FPGA, but there are still weird side cases and glitches that only happen on this setup rather than actual GBA hardware.
FPGAs aren’t some magical hardware clone that bypasses software issues.