You’re wayyy over complicating it, invisibility is just a matter of taking the light that hits you, bending it around you and putting it back in place. We can already do this on a small scale with fixed stationary setups.
Even simpler is just a very accurate and fast camera/display system that records and displays the background onto itself.