When White plays Nxg4 he gets the bishop pair. While it doesn't translate into an immediate breakthrough, this is a valuable positionnal plus that will be felt later on, when the game opens up. Having it as early as move 5, without making concession, is a small but clear achievement for White.
You might argue that this gain costed White 2 tempi (Ne5-Nxg4), but actually Black will lose time with their stranded knight on g4 too (most probably Nxg4-f6).
Concretely, after 4...e6 5.Nxg4 Nxg4, White can either :
try to exploit the floating Ng4 as you mentionned: 6.e4 Nf6! 7.e5 Nfd7 8.Bd3, intending c3, Nd2-f3, h4...
try to exploit the slight weaening of the light squares: 6.c4 intending Nc3, either e3 or e4, possibly Db3 hitting b7 and d5. This is riskier but may succeed in opening the center faster.
continue slowly as if nothing happened : e3, Bd3, Nd2, 0-0... expecting the stength of the light-squared bishop to reveal itself later in the game, possibly after c2-c3 and e3-e4.
In any case, Black's position doesn't collapse immediately but any grandmaster would gladly take White. That's the kind of position we used to evaluate as +/=, althought nowadays it's probably just +0.15...