Usually I play somewhere around move 5-Nc3 but I might start playing Nd2 more instead to get a different dynamic in the game.
Is this advised? Does this have dire consequences in QGA?
Unless you have a good reason, why would you block your own queen bishop and release tension on the important d5 square?
The idea of the gambit is to intensify pressure on the d5 square. Volunteering yourself away by Nd2 is unusual.
Agree with SmallChess answer. However, Nge2 ( you are talking about your Ng1 Knight, correct? ) is still problematic even if your bishop is out. In QGA lines where Black pushes queenside pawns the Knight on c3 often likes to retreat to e2 after ..b4 where the knight is well placed to spring back into action.