Is it OK per the rules to adjust your opponent's pieces?
The rules are clear:
4.2.1 Only the player having the move may adjust one or more pieces on their squares, provided that he first expresses his intention (for
example by saying “j’adoube” or “I adjust”).
4.2.2 Any other physical contact with a piece, except for clearly accidental contact, shall be considered to be intent.
When it is your turn, you may adjust the pieces. There is no distinction between your pieces and your opponent's pieces.
The next article makes it clear by implication that both your and your opponent's pieces are covered by this:
4.3 Except as provided in Article 4.2, if the player having the move touches on the chessboard,with the intention of moving or capturing:
4.3.1 one or more of his own pieces, he must move the first piece touched that can be moved
4.3.2 one or more of his opponent’s pieces, he must capture the first piece touched that can be captured
4.3.3 one or more pieces of each colour, he must capture the first touched opponent’s piece with his first touched piece or, if this is
illegal, move or capture the first piece touched that can be moved or
captured. If it is unclear whether the player’s own piece or his
opponent’s was touched first, the player’s own piece shall be
considered to have been touched before his opponent’s.
If you touch one of your opponent's pieces without saying "I adjust", or the equivalent in your language, then you must capture it. By implication, if you did say "I adjust" or "j'adoube", then you may adjust it.
What if he objects?
Call the arbiter. Make sure you ask for extra time (2 minutes) for the disturbance your opponent caused you. A good arbiter should give you this automatically unless there were extenuating circumstances.
When it is his move, he can readjust them to be off-center?
As long as he first says "I adjust" or "j'adoube".
Once, in a mischievous mood, when helping the arbiter set up the room before playing in a competition I deliberately positioned the knights, on all the boards I set up, either one facing forward and one backward or the left knight facing left and the right knight facing right (i.e. both knights facing away from the action). Most of the players, being at least mildly OCD, would have adjusted the pieces before the game, regardless, even if the pieces were perfectly placed :-).