Article 4.3 of the FIDE Laws of Chess (the touch-move rule) also applies if the opponent is not present. But let's split your question up a bit:
Imagine this situation, the opponent is not present on the board (far
away), you touch a piece and the opponent (or somebody else) claims
that you have to move that piece, can you deny it?
In dubio pro reo. (When in doubt, for the accused.)
Unless there is any other evidence (e. g. the arbiter or a spectator observing you), if you deny having touched a piece, the arbiter should declare that there is not enough evidence to support your opponent's claim (the burden of proof is on the claimant).
Not admitting that you touched a piece when you have in fact touched it, however, is one of the most immoral things you can do in a game - exactly because of this issue.
My idea is, while the opponent is not present on the board, it is OK to touch the pieces, but does the rule have anticipated this situation?
I would suggest a different approach that avoids all confusion: Call the arbiter to your board and tell him you intend to correct the position of some pieces - that's what the arbiter is there for. If no arbiter is in sight, a more practical approach would be to say "I adjust" clearly while e. g. making eye contact with a neutral bystander to make sure you can prove you did not intend to move a piece.