I think you cannot accurately calculate the probability a given position has been previously played by someone, somewhere. But you can make a rough estimation with a big game database.
To calculate this probability you need a really big database -I'd say it should contain some million games-. And you should have a good representation of games played by players of different levels and historical periods. Then using a database manager (like Scid), and search how many games in your database contain that position. Dividing the number of games containing the position by the total number of games, you have a rough estimation of the probability it has been previously played.
This is a rough estimation because low-level players are not very well represented in the databases, because there is less information of games played in the past and because non-annotated games (blitz, friendly games) do not appear in the database.