The Exchange Caro-Kann isn't considered critical because of one equalizing line:
And the queen doesn't have a good square as 14.Qa3 Qxa3 leads to an equal game. If white doesn't take on b7 on move 12 (which is usual) then white has the kingside attack but black is left without a bad piece and chances are equal.
I don't play the Queen's Gambit exchange lines (there's a line that is pretty slow where white plays Nf3 and a minority attack, and also an aggressive like where white plays Ne2, f3, and eventually e4 which was considered pretty dangerous last I knew) but there black often struggles because he has a difficult time finding a good square for that light square bishop. See John Nunn's 'Understanding Chess Move by Move' for a fantastic discussion on this - I'm remembering this vaguely ten years after reading the book. These might not be considered the 'most critical' but I think that Grand Masters still play this opening looking for good results, though perhaps not as often as the most mainline openings.
Fischer played that Exchange vs the Caro a few times but this was before someone - I don't remember who - found that Bd6 move. White has a perfectly good general plan in that line but it doesn't give any pressure based on a single clever reply.
I think the main point is that it's hard to know what will really happen in an opening until you see specific moves and plans and play over many games. 'Critical' openings are based on specifics which have been found during the last 150 years of competition between top players, and can be hard to predict based on the third move position.