First, I'm not convinced that engines are necessarily weak without their books. Without an opening book, modern engines might not play the most testing lines, or lines that would get them into a position where they thrive, but any top engine will still crush any human GM without an opening book.
As an interesting aside, if you leave your engine on for a while, it usually finds the Spanish Four Knights as the best line from the starting position, but I digress...
In the endgame, it totally depends on the engine and the position. Without tablebases, engines might not know that a rook pawn and wrong bishop is only a draw. If the programmers added that exception, then they'll figure it out, but otherwise, they might trade down thinking that they'll win with an extra bishop and pawn only to draw via the 50 move rule.
In more elementary endgames, like KBNvK, the stronger side will almost always win. The longest mating sequence is 33 moves if my memory serves. Most engines will stumble onto a forced mate in that time because their heuristics will have them make good moves that get closer to mate (and they can probably see 15-20 moves deep).
In terms of KQvKR, the stronger side will almost always win against a human. Humans are pretty bad at defending this position, and the computer will instantly see ways to pick off a loose rook. Against another computer, it's a little bit harder, but as Wes says in his answer, it is still likely that the stronger side would win (the Philidor maneuver is only ~5 moves long, plenty short enough for the engine to see it) given enough time.