When you have two major pieces (the major pieces are rooks and queen, so two major pieces would be two rooks or a rook and a queen), the basic checkmate pattern is called "ladder mate". This is where you force the king to the side of the board by taking away more and more space. In the second position that you posted, a good first move is to move your rook to c5. Once you do this, the king can't more to the left or below your rook because those spaces are attacked by your rook, and it can't take your rook because it's protected by the pawn. So once you do this, the king has only twelve spaces it can move two, and only three files.
Once you've done that, you can take away more files from the king, and once none are left, it's checkmate. In ladder mate, keeping your pieces away from the king makes it simpler, both because it keeps the king from threatening your pieces, and because it makes it harder to inadvertently cover the spaces around the king and give stalemate. In the stalemate position you give, both of your major pieces are just a knight's move away from the king, which means that they are covering a lot of spaces near it. They are also on opposite sides of the king, which makes it worse. You should try to keep your pieces on the same side of the king. That way, not only do they protect each other, but there is more overlap between the squares they are attacking, which means the total distinct squares attacked is lower.
Going back to the Rc5 move, once you've done this, you can move your queen to d5 and then to b2. Now the king can't stay on the b file, and it can't move to the c file because of the rook, so it has to move to the a file. You can then move your rook to c1 and then a1. This will be checkmate because the rook is taking away the a file and the queen is taking away the b file. This isn't the fastest checkmate, but it's one that can be found with simple rules.
If you let the enemy king get among your material, then it can use them to block attacks, and they get in the way of your major pieces, making ladder mates more difficult. For instance, if your knight weren't there, there'd be a simpler mate of Qd4, Rc3, Qf2, Rc1. If this happens, your options are to just let the king take the material, try to move the material away, or try to force the king away. In the case of pawns, you can just push them, and either the king will capture them, giving less cover, or eventually you'll promote them, giving you more pieces to deliver mate with. Just be careful that when you promote a piece, it doesn't create a stalemate.
If you were to promote all four of your pawns to queens, you'd have five queens, and at that point you'd have enough pieces that you could easily follow another tactic to avoid stalemates: make sure that all of your moves are checks, but don't give away material.