What you describe is known as, "half the time when we play chess." We simply get outplayed. You are probably not getting caught by clever openings. You are probably making poor moves and your opponent is seizing the opportunity.
What are the ratings of your opponents who frustrate you?
White has a slight advantage at the beginning, but this advantage is really only realized at the higher levels of the game. The reason is that amateurs waste a lot of moves so white's opening advantage goes away quickly. Don't get too hung up on who is supposed to attack and who is supposed to defend.
Learning openings is good, but don't start memorizing a bunch of openings. Your peers won't follow the script and you'll be as lost as they. Instead, focus for now on opening principles. For example, control of the center, castling, not moving the same piece twice, not bringing the queen out too early. If you've never heard of these things, we can have a worthwhile discussion.
It's hard to give concrete advise with so few details.
One thing you can do is, after you lose, ask your opponent to review the game with you, so you can see what they were thinking. Be prepared to do the same for your opponent when you win.
You can also ask a 1700-ish player (400 points above you) to review your game. They will probably be able to point out issues with your play.
edit - how to capitalize on the opening principles
To show how the opening principles can work, there's no better example than to look at the most popular e4 opening - the Ruy Lopez. This is a common variant.
[FEN ""]
1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Ba4 b5 5. Bb3 Bc5 6. O-O
White and black both claim the center with the their first moves. White's second move develops a piece and attacks an unprotected black pawn. That's 2 good things. Black responds by developing a piece and defending the pawn. That's good too. White then develops a piece and attacks the Knight on c6, threatening to win the e5 pawn. Black is unimpressed and kicks the Bishop. In this variation, White opts to retain the Bishop and moves back. Black kicks the Bishop again, gaining space but perhaps weakening his pawns a bit as the b-pawn will be unable to support the c-pawn. White retires the Bishop to a pretty good square (this attacks the center, hits the weak f7, and probably Black's castled King, soon enough.) Black then develops a Bishop. White wastes no time in castling, which defends the chronically weak f2 square. Every move has a purpose, almost every moves has a threat.
We see that White moved his Bishop a lot, and we're not supposed to do this, right? The reason it is ok is that each time, there was a purpose to the move. Further, White knew the Bishop could safely end up on b3, and that square looks right at the place where Black is likely to castle.
Also, don't freak out about the ratings. They're always an approximation and very variable at that level. You should be able to defeat a 1400-rated player about 40% of the time.
