I just play 3 to 4 games in day(sometimes online) I am average chess player with rating 1400(rating is not fide rating. It is rating given by chess.com). Will my rating increase with time? Or I need to join classes or chess clubs to improve my game?
If your rating has stagnated (not changed in a long time) then you need to change your approach. What you listed will definitely help. Even if your rating is still improving, those things will make it improve faster... or they may make your rating decrease initially as you try new things.
You can also learn by yourself without classes and clubs by using online and printed resources. However, an organized class may lead you to learn things you would not otherwise find on your own. Likewise, learning on your own will help you focus on the things you like most about the game. Do both.
At the end of the day - why do you play chess? If to have fun - why do you care if your rating improves? If to improve - why not invest yourself into classes, tactics trainers, and additional resources.