Becoming a game developer does not mean you have to be super smart, or great at maths, ok math does come into the programming part but nothing crazy. Its all down to you and what you want to programme or build. If you want to start building games on your own, its easy! and FREE! there is tons of information on the internet to get you started, but don't overwhelm yourself with too much to fast, it can be easy to loose track and give up. To get the best use out of this post follow each step in order.
First thing first, download Unity 3D a super powerful game engine that will let you build the games you want. Learn the unity layout, get used to the interface of the programme and get familiar with it as much as possible! because this will be your work station. There is tons of tutorials on YouTube or google it. You do not need to know everything, just get familiar and the rest you will learn as time comes.
Now this will be the hard part, but not as hard as you may think. Programming is just basic logic, learning the keywords and putting together code is like a puzzle, you just need to learn what the pieces look like before you put them together. If your working in Unity 3D then learn C# (C Sharp), its easy to learn, very popular and it is one of the best languages out there. Start with the easy stuff and eventually you will get the hang of it. Sticking to it can be hard but with each word you learn in programming is one more thing you can manipulate in the 3D world you want to build.
Modelling is not so hard, if your a really creative and arty person you will find it a lot easier to do. Unity 3D has basic shapes you can manipulate, but nothing advanced. To build that really cool house, or weird shaped car then you will have to use another programme. There are tons of different programs to use but Blender is one of the best, it works well with Unity 3D and controls are very similar, once again get used to the layout get familiar then start building, exporting and importing your 3D models.
Where to start?
Once you have a basic understanding of all of the above, you can start thinking about your cool idea's on what you want to make, unless you already know what you want to build. Most developers will tell you to start small, don't! start however you want, whatever makes you feel comfortable, because becoming frustrated before you even start is not what you want. Build what you want and then work on it, you will learn more on the way and slowly become the developer you want to be.
Always learn and never give up
Being a developer will be stressful, it will be hard but if it is what you truly want to do, then do not let anybody tell you different. Become the Man/Woman you want to be. All developers are different and will have there own opinions, meeting new developers is a great way to learn more and help you get others opinions on your work.