There will always be haters. Surround yourself with people who will support, encourage, and understand you.
When I first started this fitness journey, many of my friends looked at me like I was torturing myself. They now realize it's not a phase and it's something that is a part of my life. They may not agree but at least they respect my decisions now.