Missing Islamic Prayers?

Reverting to Islam almost 10 years ago, I've found it difficult to practice my religion and have always slacked in making my salats. I was the only revert in my home, but I pursued Islam wholeheartedly. I loved Islam. Today I have to hide from my family that I still believe in Islam. My family believes Muslims= Terrorists. When I do pray, I quietly bring my children into my bedroom and make salat with them. This becomes burdensome to hide my beliefs from the rest of my family( mother, brothers, uncles and aunts). Still, when I miss one salat, then I miss two and pretty soon I'm not praying at all or going to the masjid. I start to live the life of an unbeliever. Still I have so much guilt for not living the life of a good Muslim. I've lost touch with my Muslim brothers. I fear their judgement and so I stay away from the masjid. Right now it is financially impossible for me to move out of my family's home and closer to a Muslim neighborhood. Moving to a Muslim neighborhood I think is my solution. Any suggestions or has anyone else been through this?