If you are in a church that does not follow the Bible in its meetings, then the question you will need to decide for yourself is whether you should leave that church or stay and try to change things. I think the answer depends on the situation. If it's an "institutional" style church that has one man at the top of the authority structure, and he derives his living from being the only speaker, or at least the main speaker, then I believe it's very unlikely that you can do much to change the status quo, and you should just quietly leave. If they call Sunday "the Lord's day," observe Christmas and Easter, and have a hierarchical authority structure with one man at the top, then that church is entrenched in following Roman Catholicism and has very little regard for the authority of the Bible, regardless of what their official statement of faith may say. There's no chance that you'll be able to change things. Just quietly leave. If anyone treats you badly as a result, remember to respond in love, and be patient with those who do not yet see the truth.
However, if your church has already "come out from among them" and is doing some things right, then it may be a good idea to stay and try to change things. Some "house churches," for example, already do some things right, such as allowing multiple men to speak, etc.. And they may be doing only some unbiblical things, such as meeting on Sunday instead of the sabbath, allowing women to sing in church (which is just speech set to music, and is forbidden by 1 Cor. 14:34,35 and 1 Tim. 2:11-14), etc.. If that's the case, I'd say it's worth at least presenting your concerns at the next meeting, and seeing if you can persuade people. If what you say is rejected, then just politely and lovingly explain that you're going to have to quit attending, and then do so.
If you do have to leave your current church, you may be completely alone at first. Your friends in your current church will usually not understand. They may think that you've lost your way, or they may think that you left for reasons other than what you tell them. Remember to respond in love. Be prepared to be ostracized, and be prepared to be patient and loving to anyone who may be unkind to you. But on the other hand, also be prepared for blessings to come your way. God honors those who honor him. He blesses those who obey his commandments. In 2005 when I first started remembering the sabbath day to keep it holy, blessings came to me immediately, and my life changed drastically within one year. Click the About Me section of this website to see what I mean.
If you do have to leave, and you find yourself alone, then try to find at least one or two other men who will meet with you every week, and let that be your new church. Jesus, speaking about church, said in Matthew 18:20, "For where two or three are gathered together in my name, there am I in the midst of them." Also, it says in 1 Cor. 14:29 to let two or three men speak, so you've got to have at least two or three men present so they can speak in your meeting in order to follow that commandment. You may not have a fancy name and a big building to meet in, but God will consider your meeting "church," and that's all that counts, isn't it?
If you do find yourself alone at first, without anyone else you know of who is willing to obey what the Bible says about church, then there may be some in your current church who will accuse you of "forsaking the assembling of ourselves together," as it says in Hebrews 10:25. But that's not your situation, so don't worry about what they say. If you can't find anyone else who wants to obey what the Bible says about church, then what choice do you have? You can either join with those people in disobedience to God's commandments, or be without the privilege of having church for a while in your life. Look at John's situation when he wrote the book of Revelation. He was exiled to the island of Patmos because of speaking the word of God, and for testifying about Jesus Christ. And it seems that he probably didn't have any other believers with him, so he couldn't assemble with anyone on the sabbath day for church. He wasn't forsaking assembling with other believers; his circumstances just didn't allow it at that time in his life.
If you are able to find others with whom you can meet, you may find it convenient to meet in one person's home. There are several mentions in the Bible of people who held church meetings in their home. But there is no commandment that says you have to meet in someone's home. By the way, remember to focus on the things about which God has given commandments, and place less emphasis on everything else.