It's only when you upgrade your TH to level 5 that you lose the BP early...up to level 4, you're fine to build.