As others have mentioned in previous posts here, Tomatillos are LARGE plants, & thus need LARGE containers. They also need lots of water & good soil with lots of organic matter.
If you suspect your containers are too small, they probably are. Removing fruit probably won't help. Your plants simply don't have enough room to bring the fruits to maturity. They're most likely dropping the fruits prematurely out of survival.
If the plants themselves look healthy, I'd increase watering (I'm assuming you already know not to let the pots go completely dry) & perhaps add some fertilizer. Container plants - vegetables in particular - require a lot more feeding than garden plants.