Having the map filled with NPCs will be the problem now, yeah. Better just limit the number of NPCs/area, noone wants a bunch of level 1 NPCs

Reducing food upkeep for troops and/or increasing farm production should be done as well.