Your hero may have abandoned you, it depends on their loyalty. I try to keep my heroes loyalty maxed out at 100, all of the heroes loyalty.