Your comments

But why would they do that? When I spend an hour building a castle and putting archers on the towers so I don't lose walls every time an opponent with a hammer looks at my castle, I don't want my sentries to desert and waste my resources and time by getting themselves killed out in the open. If I can't make a castle keep getting stronger the more work I put into its construction then that's defeating the point of castle mode.