I always thought that them joining you was an intentional mechanic, I managed to recruit 5 barbarians (wandering ones) after helping them in a fight and won a 2v1