This seems to happen intermittently.  Restarting helps.

This griefer seems to be dragging around an invisible army with his totem.  While your totem remains unmoved, your tribe reacts to his (invisible) troops and stops whatever they're doing.  You can unfreeze things by moving your totem but as soon as you set it down again you're locked back in place.