You can recall your rally troops from marching toward the destination. Only the rally captain can do this recall.