One thing few players and DMs considered throughout many campaings we've played before is that bandits also have survival instincts and would rahter flee from a losing battle if they could. Another thing that was largely overlooked was that bandits also have memory...
That being said, those are surprisingly polite bandits though...