Keep the flag lists as IntSets rather than lists