Using category system: Set category SET, set category to NOTSET multiple categories for filtering

[nji9]

Usecase:
I want to show all files with category A set, and all other categories to NOTSET.
Select the category A.
Then setting others to UNSET is buggy:
If I press key ALT und use the left mouse hold and draw above other areas,
sometimes it does completly unexpected.