I suggest to add these items for Tree pane > Categories (as another item):
"Remove from this category" for Category
"Unrated" for Rating
"Uncolored" for Color label
Perhaps also:
"Untag" in Tag, as subitem
This way would be more intitive, just drag the file(s) to remove from category, rating, colour label.
Currently it's quite confusing, for category, remove is possible by context menu, for rating and colour label by context menu, sub-item, for Tag only in main menu and by shortcut(s).
Remove for Category, Rating, Colour label tree pane items
Moderators: helmut, XnTriq, xnview