Companion / XMP files

Older bugs which are supposed to be fixed in 0.84. *** Please try to reproduce your bug and confirm the bug fix. ***

Moderators: XnTriq, helmut, xnview, Dreamer

User avatar
m.Th.
XnThusiast
Posts: 1663
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Companion / XMP files

Post by m.Th. »

There is a setting in Settings saying "Show Companion Files" which is always enabled:
Companion-files.jpg
Please change this to the following:
Companion-files-mockup.png
Companion-files-mockup.png (6.18 KiB) Viewed 8834 times
With the obvious meaning.

Perhaps only Hide+Transfer needs explaining:

When this option is chosen, then the "companion files" (iow XMP) will be copied/moved/deleted if the main (master) file is processed in the same way.

The fix of this bug has more and more bigger importance because the "companion files" (ok, sidecars) are more and more used by the community and is a central part in the acceptance strategy for XnView MP.
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
User avatar
m.Th.
XnThusiast
Posts: 1663
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: Companion / XMP files - bug

Post by m.Th. »

I think that is much better to allow the user to customize the extensions for the companion files (with basic wildcards support). Something like this:
Companion-files-mockup.png
Companion-files-mockup.png (18.14 KiB) Viewed 8810 times
...because this will allow to support not only custom sidecars (as above RawTherapee's sidecars), but also multi-volume archives (above: Zip and Rar examples) as well as XnFrame files.

I think that this will be a quite powerful thing.
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
TD
Posts: 29
Joined: Sun Dec 02, 2007 11:41 am

Re: Companion / XMP files - bug

Post by TD »

Me too, I would really like to see this implented. I always use(d) the companion files configurations in the .ini when I work(ed) with the trad. XnView.

Regards,
t3sla
Posts: 2
Joined: Sun Jun 21, 2015 12:11 am

Re: Companion / XMP files - bug

Post by t3sla »

I'd like to pull this up again and would also love to see this feature implemented! Would like to be able to delete JPG and RW2 from my camera simultaneously!

Otherwise it's an excellent viewer under MAC with just few crashes every now and then!
dev3d
Posts: 17
Joined: Sat Sep 28, 2013 11:39 pm

Re: Companion / XMP files - bug

Post by dev3d »

+1 for CopyCompanionFile and companion extensions (eg. "Companion_00=jpg:" ) support for XnView MP, at least in .ini, and ideally exposed in Options window as m.Th. proposed.

The fact that this feature is implemented for XnView is the entire reason I began using XnView in the first place, and XnView MP's lack of support for this is the primary reason I haven't yet switched to XnView MP.
I think m.Th.'s proposal for how it should be exposed in the Options window is an excellent one.
However, please at least start with adding support for this in the XnView MP .ini file like its supported in XnView soon, allowing custom companion file extensions to be defined there, even if you don't yet have time to expose that setting in the Options window yet.

I use XnView to manage a library of media, using "Move/Copy To" to move the video preview thumbnails (such as generated using Auto Movie Thumbnailer (AMT)) or cover image, such as Movie 1.mp4.jpg and have the corresponding video file automatically moved together with the preview (by having "Companion_00=jpg:" and "Companion_01=:jpg" in XnView.ini).
User avatar
helmut
Posts: 8705
Joined: Sun Oct 12, 2003 6:47 pm
Location: Frankfurt, Germany

Re: Companion / XMP files - bug

Post by helmut »

In topic RAW+JPEG+XMP linked applying Copy/Move/Delete on both companion and accompanied file has been requested by Sirio, igorek7, Kris, m.TH. and some more users. m.Th. has even linked to this topic and I've decided to continue discussion, here.

Visibility of companion files
Currently, there is an option "Show companion files" in Tools » Settings | Browser » File list | File list.. With this option you can set whether you see the companion files or not. (m.Th. wrote in the original post that this setting is always disabled, is this correct? If yes, why is it always disabled?)

Linking actions "Copy/Move/Delete"
The companion files (JPEG and XMP) are a essential part (extensions) of the RAW image and they belong together. Linking the actions Copy/Move/Delete is a good, very reasonable and IMPORTANT request and feature.

New setting for linking actions
It's pretty clear that the user should have full control whether Copy/Move/Delete are applied on companion and accompanied file and a setting is needed. In the above posts pretty detailed drafts for a new setting have been made which is good for discussing things. Though, instead of a combined combining setting for both visibility and action I'd suggest a new, separate, and independant setting. Naming would be something like "Apply Copy/Move/Delete on companion and accompanied file". Sure enough this new setting should sit next to the setting Show companion files.

Where to place the new setting?
Already, there was some discussion were to place the settings controling the visibility and the linking of Copy/Move/Delete action. The companion files could be considered as meta data for the RAW file. On the other hand Copy/Move/Delete are file actions. There is "Browser » File list" and I wonder whether the "File list" shouldn't be it's own main category because of it's importance. This and the location of the new setting will need some more discussion and thoughts.
EDIT:
As I see now in m.Th.'s initial draft, there are even extensions listed. And if I remember correctly, currently there are UI less settings to set the relation ship. These relationships should be made visible to the users in the settings. All in all the companion files are an important and strong mechanism which might even deserve their own sub-category and tab in the settings.

Other UI modifications required?
By default, this new setting should be set. To avoid disasters and unexpected deletions, the dialogs for copying/moving/deleting might need some enhancement. But I'm not sure here because it's more or less natural, clear, and even expected that the Copy/Move/Delete actions are applied on the RAW file are also applied on the companion files.

Info icons
We have info icons for various purposes. E.g. the user can see whether a file has metadata or not. Info icons for companion files might be an option to make things even clearer.

Please note that I'm not very familiar with RAW and companion files. I just try to pick up and summarize ideas and draft possible and implementable solution.

Discussion is re-opened. ;)

Last not least: These are important suggestions - but not a bug. :arrow: Suggestions
User avatar
m.Th.
XnThusiast
Posts: 1663
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: Companion / XMP files

Post by m.Th. »

For the ones who know about the sad marriage between RAW and companions feel free to skip this post. I will post another below on-topic.

Please note that I'm not very familiar with RAW and companion files. I just try to pick up and summarize ideas and draft possible and implementable solution.
For helmut: There are at least 2 (two) pretty common and very important cases:

1. Professional cameras (many, many dSLRs) have a mode called RAW+JPEG. This mode is used for various reasons: for example one wants to have the JPEG readily available to send it somewhere (usually to a news/corporate web site, a coworker etc) or as a preview in Explorer (or somewhere else) but to have also the full image data in RAW files.

2. Image metadata. RAW files are considered „valuable assets” and, hence, untouchable. That's why they are treated read-only and, thus, their metadata cannot be changed. Ok, sometimes we can apply a star/color rating, but definitely categories (keywords in other DAMs) and Raw edits (XnView MP cannot do it but there are some very well established non-destructive Raw editors: Lr, ACDSee, CaptureOne, DxO Optics Pro, ASP etc.) are stored in these „companion files” called also sidecars. They are stored in XMP format (a variant of XML).

Hence we have two categories of companion files:

- different formats (usually a slim JPG next to a much heavier format like a variant of Raw or TIFF).
- XMP sidecars (for metadata & edits)

There is a stringent need for a graphic file manager to manage these files in a proper way: - separate (we have this already) and together (this is our topic now).
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
User avatar
m.Th.
XnThusiast
Posts: 1663
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: Companion / XMP files - bug

Post by m.Th. »

helmut wrote:In topic RAW+JPEG+XMP linked applying Copy/Move/Delete on both companion and accompanied file has been requested by Sirio, igorek7, Kris, m.TH. and some more users. m.Th. has even linked to this topic and I've decided to continue discussion, here.

Visibility of companion files
Currently, there is an option "Show companion files" in Tools » Settings | Browser » File list | File list.. With this option you can set whether you see the companion files or not. (m.Th. wrote in the original post that this setting is always disabled, is this correct? If yes, why is it always disabled?)
It is disable. Dunno why. Ask Pierre. Most probably because the underlying code is disabled/inexistent/has a problem also.

Linking actions "Copy/Move/Delete"
The companion files (JPEG and XMP) are a essential part (extensions) of the RAW image and they belong together. Linking the actions Copy/Move/Delete is a good, very reasonable and IMPORTANT request and feature.
RENAME.
You forgot Rename.

New setting for linking actions
It's pretty clear that the user should have full control whether Copy/Move/Delete are applied on companion and accompanied file and a setting is needed. In the above posts pretty detailed drafts for a new setting have been made which is good for discussing things. Though, instead of a combined combining setting for both visibility and action I'd suggest a new, separate, and independant setting. Naming would be something like "Apply Copy/Move/Delete on companion and accompanied file". Sure enough this new setting should sit next to the setting Show companion files.
Don't forget renameeee.... :)

Having two settings - hummm... ok for me. Depending on code architecture it can cause problems to Pierre. But it is ok for me.
Perhaps is better to note here that in other programs - for example in the absolute standard which is Lr - the user cannot control this: the sidecars are hidden and processed automatically.

Btw, I like your expression: „instead of a combined combining setting”.... :)


Where to place the new setting?
Already, there was some discussion were to place the settings controling the visibility and the linking of Copy/Move/Delete action.
rename, ReNaMe, re-na-meeee... tralalala...
The companion files could be considered as meta data for the RAW file.
Companion files ARE the metadata for the RAW file (and not only)
On the other hand Copy/Move/Delete are file actions. There is "Browser » File list" and I wonder whether the "File list" shouldn't be it's own main category because of it's importance. This and the location of the new setting will need some more discussion and thoughts.
Yes, most probably - many users will search for it. If we will have just few controls then no new tab is needed (IMHO) - just to put them in the 1st tab in a prominent place.

EDIT:
As I see now in m.Th.'s initial draft, there are even extensions listed. And if I remember correctly, currently there are UI less settings to set the relation ship. These relationships should be made visible to the users in the settings. All in all the companion files are an important and strong mechanism which might even deserve their own sub-category and tab in the settings.
As I said above, I agree. Please note that the very same mechanism can (and should) be used for other kinds of „companion” files - in my example from my OP I listed 3 (three) use cases:

- XMP metadata sidecars.
- Raw edits (PP3 is the extension in which Photivo saves its edits - Raw Therapee does something similar)
- multi-volume archives (requires ” * ” (wildcard / asterisk support) - but this isn't a big deal - usually there are functions for this string search case)

Other UI modifications required?
By default, this new setting should be set.
Sure.

To avoid disasters and unexpected deletions, the dialogs for copying/moving/deleting might need some enhancement. But I'm not sure here because it's more or less natural, clear, and even expected that the Copy/Move/Delete actions are applied on the RAW file are also applied on the companion files.
No need to change anything. Just copy/move/delete/rename them. The same set of business rules apply (on name conflict etc.). (it the setting is enabled) They are the same „entity”.


Info icons
We have info icons for various purposes. E.g. the user can see whether a file has metadata or not. Info icons for companion files might be an option to make things even clearer.
Yep, better to have an icon. However, I think that we are already covered(???) for XMPs. If not, then if the „Hide” is enabled a small „companion” icon should appear...


Please note that I'm not very familiar with RAW and companion files. I just try to pick up and summarize ideas and draft possible and implementable solution.
A brief intro posted above. :)


Discussion is re-opened. ;)

Last not least: These are important suggestions - but not a bug. :arrow: Suggestions



Nonononono....! This is much more a bug - and, also, can be easily cataloged to „you might lose your work”. There were countless examples in which people worked a lot to edit their Raw and after that sent only the Raw file itself to the printing house, making backup only the Raw file itself (loosing BOTH edits AND metadata), having a multi-volume Rar archive and sending only the 1st volume aso. aso. aso.
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
User avatar
Sirio
Posts: 128
Joined: Tue Oct 13, 2015 3:22 pm

Re: Companion / XMP files

Post by Sirio »

Excellent summarize, helmut .
I would like to precise one thing: Did the companions files get an hierarchy? Is it the RAW the master and JPG, xmp the slaves?
I mean, if I select the raw and delete it, XnViewMP will delete also the jpg and the xmp, ok. But if I perform this action on the JPG, It shouldn't do the same thing with the two others companions? And what about if I do this action on the xmp?

Therefore, yes for an info icon to inform us, when by e.g. I hide the raw and xmp with the filter.
Absolutely YES for a warning pop-up asking confirm to apply the action on all, or one or two of the companion files. (This warning pop-up could be disabled in the options)
Then, there could be a toggle switch (on-off) on the toolbar to perform the same action on all the companion files or not.
Sirio
dev3d
Posts: 17
Joined: Sat Sep 28, 2013 11:39 pm

Re: Companion / XMP files

Post by dev3d »

When it comes to deleting companion files, I would suggest providing the following radio button options, with defaults as shown:

Delete Companion files (eg. .XMP) when deleting Master file (eg .JPG):
(•) "Yes (or Shift+Delete to Prompt)"
( ) "No (or Shift+Delete to Prompt"
( ) "Prompt (Shift+Delete to automatically delete)"

Move/Copy/Rename Companion files (eg. .XMP) when deleting Master file (eg .JPG):
(•) "Yes (or Shift+{Hotkey} to Prompt) (eg. Shift+F2, Shift+Alt+C, Shift+Alt+M)"
( ) "No (or Shift+{Hotkey} to Prompt"
( ) "Prompt (Shift+{Hotkey} for automatic Yes)"


Delete Master files (eg. .JPG) when deleting Companion file (eg .XMP):
( ) "Yes (or Shift+Delete to Prompt)"
( ) "No (or Shift+Delete to Prompt"
(•) "Prompt (Shift+Delete to automatically delete)"

Move/Copy/Rename Companion files (eg. .XMP) when deleting owner file (eg .JPG):
( ) "Yes (or Shift+{Hotkey} to Prompt) (eg. Shift+F2, Shift+Alt+C, Shift+Alt+M)"
( ) "No (or Shift+{Hotkey} to Prompt"
(•) "Prompt (Shift+{Hotkey} for automatic Yes)"


I suggest Shift+Delete as another hotkey to perform the opposite action, as it may be familiar to those used to using Shift+Delete on Windows to Permanently Delete a file. The opposite action could be an automatic always/yes if default to prompting or showing a prompt if default to anything else.

You could create separate options for just Rename or for Rename, Move, and Copy if preferred.

I suggest always moving/copying Secondary/Slave Companion files by default, probably deleting them by default too (or else Prompt), and either using the same defaults for the reverse (Move/Copy/Delete/Rename Master when doing that to the companion/slave) or else just always defaulting to prompting in that case. For myself, I will change all those to default to Yes (otherwise, what's the point of supporting companion files?) and just use Shift+Delete, etc. in few rare cases as needed.

Also, you could provide a single checkbox at the top of a Companion Files page in the Options dialog to Disable Companion Files, for those who don't want this behavior at all, and then just have some common companion file associations supported out-of-box otherwise, and, to play it safe, could even default to Companion Files being disabled out-of-box, just as long as you keep default file/extension associations defined). Alternatively, just disabling each of the few enabled-by-default Companion File Name/Extension Association pattern/filters could suffice.
dev3d
Posts: 17
Joined: Sat Sep 28, 2013 11:39 pm

Re: Companion / XMP files

Post by dev3d »

As discussed earlier in the thread, the only reason I am forced to still use XnView Classic instead of XnView MP is the lack of companion/sidecar file support, so it would be really helpful if this is implemented ASAP (at least in XnView.ini at first, and ideally exposed in Options after that), as its the last remaining major missing feature and roadblock to using XnView MP IMO.


When implementing companion file support for XnView MP, please be aware of the issues and limitations that currently exist with XnView Classic and avoid them. I have detailed these in the "Sidecar/Companion Files and Move/Copy/Rename Dialog Issues" thread.

The limitations/issues further detailed in that thread regarding Companion/Sidecar file handling, options, and dialogs in XnView Classic cover how XnView:
  • fails to always move/copy/delete/rename all companion files together (and in all directions)
  • fails to move/copy/delete/rename consistently regardless of whether full-screen preview, browser or preview pane is focused and whether any Preview pane is visible
  • incorrectly renaming companion files via F2 Rename dialog (with multi-dot extension handling, and depending on which companion file was selected)
  • has limitations in the way that companion file extension associations are defined, especially with regards to supporting multi-dot extensions and companion files that can apply to multiple different video file types (.mkv.jpg, .mkv.lnk, .lnk, .jpg, .mkv, .mp4, .wmv, etc.).
dev3d
Posts: 17
Joined: Sat Sep 28, 2013 11:39 pm

Re: Companion / XMP files

Post by dev3d »

I suggest having settings for Companion files under a dedicated options page at Settings > Browser > Companion files. Though this could potentially be a tab nested under File list as it regards file types.

This settings page could also be a tab under Metadata but that may not be as obvious or likely to looked at in general or when going through lists of file types/filters like under File lists, especially if using it for other companion file relationships like video thumbnail images and shortcut or nfo or DVD cover images or playlist/bookmark files, etc. Therefore, I wouldn't suggest it being hidden as a tab under Metadata since companion files aren't necessarily limited to just Metadata use (eg. video thumbnail files).
I suggest providing a number of common and potentially useful companion file relationships defined out-of-box, with the ability to easily enable/disable each via checkbox instead of just deleting and recreating or forcing users to have all potentially useful out-of-box ones enabled by default.

Companion File Associations:

| Master File | Companion File | Enabled | Description/Notes | Bidirectional (Treat Companion like Master)
.jpg | .xmp | X | XMP Image Metadata
.jpg | .exf | X | JPG EXIF Metadata
.jpeg | .exf | X | JPEG EXIF Metadata
.{Ext} |.{Ext}.jpg | | Video Thumbnail Previews
.{Ext} | .jpg | | Video Thumbnail Previews without Video Extension
.{VidExt} | _thumbs.{ImgExt} | | Video Thumbnail Previews with _Thumbs Suffix
.{Ext}.lnk | .{Ext}.jpg | | Video Thumbnail Preview for Video Shortcut
.{Ext} | .pbf | | PotPlayer Bookmarks File


You could provide a grid which includes file name pattern with at least {Ext} placeholder support (possibly {Filename} and/or {VIdExt} and {ImgExt} placeholders too later, though not necessarily needed). This could include out-of-box description/notes column to clarify the usage and which could be user-editable. Most importantly, this would include a checkbox for whether the relationship applies in both directions (equal partners instead of master-slave, so Delete/Rename Master file options described above apply in both cases instead of Delete/Rename Companion file options). I suggest just having the first few (shown with X after them below) as enabled by default, while providing many defined and easily able to be enabled, including those shown below:


It may be helpful to support {VidExt} {ImgExt} in addition to {Ext} which is placeholder for any extension, or could even support this for all categories under File Types like Videos, Images, Audio, Custom, etc. In worst case, just supporting {Ext} as placeholder for everything would be good enough for my case and to start with, however.

Whatever the case, I find the current file extension pair model supported in XnView Classic's XnView.ini file, like shown below, to be very limited - not just in failing to support bi-directional relationships, but also in that its unclear when/how to use a "blank" extension and it can't support file name suffixes or multi-dot extensions or file name patterns, etc. I have the following defined in XnView.ini for XnView Classic, though even this doesn't always work as expected or in all directions.

Also, I can't define a general relationship like .jpg file or .pbf can exist for any video file (or even any file of any kind at least), without defining dozens of relationships. And, its impossible to move a .mkv file and have the associated .mkv.jpg move with it, even though you can move .mkv.jpg and have the .mkv moved with it, despite the fact that I define the rule reversed ("jpg:" and ":jpg"), so the current XnView Classic implementation isn't very consistent as-is.

[File]
Companion_00=jpg:
Companion_01=:jpg
Companion_02=pbf:mp4
Companion_03=mp4:pbf
User avatar
Sirio
Posts: 128
Joined: Tue Oct 13, 2015 3:22 pm

Re: Companion / XMP files

Post by Sirio »

:shock: dev3d, it's so complete and very interesting, but above all so complicate, No?
I mean, Lighroom manage the companions files, but have nothing of theses options.
Obviously, if we can do better than Lightroom, is the top, but more complicate mean more risk to make errors too.

I think that XnViewMP could begin easy managing the companion files. It's just my opinion.
It could be more simple a settings like this two:

Companions files linked for delete action:
(•) Yes, always
( ) Yes with warning pop-up (In the warning pop-up, it could be the possibility to check which files should be deleted.)
( ) No.

Companions files linked for copy/move/rename action:
(•) Yes, always
( ) Yes with warning pop-up (In the warning pop-up, it could be the possibility to check which files should be copied/moved.)
( ) No.
Sirio
Keith Curtis
Posts: 5
Joined: Tue Sep 27, 2016 1:51 am

Re: Companion / XMP files

Post by Keith Curtis »

I have been trying for the last two hours to hide the xmp (companion) files and as stated above the option in "File List" to "show companion files" is ticked but greyed out.

Does this mean that unlike any other program I am stuck with them always showing in my Raw file folder or have I missed something?

Cheers
Keith
User avatar
Sirio
Posts: 128
Joined: Tue Oct 13, 2015 3:22 pm

Re: Companion / XMP files

Post by Sirio »

Hello Keith Curtis
If you want to hide the xmp companion file or any type of file, do this it in the options:
- Tools> Settings> Browser>File list> Custom filter. In the string "Exclude"
add at the end of the line all the extensions that you wouldn't like to see.
e.g.:"xmp RAW RW2 ORF DNG PEF CR2 RAF ARW sfk sfl sfvp0".

In order to activate this filtering, then go in the menu: View> Filter by> Custom.
Sirio
Post Reply