Tagging JPEG creates xmp-sidecar

[oops66]

... In my workflow, I make a big distinction between image data and metadata. Storing image transformation instructions rather than burning them in is always a good idea, but this has to apply to the raw image, rather than a jpeg. To me, a jpeg is never "the original"...

Hello CameronD,
... You are right, it's better to use the Raw files as originals files (original image data + original exif metadata ...) ... but the majority of people shoots mostly in jpg (less hdd storage and time consuming), then in this case the originals data are the .jpg files.

My aim is for maximum interoperability between software packages, and to my mind that means I have no option but to embed the metadata within the jpeg image whenever possible.

Ok, in this case, I also think than it's the best way to do.

The last thing I want is to have to tick numerous options to turn that on, so if there is a single option then it should switch between:
Metadata: use sidecar files whenever possible, or
Metadata: embed within image file whenever possible

...Yes, in this case you can also have an option to tick all in one time "all write op." (and/or sub chooses to only tick the required metadata (ex): comments, icc, iptc, xmp, embedded thumb rebuild, etc ...)

With the list of override options, I can imagine a potential nightmare of situations brought about by: "When I write metadata X to image files of type Y then software Z does not read it, so I'll need to write that one the other way around".

... I see your problematic, in this case, it's the mess
EDIT, see here too:
http://newsgroup.xnview.com/viewtopic.p ... 85#p117985

[m.Th.]

The document which I promised is online at: http://is.gd/u5FIa8

I try to copy/paste it here:

---8<------8<------8<------8<------8<------8<------8<------8<------8<------8<------8<------8<------8<------8<---

About the settings in Metadata / IPTC & XMP: these settings are a little ‘fuzzy’ now. Let’s try to take a more systemic approach:

Metadata.png

We see that we have 4 metadata paths: Storage Import/Export, Export to Other Programs and Export to Internet.

We need the following options about what to update from/to XMP (either embedded either sidecar):
• Import Rating
• Import Color Labels
• Import Keywords/Categories
• Export Rating
• Export Color Labels
• Export Keywords/Categories
This should work globally: either the metadata is embedded either is a sidecar. For “Export Keywords/Rating/Color”, if the sidecar doesn’t exist and is needed, it must be created.

About how to update: We already have “Update Files from Database” and “Update Database from Files” as a manual means. We need one or two option(s):

• Automatically Export Metadata - When the RCK is changed, then the metadata is updated in a background thread
• Automatically Import Metadata - When the thumbnail thread encounters updated (newer “modified date” value) for sidecars, it will update the info in DB

About where to update: We need a drop-down box with the caption “Metadata write method:” with the values “Sidecars only”, “Try to embed in photos” and “Try to embed + sidecars”

Specifics about targets:
• Storage – it should work “as is” with the above options
• Other Programs – I would add in the dialog from Tools | Open With… | Configure Programs a checkbox saying “Automatically Export Metadata before Execute”
• Internet – (Email, Upload etc.) - perhaps something similar with the comment from “Other programs”. However, because this is more rare, we can live with the global setting from “Automatically Export Metadata” (see “About How to update” paragraph)

[oops66]

... I like this approach , (Into "what to update" we also have: time stamps modifications, Iptc keyword, xmp embedded data, embedded thumbnail modifications, icc profile, etc ... too )
... And a single option ticked ("don't touch my originals" policy by default) should disallow all of these writes files operations (database ---> to users files synchronizations)
... And an other one (single option too) to "allow all writes operations to users files" , but of course not ticked by default (to preserve the integrity of the originals users files)

[xnview]

Rating and Color label can perhaps be only one setting?

So global settings:

No change for import settings, right?

And Rating/Label - Keywords/Categories
Export to:
* DB only
* Metadata when possible
* Metadata and sidecar
* Sidecar

What do you means by "Automatically Export Metadata " & "Automatically Import Metadata "??

[m.Th.]

Rating and Color label can perhaps be only one setting?

Hmmm.... well, perhaps. It depends. For me, it doesn't matter.
But I definitely see use cases (not very often of course) in which one would want to do Import/Export the Rating part (Quality Cataloging) separated from Color Label (State/Task Cataloging).

So global settings:

No change for import settings, right?

GUI-wise, no. However, we must test(?) if it works ok from both embedded and sidecar metadata.

And Rating/Label - Keywords/Categories
Export to:
* DB only
* Metadata when possible
* Metadata and sidecar
* Sidecar

Yep, I think that this sums them up. But only for storage. Do NOT forget about the "Other Programs" case which I described in my prev post.

What do you means by "Automatically Export Metadata " & "Automatically Import Metadata "??

A small big feature. In fact it is a standard. Almost all DAMs have it.

In fact it is simple. Let's take it one by one:

"Automatically Export Metadata"

The program after each save in DB of RCK updates automatically also the metadata in image / sidecar (according with the "Export to:" settings discussed above). No need to manually go to View | Update files from database
This is very important because it is an automatic backup and an automatic way to inter-operate with the other DAMs which for us (remember, we are new and small) is critical. With the manual mode, the users can do mistakes, forget, the work can be lost when imported elsewhere etc.

Pseudo-Example:

1. Let's say that the code under the "Update files from database" is a single method/procedure/function called ExportMetadata.
2. Let's say that this option is called AutoExportMetadata
3. After the program saves the Rating, Color, Keywords in DB (at the end of the method) call:

Code: Select all

if AutoExportMetadata then ExportMetadata;

"Automatically Import Metadata"

Perhaps is better to go by example here:

1. I'm in the folder c:\f1
2. I go in the folder c:\f2 - What does should do the program? ...(among other things) it scans the folder and sees if the Modified Date of the Files (INCLUDING the metadata sidecars) matches with the ModifiedDate field from the Images table in DB. If the file is newer, and the AutoImportMetadata is checked, then the program will update automatically the metadata. In fact, I think that you must just check the case when the metadata is in sidecars and hence only the XMP file has newer modified date than the one present in the DB (yes, the sidecar, if present, should have a record in Images table).

[xnview]

No change for import settings, right?

GUI-wise, no. However, we must test(?) if it works ok from both embedded and sidecar metadata.

sidecar has a higher priority on embedded metadata, right?

And Rating/Label - Keywords/Categories
Export to:
* DB only
* Metadata when possible
* Metadata and sidecar
* Sidecar

Yep, I think that this sums them up. But only for storage. Do NOT forget about the "Other Programs" case which I described in my prev post.

All of this are needed??

What do you means by "Automatically Export Metadata " & "Automatically Import Metadata "??

But it's already like that, i don't understand...

[m.Th.]

No change for import settings, right?

GUI-wise, no. However, we must test(?) if it works ok from both embedded and sidecar metadata.

sidecar has a higher priority on embedded metadata, right?

Yes.

And Rating/Label - Keywords/Categories
Export to:
* DB only
* Metadata when possible
* Metadata and sidecar
* Sidecar

Yep, I think that this sums them up. But only for storage. Do NOT forget about the "Other Programs" case which I described in my prev post.

All of this are needed??

No, I don't think so. I would say just 3 because DB is already updated:

Export to:
* DB only
* Metadata when possible
* Metadata and sidecar
* Sidecar

Is easy to handle them with something like...

Code: Select all

if myChoice = 1 or myChoice = 2 then
	TryToEmbeddMetadata(myListOfImages);
if myChoice = 2 or myChoice = 3 then
	ExportToSidecar(myListOfImages);

What do you means by "Automatically Export Metadata " & "Automatically Import Metadata "??

But it's already like that, i don't understand...

Let's do some tests: (I will take the Rate only but the same is for Colors/Keywords)

1. I go to img_001.CR2 and I rate it with 5.
Q: Does this rating go in a newly created XMP sidecar file?
A: No.

Q: Or, if it is a jpg does it go in the embedded XMP metadata?
A: Yes.

2. I change the rating of img_001.CR2 from 5 to 4.
Q: Is this change reflected in the existing/newly created XMP sidecar file?
A: No.

Q: Or, if it is a jpg does this change propagate into the embedded XMP metadata?
A: Yes.

Just tested. If works only for the embedded part of the story. Hence all the formats which doesn't support XMP embedding, - including all the RAW files - needs fixing.

[xnview]

Currently if export is enabled, jpeg/png files are updated, and not for other
But if i add sidecar support, they will be

So like that?

Export to:
* None
* Metadata when possible
* Metadata and sidecar
* Sidecar

[hic]

Ideally, the metadata handling would be as follows (IMHO):

Jpg, Tif, PNG, DNG: Changes of keywords, ratings etc. are written to the file automatically when something is changed, no xmp-sidecar is created.
Raw files (orf, rw2, cnr etc.): Changes of keywords, ratings etc. are written to a xmp-sidecar automatically when something is changed.
An option for deliberate users: Files are not changed at all, a xmp-file is always written.

And by the way: To have an option to handle other sidecars as well (like pp3 from Rawtherapee), i.e. that they will be moved, deleted, copied when the raw is moved, deleted, copied, would be grate!

Regards,
hic

[oops66]

Ideally, the metadata handling would be as follows (IMHO):

Jpg, Tif, PNG, DNG: Changes of keywords, ratings etc. are written to the file automatically when something is changed, no xmp-sidecar is created.
Raw files (orf, rw2, cnr etc.): Changes of keywords, ratings etc. are written to a xmp-sidecar automatically when something is changed.
An option for deliberate users: Files are not changed at all, a xmp-file is always written.

And by the way: To have an option to handle other sidecars as well (like pp3 from Rawtherapee), i.e. that they will be moved, deleted, copied when the raw is moved, deleted, copied, would be grate!

Regards,
hic

Hello,
...Yes, but some people want or (doesn't want) to have the ability to change some data (or embedded EXIF data, or file attributes, etc...) even for the originals Raw files (orf, rw2, cnr, cr2 etc.)
For example:
----------------⊝ Allow writes operations for time stamps: files attributes, EXIF (to "Keep original date/time" or not even for RAW files)

See here too, for more details:
http://newsgroup.xnview.com/viewtopic.p ... 85#p117985

[m.Th.]

Currently if export is enabled, jpeg/png files are updated, and not for other
But if i add sidecar support, they will be

So like that?

Export to:
* None
* Metadata when possible
* Metadata and sidecar
* Sidecar

Yes.

[m.Th.]

Ideally, the metadata handling would be as follows (IMHO):

Jpg, Tif, PNG, DNG: Changes of keywords, ratings etc. are written to the file automatically when something is changed, no xmp-sidecar is created.
Raw files (orf, rw2, cnr etc.): Changes of keywords, ratings etc. are written to a xmp-sidecar automatically when something is changed.
An option for deliberate users: Files are not changed at all, a xmp-file is always written.

Pierre describes it much better in his...

Export to:
* None
* Metadata when possible
* Metadata and sidecar
* Sidecar

...thing.

"Metadata when possible" is your first two paragraphs. (Ok, perhaps a better formulation would be "Embedded when possible")
"Sidecar" is your last one. - the oops66's favorite (and perhaps mine) "Do not touch the originals" policy.
"Metadata and sidecar" - is a combination of the two above. Definitely useful.

And by the way: To have an option to handle other sidecars as well (like pp3 from Rawtherapee), i.e. that they will be moved, deleted, copied when the raw is moved, deleted, copied, would be grate!

Regards,
hic

Oh, sure! Is a stringent need. A must have. Already requested with details of implementations here: http://newsgroup.xnview.com/viewtopic.php?f=62&t=29532

[hic]

And by the way: To have an option to handle other sidecars as well (like pp3 from Rawtherapee), i.e. that they will be moved, deleted, copied when the raw is moved, deleted, copied, would be grate!

Regards,
hic

Oh, sure! Is a stringent need. A must have. Already requested with details of implementations here: http://newsgroup.xnview.com/viewtopic.php?f=62&t=29532

That was only partially intended. It would be good to have the opportunity to define and handle other sidecars than XMP as well, e.g. those from Rawtherapee or SilkyPix. For both, no DAM exists.

[m.Th.]

And by the way: To have an option to handle other sidecars as well (like pp3 from Rawtherapee), i.e. that they will be moved, deleted, copied when the raw is moved, deleted, copied, would be grate!

Regards,
hic

Oh, sure! Is a stringent need. A must have. Already requested with details of implementations here: http://newsgroup.xnview.com/viewtopic.php?f=62&t=29532

That was only partially intended. It would be good to have the opportunity to define and handle other sidecars than XMP as well, e.g. those from Rawtherapee or SilkyPix. For both, no DAM exists.

Aha!

I see: you have TWO requests:

1. Support other sidecars as well
2. In On-disk management layer process together the images with the sidecars (iow copy/move/delete/rename)

I responded only to point 2.

Ok, theoretically I agree with the point 1. But a question arises:

These programs (I didn't work too much with RawTherapee and I didn't saw at all SilkyPix) doesn't support XMP standard?

[hic]

These programs (I didn't work too much with RawTherapee and I didn't saw at all SilkyPix) doesn't support XMP standard?

No, they don't, nor does Rawtherapee. Rawtherapee produces a sidecar in the form "abc.orf.pp3" next to abc.orf, Silkypix produces a sidecar "abc.orf.5.spd" in a subfolder "SILKYPIX_DS". Capture One uses a similar strategy.

There is no DAM out there which is able to handle different sidecars (except JPhototagger, but that is another story). After all, one ends up with a lot of orphaned sidecar files while the corresponding raw files are already deleted/moved/renamed etc.