Rating gets wiped after every few months

[jkm]

If you delete the db and rebuild now, you'll lose the ratings on the jpeg files

this is not how it works, his JPEGs should have embedded metadata

Should, but since the user is complaining about losing ratings, I want to make sure his information is properly saved and exported before deleting the db. I can't be sure what settings have changed over time.

you likely moved files outside of the app, so app lost track of them

That was the obvious diagnosis from the outset...
Except user insisted not.

So a methodical approach is called for. There's no harm in working things through cautiously.

[MarioRossi99]

When you say "JPG rated are still there", exactly what do you mean? You're using the Catalog Filter panel to show files with a certain rating, or you're browsing to the folder they're in?n

Yeah I meant using the Catalog filter

If you delete the db and rebuild now, you'll lose the ratings on the jpeg files, so hold off for the moment.

That's not a problem, the JPG rated are very few files - I mostly rate RAW files and that's the most important

... and the paths with size different than zero are all gone now, all of the entries are zero in size,...

Hmm, this result indicates paths have changed, but you say they haven't.

I said I didnt change the path with the files rated but here are listed folders of 5/6 years ago that I have deleted. I rate files and until I edit and publish them I dont change the structure. Once I publish them I remove the rating.

By "all gone" do you mean the entries for those paths (that had non-zero sizes) were totally removed, or just that they are now of zero size?

Now they are all zero

First, please confirm that these two folders are no longer listed at all in the Catalog database directory list:
/Users/dav/Pictures/A2P PHOTOS/0. Monte Misma/
/Users/dav/Pictures

Do you mean under Preferences>Catalog>Database Directories ?
This one exist: /Users/dav/Pictures

Next, in the Mac file manager (not XnViewMP) examine two folders please:

/Users/dav/Pictures/A2P PHOTOS/0. Monte Misma/
Does that folder still exist, and are there still 19 images at that exact path?

/Users/dav/Pictures
I know the folder exists, because it still has children in your subsequent screenshot.
But are there still 19 images in that folder?

Ths first folder does not exist. I deleted it long time ago. But yeah /Users/dav/Pictures does exist and it has 0 files and 7 folders

Are you always using Details View in the Browser?

Yes I only use Details View

If you do a Help->About, it will tell you the exact path to your INI file, where your XnViewMP database is also stored. Do you have backups of that folder? Your Mac might be making backups so that you have old versions of the database. That could help recover metadata that you think you've lost.

Lastly, please do make the settings changes I provided before, and report back that/if you see the behavior I described. This will be important later.

I did make the changes in the settings as you suggested but still when I rate the RAW files in the folder there no xmp files created by the rating process. There are the RAW files only.

[jkm]

An exchange you had a year ago with Pierre makes me wonder exactly how you are rating files, and what exactly you are doing in the UI when you say "all of sudden while using app the ratings gets wiped". Some of your statements were ambiguous or not fully understandable.

But we'll move past that for now, and try and get things fixed...

... and the paths with size different than zero are all gone now, all of the entries are zero in size,...

Hmm, this result indicates paths have changed, but you say they haven't.

I said I didnt change the path with the files rated but here are listed folders of 5/6 years ago that I have deleted. I rate files and until I edit and publish them I dont change the structure. Once I publish them I remove the rating.

Next, in the Mac file manager (not XnViewMP) examine two folders please:

/Users/dav/Pictures/A2P PHOTOS/0. Monte Misma/
Does that folder still exist, and are there still 19 images at that exact path?

Ths first folder does not exist. I deleted it long time ago.

Ok, so it turns out you're deleting folders, and you're deleting them outside the XnViewMP application. You're deleting them using the Mac file manager or something else. That's part of your problem. XnViewMP is a database application, and files are tracked by their location in the filesystem. When you make filesystem changes the app doesn't know about, you can cause problems. One symptom of this is all the "old" directory entries you see in Settings:Catalog.

Either make your filesystem changes (moves/deletes/whatever) inside the app, or you will have to run database maintenance afterwards.

Do you ever move or rename or relocate any of the folders/directories that hold your image files, or the image files themselves?

3. Folder structure is created before I copy the raw files in the assigned folder and I never make any changes after that.

"I never make any changes after that" is a pretty clear statement. Except it's not true. You're at the very least making deletions. So you relocated some files/folders right into the trash. They're no longer where they originally were.

Evidently you have a very idiosyncratic workflow, and you haven't fully described what it is. Hopefully that won't become necessary. If we get everything setup correctly, you won't lose data unless you explicitly delete it.

Lastly, please do make the settings changes I provided before, and report back that/if you see the behavior I described. This will be important later.

I did make the changes in the settings as you suggested but still when I rate the RAW files in the folder there no xmp files created by the rating process. There are the RAW files only.

Sorry, but you've gone wrong somewhere, either in following the steps or reporting. If you'd made the changes I indicated, xmp sidecar files would be created.

So let's try again. Follow these steps:

1. Make these settings changes I gave you before:

So for starters I'm going to recommend that you change the following settings:
Settings:Metadata:Sidecar:Create or update XMP Sidecar=ENABLED
Settings:Metadata:Sidecar:Update master file=OFF
Settings:Metadata:Sidecar:Naming convention=filename.extension.xmp
Settings:File list:Companion files:Link companion files for the following actions=Delete & Copy both ENABLED
Settings:File list:Companion files:Show companion files=OFF

I would also encourage you to set Settings:Metadata:Exiftool to Enabled. You're missing out on a lot of functionality without that.

ALSO, in Settings:Metadata, ensure that all options under EXPORT are still enabled/checked.

2. Restart the app. Verify the settings are still as above.

3. In the XnViewMP app, rate a raw file. (How exactly do you normally do this in the app since you use details view?)
Rate it 3 stars by either:
a. drag & drop it on rating level in the catalog filter pane
b. Click the file once to select it, then use Metadata->Set Rating menu
c. Click the file once to select it, then click a rating toolbar button

4. Verify in the app the raw file shows as rated 3 stars

5. In the app, right-click the rated raw file and select "Open in File Manager" (If you don't see that option on the right-click menu, Add it it in Settings:Interface:Context menu)

6. In the Mac file manager, verify that the XMP sidecar file with the same name as your rated file exists. The sidecar file doesn't show in the app.

If you do this correctly, the sidecar file will be there, and then we can proceed.

[MarioRossi99]

An exchange you had a year ago with Pierre makes me wonder exactly how you are rating files, and what exactly you are doing in the UI when you say "all of sudden while using app the ratings gets wiped". Some of your statements were ambiguous or not fully understandable.

Not sure what ambiguos. I browse the photos in the given folder, rate them for editing or delete (CMD+1 to mark delete, CMD+4 edit candidates and then CMD+5 the ones to edit).
I then proceed to edit the photos in Photoshop and publish them. Once a photo is published I remove the rating, CMD+0

Ok, so it turns out you're deleting folders, and you're deleting them outside the XnViewMP application. You're deleting them using the Mac file manager or something else. That's part of your problem. XnViewMP is a database application, and files are tracked by their location in the filesystem. When you make filesystem changes the app doesn't know about, you can cause problems. One symptom of this is all the "old" directory entries you see in Settings:Catalog.

Either make your filesystem changes (moves/deletes/whatever) inside the app, or you will have to run database maintenance afterwards.

Yes I do move folder or delete them but that's only for older files that are not rated. I never change the structure in the file system of the photos I rate untile the rate is cleared after the photos are published. But I can do database mantainance. Once the rating is cleared I can also delete the database and start fresh.

"I never make any changes after that" is a pretty clear statement. Except it's not true. You're at the very least making deletions. So you relocated some files/folders right into the trash. They're no longer where they originally were.

Evidently you have a very idiosyncratic workflow, and you haven't fully described what it is. Hopefully that won't become necessary. If we get everything setup correctly, you won't lose data unless you explicitly delete it.

The statement is true, I never moved a rated photo, only once I cleared the ratings I might move the folder. So rated files are never moved. I explained the workflow it's pretty simple: I rate some photos the one rated to 1 are deleted within the app the ones that makes to 5 a at a later stage edited in photoshop (I right click on the files and open on photoship) after the photos are edited they stay in rating 5 will they are slowly published. Once published I rate them 0 so clearing the rating. Only then I might move the folder but it might even stay for years there. There is no correlation with some folders being moved and the disappearing of the rating. I didnt move nor delete any folde for almost one year and still the issue was recurring every few months.

So let's try again. Follow these steps:

1. Make these settings changes I gave you before:

So for starters I'm going to recommend that you change the following settings:
Settings:Metadata:Sidecar:Create or update XMP Sidecar=ENABLED
Settings:Metadata:Sidecar:Update master file=OFF
Settings:Metadata:Sidecar:Naming convention=filename.extension.xmp
Settings:File list:Companion files:Link companion files for the following actions=Delete & Copy both ENABLED
Settings:File list:Companion files:Show companion files=OFF

I would also encourage you to set Settings:Metadata:Exiftool to Enabled. You're missing out on a lot of functionality without that.

ALSO, in Settings:Metadata, ensure that all options under EXPORT are still enabled/checked.

2. Restart the app. Verify the settings are still as above.

Okay the settings didnt stick but I set them again restarted and they are still there now.

3. In the XnViewMP app, rate a raw file. (How exactly do you normally do this in the app since you use details view?)

I browse into the folder with the photos, list view and preview on the right. With arrows I brows and with CMD+1 I mark for delete and pregressvly from CMD+2 the first pass. Then I go to the catalog filter view, Go to Poor and delete the photos, the to Fair and rate those to either 0 (clearing the rating) or to CMD+5 for the ones I actually want to edit.

4. Verify in the app the raw file shows as rated 3 stars

5. In the app, right-click the rated raw file and select "Open in File Manager" (If you don't see that option on the right-click menu, Add it it in Settings:Interface:Context menu)

6. In the Mac file manager, verify that the XMP sidecar file with the same name as your rated file exists. The sidecar file doesn't show in the app.

If you do this correctly, the sidecar file will be there, and then we can proceed.

Yes the xmp file is there

I guess to start fresh I should delete the datase - what's the best way to do it?

[jkm]

4. Verify in the app the raw file shows as rated 3 stars

5. In the app, right-click the rated raw file and select "Open in File Manager" (If you don't see that option on the right-click menu, Add it it in Settings:Interface:Context menu)

6. In the Mac file manager, verify that the XMP sidecar file with the same name as your rated file exists. The sidecar file doesn't show in the app.

If you do this correctly, the sidecar file will be there, and then we can proceed.

Yes the xmp file is there

I guess to start fresh I should delete the datase - what's the best way to do it?

I'm pressed for time now so the rest can be addressed later, but to proceed for right now:

Since you now successfully have the xmp sidecars being created, now we want to ensure your metadata is saved to the filesystem.

Use the catalog filter to find all your rated/colored/tagged files. Anything that has metadata you assigned in XnViewMP that you want to save.

Once you've found them, select them all in the browser.

With the files selected, from the menu do Metadata->Update Files from Catalog. This will write XMP sidecar files with the metadata.

If you couldn't get all the files in the first pass, just repeat with different selection criteria until you've done this for all the files with metadata you want to save. It does not hurt if you do an "Update files from catalog" on the same file multiple times.

Next, go look in the OS file manager and verify the xmp files are there for all the files you just processed. They will be if you did everything right.

Now, once you're satisfied all the XMPs have been created, you're ready to take the big step.

1. First, check Settings:Integration:Paths, or look at Help->About. That will tell you where your database is.

2. Close the app. Make sure all instances of the app are closed, as in not running. Not just minimized or in the background.

3. In the OS file manager, go to the folder you identified in step 1, and rename 2 files: xnview.db and thumb.db Just change the names to .old or something.

4. Restart the app. You'll see no files in the browser, except for whatever pops up in the current directory. The directory list in Settings:Catalog will be empty except for the currently browsed folder.

5. As you browse to folders, the files in them will be added to the catalog. You can go to a top-level folder in the Folders pane, right-click on it, and do a "Show files in subfolders". That will import all the files that are children of that top level folder in one step. You can repeat that for any other top-level folder.

As your images are imported into the app, you'll see the ratings/colors/etc that were saved in the .XMP files.

As long as you keep the settings as I told you, ratings you assign will automatically be saved in the sidecars as you work. This way, in the future if a folder is deleted from the database or anything else like that, the ratings will still be in the sidecar files. If you intentionally alter or remove the rating in the app, that change will be reflected in the sidecar, and the rating will not be recoverable.

Good luck...

[MarioRossi99]

1. First, check Settings:Integration:Paths, or look at Help->About. That will tell you where your database is.

2. Close the app. Make sure all instances of the app are closed, as in not running. Not just minimized or in the background.

3. In the OS file manager, go to the folder you identified in step 1, and rename 2 files: xnview.db and thumb.db Just change the names to .old or something.

4. Restart the app. You'll see no files in the browser, except for whatever pops up in the current directory. The directory list in Settings:Catalog will be empty except for the currently browsed folder.

5. As you browse to folders, the files in them will be added to the catalog. You can go to a top-level folder in the Folders pane, right-click on it, and do a "Show files in subfolders". That will import all the files that are children of that top level folder in one step. You can repeat that for any other top-level folder.

Took me a bit to find the database as it's in a hidden folder but got it done. I will rate all the photos which the rating was lost and hopefully it will stick more than a few months. Thank you!

[MarioRossi99]

Well the rating now disappear as soon as I close the app.

I rate some photos 5. I close the app and I go to catalog filter and they are not in the 5. Some of them show up in 4 but not all of them.

[jkm]

Well the rating now disappear as soon as I close the app.

It's been 10 days since your previous post. When did this new behavior start? Immediately, or was it ok for 9 days and now this new behavior?

Since it's happening every time you close the app, it's easy to reproduce. If you want further attention on this, please do a screen recording (Snipping tool in Win11 will do it) and provide a video showing exactly what's happening, from showing the settings screens I had you adjust before, to doing the rating to "show in file manager" so we can see the .xmp files created to close/restart and showing the ratings now missing.

[xnview]

Well the rating now disappear as soon as I close the app.

I rate some photos 5. I close the app and I go to catalog filter and they are not in the 5. Some of them show up in 4 but not all of them.

you can reproduce this problem always?

[MarioRossi99]

For the fist few days everything worked as it should but then started this new behaviour where the rating is lost immediately upon closing the app.
I rate photos to 5, close the app and the rating is gone, some of those photos show up as a different rating instead (rating 3) but only some of them not all of them. In any case it's not the correct rating 5.

I made a screen recording here:
https://we.tl/t-oW9jVuTBpd9567Kh

[xnview]

For the fist few days everything worked as it should but then started this new behaviour where the rating is lost immediately upon closing the app.
I rate photos to 5, close the app and the rating is gone, some of those photos show up as a different rating instead (rating 3) but only some of them not all of them. In any case it's not the correct rating 5.

I made a screen recording here:
https://we.tl/t-oW9jVuTBpd9567Kh

Please send me your xnview.ini, and a .ARW?

[MarioRossi99]

Please send me your xnview.ini, and a .ARW?

Here they are: https://we.tl/t-7qzHhLBj80uvN0bE

[jkm]

For the fist few days everything worked as it should but then started this new behaviour where the rating is lost immediately upon closing the app.
I rate photos to 5, close the app and the rating is gone, some of those photos show up as a different rating instead (rating 3) but only some of them not all of them. In any case it's not the correct rating 5.

I made a screen recording here:
https://we.tl/t-oW9jVuTBpd9567Kh

Some points in response to the video:

-Why all the time selecting 57 files if 1 or 2 would show the issue?

-You were asked to show some specific things, but didn't show some of them. I saw no settings screens. I saw no "show in file manager" to see that the .XMP sidecar files were created. Have to see those to continue.

-It looks like the rating was not "wiped". It looks like the rating was 3 stars instead of 5. Were they rated 3 previously, and you were changing them to 5 stars? Or were they unrated before the video, and the wrong rating was stored? I saw that they seemed to be 5 before quitting. But I don't know what was written or what was pre-existing, because there are no sidecars in this video.

-If you're doing to do this in Details view, please show Rating as a column, so that we can see what the rating is before and after. All I can say for sure is at the end of the video, DSC01706 was 3 stars and not 5. But I don't know what it was when the video started because you didn't show that.

-If this happens when you tag 57 files, does it happen when you tag 1? Because of the large number of files, I wonder if XnViewMP is still writing when you quit. If it does not happen with 1 file but does happen with many files, this could be the case. But there's no way to tell what's going on because you're not showing the XMP files. So please show this again (with the minimum number of files), and after DSC01706 is tagged, before exiting the app, show the XMP file. I want to see if the modification date is "right now" as in it was just written, and show the contents, to see if the new 5 rating is in there. It had better be, if your settings are correct. Which is why I wanted you to show the settings screens I told you to set before.

[MarioRossi99]

-Why all the time selecting 57 files if 1 or 2 would show the issue?

I dont think you watched carefully. All the 57 files are affected.

-You were asked to show some specific things, but didn't show some of them. I saw no settings screens. I saw no "show in file manager" to see that the .XMP sidecar files were created. Have to see those to continue.

I can show you the settings but they are still the same I didnt change them

-It looks like the rating was not "wiped". It looks like the rating was 3 stars instead of 5. Were they rated 3 previously, and you were changing them to 5 stars? Or were they unrated before the video, and the wrong rating was stored? I saw that they seemed to be 5 before quitting. But I don't know what was written or what was pre-existing, because there are no sidecars in this video.

Yes it was wiped, Only some of them are displayed as 3. The were possibly 3 stars a while back before they were edited in photoshop, once edited they are rated as 5 and they showed as five for a week of so then all of sudden the rating is not sticky.

-If you're doing to do this in Details view, please show Rating as a column, so that we can see what the rating is before and after. All I can say for sure is at the end of the video, DSC01706 was 3 stars and not 5. But I don't know what it was when the video started because you didn't show that.

-If this happens when you tag 57 files, does it happen when you tag 1? Because of the large number of files, I wonder if XnViewMP is still writing when you quit. If it does not happen with 1 file but does happen with many files, this could be the case. But there's no way to tell what's going on because you're not showing the XMP files. So please show this again (with the minimum number of files), and after DSC01706 is tagged, before exiting the app, show the XMP file. I want to see if the modification date is "right now" as in it was just written, and show the contents, to see if the new 5 rating is in there. It had better be, if your settings are correct. Which is why I wanted you to show the settings screens I told you to set before.

Here is another video with one file only DSC01753 it has no rating, I rate it 5 close the app and open it again and it's not in the rating 5 and I go back in the folder and the file shows no rating. It doesnt matter if I do 1 file or 57. It just so happend that some of the 57 show up are rating 3 when rating 5 is wiped (to be exact it happens to 14 files.

The xmp file had the date of the rating when I last do it.

Here is the video: https://we.tl/t-Pxak3RaQCzcRFVb2

[jkm]

Thanks, this was much more informative.

-Why all the time selecting 57 files if 1 or 2 would show the issue?

I dont think you watched carefully. All the 57 files are affected.

Actually I watched very carefully. You misunderstood me. What I meant was it's a waste of time to watch you click through 57 files when the problem can be demonstrated with 1 file.

-It looks like the rating was not "wiped". It looks like the rating was 3 stars instead of 5. Were they rated 3 previously, and you were changing them to 5 stars? Or were they unrated before the video, and the wrong rating was stored? I saw that they seemed to be 5 before quitting. But I don't know what was written or what was pre-existing, because there are no sidecars in this video.

Yes it was wiped, Only some of them are displayed as 3. The were possibly 3 stars a while back before they were edited in photoshop, once edited they are rated as 5 and they showed as five for a week of so then all of sudden the rating is not sticky.

Semantics, but sometimes semantics matter. A file has one rating. And it either has a rating or it doesn't. Wiped means erased/lost/empty.
So when I see a file rated 3, it has a rating. In the first video, the rating was not wiped as in erased, because at the end the file has a rating of 3. For some reason the rating was not updated to 5. This different from what is shown in the 2nd video. The 2nd video shows no rating, then after rating the file 5 you see very strange and inconsistent results:
You browse to the 5 category in the catalog, and the file is listed, which means the app thinks it has a 5 rating. But simultaneously the 5 rating does NOT show in the column. That at the very least is a bug. And then after restart the file is no longer in 5 in the catalog filter, indicating the rating has indeed been wiped/lost in this case. But in this case, you were starting with an unrated file.

So it might be that the update to the rating is not getting recorded.

Which is why I said I actually wanted you to "show the contents" of the XMP file, in addition to the timestamp. XMP files are just text files.

The xmp file had the date of the rating when I last do it.

I need the contents too, not just the date. And before and after.

So please do this: (one last video)
Redo the exact same experiment you did in your second video, with the same file, except with these changes:
Before starting
-Set your catalog filter scope to "Current folder" instead of "Global"
-Make the rating column the first or second column, so that you don't need to scroll to see it.
-Modify your layout so that the Catalog filter pane is visible on screen simultaneously with the folders pane (you can use save/restore layout to undo this afterwards if you want)
This will make things much easier to follow.

1. Before you rate the file, capture the contents of the DSC017153.ARW.xmp file, and paste it into the forum message as "Before". If the file does not exist, note that.

Now start the video as before...

2. Immediately after rating the file, and before closing the app, capture the contents of the DSC017153.ARW.xmp file, to paste it into the forum message as "After". The timestamp on the file should again have been updated to right now, indicating it was modified.

3. After closing and restarting the app, check that xmp file again. It should be unchanged from step 2.

4. If the XMP file contained a rating of 5 (as it SHOULD), then after restart, select the DSC017153.ARW file in the app and do an Metadata->Update Catalog from files. The expectation is that if the xmp file held a rating of 5, the catalog should now be updated to show a rating of 5.

5. Restart the app again. Does DSC017153.ARW still have a rating of 5?

When I perform this experiment on Windows, as soon as I rate the file, the counts in the catalog filter update to show the file has been rated 5, and when I select the 5 category in the catalog filter, the file is there and the rating column shows as 5. And of course the rating is not lost after restart.

It looks like there's at least one bug in the Mac version that is not present in the Windows version. But I want to understand WHERE that bug might be, which is why I want to see the before and after contents of the XMP file. This video and before/after file contents should show exactly where things are going wrong.