Reducing size of my large image collection

Ask for help and post your question on how to use XnView MP.

Moderators: XnTriq, helmut, xnview

Post Reply
bdragon
Posts: 108
Joined: Mon May 01, 2017 8:06 am

Reducing size of my large image collection

Post by bdragon »

Ok, so i have a folder that has 292.330 pictures divided in 762 subfolders. This folder totals 29,3 GB.

I'd like to shrink it.

So i set up XnView MP to do the following on all these pictures:

Code: Select all

<?xml version="1.0" encoding="UTF-8"?><XnView_script version="1.0" name="st-conv">
    <Change_color_depth method="3" dither="-1" ncompo="-1" alpha="255"/>
    <Automatic_contrast/>
    <Automatic_levels/>
    <Automatic_crop method="0" position="0" color="#ffffff" tolerance="90"/>
    <Automatic_crop method="0" position="1" color="#ffffff" tolerance="90"/>
    <Automatic_crop method="0" position="3" color="#ffffff" tolerance="90"/>
    <Automatic_crop method="0" position="2" color="#ffffff" tolerance="90"/>
    <Set_DPI x="150" y="150" keep="false"/>
    <Clean_metadata flags="63"/>
    <Output folder="" filename="{Filename}" case="7" format="JPEG">
        <Options overwrite="1" orgDate="true" keepMeta="false" keepICC="false" keepFolder="true" keepParentFolder="false" delOrg="true" multipage="true" allPages="true" openExplorer="false" openBrowser="false" clearItems="true"/>
        <JPEG quality="75" progressive="true" optimizeHuffman="true" rebuildThumb="false" orgQuality="true" DCTMethod="0" subSampling="0" smoothFactor="0"/>
    </Output>
</XnView_script>
I expect that in this way i should get back about 2 to 3 GB of space. Possibly more. There are several large JPGs and PNGs in there, though most of the files are rather small.

Where is the problem?

There are several gifs in this folder. And not all of them are animations. However, with the above settings it tires to convert animations into JPEGs even though i expressly told it to keep multipage files in the original format.

I'd like to convert the static gifs into jpgs (yes in some cases i'll probably lose space, but i expect to gain in most cases) but not the animations.

Also there might be some PNG animations too (i do not remember), but i stopped it as soon as it tried to convert and animated gif into several static JPGs and prompting me for action each single time.

Is there any way to filter out animations?

Thank you.
User avatar
m.Th.
XnThusiast
Posts: 1663
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: ok, i'm trying to do something...

Post by m.Th. »

Ok, I think that I can give you a solution. :)

1. PNG does not support animation. MNG or APNG does. (see https://en.wikipedia.org/wiki/Portable_Network_Graphics ).
2. GIFs do not need to be converted - waste of time and (perhaps?) space.

So, our problem turns to:

a. Exclude (temporarily) APNGs and GIFs.

How to do it: Go to Tools | Settings | File List | Custom Filter and in the "Exclude" row enter at the end " apng gif" (without the quotes) in order to hide these files globally. I don't know if the program needs a restart.

b. Go to your folder and select "a large batch" of files + folders (I think that selecting all 292.330 pictures) can lead to a crash - especially if you have the 32-bit version.

c. Before going to sleep press Ctrl+U and select JPG (ok, other formats perhaps are better theoretically but let us just stick with JPG). On {Settings} button choose:
- Quality - as you wish (it is your responsibility here. :) )
- Progressive: False
- Optimize Huffman: True
- DCT method: Float
- Smoothing factor: <depends> (ok, you can leave it to 0)
- Subsampling: 1x1, 1x1, 1x1
- Rebuild embedded thumb: False
- Use estimated quality: False
- Keep(s) : depending on what you have you can have them true or false. No big space needed by these, generally speaking, do not take a lot of space. I see that you stripped them off, tough.

Note: If you use the DCT method and Subsampling which I suggested, you can lower the Quality more without seeing artifacts.

Setup the other dialog fields accordingly (Output folder etc.).

Other notes about actions:

Change color depth: 24 bits. Not really needed. JPEG does this by itself.

Automatic contrast, levels, crop: well, ok. Depends on what you have. Cartoons? :)

DPI: 150. Meh. Not relevant at all. (this is only for print).

Yes, you can gain space (especially by converting from PNG to JPG) but what would help the situation more is to Resize the bigger ones. If you intend to look at them only on screen and you have images bigger than eg. 2800x1800 you can add an Resize action with:

Mode: Fit
Width: 2800 (put something bigger than your screen but not much bigger)
Height: 1800
Keep Ratio: True (!!)
Enlarge/Reduce: Reduce Only (!!)
Use gamma correction: False (meh)
Follow orientation: True (! - it is better like this IMHO)
Resample: Lanczos (it is ok)

Beware that Resizing is a destructive operation. Use with care and only if you know that you don't need the bigger originals.

PS: Do not forget at the end to un-hide the APNGs and GIFs. :)
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
bdragon
Posts: 108
Joined: Mon May 01, 2017 8:06 am

Re: ok, i'm trying to do something...

Post by bdragon »

It's my collection of lost webtoons. Cannot resize them or they become unreadable. But yes. Removing white spave is something that can be done.

Changing to 24 bits removes a bad mistake where some versions of XnConvert will use a different color (usually fuchsia black or random garbage) instead of the background color i chose. This usually happens when somebody has saved the picture with transparent bits that are not of a single background color but either several different colors or they just made their random doodles or palette pickups transparent.

By converting to 24 bits i make sure it uses the background color i chose.

Also selecting almost 300.000 files does not crash it but sure does make it sluggish. It's times like this that i like having bought an alienware even though i no longer have time to play
User avatar
m.Th.
XnThusiast
Posts: 1663
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: ok, i'm trying to do something...

Post by m.Th. »

Got the 24bits thing. Never saw it. Anyway it is better to have it.
Also, got the Alienware thing. :)

Hmmm.... it seems that there are some APNGs with PNG extension. Toooooo bad. :( You must filter them out „by hand”. Do you have many of them?
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
bdragon
Posts: 108
Joined: Mon May 01, 2017 8:06 am

Re: ok, i'm trying to do something...

Post by bdragon »

I have no idea. Should try and cancel every gif and i do not wish to do that because with300000 pictures i'd be done by tomorrow or a week from now
bdragon
Posts: 108
Joined: Mon May 01, 2017 8:06 am

Re: ok, i'm trying to do something...

Post by bdragon »

ok, i just excluded the 30 thousand or so gifs and tried to let it brew... about half an hour later it finished the above steps and now... welll the folder is 28,3 GB in size... not as much loss as i expected, but anyway, saving up 1.6 gigabytes (about 5%) from images by doing very little manipulation and keeping almost everything intact is still quite satisfying.

Also with this i confirmed that there were no png animations. :D

Now ì will have to deal with the 30 thousand gifs... which are still too many to check manually for animations.

While one might say that converting a gif to a jpeg will make me lose space... i disagree. Gifs are quite space devouring, especially beyond a certain pixel size, since they are lossless compression. Small gifs might be left alone, but bigger gifs are better compressed as jpegs.

problem is... i have no idea how to see which is which.

i picked up one at random ... it was sluggy freelance's strip from 2016-01-14. That's a static gif of about 200kb in size and with a pixel size of 750x617... so yeah, compressing that makes it become... 179 kb in size... true... not a big change... but there's 35.199 gifs left and they weigh 2.2 GB. uhm... yeah... i might squeeze about 200 megabytes out of them if they do compact this much...

but...

all this work for 200 mb? Yeeeeeeah... no.

Unless i can filter them easily in xnview of xnconvert i'll just skip it, thank you.
User avatar
m.Th.
XnThusiast
Posts: 1663
Joined: Wed Aug 16, 2006 6:31 am
Contact:

Re: Reducing size of my large image collection

Post by m.Th. »

all this work for 200 mb? Yeeeeeeah... no.
:)

Son, can I give a humble hint?

Perhaps a bigger HDD would make a difference? :)

Also, you can use this thing http://www.uderzo.it/main_products/space_sniffer/ if you're on Windows (Linuxes usually have such things in distros by default) in order to see where your space is wasted.

Hint #2: You can safely delete your CBS and Temp folders (again, if you're on Windows).
m. Th.

- Dark Themed XnViewMP 1.6 64bit on Win11 x64 -
bdragon
Posts: 108
Joined: Mon May 01, 2017 8:06 am

Re: Reducing size of my large image collection

Post by bdragon »

m.Th. wrote:Perhaps a bigger HDD would make a difference? :)
i already have 2 main hard drives, 1 tb each, and 4 backup hard drives, 2 tb each.
m.Th. wrote:Also, you can use this thing http://www.uderzo.it/main_products/space_sniffer/ if you're on Windows (Linuxes usually have such things in distros by default) in order to see where your space is wasted.

Hint #2: You can safely delete your CBS and Temp folders (again, if you're on Windows).
To check which stuff takes too much space i usually use VisDir. Old as [CENSORED] but very useful.

But... Okay, let's make it a full explanation, because i'm receiving wild guesses at what i'm trying to do. And that does not help. So yeah lenghty explanation below:

Basically in the last month each weekend i went on a "spring cleaning" of my computer. But i'm using batch converters like handbrake or XnConvert to compact the stuff i don't really need while still keeping most details, and stuff like AntiDupl.Net to find out wildly misplaced stuff and either check those folders or delete the "stragglers". I also spent time in this last month to sort up my hard drives by putting unsorted files into proper directories, delete junk, remove programs i was not using anymore or update programs i was using often.

In this specific weekend, since it was a long one and i took a "bridge" vacation what with liberation day and laborer's day i went into the folder of "memorabilia" and started to compact stuff i did not need much but still wanted to keep as a memento. Like lowering resolutions of old cartoon shows (there's still a queue of 180 cartoon episodes on handbrake even after 4 days :P ), or compacting old webcomics for which i kept an archive. I might be 40 but i still love my sluggy freelance or my gargoyles or my dominion tank police. :D call me a baby, i won't be offended. :wink:

So yeah, i have about half terabyte of stuff i need for actual work and about 400+ gb of stuff i kept for work archive and 200+ gb of old comics or old cartoons for sentimental value. I just needed to know how to do a bit of cleaning without "damaging" the original files too much and by letting the computer crunch stuff in the background without distracting me too much from enjoying my vacation and/or sorting out other stuff.

That's all.
User avatar
XnTriq
Moderator & Librarian
Posts: 6336
Joined: Sun Sep 25, 2005 3:00 am
Location: Ref Desk

Re: Reducing size of my large image collection

Post by XnTriq »

In details mode, we can sort files by type. XnView differentiates JPEGs by JFIF version
  • JPEG TrueColour (v1.1)
  • JPEG TrueColour (v1.2)
  • JPEG CMYK (v1.1)
  • JPEG CMYK (v1.2)
… and GIFs by their “magic number” (if I'm not mistaken):
  • Compuserve GIF (Version 87a)
  • Compuserve GIF (Version 87a - Interlaced)
  • Compuserve GIF (Version 89a)
  • Compuserve GIF (Version 89a - Interlaced)
GIF87a = non-animated (only one frame)
GIF89a = non-animated (only one frame) or animated (more than one frame)
bdragon
Posts: 108
Joined: Mon May 01, 2017 8:06 am

Re: Reducing size of my large image collection

Post by bdragon »

Thanks. That is very useful. :D
Post Reply