Hi!
I just started using XnView after switching from IrfanView - both great products.
I'm missing one thing from both though: Batch processing running over multiple threads.
If you're using a dual-core cpu, or a quad-core cpu, you'll only see 50% / 25% usage when running a batch job.
Using one thread per cpu-core could really speed up this process.
// hermandsen
Batch Processing running on multiple cores
Moderators: XnTriq, helmut, xnview
-
- Posts: 1
- Joined: Sat Oct 29, 2011 11:51 am
-
- Author of XnView
- Posts: 44759
- Joined: Mon Oct 13, 2003 7:31 am
- Location: France
Re: Batch Processing running on multiple cores
XnConvert uses multiple cores to process a picture.
Pierre.
-
- Posts: 242
- Joined: Tue May 17, 2005 9:24 pm
- Location: Bratislava, Slovakia
Re: Batch Processing running on multiple cores
I wholeheartedly support this request. It would be very nice if XnView could eg. resize four images (in case of a quad-core processor) at the same time instead of just one after another.
Roman
Roman
-
- Posts: 8705
- Joined: Sun Oct 12, 2003 6:47 pm
- Location: Frankfurt, Germany
Re: Batch Processing running on multiple cores
Sure enough, I also support this request. According to Pierre multiple processors are supported by XnConvert, already.
A note:
When looking at the processor load one has to take the file accessing into account. Depending on the (file) size of the images and the type of processing, the bottle neck might not be the processor(s) which perform the unpacking, the processing, and packing of image data, but the hard disk which has to supply the image data and make it persistent after processing.
A note:
When looking at the processor load one has to take the file accessing into account. Depending on the (file) size of the images and the type of processing, the bottle neck might not be the processor(s) which perform the unpacking, the processing, and packing of image data, but the hard disk which has to supply the image data and make it persistent after processing.
-
- Posts: 3
- Joined: Tue Oct 01, 2013 7:36 am
Re: Batch Processing running on multiple cores
(I apologize for any inconveniences that my poor English may cause.)
I also support this request.
In my opinion, processing multiple files simultaneously is probably the easiest and most effective way of utilizing multicore cpus 100%.
Because, it is not easy to parallelize standard algorithms or make any parallelized algorithms be able to use resource of cpu 100%.
If multiple files are to be processed simultaneously, multi-core cpus can be utilized without having any notable bottlenecks since each files are modified and saved independently.
Pixillion Image Converter and Contenta Converter use this method. And during the batch processing, the cpu usage graph in Windows Task Manager shows almost 100%. Also, it is really fast.
In regard to this method, sometimes many Lightroom users do what they call 'manual parallel processing method.'
Even though many of processing algorithms in Lightroom has already been parallelized, many people does not seem to be fully satisfied with Lightroom because it does not use their cpus 100%.
http://feedback.photoshop.com/photoshop ... n_parallel
Thus, Lightroom users do this in order to utilize their cpus 100%, separating bunch of pictures into a few independent groups and processing them simultaneously.
I think if XnView can separate files of the list of the batch window into a few group and process them simultaneously, it can easily utilize any multi-core cpus, regardless of parallel efficiency of processing algorithm.
Best Regards,
Joonsuk Kim
I also support this request.
In my opinion, processing multiple files simultaneously is probably the easiest and most effective way of utilizing multicore cpus 100%.
Because, it is not easy to parallelize standard algorithms or make any parallelized algorithms be able to use resource of cpu 100%.
If multiple files are to be processed simultaneously, multi-core cpus can be utilized without having any notable bottlenecks since each files are modified and saved independently.
Pixillion Image Converter and Contenta Converter use this method. And during the batch processing, the cpu usage graph in Windows Task Manager shows almost 100%. Also, it is really fast.
In regard to this method, sometimes many Lightroom users do what they call 'manual parallel processing method.'
Even though many of processing algorithms in Lightroom has already been parallelized, many people does not seem to be fully satisfied with Lightroom because it does not use their cpus 100%.
http://feedback.photoshop.com/photoshop ... n_parallel
Thus, Lightroom users do this in order to utilize their cpus 100%, separating bunch of pictures into a few independent groups and processing them simultaneously.
I think if XnView can separate files of the list of the batch window into a few group and process them simultaneously, it can easily utilize any multi-core cpus, regardless of parallel efficiency of processing algorithm.
Best Regards,
Joonsuk Kim
-
- Posts: 17
- Joined: Sat Jul 18, 2009 11:31 pm
Re: Batch Processing running on multiple cores
To my experience a big bottleneck is the storage. I am used to store on a hard drive or a (fast) NAS. Anyway it is not fast enough. Before processing I move all my pic to a local ssd and work on it. Batch rename is then almost instant and so is timestamp change. For timestamp change it is a huge improvment as Xnview would look frozen for a long time if pics were on the NAS.
-
- Posts: 1
- Joined: Thu Dec 26, 2013 8:06 am
Re: Batch Processing running on multiple cores
multiple cores/cpus computers have been available since 2006 and this feature has been asked for in this forum since 2011.
we're almost in 2014 now and my cpu is only being used at 13% (1/8) of its potential which means I have to wait 8 times as much as I should for the convert steps to take place.
please implement this feature.
it's definitely a feature worth buying a licence for. you can quote me on this one.
we're almost in 2014 now and my cpu is only being used at 13% (1/8) of its potential which means I have to wait 8 times as much as I should for the convert steps to take place.
please implement this feature.
it's definitely a feature worth buying a licence for. you can quote me on this one.
-
- Posts: 226
- Joined: Wed Dec 25, 2013 7:08 pm
Re: Batch Processing running on multiple cores
[quote="maerean"]my cpu is only being used at 13% (1/8) of its potential
You are lucky! New Intel's 8 core/16 threads CPU use 6% of CPU potential, I suppose.
You are lucky! New Intel's 8 core/16 threads CPU use 6% of CPU potential, I suppose.
-
- Posts: 3
- Joined: Tue Oct 01, 2013 7:36 am
Re: Batch Processing running on multiple cores
Actually, the storage is not a big bottleneck. To fully utilize my cpu, I decided to use nconvert, which is command line batch image processor, instead of XnView. In addition, thanks to Joey from stackoverflow.com http://stackoverflow.com/questions/6727 ... -processes, I could run multiple instances of nconvert simultaneously to convert image sequences.loopingz wrote:To my experience a big bottleneck is the storage.
Here is the code ...
Code: Select all
cls
@echo off
for /f %%i in ('dir "../2_Modified/" ^| find "파일"') do set numFiles=%%i
setlocal EnableDelayedExpansion
set /a "cntStart = 1"
set /a "cntFinal = 15"
set /a "maxIdx = numFiles/15"
set /a "remIdx = numFiles-maxIdx*15"
set /a "currIdx = maxIdx*15+1"
for /l %%i in (1,1,%maxIdx%) do (
call :loop !cntStart! !cntFinal!
set /a "cntStart += 15"
set /a "cntFinal += 15"
)
if %remIdx% gtr 0 (
echo Starting processing instance for %currIdx% ~ %numFiles%
start /b /high nconvert -quiet -out lwfjp2 -n %currIdx% %numFiles% 1 -o ../3_JPEG2000/Page###.jp2 ../2_Modified/Page###.tif
)
goto :eof
:loop
call :checkinstances
if %INSTANCES% leq 12 (
echo Starting processing instance for %1 ~ %2
start /b /high nconvert -quiet -out lwfjp2 -n %1 %2 1 -o ../3_JPEG2000/Page###.jp2 ../2_Modified/Page###.tif
goto :eof
)
echo Waiting for instances to close ...
ping -n 1 ::1 >nul 2>&1
goto loop
goto :eof
:checkinstances
for /f %%t in ('tasklist /fo csv /fi "IMAGENAME eq nconvert.exe"^|find /c "nconvert.exe"') do set INSTANCES=%%t
goto :eof