Coding is very slow

Anything and everything to do with DCP-o-matic.
bnm14777
Posts: 40
Joined: Fri Jun 08, 2018 8:12 am

Coding is very slow

Post by bnm14777 »

I used to create a MOV high-definition file with a bit rate of 3840X2160 and 900M. Creating a 4KDCP file (of course, the same problem existed with creating a 2KDCP file before) can be very slow, and it takes about 30 hours to complete,

The system version is windows10 X64

DOM uses the version 2.14.59 And 2.16.46、2.16.47

Hardware for MAC Pro 2012
DCP Intel X5680 3.33GHz x 24 cores
Memory 64G
Graphics card AMD RX590

During the transcoding process, the CPU usage will not exceed 80% at its highest, but it is unstable and often stays around 20%.


Is there something wrong with this?
carl
Site Admin
Posts: 2330
Joined: Thu Nov 14, 2013 2:53 pm

Re: Coding is very slow

Post by carl »

That's strange, it should be able to keep your CPUs busy most of the time. Although it's possible that the MOV decoding thread is too slow and can't feed the DCP encoder threads with enough data.

Are you using a local hard drive for your source and DCP?
Carsten
Posts: 2648
Joined: Tue Apr 15, 2014 9:11 pm
Location: Germany

Re: Coding is very slow

Post by Carsten »

I have seen that happen when DCP-o-matic turns to swapping, which means, number of encoding threads and RAM is not balanced. And often when slower external discs (USB) are used.

How many encoding threads have you set? 4k needs four times as much memory as 2k for every thread. Is this a single or dual CPU system? The 5680 is a 6core HT CPU, so, for a dual 5680, something like 24-32 encoding threads should be the sweet spot, but with 4k this could already result into swapping. Check memory usage during encoding.


What does '900M' mean? Did you set a 900MBit/s J2K/DCP data rate? Or is that the source/master file data rate?

What is the frame rate of the DCP?


- Carsten
bnm14777
Posts: 40
Joined: Fri Jun 08, 2018 8:12 am

Re: Coding is very slow

Post by bnm14777 »

carl wrote: Wed Mar 22, 2023 4:36 pm That's strange, it should be able to keep your CPUs busy most of the time. Although it's possible that the MOV decoding thread is too slow and can't feed the DCP encoder threads with enough data.

Are you using a local hard drive for your source and DCP?
All running on local disk

The DOM encoding thread has been set to 64.
bnm14777
Posts: 40
Joined: Fri Jun 08, 2018 8:12 am

Re: Coding is very slow

Post by bnm14777 »

Carsten wrote: Wed Mar 22, 2023 6:21 pm I have seen that happen when DCP-o-matic turns to swapping, which means, number of encoding threads and RAM is not balanced. And often when slower external discs (USB) are used.

How many encoding threads have you set? 4k needs four times as much memory as 2k for every thread. Is this a single or dual CPU system? The 5680 is a 6core HT CPU, so, for a dual 5680, something like 24-32 encoding threads should be the sweet spot, but with 4k this could already result into swapping. Check memory usage during encoding.


What does '900M' mean? Did you set a 900MBit/s J2K/DCP data rate? Or is that the source/master file data rate?

What is the frame rate of the DCP?


- Carsten
The same phenomenon with both local and USB drives

Encoding threads are 64
Maximum number of frames processed per thread is 12

Source file Bit rate 947MB/s
DCP code rate set to 220MB/s

There is also a strange phenomenon
Running DOM Encode Server on another MAC instead prolongs the encoding time
carl
Site Admin
Posts: 2330
Joined: Thu Nov 14, 2013 2:53 pm

Re: Coding is very slow

Post by carl »

Maybe you could post the "log" file from inside one of your projects, or email it to carl@dcpomatic.com
bnm14777
Posts: 40
Joined: Fri Jun 08, 2018 8:12 am

Re: Coding is very slow

Post by bnm14777 »

carl wrote: Thu Mar 23, 2023 8:50 am Maybe you could post the "log" file from inside one of your projects, or email it to carl@dcpomatic.com
Hi Car Log has been sent to the mailbox. Please check
Carsten
Posts: 2648
Joined: Tue Apr 15, 2014 9:11 pm
Location: Germany

Re: Coding is very slow

Post by Carsten »

64 threads is too much for a dual x5680. Pretty sure that this causes the swapping when encoding 4k. Swapping is also the reason why the CPU cores are seen idling.
bnm14777
Posts: 40
Joined: Fri Jun 08, 2018 8:12 am

Re: Coding is very slow

Post by bnm14777 »

Carsten wrote: Thu Mar 23, 2023 11:43 pm 64 threads is too much for a dual x5680. Pretty sure that this causes the swapping when encoding 4k. Swapping is also the reason why the CPU cores are seen idling.
The same is true when configured at 24 or 32 threads
Carsten
Posts: 2648
Joined: Tue Apr 15, 2014 9:11 pm
Location: Germany

Re: Coding is very slow

Post by Carsten »

Check CPU activity and swap size in activity monitor. Even with modern SSD discs, swapping definitely kills encoding performance.


But it could also be the source file, at 900MBit/s and 4k, decoding will certainly take a significant part. You could easily check that out with different source files. Stay with max 32threads.
Post Reply