Coding is very slow

[bnm14777]

I used to create a MOV high-definition file with a bit rate of 3840X2160 and 900M. Creating a 4KDCP file (of course, the same problem existed with creating a 2KDCP file before) can be very slow, and it takes about 30 hours to complete,

The system version is windows10 X64

DOM uses the version 2.14.59 And 2.16.46、2.16.47

Hardware for MAC Pro 2012
DCP Intel X5680 3.33GHz x 24 cores
Memory 64G
Graphics card AMD RX590

During the transcoding process, the CPU usage will not exceed 80% at its highest, but it is unstable and often stays around 20%.


Is there something wrong with this?

[carl]

That's strange, it should be able to keep your CPUs busy most of the time. Although it's possible that the MOV decoding thread is too slow and can't feed the DCP encoder threads with enough data.

Are you using a local hard drive for your source and DCP?

[Carsten]

I have seen that happen when DCP-o-matic turns to swapping, which means, number of encoding threads and RAM is not balanced. And often when slower external discs (USB) are used.

How many encoding threads have you set? 4k needs four times as much memory as 2k for every thread. Is this a single or dual CPU system? The 5680 is a 6core HT CPU, so, for a dual 5680, something like 24-32 encoding threads should be the sweet spot, but with 4k this could already result into swapping. Check memory usage during encoding.


What does '900M' mean? Did you set a 900MBit/s J2K/DCP data rate? Or is that the source/master file data rate?

What is the frame rate of the DCP?


- Carsten

[bnm14777]

That's strange, it should be able to keep your CPUs busy most of the time. Although it's possible that the MOV decoding thread is too slow and can't feed the DCP encoder threads with enough data.

Are you using a local hard drive for your source and DCP?

All running on local disk

The DOM encoding thread has been set to 64.

[bnm14777]

I have seen that happen when DCP-o-matic turns to swapping, which means, number of encoding threads and RAM is not balanced. And often when slower external discs (USB) are used.

How many encoding threads have you set? 4k needs four times as much memory as 2k for every thread. Is this a single or dual CPU system? The 5680 is a 6core HT CPU, so, for a dual 5680, something like 24-32 encoding threads should be the sweet spot, but with 4k this could already result into swapping. Check memory usage during encoding.


What does '900M' mean? Did you set a 900MBit/s J2K/DCP data rate? Or is that the source/master file data rate?

What is the frame rate of the DCP?


- Carsten

The same phenomenon with both local and USB drives

Encoding threads are 64
Maximum number of frames processed per thread is 12

Source file Bit rate 947MB/s
DCP code rate set to 220MB/s

There is also a strange phenomenon
Running DOM Encode Server on another MAC instead prolongs the encoding time

[carl]

Maybe you could post the "log" file from inside one of your projects, or email it to carl@dcpomatic.com

[bnm14777]

Maybe you could post the "log" file from inside one of your projects, or email it to carl@dcpomatic.com

Hi Car Log has been sent to the mailbox. Please check

[Carsten]

64 threads is too much for a dual x5680. Pretty sure that this causes the swapping when encoding 4k. Swapping is also the reason why the CPU cores are seen idling.

[bnm14777]

64 threads is too much for a dual x5680. Pretty sure that this causes the swapping when encoding 4k. Swapping is also the reason why the CPU cores are seen idling.

The same is true when configured at 24 or 32 threads

[Carsten]

Check CPU activity and swap size in activity monitor. Even with modern SSD discs, swapping definitely kills encoding performance.


But it could also be the source file, at 900MBit/s and 4k, decoding will certainly take a significant part. You could easily check that out with different source files. Stay with max 32threads.