Speed of exporting to MP4

[sup]

What speed can I expect when exporting to an MP4?

My computer is reasonably powerful (Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz and 32GB of RAM) and can preview the film in real time when played on a FullHD screen. When played on 4K screen, it is laggy, the film is 4K. It encodes to MP4 (I set CRF to 16) around 3-4 FPS. Is that to be expected?

Is there a way to make it faster (for example by scaling to 2K or using my discrete nvidia card for encoding the MP4)? I am on 2.14.54, would a new development version change anything in this regard?

[Carsten]

Not based on solid numbers (as I rarely export), but from my memory the export speed had huge variations over different versions of DCP-o-matic. It is true that currently it seems as if export to MP4 and Prores is actually slower than J2K encoding. Using 4k certainly doesn't help. Do you actually need 4k exports? You can create 2k exports from 4k DCPs if you set the DCP to 2k. Export uses current DCP settings.

- Carsten

[sup]

Indeed, setting it to 2K speeds up things (to around 8 FPS). In in the timings tab, I can trim it to export jhust three minutes to inspect the result before committing to exporting the whole film, perfect, so this helps. Carl himself opened a bug to implement this in the export window: https://dcpomatic.com/mantis/view.php?id=1793

Is decoding J2K faster than encoding? Because if so, it is weird that exporting takes more time than creating a DCP. I wonder what ffmpeg switches are used internally. It would be great if one could set more settings to ffmpeg - for example preset (is it not set to ultraslow now:) and profile and even codec, I would love to use h264_nvenc (so my NVidia card), as it is quite a lot faster than CPU.

Also, 3 minutes at CFR 18 into MP4 were 1.6 GB big. Some tweaking to the parameters to make the file smaller would be welcome. Maybe this is the reason it is so slow?

I actually requested something similar three years ago (last comment): https://dcpomatic.com/mantis/view.php?id=1103

It seems that the MP4 version is still more for reuse use case and not a screener.

[sup]

In any case, a 2K MP4 export of 135min movie with CRF 17 has bitrate over 100 000 (one hundred thousand) kbits and is around 96 GB. That is about ten times more than would be more than enough.

[sup]

Hm, this seems to be DCP specific. I am ripping another three DCPs with CRF around 28 and the sizes will be all under 10 GB. Or can a difference in CRF of 10 make a 90 GB difference in size of the resulting export? A rule of thumb for CRF when converting normal h264 files is to use a value of around 17 to keep perceptually the same quality. Is there so much more data in a DCP that using this value just adds to duplicating the DCP size? I always thought DCPs are quite inefficient in this regard and that a 10-20 GB mp4 file can encode as much as a 150 GB DCP.

Some rudimentary info about this might be handy in here: in https://dcpomatic.com/manual/html/ch09.html

For example a recomended value of CRF for certain results and maybe a hint that what gets exported is what is set - ie. that setting the DCP to 2K will speed things up, or that burning subtitles is also possible here. It was not actually obvious to me at all.

[Carsten]

A long time ago did some tests with different quality indicators and they indeed appeared not to be very intuitive. However, I usually take the highest number for low quality and usually 'something' lower (around 30) for medium quality and am fine with it. The lowest quality will usually already show considerable blocking, which can be used even as a bare minimum copy protection scheme if needed, almost like a watermark.

Don't know which version you are using, but in the current test version, all the indicators (like 'MP4/h264') look okay to me. They are not chosen for logical consistency, but so that the ordinary user understands what to expect.

Both ProRes and MP4 result in quicktime containers being created, so, telling both apart the way it is now looks 'useful' to me. Given that MP4 will usually use h.264.

We never wanted the export dialog to look as messy as the typical video editor/converter codec and settings mess. As a matter of fact, there are already too many settings now. And more could be useful (e.g. one to get rid of pillar/letter boxing), but...

- Carsten

[sup]

I am using the latest stable version. I as an ordinary user (who tends to be quite systematic though) am not sure what to expect of "Prores": "MOV/ProRes" would be a lot clearer to me. I do not understand your remark about "quicktime containers" - I thought MOV is a quicktime container, but not MP4.

I understand the need not to confuse the user. If the defaults were changed to make it faster, I would not mind at all. The issue I am dealing with is 1) slow export and 2) big file size.

1) The slow export was partially solved by setting the movie to 2K and then exporting. But it could be further tweaked by using hardware acceleration for mp4 creation and tweaking some ffmpeg settings. Best would be to change the defaults, but given people have different hardware etc., I would not mind a preference in "advanced" that would allow me to set my own ffmpeg parametres. Do you think that would be a sensible thing to do?

Setting CRF lower also makes it faster, though not as much as using 2K instead of 4k.

2) The big file sizes puzzles me. I just tried to export one minute of a 4K source at 2K at 16 and 28 CRF and the resulting size is around 667 to 113 MBs. That is five times more. When I then run the results through

Code: Select all

ffmpeg -i 667MBFILE.mp4 -c:v libx264 -preset slow -crf 16 -c:a copy -y OUTPUT1.mp4
ffmpeg -i 667MBFILE.mp4 -c:v libx264 -preset slow -crf 28 -c:a copy -y OUTPUT2.mp4
ffmpeg -i 113MBFILE.mp4 -c:v libx264 -preset slow -crf 16 -c:a copy -y OUTPUT3.mp4
ffmpeg -i 113MBFILE.mp4 -c:v libx264 -preset slow -crf 28 -c:a copy -y OUTPUT4.mp4

I get the following sizes: 150 MB, 100 MB,100 MB and 6 MB. That last 6 MB is actually surprising me the most (and the result is still not especially pixelated). I always thought that setting a CRF around 17 or so pretty much preserves perceptual quality while offering a reasonable size as opposed to pseudo(?)lossless formats as DCP. Bu I never realized the input parametres can influence the resulting video so much if the scene is exactly the same. Now I see that when I transcode the resultign video several times, always feeding the result of the transcode to the next one, it gradually reduces the size of the video from 150 MB to 130 MB, then to 120 MB etc.

So, the result is that the rule of thumb to use CRF 17 to preserve perceptual equality does not really hold for DCP - CRF at around 28 is more thna enough (to my eyes).

Would it not be a good idea to mention some of these things in the manual?

[Carsten]

As a matter of fact, MP4 is an officially standardised version of MOV - MP4 may only contain a subset of features that Quicktime supports (e.g. no Prores ). But, technically, MP4 is a Quicktime container. Not too important, though.


I'm not familiar with using CRF. I understood that the quality setting in the export dialog uses some more or less arbitrary parameter from the FFMPEG code base. Admittedly, something between 10 (best) and 0 (worst) would probably be better for user expectation.

I can't remember wether the choices had to do with it, but, Prores is needed for some Pro-Applications, but (especially under Windows), Prores codec is not supported in many applications. Hence I see a need for a high quality MP4/h.264 for those who want to preserve quality (e.g. for editing and DCP re-conversion), but are limited to h.264.

Agreed, we could use MOV/ProRes, and MP4/h.264.

- Carsten

[sup]

So I filed a but to rename the ProRes export option here: https://dcpomatic.com/mantis/view.php?id=2067

CRf is Constatnt Rate Factor and I am pretty sure that is what the setting in DOM is. The range (0-51) is the same. More info here: https://trac.ffmpeg.org/wiki/Encode/H.264 That also explains the different sizes of the files from DCP-o-Matic - a difference of 12 in CRF should correspond in roughly four-time difference in file size.

Thanks for claryfying about the MOV container, I had no idea MP4 is a subset of it.

I also filled a bug to expand the documentation of the export dialog: https://dcpomatic.com/mantis/view.php?id=2068

Not sure if I should open a bug about the request to be able to pass one!s own settings to ffmpeg or if the discussion in bug https://dcpomatic.com/mantis/view.php?id=1103 is enough?

[carl]

Passing custom settings to ffmpeg isn't so simple because DCP-o-matic does not call the ffmpeg binary but instead it uses the libavcodec/libavformat C API.