Strange Encoding Artifacts of DOM

[freezer]

I have a very strange issue with one particular scene in a movie where the encoding by DOM does lead into single frame compression artifacts/glitches.

The master file is a DCI 2K rec709 DNxHR HQX 12bit MXF exported from Davinci Resolve, the whole footage went to heavy Dehancer film grain process. The master file however does not show any of those glitches.

The scene is a night shot in the woods against the sky - which has been graded to have a deeply saturated blue.
In an older version of the master file the DCP (120 Mbit/s) glitches got even stronger, I have countered them by reducing the saturation in that shot and also raising the bitrate of the DCP to 150 MBit/s.

Nightsky J2k Glitch saturated blue version.jpg

Nightsky J2k Glitch less saturated blue version.jpg

I will prepare two MXFs of that particular shot tomorrow for testing and place an upload link here.

To me it looks like the J2k encoder hits an edge case with this particular shot resulting in compression glitches. I have never seen that before with DOM.

DOM version: 2.16.39

[Carsten]

Are you sure this is not caused/triggered by the DNxHR HQX codec? Can you try Davincis integrated Kakadu J2K encoder, or some uncompressed format for the few seconds where this issue shows up?

Why do you limit yourself to 150MBit/s?

Whatever, it seems to occur because of added noise/grain. Color noise is the only component that can bring J2K to it's knees. However, even then I usually only see softening, but not color artefacts.

[freezer]

Are you sure this is not caused/triggered by the DNxHR HQX codec?

The DNxHR codec master itself does not show any glitches in this shot.

Can you try Davincis integrated Kakadu J2K encoder, or some uncompressed format for the few seconds where this issue shows up?

Yes, good idea. I will test this.

Why do you limit yourself to 150MBit/s?

Because I have to send the DCP over WeTransfer to the cinema for testing and this way I get a manageable size. Not so much a restriction on my upload side but the cinemas do have worse bandwidth for downloads. Quite often they have slower download rates then my upload rate (which is 50 MBit/s).

Whatever, it seems to occur because of added noise/grain. Color noise is the only component that can bring J2K to it's knees. However, even then I usually only see softening, but not color artefacts.

My guess is that here a very specific combination of factors came together and created an edge case for the encoder forcing it to break.

I'll prepare some different samples (also with less colored grain) and codecs to test.

[freezer]

Just an update of my test series:

1. Direct encoding to DCP via Davinci Resolve Kakadu J2K encoder at 150 Mbit/s does not have any glitches
2. Encoding at 150 Mbit/s in DOM from 3 different file types always results in those glitches. Tested with 12bit DNxHR MXF, 10bit DNxHR MXF and 16bit TIFF image sequence
3. Encoding at 200 Mbit/s in DOM did result in no visual glitches anymore

I uploaded all 3 files types (12bit DNxHR MXF, 10bit DNxHR MXF and 16bit TIFF image sequence) for others to test, link here:
https://we.tl/t-52aCiuuDRM

EDIT:
I did a test with the grain completely off and the glitches are still there at 150 Mbit/s
So it is save to say that the grain does not trigger this but the complexity of the image content - it seems to happen around 2 stars and the branches when they are very sharp for a frame.

Link to the file without grain:
https://we.tl/t-BDwAp1BoE4

[Carsten]

High contrast high detail. Although I have never before seen artifacts bleeding out like this into larger areas.

It's also pretty weird that this happens only in one area of the frame, whereas other parts of the sky have similar features but don't show these artefacts.

BTW - the TIFF in the collection you linked to above is unpacked into 'Nightsky_less_sat_TIFF.RAR' - why RAR - and my Mac can not unRAR it.

[freezer]

"The Unarchiver" is able to extract RAR5:
https://apps.apple.com/at/app/the-unarc ... 4353?mt=12

Why RAR? Because it is far superior to ZIP. 256-bit BLAKE2 file hash, optional data redundancy is provided in the form of Reed–Solomon recovery records and recovery volumes, allowing reconstruction of damaged archives (including reconstruction of entirely missed volumes) and it also recognizes identical image sequences like in a DCP with a still frame and is able to compress that down from e.g. 10 GB to a few MB.