DCP Ingesting stops at 99% on cinema server

[scorpio81]

I believe it could also be bad sectors on first disk or bad copying to the first disk.

I'm gonna tell you something that we have experienced with DCP ingest.
We have a portable HDD without external power supply. For a year and maybe more everything was fine, ingests were done without a glitch. But all of a sudden ingests started to fail - local and copied files had different hashes. That means that copied file hash didn't match one in the DCP description files. We copied our DCPs to other parts of the disk, copied only files that failed to ingest... After that ingest could fail on other file or complete without a glitch. So we started to think our HDD had bad sectors. Checked it with different software and no bad sectors were found...
And I tried to connect this disk to another USB port and voila! All ingests are okay from this time!

So even USB port means something!

[kmeh7]

Hi scorpion81

Ya even that is a possibility !!!!

[kmeh7]

Hi scorpion81,

What do mean by incoreect file hashes ????

One of my dxp again stuck ay 99%

Kmeh7

[kmeh7]

Hi ,

Is there any software to check the verification of dip for mac ??

thanks again

Kmeh7

[scorpio81]

Hi kmeh7!

Each file in DCP has its hash (aka checksum or fingerprint, calculated using SHA1 algorithm and converted to Base64 format) written in PKL and CPL files. Servers check those hashes and if copied file hash doesn't match the one in PKL/CPL ingest stops with an error.
The reasons could be: a) bad sectors on drive used to create DCP, drive where DCP is copied to for ingest or drive(s) on server; b) errors during copying process; c) errors in hash calculation.

You can check hashes by yourself. Find text between <hash>and </hash> tags in PKL file, copy it and convert from Base64 to HEX without spaces(use trial version of Text Conversion Software from HotHotSoftware). Do it for every file and create text file in your DCP folder with .sha extension with the following look:

Code: Select all

7B763B5E184C0D377355B50312FEAAB56F8052EF *j2c_1a8214d2-143c-4f5e-aeb5-384672b0db21.mxf
D58AE8DCDD040513AFA1AC513FF5015BF8D93897 *pcm_bbc59028-18bf-48ed-a39a-4b2ec7d1a019.mxf
AE403D650E7ADA0F10F8F1B3783F85FD37651AF7 *cpl_b0dc8fed-235d-4319-b833-add419f4a908.xml

Each file should be on separate line. Then use any software capable of creating and verifying file checksums.

P.S. Filename is taken from Assetmap, hash is taken from PKL by file UUID assigned in Assetmap. That is in general for any DCP. DOM projects have file UUIDs in their filenames, so there's no need to look for filenames in Assetmap.
P.P.S. It is not very quick and easy procedure but you can check integrity of any DCP without need to ingest it.

[kmeh7]

Thanks scorpio81 for your detailed inofrmation.

[kmeh7]

Hey Scorpion 81,

Can you make a video tutorial or something that expalin your information to me more clearly

I am not from tech background but i can pick it up if you be little clearer.

Thanks in advance
Kmeh7

[scorpio81]

Hi kmeh7!

All right, I'll try.

This is part of Assetmap:

Code: Select all

<AssetList>
<Asset>
<Id>urn:uuid:7e403d50-4606-4493-a13c-702712b41f3a</Id>
<PackingList>true</PackingList>
<ChunkList>
<Chunk>
<Path>pkl_7e403d50-4606-4493-a13c-702712b41f3a.xml</Path>
<VolumeIndex>1</VolumeIndex>
<Offset>0</Offset>
<Length>8207</Length>
</Chunk>
</ChunkList>
</Asset>

1. Look at <Id> tag - this is the UUID for this asset. The filename is under <Path> tag. Write down this pair: filename-UUID. Since UUID is unique so there's no need to write down all symbols, the first 4 to 8 symbols would be enough. Do it for every asset in the Assetmap (except PKL) so you'll get a list of at least 3 pairs: one for CPL, one for video mxf and one for audio mxf.

This is part of PKL:

Code: Select all

<Asset>
<Id>urn:uuid:4729ba79-f13f-4f80-8259-ffbb5936ff33</Id>
<AnnotationText>pcm_4729ba79-f13f-4f80-8259-ffbb5936ff33.mxf</AnnotationText>
<Hash>kvn6DR+lSzem9cqyBnW0snJQsaw=</Hash>
<Size>369978114</Size>
<Type>application/x-smpte-mxf;asdcpKind=Sound</Type>
</Asset>

2. Look at <Id> tag: this is the UUID you need to choose the right file from the pair filename-UUID you wrote before. Please note that this PKL has <AnnotationText> tag with filename so in this case you don't need Assetmap at all. And even if there is no <AnnotationText> but filename contains UUID (all these hexadecimal values) then you don't need Assetmap also. But I'm explaining you the general basics so Assetmap is used.
Find the <Hash> tag and copy its value (everything between <Hash> and </Hash> tags.

3. Open Text Conversion Software from HotHotSoftware, paste the hash to the left window, choose Base64 for the left window and Hex for the right window, uncheck "Add spaces between values" and click "Decode/Encode Content" - in the right window you'll get hexadecimal value of file hash. Copy it.

4. Open Notepad, paste hash, add space and asterisk, then copy and paste filename - the example of what you should get I've written earlier.

5. Do steps 2-4 for every asset (at least two more times), so in the end you will get at least 3 pairs hash-filename. Save the file with extension .sha1 (possibly you will have to manually remove the .txt extension that Notepad adds by default) to your DCP folder. Name doesn't matter.

6. I use TotalCommander to check hashes. Just open your DCP folder and doubleclick your .sha1 file and wait for result.

By the way, I have added here a feature request so future versions of DOM could do this verification for any DCP automatically. And it doesn't matter if DCP is encrypted since only hashes do matter.

[kmeh7]

Hi scorpion81

Thankyou you so much . You are Fantastic.

Anyways i wanted to share something funny
I hash checked a dcp in easydcp player which showed me error in video mxf file.

But when it went to loading at server it ingested Successfully.

It was a INTEROP dcp.

So does it mean interOp dcps are more safer drom validation point of view ???

Thanks again

Kmeh7

[scorpio81]

Hi again!

Did you stop and then restart your encoding at the beginning? What kind of error did EasyDCP player show? And did you try to playback your DCP from the server?

I once had a somewhat similar situation. EasyDCP showed an error but DCP ingested succesfully on Dolby server. But playback failed to start on Dolby, it just freezed. Playback could only be started if I manually fast forwarded a few seconds into the clip.
This was due to some kind of bug in DOM 2.8.0 - if encoding was stopped after a few seconds fro the start and then restarted, it seemed that video mxf was corrupted or something, right at the point of stop/restart. Don't know. There was a problem with hashes also. Carl sent me a test version where this bug was fixed. At least I never encountered this bug after that.