View Bug Details

IDProjectCategoryView StatusLast Update
0001607DCP-o-maticBugspublic2021-01-08 00:37
Reporteroverlookmotel Assigned Tocarl  
PriorityurgentSeverityminorReproducibilityalways
Status closedResolutionfixed 
PlatformMacOSOS XOS Version10.12
Product Version2.14.0 
Target Version2.14.11 
Summary0001607: Inconsistency in calculation of Position, TrimStart and TrimEnd in metadata.xml
Description

My understanding is that <Position>, <TrimStart> and <TrimEnd> in metadata.xml are calculated as: 96000 / frame rate x number of frames.

But is that the frame rate of the content or the DCP?

Using a video file which is 23.98fps (well 24000/1001fps), when I set Position, Trim Start and Trim End all to 10 frames, with DCP being made at 24fps, I get the following values in metadata.xml:

Position: 40000
TrimStart: 40040
TrimEnd: 40000

And if DCP frame rate is set to 25fps:

Position: 38400
TrimStart: 40040
TrimEnd: 38400

It appears that TrimStart is calculated off frame rate of content, TrimEnd off frame rate of DCP.

Is this the intended behavior? And if not, would it cause the content to be trimmed incorrectly?

Even weirder:

  • Set DCP frame rate to 25fps
  • Set trim start + trim end to 10 frames
    -> TrimStart = 40040, TrimEnd = 38400
  • Change DCP frame rate to 24fps
    -> TrimStart = 40040, TrimEnd = 38400 (no change)
  • Set trim start + trim end again (I mean don't change the value, but delete a digit and add it again so you can click "Set")
    -> TrimStart = 40040, TrimEnd = 40000 (TrimEnd changes)

i.e value of TrimEnd depends on whether you set Trim before or after setting the DCP frame rate.

I ask all this for a couple of reasons:

  1. I have a script which runs files through FFMPEG to identify black+silence at start and end and edits metadata.xml to set trim accordingly. It works fine for 24/25fps sources, but goes wrong for 23.98fps sources because I don't understand the logic.

  2. I programmatically add a frame of black to start and end of all DCPs, again by a script which edits metadata.xml. Calculating the right position for the content + the black frame is also complicated by this.

One other question: If you have two pieces of video content overlapping, i.e. Position of 2nd content is less than where 1st content ends, what will happen? Will DCP-o-matic correct it, ignore it, or may it lead to unpredictable behavior? Obviously this will never happen if you're using the GUI, but if you're editing metadata.xml outside DCP-o-matic, it could be the case.

TagsNo tags attached.
Branch
Estimated weeks required
Estimated work requiredUndecided

Activities

carl

2019-09-20 09:24

administrator   ~0003417

As you say, the times are stored in seconds multipled by 96000, so a time of 96000 is 1 second.

Position is stored at the DCP rate.
Trim is stored at the content rate.

Also, when you set position it is rounded to the nearest DCP frame boundary.
Start trim is rounded to the nearest content frame boundary.

I think this explains why you are seeing the differences in position depending on the DCP rate.

Finally, it looks like there's a bug where it uses the DCP frame rate (rather than the content one)
when it reads the frames part of start/end trim. I've fixed that in my local copy.

As to the overlapping thing; I'm not sure. I suspect it may lead to problems but we can test it and work that stuff out if you need.

Does that make sense?

overlookmotel

2019-09-20 14:18

developer   ~0003418

Thanks. That does make sense.

Can you clarify one thing: Is the bug you mention the reason why I'm seeing TrimEnd of 40000/38400 for 10 frames (rather than 40040)?

Also, two other related questions, if you have time to explain:

  1. When calculating TrimStart value based on content frame rate, does DCP-o-matic calculate this simply as 96000 / frame rate x number of frames? Or does whether the file uses drop frame/non-drop frame timecode come into it? i.e. are timestamps in the video file taken into account, or is it purely about number of frames?

  2. For WAV files, does value of <Length> depend on the sample rate of the WAV? Or is it always calculated as 48000 x number of seconds? I haven't had a 96k WAV file recently to test this.

overlookmotel

2019-09-20 14:21

developer   ~0003419

Overlapping: I was only asking as I'm worried with my miscalculations of Trim etc, I might have done this by accident. If I can figure out the correct values for trim fields, this will be a non-issue. We've checked all DCPs we've made with non-24/25fps content and not seen any glitches as yet.

overlookmotel

2019-09-20 14:32

developer   ~0003420

Sorry, one last question:

To work around TrimEnd bug (it only seems to apply to TrimEnd, not TrimStart, as far as I can see):

  • Should I emulate the bug's behavior and calculate TrimEnd value based on frames part at DCP frame rate rather than content rate i.e. so my script produces same XML as DOM GUI?
    or:
  • Should I calculate it correctly, ignoring the bug?

Put another way: Is the bug just in the DCP-o-matic GUI writing miscalculated trimEnd values, or does the encoder also have same bug and therefore will miscalculate where to cut the content when reading the TrimEnd field? I assume the former, but just want to make sure.

I know this is all severe nit-picking. I just want to get it exactly right to avoid any glitches due to overlapping content.

carl

2019-09-20 20:30

administrator   ~0003421

Can you clarify one thing: Is the bug you mention the reason why I'm seeing TrimEnd of 40000/38400 for 10 frames (rather than 40040)?

I think so, yes.

When calculating TrimStart value based on content frame rate, does DCP-o-matic calculate this simply as 96000 / frame rate x number of frames? Or does whether the file uses drop frame/non-drop frame timecode come into it? i.e. are timestamps in the video file taken into account, or is it purely about number of frames?

It's just about the number of frames.

For WAV files, does value of <Length> depend on the sample rate of the WAV? Or is it always calculated as 48000 x number of seconds? I haven't had a 96k WAV file recently to test this.

<Length> is in samples. So for a 48kHz file 1 second would be 48000; for a 96kHz file it would be 96000.

To work around TrimEnd bug (it only seems to apply to TrimEnd, not TrimStart, as far as I can see):

Should I calculate it correctly, ignoring the bug?

Yes, I would do this. It's just the GUI writing incorrect values.

overlookmotel

2019-09-25 13:51

developer   ~0003435

Thanks for all the answers.

Would you be willing to merge the bug fix for trim calculation into stable branch and make a release? (feel free to say no - I have no idea how much work it is to cut a release, or whether you fear this change might have other knock-on effects and so don't want to merge into stable yet)

carl

2019-09-25 21:11

administrator   ~0003436

I think that would be fine. I am still without a build server for another few days (or could be longer) so I can't do it immediately.

overlookmotel

2019-09-26 14:05

developer   ~0003437

OK great. Thank you.

By the way, have you considered building in the cloud using something like Travis CI? It's free for open source. Maybe something I could help with if you wanted to try it.

carl

2019-09-27 22:09

administrator   ~0003438

I'll have a look at travis. I didn't know it was free...

carl

2019-10-04 00:20

administrator   ~0003445

I had a go with travis. A build for even one Linux target overran their 50 minute limit, so I think that's a no-go.

overlookmotel

2019-10-23 00:53

developer   ~0003494

Thanks very much for including this in stable release. I'll give it a go and see if I can fix my trimming malfunctions!

I had no idea DCP-o-matic build would take so long. 50 mins+ is crazy. Sorry that was a red herring with Travis.

Bug History

Date Modified Username Field Change
2019-09-20 01:43 overlookmotel New Bug
2019-09-20 09:24 carl Assigned To => carl
2019-09-20 09:24 carl Status new => feedback
2019-09-20 09:24 carl Note Added: 0003417
2019-09-20 14:18 overlookmotel Note Added: 0003418
2019-09-20 14:18 overlookmotel Status feedback => assigned
2019-09-20 14:21 overlookmotel Note Added: 0003419
2019-09-20 14:32 overlookmotel Note Added: 0003420
2019-09-20 20:30 carl Note Added: 0003421
2019-09-25 13:51 overlookmotel Note Added: 0003435
2019-09-25 21:11 carl Note Added: 0003436
2019-09-26 14:05 overlookmotel Note Added: 0003437
2019-09-27 22:09 carl Note Added: 0003438
2019-09-28 09:03 carl Priority normal => urgent
2019-09-28 09:03 carl Target Version => 2.14.x
2019-10-04 00:20 carl Note Added: 0003445
2019-10-14 16:28 carl Target Version 2.14.x => 2.14.11
2019-10-14 16:28 carl Status assigned => resolved
2019-10-14 16:28 carl Resolution open => fixed
2019-10-23 00:53 overlookmotel Note Added: 0003494
2021-01-08 00:37 carl Status resolved => closed