View Bug Details

IDProjectCategoryView StatusLast Update
0001272DCP-o-maticBugspublic2018-10-17 20:16
Reportercarl Assigned Tocarl  
PriorityhighSeverityminorReproducibilityalways
Status closedResolutionfixed 
Target Version2.14.0 
Summary0001272: No detection of source files changing if they have the same name and location
Description

[]

TagsNo tags attached.
Branch
Estimated weeks required
Estimated work requiredUnknown

Activities

overlookmotel

2018-04-28 15:17

developer   ~0002405

Checking last modification timestamp on file(s) would be good, I think.

Could also re-run ffprobe and check nothing has changed, though that would require handling any change in output from ffprobe between versions - to prevent "file has changed" being triggered when opening project files created on an old version.

But I think it'd be annoying if source file changing was verified by calculating a hash as it would slow down opening projects. For features made from a 100GB ProRes file, the time taken to calculate the hash would be very long.

Carsten

2018-04-30 14:32

manager   ~0002408

Last edited: 2018-05-01 00:13

I often have that happening when creating slides or doing subtitle tests - change the source file while DCP-o-matic is still open. Currently, there is no way to refresh manually, so, to be sure, I either have to delete and reload the content, or quit and restart DCP-o-matic.

I remember that DCP-o-matic once had a source file consistency check I think, but if I remember right, it was modified because it needed to many resources? I remember that folder with the many small files...

Checking a file for modification is probably the easiest way. Or a manual 'file refresh' option.

  • Carsten

carl

2018-05-16 23:57

administrator   ~0002434

Does right-click -> Re-examine in the content list work for the change-while-DoM-is-open case?

Maybe we should re-examine all content on loading a project: it doesn't hash the whole file, just the start and the end of it.

carl

2018-08-16 22:24

administrator   ~0002621

I'm thinking of a quick(-ish) job which runs on project load and before making a DCP which checks all the content files to see if they have changed since they were added to the project; if they have, it re-examines them. Does that sound about right? The check would probably be DoM's "poor man's hash" (i.e. hash of the first and last Mb of each file plus its length) and also last modification time.

Carsten

2018-08-17 00:09

manager   ~0002624

Last edited: 2018-08-17 01:34

Sounds okay to me. I do background changes to sources occasionally, e.g. when I notice a bitmap image is not right, I recreate it, overwritting the first one(s). I do expect DCP-o-matic to notice that change when I reopen the project or recreate the DCP.
I would also love to edit subtitle files in an external editor, and have a way to update the changes in DCP-o-matic.
If that's not possible, an internal subtitle editor will do... ;-)

  • Carsten

carl

2018-08-21 23:19

administrator   ~0002633

Last edited: 2018-08-21 23:19

The basics of this are implemented in f0f4b2371ff5cd6fc39239a3c201bae0c033d21f and preceding commits (will be in 2.13.45).

Carsten

2018-08-22 16:43

manager   ~0002635

Had it just yesterday - had to adjust a date in a slide I made a DCP from a few minutes before. After overwriting the PNG file, reopened the project in DCP-o-matic, saw the new/correct slide in preview, but creating the DCP of course reused the preexisting MXF in video, even after being asked wether to overwrite the DCP. Had to delete the video mxf manually before I could create a new DCP using the new slide. Now, I know this, I know how to deal with this, and it doesn't bother me much..
The question is, is this unexpected or problematic behaviour for the average user? And if we change it, do we create different issues?

  • Carsten

carl

2018-08-22 19:26

administrator   ~0002636

It feels problematic to me. The only fly in the ointment here is that the decision of whether or not to remake the video is based on a quite bad hash function: a hash of the first and last megabytes of the file, plus the file size. It's pretty unlikely that you could change the file without this hash changing, but not impossible (and much more likely than with a proper hash). But we can't really do full hashes when checking things as they could be very slow.

We could add last-write-time into the mix but this may cause some false re-makes if files are updated / copied but not changed.

Even with the "bad" hash I think it's better to stop the situation you describe even if it's only most of the time.

Carsten

2018-08-23 14:47

manager   ~0002637

If all that could happen is the creation of a fomally correct DCP with 'wrong' content (e.g. reused from previous encoding), it could be worse. I don't know how the encoding following a prior analysis works - is it possible that, if content is changed without DCP-o-matic getting it, the encoding uses wrong parameters, thus creating formally bad DCP/MXFs? What if e.g. a video file that was previously 16:9 has been replaced with one that is flat? Whatever, as that can already happen now, everything that catches 'some' or 'most' cases would be an improvement. Again, for me and probably others, it would be quite typical to change subtitle files in an external editor and expect DCP-o-matic to use these changes immediately. When they are set as burn-in, they need to trigger a re-encode.

  • Carsten

Bug History

Date Modified Username Field Change
2018-04-10 23:51 carl New Bug
2018-04-28 15:17 overlookmotel Note Added: 0002405
2018-04-30 14:32 Carsten Note Added: 0002408
2018-05-01 00:13 Carsten Note Edited: 0002408
2018-05-16 23:08 carl Priority normal => high
2018-05-16 23:57 carl Note Added: 0002434
2018-08-16 22:24 carl Assigned To => carl
2018-08-16 22:24 carl Status new => feedback
2018-08-16 22:24 carl Note Added: 0002621
2018-08-17 00:09 Carsten Note Added: 0002624
2018-08-17 01:34 Carsten Note Edited: 0002624
2018-08-21 23:19 carl Status feedback => resolved
2018-08-21 23:19 carl Resolution open => fixed
2018-08-21 23:19 carl Note Added: 0002633
2018-08-21 23:19 carl Note Edited: 0002633
2018-08-22 16:43 Carsten Note Added: 0002635
2018-08-22 19:26 carl Note Added: 0002636
2018-08-23 14:47 Carsten Note Added: 0002637
2018-10-17 20:16 carl Status resolved => closed