View Bug Details

IDProjectCategoryView StatusLast Update
0002445DCP-o-maticFeaturespublic2023-05-30 00:05
Reportermhm Assigned Tocarl  
PrioritynormalSeverityfeatureReproducibilityhave not tried
Status resolvedResolutionfixed 
Target Version2.16.56 
Summary0002445: Please create CLI tool to "map" a CPL into a complete package.
Description

If you have a multi CPL package or an OV package and a VF package it would be really useful to have a CLI tool to "map" a specific set of CPL:s into a complete package referencing just the CPL:s you want (and generating new PKL, ASSETMAP and VOLINDEX).

E.g. a tool like this:

dcpomatic2_map -o <output directory> [-r] [-l] [-s] [-d search-directory] <cpl file(s), ...>

-l: use hard links instead of copying
-s: use symlinks instead of copying assets
-r : rename all files (except ASSETMAP and VOLINDEX) to <uuud>.<mxf|xml>
-d: directory to search for assets (could be specified multiple times)

SMPTE or Interop format of the regenerated files would be taken from the CPL format (Interop or SMPTE version)

Extra care must be taken with Interop subtitle files and directories of course..

So for example if I have an "OV" DCP in the directory feature_OV and an supplemental VF package for this OV in the directory feature_VF (and a feature_VF/cpl.xml CPL file) i could run:

dcpomatic2_map -o complete_VF -r -l -d feature_VF -d feature_OV feature_VF/cpl.xml

To generate a complete DCP package in the directory "complete_VF" containing only the cpl originally at feature_VF/cpl.xml (using hard links).

This feature would be immensly useful when distributing DCPs for the festivals I work with. To make it easier for the projectionists to have only one version to choose from and also have a complete DCP of only the one CPL that is to be screened.

I realise that this may be a lot of work and I am willing to monetarily sponsor this feature if it will make it more likely to happen.

Thanks for considering!

And hope I made the above feature request clear. Otherwise just ask.

Thanks!

Best,
Mattias

TagsNo tags attached.
Branchdcpmap
Estimated weeks required
Estimated work requiredUndecided

Relationships

related to 0000243 acknowledged Ability to (re-)create PKL, ASSETMAP[.xml] and VOLINDEX[.xml] 

Activities

mhm

2023-02-12 04:33

reporter   ~0005473

Also somewhat related to my ancient feature request in Issue 0000243

mhm

2023-02-12 05:20

reporter   ~0005476

Something I didn't mention that maybe is not obvious. For this dcpomatic2_map command everything except the newly generated PKL, ASSETMAP and VOLINDEX is to be kept untouched. Just (optinally) renamed files and moved/linked to another directory location.

carl

2023-02-12 13:01

administrator   ~0005481

Is it important that this tool is command-line based? I think the DCP-o-matic Combiner might also do some of this stuff, and could probably be extended to do the rest.

mhm

2023-02-13 02:11

reporter   ~0005490

For me a CLI tool is essential, yes. As I would probably want to use it in scripts and on headless machines.

mhm

2023-02-13 03:33

reporter   ~0005491

What I mean is probably clear by now, but just for completeness here is some explaining of this feature I did some time ago: https://github.com/Ymagis/ClairMeta/issues/188#issuecomment-792380106

As you can see this is functionality I really, really want :)

carl

2023-04-17 22:05

administrator   ~0005642

I've got a basic test version of this ready to go now - what operating system would be easiest for you to test on?

mhm

2023-04-18 06:52

reporter   ~0005644

Linux would be the easiest to test on for me thanks!

carl

2023-04-18 08:42

administrator   ~0005646

Which one?

carl

2023-05-13 07:57

administrator   ~0005681

3a1ddb9e182be28a22c5bbc85c06ee6629b72fe6

overlookmotel

2023-05-27 16:06

reporter   ~0005697

Can I ask how the "search for assets" step works? Does it look for ASSETMAPs in the directories specified with -d, and locate the picture/sound/subs assets specified in the CPL from those? Or is it finding the assets by some other method based on filename etc.

@mhm mentions in https://github.com/Ymagis/ClairMeta/issues/188#issuecomment-792380106 that ASSETMAP and PKL are redundant information, and CPL contains everything you need to parse a DCP. But I don't quite understand how that works - doesn't only the ASSETMAP contain the mapping from UUID to file path?

Perhaps you can rely on other heuristics to locate assets in well-formed, valid DCPs made by common software. But from my experience on film festivals, we get sent all kinds of weird stuff! I've seen some cases where DCP folders contain detritus from a previous encode. i.e. amongst the "real" assets, you have other files which look like the ones you're looking for, but are actually from some other encode.

This new tool looks incredibly useful. It'd just be good to understand how it's working before putting it into service. I tried to figure it out from the commit, but I'm afraid I got lost where it calls into dcp.read(). I don't speak C++!

carl

2023-05-28 20:06

administrator   ~0005700

Can I ask how the "search for assets" step works

It looks for ASSETMAPs in the directories specified with -d.

@mhm mentions in https://github.com/Ymagis/ClairMeta/issues/188#issuecomment-792380106 that ASSETMAP and PKL are redundant information, and CPL contains everything you need to parse a DCP. But I don't quite understand how that works - doesn't only the ASSETMAP contain the mapping from UUID to file path?

I think you are right, though I think the only part of a DCP that does not contain its own UUID is an interop font file, so for SMPTE I think you could scan files, work out what they are (PKL, CPL, video MXF etc.) then not need the ASSETMAP any more.

overlookmotel

2023-05-29 11:43

reporter   ~0005702

Thanks for coming back. OK, that's clear. I'll give it a whirl in next few days.

If I've understood right, this tool also allows adding subtitles to an OV DCP without copying any files, which should be much faster than current method, which requires remaking the video+audio MXFs:

  1. Create VF DCP using an SRT, and video+audio from original DCP just as references.
  2. Use dcpomatic2_map with -l option to create a new OV with the subs from the VF's CPL.

Should take a matter of seconds, rather than current 0000001:0000001 hour. Nice!

carl

2023-05-30 00:05

administrator   ~0005707

I think that should work, indeed - not to say that the main GUI shouldn't do a better job in these cases!

Bug History

Date Modified Username Field Change
2023-02-12 04:00 mhm New Bug
2023-02-12 04:33 mhm Note Added: 0005473
2023-02-12 05:20 mhm Note Added: 0005476
2023-02-12 13:01 carl Assigned To => carl
2023-02-12 13:01 carl Status new => feedback
2023-02-12 13:01 carl Note Added: 0005481
2023-02-13 02:11 mhm Note Added: 0005490
2023-02-13 02:11 mhm Status feedback => assigned
2023-02-13 03:33 mhm Note Added: 0005491
2023-02-13 07:54 carl Target Version => 2.16.43
2023-02-13 07:54 carl Estimated work required => Undecided
2023-02-16 18:53 carl Target Version 2.16.43 => 2.16.44
2023-02-17 19:25 carl Branch => libdcp: dcpmap
2023-02-17 23:00 carl Branch libdcp: dcpmap => dcpmap
2023-02-17 23:15 carl Status assigned => in progress
2023-02-18 23:54 carl Target Version 2.16.44 => 2.16.45
2023-02-21 09:48 carl Relationship added related to 0000243
2023-03-03 22:51 carl Target Version 2.16.45 => 2.16.46
2023-03-05 20:36 carl Target Version 2.16.46 => 2.16.47
2023-03-08 00:27 carl Target Version 2.16.47 => 2.16.48
2023-03-23 20:24 carl Target Version 2.16.48 => 2.16.49
2023-03-27 18:36 carl Target Version 2.16.49 => 2.16.51
2023-03-30 12:33 carl Target Version 2.16.51 => 2.16.52
2023-04-11 00:03 carl Target Version 2.16.52 => 2.16.54
2023-04-17 22:05 carl Note Added: 0005642
2023-04-18 00:35 carl Status in progress => feedback
2023-04-18 06:52 mhm Note Added: 0005644
2023-04-18 06:52 mhm Status feedback => assigned
2023-04-18 08:42 carl Status assigned => feedback
2023-04-18 08:42 carl Note Added: 0005646
2023-04-22 20:03 carl Target Version 2.16.54 => 2.16.56
2023-05-01 22:48 carl Target Version 2.16.56 =>
2023-05-01 22:49 carl Target Version => 2.16.x
2023-05-12 13:12 carl Target Version 2.16.x => 2.16.56
2023-05-13 07:57 carl Status feedback => resolved
2023-05-13 07:57 carl Resolution open => fixed
2023-05-13 07:57 carl Note Added: 0005681
2023-05-27 16:06 overlookmotel Note Added: 0005697
2023-05-28 20:06 carl Note Added: 0005700
2023-05-29 11:43 overlookmotel Note Added: 0005702
2023-05-30 00:05 carl Note Added: 0005707