View Bug Details

IDProjectCategoryView StatusLast Update
0001351DCP-o-maticFeaturespublic2024-02-07 23:32
ReporterIgor.Voyt Assigned To 
PrioritynormalSeverityfeatureReproducibilityhave not tried
Status acknowledgedResolutionopen 
Summary0001351: Option to select destination on make DCP
Description

[]

TagsNo tags attached.
Branch
Estimated weeks required
Estimated work requiredUnknown

Relationships

related to 0002352 acknowledgedcarl Allow configuration of a DCP store directory 
related to 0002755 confirmedcarl Option to delete/not write all "extra" files after a successful DCP build 
related to 0002756 in progresscarl Stop used video directory and hard-linking 

Activities

carl

2018-08-12 00:26

administrator   ~0002595

Only problem I can forsee is being unable to hardlink from the project directory to the DCP directory due to them begin on different filesystems.

Carsten

2018-08-12 13:53

manager   ~0002599

Last edited: 2018-08-12 13:54

One positive thing is that you can write and hash check on the distribution drive directly, without adding the whole project folder. Does the final hash check/creation take place before or after linking the files into the dcp directory?

It could work in a similar way as for TMS transfer?

But yes, it would conflict with e.g. reusing already encoded data. If there is an additional explicit copy step, it doesn't make that much sense - the space is needed within the project folder anyway, and it takes extra time to copy to the final destination. Being there, it would make more sense to 'strip' unneeded project files from a target location ;-)

  • Carsten

Carsten

2018-08-12 13:59

manager   ~0002600

Maybe ask Igor for his main intention? It does make some sense to keep the full project file on a local drive, and only create the DCP on an external drive. E.g. there are many fast MacBooks now with very fast but rather small internal SSDs.

  • Carsten

Igor.Voyt

2018-08-13 14:05

reporter   ~0002603

The main goal is to have possibility to encode DCP to any drive. I have one drive exceptionally for DCP projects (let's call it 'main') and couple of drives for misc data. Sometimes free space on the main drive is ending. I don't have practice to create DCP-project on any drive other from the main. So I have imagined that it could be nice to create DCP-project on the main drive and generate DCP (only DCP without project files) on any drive to save space on the main drive where project files located.

I have forgotten that there is taking place hardlinking. What is it used for?

carl

2018-08-13 14:16

administrator   ~0002604

It's something that's been in there since early on and I'm never quite sure whether it's a good idea or not. When DoM writes the video asset it uses a filename which is depends uniquely on all the settings which affect the video asset (e.g. content, size, whether subs have been burnt into it etc. etc.)

If a subsequent make-dcp operation happens on the same project DoM looks for a file with the magic name for the current settings and re-uses it if it finds it. This allows for 1) resumption of interrupted transcodes and 2) quick re-builds of DCPs if only non-video settings have changed (e.g. if you alter the sound settings it will make a new DCP without transcoding the video data again).

When the DCP is finished it hard-links the "magic" filename into the DCP with its proper name. However hard linking only works on the same filesystem, so if the DCP is made on a different disk to the project the current code would copy the video asset into the DCP.

One possible solution that has just come to mind is to write the video asset direct into the DCP and keep a note in DoM's configuration of the UUID of the asset and all the video settings. Then, if we do a rebuild, we look for the asset and re-use it if we can.

I feel like there's a better one that I haven't thought of.

Igor.Voyt

2018-08-13 15:31

reporter   ~0002606

"write the video asset direct into the DCP and keep a note in DoM's configuration of the UUID of the asset"
I think this solution is the best

Carsten

2018-08-13 15:43

manager   ~0002608

Last edited: 2018-08-13 15:46

Well, I haven't though this through fully, but I am not sure wether this would be perfectly clean for situations where (quite typical) DCPs are then created on removable external media and then that drive is removed. For that reason, I suggested to do this as a copy after DCP creation (similar to send to TMS). I understand that doesn't solve the disc capacity issue.

In general, I have no trouble to create project files on any external media, so the DCP is immediately generated on the target disc. Just that this leaves all other project files on the target/ingest disc as well, and makes them inaccessible once that drive is removed.

How would we enable this option in the GUI? I guess it should be on a per-case basis? Or a checkbox with a file/path selector dialog somewhere under the DCP tab? Will this work into batch converter as well?

  • Carsten

carl

2018-08-13 16:06

administrator   ~0002609

Well, I haven't though this through fully, but I am not sure wether this would be perfectly clean for situations where (quite typical) DCPs are then created on removable external media and then that drive is removed. For that reason, I suggested to do this as a copy after DCP creation (similar to send to TMS). I > understand that doesn't solve the disc capacity issue.

I don't quite follow you here. What's the problem with a drive being removed?

How would we enable this option in the GUI? I guess it should be on a per-case basis? Or a checkbox with a file/path selector dialog somewhere under the DCP tab? Will this work into batch converter as well?

I think it would be transparent to the user. On making a DCP it would look for an existing file, and if it's not found it would start transcoding from scratch.

Carsten

2018-08-14 00:04

manager   ~0002612

What if you actually reuse content multiple times - typically they'd queue up in the video mxf folder, and the final DCPs receive only the needed part. How will that work with a dedicated DCP drive - they'd create 'waste' in the dcp path, and they can't be reused if that drive is gone?

  • Carsten

carl

2018-08-14 00:13

administrator   ~0002613

Good point. You'd have to delete old ones or they'd stack up, like you say. I'm not sure how much of a problem that would be. How many times are you going to make a DCP with video settings A, change to video settings B, make it again, then decide to go back to A?

Carsten

2018-08-14 00:49

manager   ~0002614

Last edited: 2018-08-14 01:30

We would have to tell people to go into the final DCP folder and delete 'some' MXF files - that doesn't sound good - think about a multireel feature. But if we don 't, one out of x DCP-o-matic DCPs will carry trash media assets with them.

I do use the reuse option quite often for versioning, also, often people cancel an encoding a few seconds or minutes after start because they notice a setting is wrong. That's why I like the separate video folder approach (also temp audio files in project path) - it makes sure the final DCP is always clean and doesn't need to be touched again.
Of course I have the same trouble as Igor when I create DCPs on my notebook - there is never enough space, and I need to relocate the project folder to an external drive regularly - but there is 'duplicate' now, which makes it a bit easier.

  • Carsten

Igor.Voyt

2023-12-24 10:48

reporter   ~0006165

What do you think about the following solution - combination of selecting custom export path (removable or another internal drive) and storing generated DCP files UUIDs in project.

  1. IF there are made some changes to project
    and DCP was previously generated (there are DCP files UUIDs in project)
    and UUIDs are valid (DCP files present in a target location and are related to the project)
    THEN use existing path to re-generate DCP

  2. IF there are made some changes to project
    and DCP was previously generated (there are DCP files UUIDs in project)
    and UUIDs are NOT valid (DCP files absent in a target location or location does not exist)
    THEN prompt user to set custom location
    IF UUIDs are valid (DCP files present in a custom location and are related to the project)
    THEN use custom path to re-generate DCP
    ELSE generate new DCP

Bug History

Date Modified Username Field Change
2018-07-30 07:49 Igor.Voyt New Bug
2018-07-31 00:23 carl Target Version => 2.14.0
2018-08-12 00:26 carl Note Added: 0002595
2018-08-12 13:53 Carsten Note Added: 0002599
2018-08-12 13:54 Carsten Note Edited: 0002599
2018-08-12 13:59 Carsten Note Added: 0002600
2018-08-13 14:05 Igor.Voyt Note Added: 0002603
2018-08-13 14:16 carl Note Added: 0002604
2018-08-13 15:31 Igor.Voyt Note Added: 0002606
2018-08-13 15:43 Carsten Note Added: 0002608
2018-08-13 15:44 Carsten Note Edited: 0002608
2018-08-13 15:46 Carsten Note Edited: 0002608
2018-08-13 16:06 carl Note Added: 0002609
2018-08-14 00:04 Carsten Note Added: 0002612
2018-08-14 00:13 carl Note Added: 0002613
2018-08-14 00:49 Carsten Note Added: 0002614
2018-08-14 01:30 Carsten Note Edited: 0002614
2019-01-06 19:22 carl Target Version 2.14.0 =>
2022-10-04 11:32 carl Relationship added related to 0002352
2023-12-22 22:32 carl Status new => acknowledged
2023-12-24 10:48 Igor.Voyt Note Added: 0006165
2024-01-03 11:53 carl Severity minor => feature
2024-02-07 23:24 carl Relationship added related to 0002755
2024-02-07 23:32 carl Relationship added related to 0002756