Let's Talk About "The Process", Shall We?

Anything and everything to do with DCP-o-matic.
Liberated Images
Posts: 15
Joined: Tue Aug 30, 2022 5:00 pm

Re: Let's Talk About "The Process", Shall We?

Post by Liberated Images »

Okay, awesome. There is certainly some very helpful info here. Thank you for the link as well. That sent me down a whole rabbit hole I'd never been down before. I do love to learn so thank you for breaking some stuff down and giving me a bit of homework, so to speak.

jdent02 wrote: Thu Oct 06, 2022 7:28 pm DCP-o-Matic also has a disk writer that'll do it for you. Be careful with it though, as you can accidentally wreck your computer if you use it wrong.
What do you mean by this?
- Kyle :D
Liberated Images
Posts: 15
Joined: Tue Aug 30, 2022 5:00 pm

Re: Let's Talk About "The Process", Shall We?

Post by Liberated Images »

Carsten wrote: Thu Oct 06, 2022 9:13 pm While NTFS is not a formally accepted format (actually, even ext2/ext3 is only a defacto standard in the DCI world), it is factually guaranteed to work on all DCI servers as well.
I am in no way implying that I do not believe you, but I am curious as to what makes you say it is factually guaranteed to work. I mean no disrespect.
- Kyle :D
Liberated Images
Posts: 15
Joined: Tue Aug 30, 2022 5:00 pm

Re: Let's Talk About "The Process", Shall We?

Post by Liberated Images »

Carsten wrote: Thu Oct 06, 2022 9:13 pm The whole ext2/3 thing is unnecessarily complicated if you don't work in a Linux environment. Of course, you are free to use DCP-o-matic disk writer as well. It's not an ext2/3 driver, but I will give you an ext2/3 disk with a DCP on it.

You are correct in your suspicion. I am totally unfamiliar with Linux and would be more comfortable with the route of using NTFS with Tuxera on my Mac. However, I am willing to learn and do whatever is necessary to do the best work possible in putting out working DCPs every time possible.

Is there truly a risk with running the disk writer in DOM?

It seems I may need to experiment with NTFS for now and be ready to go the ext2/3 route if I encounter a failure and see what happens.

Thank you all for your input. I am open to continuing this discussion all the time so we can all learn and grow together. I think having a solid process is my biggest roadblock and this is helping me get past that. You're appreciated.
- Kyle :D
Carsten
Posts: 2668
Joined: Tue Apr 15, 2014 9:11 pm
Location: Germany

Re: Let's Talk About "The Process", Shall We?

Post by Carsten »

The number of DCI compliant servers is small, I can name any brand and type from my head if you'd rock me up in the middle of the night. Not one has ever been reported to reject an NTFS formatted disk. I also have seen NTFS being used for wide distribution by some smaller commercial companies.

Again - partition format (NTFS) is different from partition scheme (e.g. MBR vs. GPT). You can screw up an ext2/3 disc with the wrong partition scheme as well. On a Mac, I would recommend using Tuxera NTFS, and if ext2/3 becomes a real necessity, to try DCP-o-matic disk writer. Not saying that Paragon is forever unable to solve their issues, but, we simply have seen too many problems with it.

On a Mac, it is very easy to create a disk with all the necessary options with disk utility. Tuxera comes with it's own management program, but also integrates into the Mac disk utility, so, if you initialise a disk, it is straight forward to choose MBR and NTFS/Tuxera for it. It's a lot more complicated in Windows, unfortunately. You need to resort to a picky command line utility to force a disk into MBR now.

Disk writer is only a risk so far as the software is writing raw to an external disk. And with all software that reformats a disk, there is a risk of data loss if e.g. you point disk writer to the wrong target device.
rl1983
Posts: 14
Joined: Thu Jul 20, 2017 9:48 pm

Re: Let's Talk About "The Process", Shall We?

Post by rl1983 »

NTFS works great. Haven't had any issues with it in the field.

EXT 2/3 is the standard because you can run fsck and get a previous non-functional drive functioning well enough to transfer data off the drive. NTFS I don't think has that level of error correction.

Working with EXT 2/3 in Linux (Ubuntu at least) can be frustrating at times when you have to chmod 777 every external drive that you want to write to. It's not so plug and play like Windows NTFS.
RaquelGomez
Posts: 8
Joined: Tue Apr 12, 2022 4:14 pm

Re: Let's Talk About "The Process", Shall We?

Post by RaquelGomez »

This is not an answer, but a question that builds on what you are saying. I have been trying to understand all this formatting staff. I am also a mac user. I went the NTFS way, because even if I got to start and format the disc in linux. .I was not sure about how do the chmod 777 (If any body has a link for how to this in the terminal, I will appreciate it).

But my question is far more evident. Or surely it is for many, as I do not get anybody asking about it.

When I built the DCP (write for first time) I am using a NTFS/MBR disc.. and then I copy the resulted DCP on a linux??? Should I write direct on a linux..

[/b]
Or if in NTSF .
A) Could I write it in a Mac formatted disc and then copy it to a NTFS/MBR disk
B) Could I write it in NTFS (not MBR) formatted disc and then copy it to a NTFS/MBR disk

I hope you undestand what ai talk about. Thank you. for your response
Carsten
Posts: 2668
Joined: Tue Apr 15, 2014 9:11 pm
Location: Germany

Re: Let's Talk About "The Process", Shall We?

Post by Carsten »

Yes, you can write to a Mac format drive with DCP-o-matic, and copy to an NTFS formatted distribution drive (usually, an external USB connected drive). But you need a Mac NTFS driver that allows to write to an NTFS drive (e.g. Tuxera).

Basically, you can write/copy a DCP from any format to any format, as long as the format follows current OS schemes. No problem to copy from a GPT/UUID native Mac or Windows disc to an MBR/NTFS distribution drive.

You will probably fail to write a large DCP to e.g. the older FAT32 file system, because it has a 4GB file size limit. A FAT32 drive can still transport a DCP if no file contained in the DCP is larger than 4GB. E.g. shorts or trailers.

But the distribution drive that is to be connected to the DCI server for DCP ingest MUST carry an MBR partition table at first. The nuissance is that most modern operating systems nowadays create UUID/GPT partition tables by default, at least for larger drives. Some drives even come preformatted as GPT/UUID right out of the box. In that case, you have to repartition the disc to an MBR partition table. Which, currently is quite easy on a Mac, but more complicated in Windows or Linux.
RaquelGomez
Posts: 8
Joined: Tue Apr 12, 2022 4:14 pm

Re: Let's Talk About "The Process", Shall We?

Post by RaquelGomez »

Thank you very much. Knowing that helps a lot. And thanks again Carsten for your care.

Just for other users. This was important to me as my disk is bigger than 2TB and those are not formatable to NTFS/MBR:
MBR supports a Maximum of 2TB … .
Carsten
Posts: 2668
Joined: Tue Apr 15, 2014 9:11 pm
Location: Germany

Re: Let's Talk About "The Process", Shall We?

Post by Carsten »

I would strongly advise against using discs larger than 2TB for distribution. Many servers do not recognise larger discs, and as you say, you can not use MBR. Creating a 2TB partition on such a drive will not help.

Of course, you can store an unlimited number of DCPs on a larger drive, e.g. on an 8 or 16TB GPT/UUID disc without a problem. Just that for distribution/connecting it to a DCP server, it needs to be copied to a max 2TB MBR drive. You could also ingest through e.g. FTP/network from a larger drive or NAS. FTP ingest follows completely other rules than direct local disc connection.

Yes, I know that SOME servers support GPT and larger discs, but for distribution, we need to follow the minimal common standard. 'Semiofficial' document here: https://files.isdcf.com/papers/ISDCF-Do ... y-Recs.pdf
Note, this document does not mention or recommend NTFS (as I do). NTFS will still work IF IT IS USED ON A MAX 2TB MBR PARTITIONED DRIVE. Don't blame NTFS when using a GPT/UUID partitioned disc, or a disc larger than 2TB!
ancient-skyscraper
Posts: 9
Joined: Thu Feb 16, 2023 11:55 pm

Re: Let's Talk About "The Process", Shall We?

Post by ancient-skyscraper »

This has been a crazy-informative thread to read! Thank you, everyone! As per usual, I can't really help much with the most significant stuff, but I did find one area where I can chime in:
RaquelGomez wrote: Tue Jul 18, 2023 7:43 pm I was not sure about how do the chmod 777 (If any body has a link for how to this in the terminal, I will appreciate it).
So, chmod is actually pretty easy. Open a terminal window, navigate to the directory above what you want to change, and then run:

Code: Select all

ls -lah               # List files (show permissions) before you start
chmod -R 777 DCP_DIR  # Change permissions on DCP_DIR (and all its contents)
ls -lah               # Admire your work
  • chmod itself stands for "change mode of access" [2] (we now know of "mode of access" as "file permissions," but the command name never changed)
  • The -R bit tells the command to operate recursively (which is just fancy Linux-speak for "do it for every file, and every file within every subdirectory, etc."
  • 777 means "give read, write and execute access to the user that owns the file, members of the group that owns the file, and everyone else on the system"
    • It's really three options in a row: the first number is access for the owner, second is for the group, third is for everyone else
    • A 7 means "read + write + execute" access
    • Three sevens in a row means all three classes of users gets the same access ("read + write + execute")
  • DCP_DIR is just the target of the chmod command; what file/directory should it change the mode of access on?
You'll see I also added an ls -lah before and after the chmod command. You're probably familiar with that one already (it just lists the contents of your current directory, like the dir command from MS-DOS/command shell/PowerShell). On Linux, it will also show you the file permissions, so I like to run it before and after I modify those just to make sure I did it right.

Hope that helps! Sorry for the long-winded response!

More reading:
[1] Great article about chmod with a lot of examples
[2] Article about file permissions in general