Page 1 of 1

Problems importing a large J2C sequence

Posted: Fri May 26, 2023 1:01 pm
by gi-jones
Hi all
I'm trying to make a DCP of a full feature film, but DOM becomes unresponsive when I try to import the file sequence.

The sequence consiste of roughly 130000 J2C files. They are DCI compliant in 4k and made from Resolve. I have tested the full workflow with the first minute, and that works fine. But when I point DOM to the folder containing the full file set, it spends roughly 2 hours indexing (or whatnot). When that is done, the program simply crashes. One time I managed to save the project, but every time I opened it afterwards it crashed again.

Is this not a reasonable use case? Or am I doing something stupid?

Re: Problems importing a large J2C sequence

Posted: Fri May 26, 2023 1:05 pm
by carl
This should work - it's a shame it took 2 hours. Were the files stored on a local drive or somewhere else?

Re: Problems importing a large J2C sequence

Posted: Fri May 26, 2023 1:08 pm
by gi-jones
Should it happen significantly faster than 2 hours?

The files are on an external USB-C SSD disk. ExFAT

The thing is - this morning I tried using ASDCP (command line) to wrap an MXF, which worked fine. So I'm pretty sure the disk is ok.

Re: Problems importing a large J2C sequence

Posted: Fri May 26, 2023 7:41 pm
by gi-jones
So I tried moving the files from the external drive to my internal SSD, and suddenly the ingestion only took a couple of minutes.

I guess the problem must be with using the external drive somehow. It's a very fast one though - Samsung T7. The ExFat formatting might be a factor I guess...

Re: Problems importing a large J2C sequence

Posted: Sun May 28, 2023 7:13 pm
by carl
Did it crash when you used the internal SSD?

It's odd that it took so long with the external. There shouldn't be too much going on with the files; just finding them all, basically. I'll see if I can see any inefficiency in that code.

Re: Problems importing a large J2C sequence

Posted: Mon May 29, 2023 12:33 pm
by Carsten
Maybe - just for the fun of it, you could try to format the same external disk to NTFS (or whatever the native OS filesystem is), and just try the sequence import again. Maybe something weird happens because of ExFAT.

Re: Problems importing a large J2C sequence

Posted: Mon May 29, 2023 11:40 pm
by IoannisSyrogiannis
Working with Windows at work, I get a lot of ExFAT drives.
Someone had the great idea to use a couple for a film festival.
Copying a couple of terabytes took almost a day each time.
Bottom line: my bet is on ExFAT, not external vs internal drive.

Re: Problems importing a large J2C sequence

Posted: Thu Jun 01, 2023 9:43 am
by gi-jones
carl wrote: Sun May 28, 2023 7:13 pm Did it crash when you used the internal SSD?
Nope, everything worked fine from there.

I agree that ExFat is probably the issue. It's not possible for me to reformat the drive right now unfortunately.

Re: Problems importing a large J2C sequence

Posted: Mon Jun 05, 2023 6:48 am
by worldofcrap
You don’t mention if you’re on a Mac, windows or Linux machine. If it’s MacOS, the support for ExFAT file system is DOG slow, particularly with many small files. Windows is slightly better, but it’s still nowhere near as fast as NTFS.