View Bug Details

IDProjectCategoryView StatusLast Update
0001431DCP-o-maticFeaturespublic2020-12-16 00:16
Reportercarl Assigned Tocarl  
PriorityhighSeveritymajorReproducibilityN/A
Status closedResolutionfixed 
Target Version2.16.0 
Summary0001431: Investigate opengl canvas for rendering preview/player.
Description

[]

TagsNo tags attached.
Branch
Estimated weeks required
Estimated work requiredMajor

Relationships

related to 0001472 acknowledgedcarl Sudden video freezes in DCP-o-matic Player, followed by crash 
related to 0001546 acknowledged Option to set playback size to a multiple of the original 

Activities

carl

2019-01-20 00:59

administrator   ~0002910

Last edited: 2019-01-26 00:44

Basically functional in opengl branch.

Other stuff;

  1. can OpenGL do the final scale and colourspace conversion (LUT) rather than the CPU?
  2. does it work on not-Linux?
  3. #if 0 stuff in that branch (mostly swaroop)
  4. vsync?

carl

2019-03-04 01:19

administrator   ~0003116

Works on OSX, not Windows; seems to be because glTexImage2D has limitations on the size (either too big or not a power of 2).

carl

2019-03-04 23:46

administrator   ~0003121

Would be interesting to make GLView take a PlayerVideo (rather than an Image) and do the scaling and compositing using GL.

Carsten

2019-03-26 01:19

manager   ~0003190

Last edited: 2019-03-26 01:42

Is there a chance we get a better player performance for many cores/high resolutions in 2.14? It seems there is a serious bottleneck in the current implementation. I rarely work on high res displays, but it seems there is something weird happening.
I have seen some of the current player logs, but can't read much out of it ('delayed'). I was hoping to get an idea about:

  • how much time it takes to decode a J2K frame
  • how much time it takes to scale it to display resolution
  • how much time it takes for the color conversion
  • how much time it takes to actually draw the image to the screen
    I would think that, except maybe for the final display, all of these operations can run multithreaded per frame.

I somehow suspect the final display writing as the bottleneck (as it appears the issue is somehow correlated to higher res displays), but maybe I am on a completely wrong track here. Maybe there is something going on with a higher number of threads/cores.

carl

2019-03-26 21:02

administrator   ~0003193

@carl some timings on my mac mini

DrawBitmap: 50ms
wxBitmap::wxBitmap: 15ms
Draw masking etc.: 5ms
wxImage constructor: 0.1ms

carl

2019-03-26 23:56

administrator   ~0003196

Last edited: 2019-03-27 00:01

I added some basic timing to version 2.15.0, which you can find here

https://dcpomatic.com/test-download?v=2.15.0

After playing some stuff, go to Tools -> Timing and a window is displayed with timing info. It would be interesting to see what you get on any Macs you have around (especially high-res and/or high-CPU-performance ones).

If we can find a simple-ish cause of the slowdown in 2.15.0 I am happy to backport to 2.14.x. An opengl renderer may be a bridge too far, though. That being said, I'm hopeful that the release cycles will get shorter from now on.

Carsten

2019-03-27 00:46

manager   ~0003199

Ah, that's interesting, will give it a try.

  • Carsten

Carsten

2019-03-29 08:46

manager   ~0003213

Can you tell us what these different timings mean?

carl

2019-03-29 14:32

administrator   ~0003214

get() calls is the number of times a new frame has been requested since program start.

The other things are all timings. The first number in the row is the number of seconds (total) spent doing that thing. The second number is the count of times that thing has been done. The third number is the time in seconds per call.

paint-panel is putting the image onto the screen. This seems slow on OS X (hopefully OpenGL might help)

refresh-panel should be very quick; if it's more than 1ms something very strange is happening.

get-image is time spent preparing (scaling etc.) the image in the main thread. If this is low it suggests that the background threads had time to do the preparation before the image was needed (so the main thread had nothing to do, which is the ideal). If is tens of ms it suggests that the background threads are overloaded (i.e. we are CPU bound).

ImageChanged should always be very quick, I just put that in to check.

carl

2019-05-07 00:09

administrator   ~0003321

Last edited: 2019-05-07 11:59

@carl using wx-3.0.3 branch of mxe to allow WX_GL_MAJOR_VERSION attribute to wxGLCanvas; then trying to force v3 to get more allowed texture sizes. 1024x1024 works.
WX_GL_MAJOR_VERSION -> 3 makes no difference.

carl

2019-05-09 00:39

administrator   ~0003328

Last edited: 2019-05-10 01:37

@carl looks like just the VM not supporting OpenGL2; ok on laptop. Making opengl2 branch with tidied-up version; investigate why paint never happens in GLView during a PH drag (does it happen in simple? simple updates much more responsively...)

carl

2019-05-11 00:23

administrator   ~0003335

Initial steps are now in v2.15.x.

Carsten

2019-05-12 12:45

manager   ~0003336

Last edited: 2019-05-12 17:06

Seems to work for me, but on my specific MacBook, don't see a difference to 2.14.x (it was performing good before). A simple test shows slightly better fps in OpenGL in full res than 'simple'. I will ask a friend to test it on her Retina Mac Book, which is a nightmare so far.

See timings. This is for a flat DCP at half decoding resolution, playback with zero dropped frames on this machine. Here, clearly J2K is the bottleneck, not drawing.

  • Carsten

Carsten

2019-05-12 12:50

manager  

Timing_MBP15_OpenGL.png (45,856 bytes)   
Timing_MBP15_OpenGL.png (45,856 bytes)   
Timing_MBP15_Simple.png (40,538 bytes)   
Timing_MBP15_Simple.png (40,538 bytes)   

Carsten

2019-05-12 17:15

manager   ~0003338

Last edited: 2019-05-12 17:30

I guess I wasted my timing results with player 2.15 on a Retina MacBook here:

https://dcpomatic.com/mantis/view.php?id=1546

Carl - why does the timing window sometimes show a 'paint panel row', and sometimes not?

  • Carsten

carl

2019-05-12 23:13

administrator   ~0003339

How do you mean "wasted my timing results"?

The paint-panel row got lost in the GL patch; it should be back in 2.15.2.

carl

2019-05-12 23:15

administrator   ~0003340

Currently OpenGL is only used for the final blit; hopefully I can get it to do the scale as well, which should free up some CPU time and hence allow more J2K decoding to happen per second...

Carsten

2019-05-12 23:44

manager   ~0003341

Last edited: 2019-05-12 23:47

Really wondering what is so special about these Retina MacBooks and 5k displays. Yes, more pixels, but not so much more that you would think it costs so much performance. Could it be the OS X version? I can't remember what this 5k iMac ran. I think El Capitan.
The two Retina MacBooks both ran HighSierra.

Remember, that i7-6coreHT MacBook could not even preview BigBuckBunny.mov source without serious frame dropping in DCP-o-matic.

  • Carsten

Carsten

2019-05-13 11:49

manager   ~0003343

'How do you mean "wasted my timing results"?'

I meant it probably went to the wrong bug entry. I now created relationships, as they all have something in common about the current drawing performance.

Bug History

Date Modified Username Field Change
2018-12-13 22:15 carl New Bug
2019-01-11 00:57 carl Target Version 2.14.0 => 2.16.0
2019-01-20 00:59 carl Note Added: 0002910
2019-01-26 00:44 carl Note Edited: 0002910
2019-03-04 01:19 carl Note Added: 0003116
2019-03-04 23:46 carl Note Added: 0003121
2019-03-26 00:17 carl Priority normal => high
2019-03-26 00:17 carl Status new => acknowledged
2019-03-26 01:19 Carsten Note Added: 0003190
2019-03-26 01:36 Carsten Note Edited: 0003190
2019-03-26 01:37 Carsten Note Edited: 0003190
2019-03-26 01:42 Carsten Note Edited: 0003190
2019-03-26 21:02 carl Note Added: 0003193
2019-03-26 23:56 carl Note Added: 0003196
2019-03-27 00:01 carl Note Edited: 0003196
2019-03-27 00:46 Carsten Note Added: 0003199
2019-03-29 08:46 Carsten Note Added: 0003213
2019-03-29 14:32 carl Note Added: 0003214
2019-05-01 10:36 carl Severity minor => major
2019-05-01 10:36 carl Reproducibility have not tried => N/A
2019-05-01 10:36 carl Estimated work required Unknown => Major
2019-05-07 00:09 carl Note Added: 0003321
2019-05-07 11:59 carl Note Edited: 0003321
2019-05-09 00:39 carl Note Added: 0003328
2019-05-09 00:39 carl Note Edited: 0003328
2019-05-09 00:40 carl Note Edited: 0003328
2019-05-09 01:41 carl Note Edited: 0003328
2019-05-10 01:37 carl Note Edited: 0003328
2019-05-10 01:37 carl Relationship added related to 0001472
2019-05-11 00:23 carl Assigned To => carl
2019-05-11 00:23 carl Status acknowledged => resolved
2019-05-11 00:23 carl Resolution open => fixed
2019-05-11 00:23 carl Note Added: 0003335
2019-05-12 12:45 Carsten Note Added: 0003336
2019-05-12 12:50 Carsten File Added: Timing_MBP15_OpenGL.png
2019-05-12 12:50 Carsten File Added: Timing_MBP15_Simple.png
2019-05-12 12:52 Carsten Note Edited: 0003336
2019-05-12 17:03 Carsten Note Edited: 0003336
2019-05-12 17:06 Carsten Note Edited: 0003336
2019-05-12 17:13 Carsten Relationship added related to 0001546
2019-05-12 17:15 Carsten Note Added: 0003338
2019-05-12 17:30 Carsten Note Edited: 0003338
2019-05-12 23:13 carl Note Added: 0003339
2019-05-12 23:15 carl Note Added: 0003340
2019-05-12 23:44 Carsten Note Added: 0003341
2019-05-12 23:47 Carsten Note Edited: 0003341
2019-05-13 11:49 Carsten Note Added: 0003343
2020-12-16 00:16 carl Status resolved => closed