View Revisions: Issue #1477

Summary 0001477: Introduce Autocrop/Letter/Pillarbox detection
Revision 2019-02-28 10:31 by Carsten
Description

I think there have been feature suggestions towards this before, however, as I now have better ideas on how to do it, I'll file a new one ;-)

Many people have trouble finding proper scaling and cropping parameters for letter- or pillarboxed content. While the preview allows to set cropping visually, the preview window is usually downscaled and not pixel accurate. So it would be good if DCP-o-matic could detect active pixels automatically. We know that automatic aspect ratio detection can sometimes fail, because there is too much black content in the active image area. Also, some (few) movies change aspect ratio during runtime. However, I think a good way to do is is to do a one-time active pixel detection based on the current preview image selection. Then adjust cropping/scaling based on that image. From then on, the user should be allowed to use all common controls to adjust, reset, improve the cropping and scaling. So to say, the automatic crop would just result into a cropping and scaling 'suggestion'. I guess that would suit most peoples expectations.

A few things to consider:

  • anamorphic content needs to be taken into account, e.g. Scope DVDs in 16:9
  • not all content has letterbox/pillarboxes at 'pure' black levels. Some blacks may be noisy. The detection still needs to be able to find these bars
  • many codecs add 'smeared' edges around active image area (see attached image). These smeared edges should be added to the active image area. So, some sort of a dynamic threshold detection should be there.
  • some content has misleading leaders - I know e.g. one popular studio logo which looks like scope, but is wider. If users do not understand this, the autocrop will fail if applied to the beginning of the movie. Maybe check plausibility of the crop by looking into later parts of the movie? Maybe take multiple samples at least from the beginning of the movie? Having weird stuff around the beginning (opening credits) is pretty common. Maybe just skip over the first minutes, show the chosen preview image, display the chosen crop frame (e.g. in green), before the crop is accepted - so, some sort of an automatic, yet interactive process? A simple 'Yes/No' confirmation box to apply cropping/scaling?

The function to detect letter/pillarboxing could either go to the scale to/aspect ratio list, or, into the 'Content' main drop down menu below 'scale to fit width/height'.

I understand that ffmpeg has some sort of letterbox detection ('cropdetect'). Maybe it can be used for this.

  • Carsten
Revision 2019-02-28 10:29 by Carsten
Description

I think there have been feature suggestions towards this before, however, as I now have better ideas on how to do it, I'll file a new one ;-)

Many people have trouble finding proper scaling and cropping parameters for letter- or pillarboxed content. While the preview allows to set cropping visually, the preview window is usually downscaled and not pixel accurate. So it would be good if DCP-o-matic could detect active pixels automatically. We know that automatic aspect ratio detection can sometimes fail, because there is too much black content in the active image area. Also, some (few) movies change aspect ratio during runtime. However, I think a good way to do is is to do a one-time active pixel detection based on the current preview image selection. Then adjust cropping/scaling based on that image. From then on, the user should be allowed to use all common controls to adjust, reset, improve the cropping and scaling. So to say, the automatic crop would just result into a cropping and scaling 'suggestion'. I guess that would suit most peoples expectations.

A few things to consider:

  • anamorphic content needs to be taken into account, e.g. Scope DVDs in 16:9
  • not all content has letterbox/pillarboxes at 'pure' black levels. Some blacks may be noisy. The detection still needs to be able to find these bars
  • many codecs add 'smeared' edges around active image area (see attached image). These smeared edges should be added to the active image area. So, some sort of a dynamic threshold detection should be there.
  • some content has misleading leaders - I know e.g. one popular studio logo which looks like scope, but is wider. If users do not understand this, the autocrop will fail if applied to the beginning of the movie. Maybe check plausibility of the crop by looking into later parts of the movie? Maybe take multiple samples at least from the beginning of the movie? Having weird stuff around the beginning (opening credits) is pretty common. Maybe just skip over the first minutes, show the chosen preview image.

The function to detect letter/pillarboxing could either go to the scale to/aspect ratio list, or, into the 'Content' main drop down menu below 'scale to fit width/height'.

I understand that ffmpeg has some sort of letterbox detection ('cropdetect'). Maybe it can be used for this.

  • Carsten
Revision 2019-02-28 10:28 by Carsten
Description

I think there have been feature suggestions towards this before, however, as I now have better ideas on how to do it, I'll file a new one ;-)

Many people have trouble finding proper scaling and cropping parameters for letter- or pillarboxed content. While the preview allows to set cropping visually, the preview window is usually downscaled and not pixel accurate. So it would be good if DCP-o-matic could detect active pixels automatically. We know that automatic aspect ratio detection can sometimes fail, because there is too much black content in the active image area. Also, some (few) movies change aspect ratio during runtime. However, I think a good way to do is is to do a one-time active pixel detection based on the current preview image selection. Then adjust cropping/scaling based on that image. From then on, the user should be allowed to use all common controls to adjust, reset, improve the cropping and scaling. So to say, the automatic crop would just result into a cropping and scaling 'suggestion'. I guess that would suit most peoples expectations.

A few things to consider:

  • anamorphic content needs to be taken into account, e.g. Scope DVDs in 16:9
  • not all content has letterbox/pillarboxes at 'pure' black levels. Some blacks may be noisy. The detection still needs to be able to find these bars
  • many codecs add 'smeared' edges around active image area (see attached image). These smeared edges should be added to the active image area. So, some sort of a dynamic threshold detection should be there.
  • some content has misleading leaders - I know e.g. one popular studio logo which looks like scope, but is wider. If users do not understand this, the autocrop will fail if applied to the first images. Maybe check plausibility of the crop by looking into later parts of the movie? Maybe take multiple samples at least from the beginning of the movie? Having weird stuff around the beginning (opening credits) is pretty common. Maybe just skip over the first minutes, show the chosen preview image.

The function to detect letter/pillarboxing could either go to the scale to/aspect ratio list, or, into the 'Content' main drop down menu below 'scale to fit width/height'.

I understand that ffmpeg has some sort of letterbox detection ('cropdetect'). Maybe it can be used for this.

  • Carsten
Revision 2019-02-28 10:23 by Carsten
Description

I think there have been feature suggestions towards this before, however, as I now have better ideas on how to do it, I'll file a new one ;-)

Many people have trouble finding proper scaling and cropping parameters for letter- or pillarboxed content. While the preview allows to set cropping visually, the preview window is usually downscaled and not pixel accurate. So it would be good if DCP-o-matic could detect active pixels automatically. We know that automatic aspect ratio detection can sometimes fail, because there is too much black content in the active image area. Also, some (few) movies change aspect ratio during runtime. However, I think a good way to do is is to do a one-time active pixel detection based on the current preview image selection. Then adjust cropping/scaling based on that image. From then on, the user should be allowed to use all common controls to adjust, reset, improve the cropping and scaling. So to say, the automatic crop would just result into a cropping and scaling 'suggestion'. I guess that would suit most peoples expectations.

A few things to consider:

  • anamorphic content needs to be taken into account, e.g. Scope DVDs in 16:9
  • not all content has letterbox/pillarboxes at 'pure' black levels. Some blacks may be noisy. The detection still needs to be able to find these bars
  • many codecs add 'smeared' edges around active image area (see attached image). These smeared edges should be added to the active image area. So, some sort of a dynamic threshold detection should be there.

The function to detect letter/pillarboxing could either go to the scale to/aspect ratio list, or, into the 'Content' main drop down menu below 'scale to fit width/height'.

I understand that ffmpeg has some sort of letterbox detection ('cropdetect'). Maybe it can be used for this.

  • Carsten
Revision 2019-02-28 10:21 by Carsten
Description

I think there have been feature suggestions towards this before, however, as I know have better ideas how to do it, I'll file a new one ;-)

Many people have trouble finding proper scaling and cropping parameters for letter- or pillarboxed content. While the preview allows to set cropping visually, the preview window is usually downscaled and not pixel accurate. So it would be good if DCP-o-matic could detect active pixels automatically. We know that automatic aspect ratio detection can sometimes fail, because there is too much black content in the active image area. Also, some (few) movies change aspect ratio during runtime. However, I think a good way to do is is to do a one-time active pixel detection based on the current preview image selection. Then adjust cropping/scaling based on that image. From then on, the user should be allowed to use all common controls to adjust, reset, improve the cropping and scaling. So to say, the automatic crop would just result into a cropping and scaling 'suggestion'. I guess that would suit most peoples expectations.

A few things to consider:

  • anamorphic content needs to be taken into account, e.g. Scope DVDs in 16:9
  • not all content has letterbox/pillarboxes at 'pure' black levels. Some blacks may be noisy. The detection still needs to be able to find these bars
  • many codecs add 'smeared' edges around active image area (see attached image). These smeared edges should be added to the active image area. So, some sort of a dynamic threshold detection should be there.

The function to detect letter/pillarboxing could either go to the scale to/aspect ratio list, or, into the 'Content' main drop down menu below 'scale to fit width/height'.

I understand that ffmpeg has some sort of letterbox detection ('cropdetect'). Maybe it can be used for this.

  • Carsten
Revision 2019-02-28 10:20 by Carsten
Description

I think there have been feature suggestions towards this before, however, as I know have better ideas how to do it, I'll file a new one ;-)

Many people have trouble finding proper scaling and cropping parameters for letter- or pillarboxed content. While the preview allows to set cropping visually, the preview window is usually downscaled and not pixel accurate. So it would be good if DCP-o-matic could detect active pixels automatically. We know that automatic aspect ratio detection can sometimes fail, because there is too much black content in the active image area. Also, some (few) movies change aspect ratio during runtime. However, I think a good way to do is is to do a one-time active pixel detection based on the current preview image selection. Then adjust cropping/scaling based on that image. From then on, the user should be allowed to use all common controls to adjust, reset, improve the cropping and scaling. So to say, the automatic crop would just result into a cropping and scaling 'suggestion'. I guess that would suit most peoples expectations.

A few things to consider:

  • anamorphic content needs to be taken into account, e.g. Scope DVDs in 16:9
  • not all content has letterbox/pillarboxes at 'pure' black levels. Some blacks may be noisy. The detection still needs to be able to find these bars
  • many codecs add 'smeared' edges around active image area (see attached image). These smeared edges should be added to the active image area. So, some sort of a dynamic threshold detection should be there.

The function to detect letter/pillarboxing could either go to the scale to/aspect ratio list, or, into the 'Content' main drop down menu below 'scale to fit width/height'.

I understand that ffmpeg has some sort of letterbox detection. Maybe it can be used for this.

  • Carsten
Revision 2019-02-28 10:18 by Carsten
Description

I think there have been feature suggestions towards this before, however, as I know have better ideas how to do it, I'll file a new one ;-)

Many people have trouble finding proper scaling and cropping parameters for letter- or pillarboxed content. While the preview allows to set cropping visually, the preview window is usually downscaled and not pixel accurate. So it would be good if DCP-o-matic could detect active pixels automatically. We know that automatic aspect ratio detection can sometimes fail, because there is too much black content in the active image area. Also, some (few) movies change aspect ratio during runtime. However, I think a good way to do is is to do a one-time active pixel detection based on the current preview image selection. Then adjust cropping/scaling based on that image. From then on, the user should be allowed to use all common controls to adjust, reset, improve the cropping and scaling. So to say, the automatic crop would just result into a cropping and scaling 'suggestion'. I guess that would suit most peoples expectations.

A few things to consider:

  • anamorphic content needs to be taken into account, e.g. Scope DVDs in 16:9
  • not all content has letterbox/pillarboxes at 'pure' black levels. Some blacks may be noisy. The detection still needs to be able to find these bars
  • many codecs add 'smeared' edges around active image area (see attached image). These smeared edges should be added to the active image area. So, some sort of a dynamic threshold detection should be there.

The function to detect letter/pillarboxing could either go to the scale to/aspect ratio list, or, into the 'Content' main drop down menu below 'scale to fit width/height'.

  • Carsten
Revision 2019-02-28 10:14 by Carsten
Description

I think there have been feature suggestions towards this before, however, as I know have better ideas how to do it, I'll file a new one ;-)

Many people have trouble finding proper scaling and cropping parameters for letter- or pillarboxed content. While the preview allows to set cropping visually, the preview window is usually downscaled and not pixel accurate. So it would be good if DCP-o-matic could detect active pixels automatically. We know that automatic aspect ratio detection can sometimes fail, because there is too much black content in the active image area. Also, some (few) movies change aspect ratio during runtime. However, I think a good way to do is is to do a one-time active pixel detection based on the current preview image selection. Then adjust cropping/scaling based on that image. From then on, the user should be allowed to use all common controls to adjust, reset, improve the cropping and scaling. So to say, the automatic crop would just result into a cropping and scaling 'suggestion'. I guess that would suit most peoples expectations.

A few things to consider:

  • anamorphic content needs to be taken into account, e.g. Scope DVDs in 16:9
  • not all content has letterbox/boxes at 'pure' black levels. Some blacks may be noisy. The detection still needs to be able to find these bars
  • many codecs add 'smeared' edges around active image area (see attached image). These smeared edges should be added to the active image area. So, some sort of a dynamic threshold detection should be there.

The function to detect letter/pillarboxing could either go to the scale to/aspect ratio list, or, into the 'Content' main drop down menu below 'scale to fit width/height'.

  • Carsten