WebP/Lossy: Add a missing clamp

The spec says that the AC dequantization factor for Y2 data should
be at least 8, so do that.

This only has a very small effect (only the first two AC table
entries are < 8 after multiplying with 155 / 100, so this would
have only a small effect on brightness), and this case is hit
exactly 0 times in all my test images.  But it's still good to match
the spec.
This commit is contained in:
Nico Weber 2023-06-01 08:27:27 -04:00 committed by Andreas Kling
parent 13daa29d81
commit 287e2655cb
Notes: sideshowbarker 2024-07-17 01:12:07 +09:00

View file

@ -609,7 +609,7 @@ i16 dequantize_value(i16 value, bool is_dc, QuantizationIndices const& quantizat
// can be found in related lookup functions in dixie.c (Section 20.4)."
// Apparently spec writing became too much work at this point. In section 20.4, in dequant_init():
// * For y2, the output (!) of dc_qlookup is multiplied by 2, the output of ac_qlookup is multiplied by 155 / 100
// * Also for y2, ac_qlookup is at least 8 for lower table entries (XXX!)
// * Also for y2, ac_qlookup is at least 8 for lower table entries
// * For uv, the dc_qlookup index is clamped to 117 (instead of 127 for everything else)
// (or, alternatively, the value is clamped to 132 at most)
@ -646,7 +646,7 @@ i16 dequantize_value(i16 value, bool is_dc, QuantizationIndices const& quantizat
if (is_dc)
dequantization_factor *= 2;
else
dequantization_factor = (dequantization_factor * 155) / 100;
dequantization_factor = max((dequantization_factor * 155) / 100, 8);
}
return dequantization_factor * value;