7.3-stable review patch. If anyone has any objections, please let me know. ------------------ From: Keita Morisaki [ Upstream commit 7b23bf82ccbdb4cf0752f2b8686e7319a7ce497f ] median3() is meant to return the median of three values, but its last compare-and-swap step swaps b and c instead of a and b. When c is smaller than both a and b, it returns max(a, b) instead of min(a, b). For example, median3(2, 3, 1) returns 3 instead of 2. _do_calc_rc_params() always passes 0 as the first argument, and the second argument is never below 12. So whenever the third argument is negative, first_line_bpg_offset (and second_line_bpg_offset for 4:2:0) is set to the second argument instead of 0. Fix this by swapping a and b in the last step, as intended. Note that this change does not affect valid DSC configurations. The third argument is negative only when bits per pixel exceeds the uncompressed rate of the colour format: - 3 * bpc for 4:4:4, 3 * bpc + 2 for RGB - 2 * bpc for 4:2:2 - 1.5 * bpc for 4:2:0 Fixes: 97bda0322b8a ("drm/amd/display: Add DSC support for Navi (v2)") Signed-off-by: Keita Morisaki Tested-by: Dan Wheeler Reviewed-by: Roman Li Signed-off-by: Alex Deucher (cherry picked from commit 1ddc4bc97a62fc89d9d3681413d9f977e1fa1a83) Cc: stable@vger.kernel.org Signed-off-by: Sasha Levin --- drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c index b8fc27a5361e8..657173d8081c3 100644 --- a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c +++ b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c @@ -46,7 +46,7 @@ static int median3(int a, int b, int c) if (b > c) swap(b, c); if (a > b) - swap(b, c); + swap(a, b); return b; } -- 2.53.0