Once the firmware has acknowledged the POW_OFF request, a runtime suspend disables the IRQ and stops the firmware straight away, even if the firmware has raised an interrupt the host has not taken yet. On an MT8173 (GX6250, META firmware), the SIDEKICK_IDLE.GARTEN poll at the end of pvr_fw_stop() can then time out. The suspend fails with -ETIMEDOUT, runtime PM latches the error, and every later pvr_power_get() returns -EINVAL. The recovery reset that follows can fail the same way and lose the device. The firmware counts the interrupts it raises in its OS data. Sample that count every time the host clears the firmware interrupt, and before disabling the IRQ for a runtime suspend, wait until the host has seen every interrupt the firmware counted, as Imagination's DDK does. Sampling in pvr_fw_irq_clear() rather than in the IRQ thread also covers the clears at probe and in a reset, whose interrupts never reach the thread, and picks up the zeroed counts after a hard reset. Fixes: cc1aeedb98ad ("drm/imagination: Implement firmware infrastructure and META FW support") Cc: stable@vger.kernel.org Signed-off-by: Ryan Brue --- Found on an Amazon Fire HD 10 (2017), an MT8173 tablet that's not upstream yet. Its GX6250 (4.40.2.51) is not on the driver's list and runs with exp_hw_support. With a compositor running, about one desktop start in five hit this before the patch. A debug build that logs timed-out register polls named the poll, SIDEKICK_IDLE read 0x7a, GARTEN not idle. Only META firmware on this one GPU was tested. Each failing suspend in the instrumented runs was off by exactly one interrupt (for example firmware 14320, host 14319). Why the META core then can't go idle is inferred from the DDK, which waits at this point; a fixed 1s delay in the same place also stopped the failures here. The reset paths were tested with a debug build that triggers a soft reset and can drop one count before the reset's clear. With v1's sampling, the next runtime suspend then waited the full second and warned, twice out of two; with v2 it didn't. A hard reset of this GPU times out and loses it, so the hard-reset path is still untested at runtime. It reproduces within a minute with vkcube --present_mode 1 on the compositor's display and the GPU's power/autosuspend_delay_ms set to 1. --- This patch was developed with the help of an LLM. I know there's a Mesa policy against LLM use for the powervr driver there, so if there's a policy for drm/imagination too, let me know. --- Changes in v2: - Sample the firmware's interrupt counts in pvr_fw_irq_clear() rather than in the IRQ thread, so that the clears at probe and in pvr_power_reset() update the host's copy too (Sashiko). This replaces the zeroing in fw_osdata_init(): after a hard reset, the reset path's clear samples the zeroed counts. - Fixes: cc1aeedb98ad, which put pvr_fw_stop() into the runtime suspend path; at 727538a4bbff it only gated the clocks (Sashiko). - Rebased on drm-misc-next. - Link to v1: https://lore.kernel.org/r/20261110-rbrue-suez-upstreaming-pvr-fw-irq-handshake-v1-1-8872686d14d0@gmail.com --- With continuous rendering and a 1 ms autosuspend delay, a suspend failed this way within 22 seconds in 7 of 7 unpatched runs. Three of them were instrumented: between them they made 2711 suspends, and the three that failed were the only ones with an interrupt still outstanding after disable_irq(). With this patch, 8 of 8 runs lasted 60 seconds without a failure, and in a further run the wait was needed in 15 of 6925 suspends, for at most 6.1 ms. --- drivers/gpu/drm/imagination/pvr_device.h | 15 +++++++++++++++ drivers/gpu/drm/imagination/pvr_power.c | 27 +++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/drivers/gpu/drm/imagination/pvr_device.h b/drivers/gpu/drm/imagination/pvr_device.h index c246122b4caf..9d46c41140b7 100644 --- a/drivers/gpu/drm/imagination/pvr_device.h +++ b/drivers/gpu/drm/imagination/pvr_device.h @@ -176,6 +176,12 @@ struct pvr_device { /** @irq: IRQ number. */ int irq; + /** + * @fw_irq_count: Firmware interrupt counts, as last sampled after + * clearing the interrupt. + */ + u32 fw_irq_count[MAX_THREAD_NUM]; + /** @fwccb: Firmware CCB. */ struct pvr_ccb fwccb; @@ -813,6 +819,15 @@ static __always_inline void pvr_fw_irq_clear(struct pvr_device *pvr_dev) { pvr_dev->fw_dev.defs->irq_clear(pvr_dev); + + /* A cleared interrupt counts as taken, whoever cleared it. */ + if (READ_ONCE(pvr_dev->fw_dev.initialised)) { + struct rogue_fwif_osdata *osdata = pvr_dev->fw_dev.fwif_osdata; + + for (u32 i = 0; i < MAX_THREAD_NUM; i++) + WRITE_ONCE(pvr_dev->fw_irq_count[i], + READ_ONCE(osdata->interrupt_count[i])); + } } #endif /* PVR_DEVICE_H */ diff --git a/drivers/gpu/drm/imagination/pvr_power.c b/drivers/gpu/drm/imagination/pvr_power.c index eb4b6ecdf4f4..2d0fbcf457f6 100644 --- a/drivers/gpu/drm/imagination/pvr_power.c +++ b/drivers/gpu/drm/imagination/pvr_power.c @@ -89,6 +89,26 @@ pvr_power_request_pwr_off(struct pvr_device *pvr_dev) return pvr_power_send_command(pvr_dev, &pow_cmd); } +static bool +pvr_power_fw_irqs_taken(struct pvr_device *pvr_dev, u32 thread) +{ + return READ_ONCE(pvr_dev->fw_dev.fwif_osdata->interrupt_count[thread]) == + READ_ONCE(pvr_dev->fw_irq_count[thread]); +} + +static void +pvr_power_wait_for_fw_irqs(struct pvr_device *pvr_dev) +{ + bool taken; + + for (u32 i = 0; i < MAX_THREAD_NUM; i++) { + if (read_poll_timeout(pvr_power_fw_irqs_taken, taken, taken, 100, + POWER_SYNC_TIMEOUT_US, false, pvr_dev, i)) + drm_warn(from_pvr_device(pvr_dev), + "FW thread %u has IRQs the host has not taken\n", i); + } +} + static int pvr_power_fw_disable(struct pvr_device *pvr_dev, bool hard_reset, bool rpm_suspend) { @@ -107,6 +127,13 @@ pvr_power_fw_disable(struct pvr_device *pvr_dev, bool hard_reset, bool rpm_suspe } if (rpm_suspend) { + /* + * Let the IRQ thread take every interrupt the firmware has raised + * before stopping it, as the vendor driver does. Without this, the + * META firmware processor can fail to go idle in pvr_fw_stop(). + */ + pvr_power_wait_for_fw_irqs(pvr_dev); + /* This also waits for late processing of GPU or firmware IRQs in other cores */ disable_irq(pvr_dev->irq); } --- base-commit: 41b5458595c60143d0ebaa1e032c9cda5d764dff change-id: 20261004-rbrue-suez-upstreaming-pvr-fw-irq-handshake-b80953eb904b Best regards, -- Ryan Brue