Some NVMe devices need more than 5 seconds to recover after DPC containment, but pci_dpc_recovered() waits only 4 seconds. If the device hasn't recovered by then, pciehp reports "device recovery failed" and the device is re-enumerated. Move PCIE_RESET_READY_POLL_MS definition from pci.c to pci.h so it can be used in dpc.c. Use PCIE_RESET_READY_POLL_MS (60 seconds) as the timeout so pciehp waits as long as pci_dev_wait() does, which is sufficient for all known devices. Tested with INNOGRIT NVMe SSD [1dbe:5668] with ACS Fatal error injection: Upstream kernel (7.3.0-master+, 4s timeout): [123.018691] nvme: not ready 1023ms after DPC; waiting [124.106690] nvme: not ready 2047ms after DPC; waiting [125.642703] pciehp: Slot(64): Link Down [126.218696] AER: subordinate device reset failed [126.218700] AER: device recovery failed Device re-enumerated as nvme1 With this patch (60s timeout): [74.894811] nvme: not ready 1023ms after DPC; waiting [75.982813] nvme: not ready 2047ms after DPC; waiting [78.094810] nvme: ready 4095ms after DPC [78.094816] nvme: restart after slot reset [78.111620] AER: device recovery successful Device works normally Fixes: a97396c6eb13 ("PCI: pciehp: Ignore Link Down/Up caused by DPC") Cc: stable@vger.kernel.org Signed-off-by: Ziyou Wang (Lenovo) --- drivers/pci/pci.c | 1 - drivers/pci/pci.h | 10 ++++++++++ drivers/pci/pcie/dpc.c | 11 ++++++----- 3 files changed, 16 insertions(+), 6 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 8cc6a89130b5..08f08f25e597 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -80,7 +80,6 @@ struct pci_pme_device { * limit, but 60 sec ought to be enough for any device to become * responsive. */ -#define PCIE_RESET_READY_POLL_MS 60000 /* msec */ static void pci_dev_d3_sleep(struct pci_dev *dev) { diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index ba3c3fddddc2..1e8ddcc013ff 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -61,6 +61,16 @@ struct pcie_tlp_log; */ #define PCIE_RESET_CONFIG_WAIT_MS 100 +/* + * Time to wait for a device to become ready after a reset. + * + * Devices may extend the 1 sec period through Request Retry Status + * completions (PCIe r6.0 sec 2.3.1). The spec does not provide an upper + * limit, but 60 sec ought to be enough for any device to become + * responsive. + */ +#define PCIE_RESET_READY_POLL_MS 60000 /* msec */ + /* Parameters for the waiting for link up routine */ #define PCIE_LINK_WAIT_MAX_RETRIES 10 #define PCIE_LINK_WAIT_SLEEP_MS 90 diff --git a/drivers/pci/pcie/dpc.c b/drivers/pci/pcie/dpc.c index 2b779bd1d861..df1c469fa24c 100644 --- a/drivers/pci/pcie/dpc.c +++ b/drivers/pci/pcie/dpc.c @@ -116,12 +116,13 @@ bool pci_dpc_recovered(struct pci_dev *pdev) return false; /* - * Need a timeout in case DPC never completes due to failure of - * dpc_wait_rp_inactive(). The spec doesn't mandate a time limit, - * but reports indicate that DPC completes within 4 seconds. + * Wait for DPC recovery completion. Use PCIE_RESET_READY_POLL_MS + * (60s) to match pci_dev_wait() timeout. Some devices need >5 seconds + * to recover after DPC containment. The 4 second timeout is too short. */ - wait_event_timeout(dpc_completed_waitqueue, dpc_completed(pdev), - msecs_to_jiffies(4000)); + wait_event_timeout(dpc_completed_waitqueue, + dpc_completed(pdev), + msecs_to_jiffies(PCIE_RESET_READY_POLL_MS)); return test_and_clear_bit(PCI_DPC_RECOVERED, &pdev->priv_flags); } -- 2.53.0