From: Florian Maurer On IPQ8074 a firmware crash is handled by remoteproc, which stops the remote processor and thereby resets WCSS. ath11k sets ATH11K_FLAG_CRASH_FLUSH, which stops the TX paths, only when it handles the QMI server exit event. That event results from the glink teardown in rproc_stop() and is handled asynchronously, so it can be processed after WCSS is already in reset. A TX in that window writes a ring head pointer to the reset block, which ends in an asynchronous SError: remoteproc remoteproc0: stopped remote processor cd00000.q6v5_wcss SError Interrupt on CPU3, code 0x00000000bf000002 -- SError pc : ath11k_dp_tx+0x45c/0x878 [ath11k] Kernel panic - not syncing: Asynchronous SError Interrupt remoteproc sends a QCOM_SSR_BEFORE_SHUTDOWN notification with crashed set before it stops the processor. Register for it, like ath10k does on SNOC, set ATH11K_FLAG_CRASH_FLUSH there and wait with synchronize_rcu() for TX paths which have not seen the flag yet. ath11k_dp_tx() is called by mac80211 within RCU read-side critical sections or with BHs disabled. Check the flag in ath11k_ce_send() again under ce_lock, so that this also covers WMI and HTC. This stops new TX descriptors, for data and for everything sent through CE. Ring pointer updates from the RX and TX completion paths (NAPI, CE tasklets) are not stopped by this. A crash during the restart work of an earlier crash, after the QMI firmware start and before ath11k_core_qmi_firmware_ready() clears ATH11K_FLAG_CRASH_FLUSH after ath11k_dp_alloc(), is not covered yet: the flag set by the notifier is cleared again, and CE/HTC sends from the restart work can still reach the reset WCSS. mac80211 data queues are still stopped at that point. That window is a few milliseconds and will be addressed separately. The SSR subsystem name depends on the remoteproc driver, so add it to hw_params. Only IPQ8074 sets it for now. qcom_register_ssr_notifier() must be reachable from ath11k_ahb, so add the same Kconfig dependency on QCOM_RPROC_COMMON as ath10k SNOC has. Tested-on: IPQ8074 hw2.0 AHB WLAN.HK.2.9.0.1-02146-QCAHKSWPL_SILICONZ-1 Signed-off-by: Florian Maurer Signed-off-by: Julius Bairaktaris --- v3: new in this series, from Florian Maurer's series [1], unchanged. [1] https://lore.kernel.org/r/cover.1793531790.git.fmaurer@disroot.org drivers/net/wireless/ath/ath11k/Kconfig | 1 + drivers/net/wireless/ath/ath11k/ahb.c | 59 +++++++++++++++++++++++++ drivers/net/wireless/ath/ath11k/ahb.h | 2 + drivers/net/wireless/ath/ath11k/ce.c | 7 +++ drivers/net/wireless/ath/ath11k/core.c | 1 + drivers/net/wireless/ath/ath11k/hw.h | 1 + 6 files changed, 71 insertions(+) diff --git a/drivers/net/wireless/ath/ath11k/Kconfig b/drivers/net/wireless/ath/ath11k/Kconfig index 2b608d4ed263..7dcc2c3543eb 100644 --- a/drivers/net/wireless/ath/ath11k/Kconfig +++ b/drivers/net/wireless/ath/ath11k/Kconfig @@ -13,6 +13,7 @@ config ATH11K_AHB tristate "Atheros ath11k AHB support" depends on ATH11K depends on REMOTEPROC + depends on QCOM_RPROC_COMMON || QCOM_RPROC_COMMON=n depends on RPMSG select QRTR select QRTR_SMD diff --git a/drivers/net/wireless/ath/ath11k/ahb.c b/drivers/net/wireless/ath/ath11k/ahb.c index 3ff7d5f27f5e..893a3270394e 100644 --- a/drivers/net/wireless/ath/ath11k/ahb.c +++ b/drivers/net/wireless/ath/ath11k/ahb.c @@ -20,6 +20,7 @@ #include "pcic.h" #include #include +#include static const struct of_device_id ath11k_ahb_of_match[] = { /* TODO: Should we change the compatible string to something similar @@ -832,6 +833,60 @@ static const struct ath11k_hif_ops ath11k_ahb_hif_ops_wcn6750 = { .ce_irq_disable = ath11k_pci_disable_ce_irqs_except_wake_irq, }; +static int ath11k_ahb_ssr_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct ath11k_ahb *ab_ahb = container_of(nb, struct ath11k_ahb, ssr_nb); + struct ath11k_base *ab = container_of((void *)ab_ahb, struct ath11k_base, + drv_priv); + struct qcom_ssr_notify_data *notify = data; + + if (action != QCOM_SSR_BEFORE_SHUTDOWN || !notify->crashed) + return NOTIFY_DONE; + + /* WCSS can be reset before the QMI server exit event sets the flag. + * ath11k_dp_tx() runs in RCU read-side sections or with BHs disabled + * and ath11k_ce_send() checks the flag under ce_lock, so after + * synchronize_rcu() no new TX descriptors are queued. + */ + ath11k_dbg(ab, ATH11K_DBG_AHB, "firmware crashed, stopping tx\n"); + set_bit(ATH11K_FLAG_CRASH_FLUSH, &ab->dev_flags); + synchronize_rcu(); + + return NOTIFY_OK; +} + +static void ath11k_ahb_register_ssr_notifier(struct ath11k_base *ab) +{ + struct ath11k_ahb *ab_ahb = ath11k_ahb_priv(ab); + void *handle; + + if (!ab->hw_params.ssr_name) + return; + + ab_ahb->ssr_nb.notifier_call = ath11k_ahb_ssr_notifier; + handle = qcom_register_ssr_notifier(ab->hw_params.ssr_name, + &ab_ahb->ssr_nb); + if (IS_ERR(handle)) { + ath11k_warn(ab, "failed to register ssr notifier: %ld\n", + PTR_ERR(handle)); + return; + } + + ab_ahb->ssr_handle = handle; +} + +static void ath11k_ahb_unregister_ssr_notifier(struct ath11k_base *ab) +{ + struct ath11k_ahb *ab_ahb = ath11k_ahb_priv(ab); + + if (!ab_ahb->ssr_handle) + return; + + qcom_unregister_ssr_notifier(ab_ahb->ssr_handle, &ab_ahb->ssr_nb); + ab_ahb->ssr_handle = NULL; +} + static int ath11k_core_get_rproc(struct ath11k_base *ab) { struct ath11k_ahb *ab_ahb = ath11k_ahb_priv(ab); @@ -1230,6 +1285,8 @@ static int ath11k_ahb_probe(struct platform_device *pdev) if (ret) goto err_ce_free; + ath11k_ahb_register_ssr_notifier(ab); + ret = ath11k_core_init(ab); if (ret) { ath11k_err(ab, "failed to init core: %d\n", ret); @@ -1247,6 +1304,7 @@ static int ath11k_ahb_probe(struct platform_device *pdev) return 0; err_ce_free: + ath11k_ahb_unregister_ssr_notifier(ab); ath11k_ce_free_pipes(ab); err_hal_srng_deinit: @@ -1288,6 +1346,7 @@ static void ath11k_ahb_free_resources(struct ath11k_base *ab) { struct platform_device *pdev = ab->pdev; + ath11k_ahb_unregister_ssr_notifier(ab); ath11k_ahb_free_irq(ab); ath11k_hal_srng_deinit(ab); ath11k_ahb_release_smp2p_handle(ab); diff --git a/drivers/net/wireless/ath/ath11k/ahb.h b/drivers/net/wireless/ath/ath11k/ahb.h index 8c1eb1e8e6b1..d7ea47f42edf 100644 --- a/drivers/net/wireless/ath/ath11k/ahb.h +++ b/drivers/net/wireless/ath/ath11k/ahb.h @@ -23,6 +23,8 @@ struct ath11k_base; struct ath11k_ahb { struct rproc *tgt_rproc; + struct notifier_block ssr_nb; + void *ssr_handle; struct { struct device *dev; struct iommu_domain *iommu_domain; diff --git a/drivers/net/wireless/ath/ath11k/ce.c b/drivers/net/wireless/ath/ath11k/ce.c index af08a2a24c6d..08fd0ab353e4 100644 --- a/drivers/net/wireless/ath/ath11k/ce.c +++ b/drivers/net/wireless/ath/ath11k/ce.c @@ -745,6 +745,12 @@ int ath11k_ce_send(struct ath11k_base *ab, struct sk_buff *skb, u8 pipe_id, spin_lock_bh(&ab->ce.ce_lock); + /* Check again with BHs disabled, see ath11k_ahb_ssr_notifier() */ + if (test_bit(ATH11K_FLAG_CRASH_FLUSH, &ab->dev_flags)) { + ret = -ESHUTDOWN; + goto err_ce_unlock; + } + write_index = pipe->src_ring->write_index; nentries_mask = pipe->src_ring->nentries_mask; @@ -791,6 +797,7 @@ int ath11k_ce_send(struct ath11k_base *ab, struct sk_buff *skb, u8 pipe_id, err_unlock: spin_unlock_bh(&srng->lock); +err_ce_unlock: spin_unlock_bh(&ab->ce.ce_lock); return ret; diff --git a/drivers/net/wireless/ath/ath11k/core.c b/drivers/net/wireless/ath/ath11k/core.c index e8d245f1023a..d4845a1300d9 100644 --- a/drivers/net/wireless/ath/ath11k/core.c +++ b/drivers/net/wireless/ath/ath11k/core.c @@ -91,6 +91,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .coldboot_cal_mm = true, .coldboot_cal_ftm = true, .cbcal_restart_fw = true, + .ssr_name = "q6wcss", .fw_mem_mode = 0, .num_vdevs = 16 + 1, .num_peers = 512, diff --git a/drivers/net/wireless/ath/ath11k/hw.h b/drivers/net/wireless/ath/ath11k/hw.h index 4996536fbd14..ce52c61d661d 100644 --- a/drivers/net/wireless/ath/ath11k/hw.h +++ b/drivers/net/wireless/ath/ath11k/hw.h @@ -191,6 +191,7 @@ struct ath11k_hw_params { bool coldboot_cal_mm; bool coldboot_cal_ftm; bool cbcal_restart_fw; + const char *ssr_name; int fw_mem_mode; u32 num_vdevs; u32 num_peers; -- 2.53.0