Since commit 13210fc63f35 ("netfilter: nf_tables: imbalance in flowtable binding"), FLOW_BLOCK_UNBIND is called when the flowtable hooks are unregistered, in the commit phase, and nf_flow_table_free() only tears the flows down later. Their FLOW_CLS_DESTROY requests then find no block callback, so the rules stay in hardware: the flows keep being forwarded, and drivers such as mtk_eth_soc and mlx5 keep their entries and reject a later flow with the same cookie with -EEXIST. Commit 5acab91458ce ("netfilter: nf_tables: unbind callbacks from flowtable destroy path") had placed the unbind after nf_flow_table_free() for this reason. Remove the flows of the device from the hardware in nf_flow_table_offload_setup() before unbinding it, as nf_flow_table_indr_cleanup() does for indirect blocks. The GC run that nf_flow_table_gc_cleanup() waits for may skip flows with pending offload work, or have started before the teardown, so queue the GC work once more, wait for it and flush the delete work again. A direct nf_flow_table_gc_run() would race with the GC worker. No BIND or UNBIND call is added. Found on a router with mtk_eth_soc flow offloading, where each firewall reload left the flows of the old flowtable bound in the PPE. Jalen reproduced it on mainline with mtk_eth_soc and mlx5 and tested this change on both, see the report. Build-tested with W=1 and sparse on arm64 and x86_64. In an arm64 guest with lockdep and KASAN, with TCP flows through an offload flowtable on netdevsim ports, the delete work of a deleted flowtable now runs before FLOW_BLOCK_UNBIND returns instead of after it, and nft_offload.sh passes. Fixes: 13210fc63f35 ("netfilter: nf_tables: imbalance in flowtable binding") Reported-by: Jalen Closes: https://lore.kernel.org/netfilter-devel/CAHqkChbVoGHrWtaFyGFaLQDiAunWw1qOpW-vKSgkKxWEfQ_tCg@mail.gmail.com/ Cc: stable@vger.kernel.org Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Paul-Louis Charrière --- net/netfilter/nf_flow_table_offload.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/net/netfilter/nf_flow_table_offload.c b/net/netfilter/nf_flow_table_offload.c index 4365859220e..3844a23af5b 100644 --- a/net/netfilter/nf_flow_table_offload.c +++ b/net/netfilter/nf_flow_table_offload.c @@ -1279,6 +1279,19 @@ int nf_flow_table_offload_setup(struct nf_flowtable *flowtable, if (!nf_flowtable_hw_offload(flowtable)) return nf_flow_offload_xdp_setup(flowtable, dev, cmd); + /* Hardware rules are deleted through the block callbacks, so delete + * the rules of this device's flows before unbinding it. Run the GC + * twice: the first run may skip flows with pending offload work, or + * have started before the teardown. + */ + if (cmd == FLOW_BLOCK_UNBIND) { + nf_flow_table_gc_cleanup(flowtable, dev); + mod_delayed_work(system_power_efficient_wq, + &flowtable->gc_work, 0); + flush_delayed_work(&flowtable->gc_work); + flush_workqueue(nf_flow_offload_del_wq); + } + if (dev->netdev_ops->ndo_setup_tc) err = nf_flow_table_offload_cmd(&bo, flowtable, dev, cmd, &extack); base-commit: 734bef548a96e8774d6eef5dfa2026c6edc4e80b -- 2.39.5