rxe_invalidate_mr() and rxe_invalidate_mw() look the object up by index in the device-wide pool and then only compare the key, so a QP can invalidate an MR or MW that belongs to a different protection domain on the same rxe device. Both entry points are affected: IB_WR_LOCAL_INV from the requester and SEND_WITH_INVALIDATE from the responder, where the rkey comes off the wire. Everything else that resolves one of these objects already refuses a foreign PD. lookup_mr() rejects mr_pd(mr) != pd, rxe_reg_fast_mr() rejects qp->ibqp.pd != mr->ibmr.pd, and rxe_check_bind_mw() was given the same check in commit fbbe1d5cd7d4 ("RDMA/rxe: Check PDs for memory window binds"). Remote access is therefore scoped to a PD while remote invalidation is not. A peer on one connection can invalidate a kernel ULP's fast-reg MR on another, which ends that ULP's I/O. Reject a foreign PD in both functions; -EINVAL turns into RESPST_ERR_INVALIDATE_RKEY for the responder and a failed work request for the requester. The MR side has behaved this way since the driver was merged, where do_complete() freed the MR it found by index; rxe_invalidate_mw() arrived later carrying the same omission. Fixes: 8700e3e7c485 ("Soft RoCE driver") Cc: stable@vger.kernel.org Signed-off-by: Yehyeong Lee --- Reproduced on rxe with two protection domains on one device. PD-A holds the attacking QP, nvme-rdma holds PD-B and is running I/O. The RXEINV/RXEPD lines below are instrumentation I added to observe it and are not part of this patch: RXEINV prints the two PDs and the MR state inside rxe_invalidate_mr(), and the attacker takes the target rkey from rxe_reg_fast_mr() rather than guessing it. Before, a SEND_WITH_INVALIDATE naming nvme-rdma's rkey is accepted: RXEPD: hunter: TARGET is a foreign-PD fast-reg MR: rkey=0x29ba mr_pd=00000000185f3e0e (mine: A=00000000d715b639 B=000000001440b63b) iova=0xffff888065174000 RXEPD: hunter: firing SEND_WITH_INV from QP-A(PD-A=00000000d715b639) at rkey=0x29ba rdma_rxe: RXEINV: REMOTE entry rxe_resp.c execute() qp=27 wire_rkey=0x29ba rdma_rxe: RXEINV: enter rxe_invalidate_mr qp=27 qp_pd=00000000d715b639 mr_pd=00000000185f3e0e cross_pd=1 key=0x29ba state_before=2 rdma_rxe: RXEINV: done rxe_invalidate_mr key=0x29ba state_after=1 (RXE_MR_STATE_FREE=1) RXEPD: hunter SEND_WITH_INV -> wc.status=0 (success) and nvme-rdma's I/O stops: nvme nvme0: RECV for CQE 0x00000000448b8c67 failed with status WR flushed (5) nvme nvme0: starting error recovery nvme nvme0: Reconnecting in 10 seconds... The controller does not recover, because each reconnect registers a new MR that is invalidated the same way. After, the same attack is refused: RXEPD: hunter: TARGET is a foreign-PD fast-reg MR: rkey=0x90a0 mr_pd=0000000049ba8cc5 (mine: A=00000000a32c3059 B=000000006c5e8389) iova=0xffff88806bbb0000 RXEPD: hunter: firing SEND_WITH_INV from QP-A(PD-A=00000000a32c3059) at rkey=0x90a0 RXEPD: hunter SEND_WITH_INV -> wc.status=5 (WR flushed) rxe_invalidate_mr() is not reached for a foreign PD, so no cross_pd=1 entry is traced (4 before, 0 after), nvme-rdma keeps running, BCHECK[D] DONE iters=524 clean_p0=524 ioerr=0 CORRUPT_HITS=0 POST_COMPLETION=0 Only the MR path is exercised above. The rxe_invalidate_mw() hunk is the same omission in the MW path and is fixed for symmetry; I did not reproduce a cross-PD MW invalidation. nvme-rdma's own same-PD invalidations keep working: 3180 of them are traced with the check in place. drivers/infiniband/sw/rxe/rxe_mr.c | 6 ++++++ drivers/infiniband/sw/rxe/rxe_mw.c | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index 71d9ea4772890..2da5134560730 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -743,6 +743,12 @@ int rxe_invalidate_mr(struct rxe_qp *qp, u32 key) goto err_drop_ref; } + if (unlikely(qp->ibqp.pd != mr->ibmr.pd)) { + rxe_dbg_mr(mr, "qp->pd and mr->pd don't match\n"); + ret = -EINVAL; + goto err_drop_ref; + } + if (atomic_read(&mr->num_mw) > 0) { rxe_dbg_mr(mr, "Attempt to invalidate an MR while bound to MWs\n"); ret = -EINVAL; diff --git a/drivers/infiniband/sw/rxe/rxe_mw.c b/drivers/infiniband/sw/rxe/rxe_mw.c index bddb7a2578313..ab2212eabd71e 100644 --- a/drivers/infiniband/sw/rxe/rxe_mw.c +++ b/drivers/infiniband/sw/rxe/rxe_mw.c @@ -276,6 +276,11 @@ int rxe_invalidate_mw(struct rxe_qp *qp, u32 rkey) goto err_drop_ref; } + if (unlikely(qp->pd != rxe_mw_pd(mw))) { + ret = -EINVAL; + goto err_drop_ref; + } + spin_lock_bh(&mw->lock); ret = rxe_check_invalidate_mw(qp, mw); -- 2.43.0