Similar to the skb path, the XDP transmit path may put frames shorter than ETH_ZLEN in the TX ring, and may put frames whose first buffer is shorter than 16 bytes. The former relies on mEMAC hardware padding, which we no longer want to rely upon, and the latter is not supported by ENETC at all. There are two callers of enetc_xdp_tx(): - ndo_xdp_xmit(), which can transmit short frames for example when redirecting from a veth interface pair through a devmap. - XDP_TX from enetc_clean_rx_ring_xdp(). With RX VLAN extraction enabled (the default), a minimum-sized VLAN-tagged frame is 56 bytes long when it is sent back. Furthermore, once the driver honours bpf_xdp_adjust_tail() and bpf_xdp_adjust_head() for XDP_TX, the BPF program can make the frame arbitrarily short. Handle both in enetc_xdp_tx(). When the frame is shorter than ETH_ZLEN, append a BD pointing to the shared read-only tx_pad zero buffer, the same as for short TSO segments, and advertise the padded frame length in the BDs. The pad BD becomes the frame's last BD, so the F flag, the is_eof marker and the xdp_frame pointer (for XDP_REDIRECT) move onto it. The pad BD has no DMA mapping of its own, and it is not an XDP_TX buffer, so TX completion neither unmaps nor recycles it. It is also not accounted in xdp_tx_in_flight, which counts RX buffers. The 16 byte limitation for the first buffer cannot be addressed by the pad BD, so frames with a head buffer shorter than that are dropped. Both callers already treat a false return code from enetc_xdp_tx() as a drop. Doing this in the driver rather than padding the xdp_frame in place also covers multi-buffer frames shorter than ETH_ZLEN, and avoids writing to the tailroom of buffers that, in the XDP_TX case, are recycled into the RX ring. Fixes: 7ed2bc80074e ("net: enetc: add support for XDP_TX") Fixes: 9d2b68cc108d ("net: enetc: add support for XDP_REDIRECT") Signed-off-by: Vladimir Oltean --- v3->v4: - patch is new. It replaces "net: enetc: pad short XDP frames coming from devmap", which padded the xdp_frame in place using a new generic xdp_frame_pad() helper, and only covered ndo_xdp_xmit(). The XDP_TX path is now covered too, using the tx_pad buffer instead. --- drivers/net/ethernet/freescale/enetc/enetc.c | 36 +++++++++++++++++--- 1 file changed, 32 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c index 08870df613b8..15c48c938a4e 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc.c +++ b/drivers/net/ethernet/freescale/enetc/enetc.c @@ -1747,8 +1747,11 @@ static bool enetc_xdp_tx(struct enetc_bdr *tx_ring, { struct enetc_tx_swbd *tmp_tx_swbd = xdp_tx_arr; int i, k, frm_len = tmp_tx_swbd->len; + struct enetc_tx_swbd *last_swbd; + int pad_len; - if (unlikely(enetc_bd_unused(tx_ring) < ENETC_TXBDS_NEEDED(num_tx_swbd))) + /* Hardware does not support a first buffer size of less than 16 bytes */ + if (unlikely(xdp_tx_arr[0].len < ENETC_MIN_BUFF_SIZE)) return false; while (unlikely(!tmp_tx_swbd->is_eof)) { @@ -1756,15 +1759,24 @@ static bool enetc_xdp_tx(struct enetc_bdr *tx_ring, frm_len += tmp_tx_swbd->len; } + /* Pad frames below ETH_ZLEN so the mEMAC does not pad them. */ + pad_len = max_t(int, ETH_ZLEN - frm_len, 0); + + if (unlikely(enetc_bd_unused(tx_ring) < + ENETC_TXBDS_NEEDED(num_tx_swbd + !!pad_len))) + return false; + i = tx_ring->next_to_use; for (k = 0; k < num_tx_swbd; k++) { struct enetc_tx_swbd *xdp_tx_swbd = &xdp_tx_arr[k]; - enetc_xdp_map_tx_buff(tx_ring, i, xdp_tx_swbd, frm_len); + enetc_xdp_map_tx_buff(tx_ring, i, xdp_tx_swbd, + frm_len + pad_len); + last_swbd = &tx_ring->tx_swbd[i]; - /* last BD needs 'F' bit set */ - if (xdp_tx_swbd->is_eof) { + /* last BD needs 'F' bit set, unless a pad BD follows */ + if (xdp_tx_swbd->is_eof && !pad_len) { union enetc_tx_bd *txbd = ENETC_TXBD(*tx_ring, i); txbd->flags = ENETC_TXBD_FLAGS_F; @@ -1773,6 +1785,22 @@ static bool enetc_xdp_tx(struct enetc_bdr *tx_ring, enetc_bdr_idx_inc(tx_ring, &i); } + /* The pad BD becomes the last one, so it takes over the xdp_frame + * (if any) from the last data BD. The pad BD has no DMA mapping of + * its own, and it is not an XDP_TX buffer to be recycled. + */ + if (unlikely(pad_len)) { + struct enetc_tx_swbd *pad_swbd = &tx_ring->tx_swbd[i]; + + enetc_map_tx_pad(tx_ring, i, pad_len); + pad_swbd->is_xdp_redirect = last_swbd->is_xdp_redirect; + pad_swbd->xdp_frame = last_swbd->xdp_frame; + last_swbd->xdp_frame = NULL; + last_swbd->is_eof = false; + + enetc_bdr_idx_inc(tx_ring, &i); + } + tx_ring->next_to_use = i; return true; -- 2.43.0