From: Willem de Bruijn When hardware offload is enabled, FQ will forward packets to the netdevice for pacing. The device has to test that skb->tstamp is in the future. Avoid this cost for packets whose txtime has already passed, by clearing skb->tstamp. Also clear slightly into the future, for EDT timestamps that are so close to now that they fall within a reasonable normal Tx latency. This slack is set to 400 nsec. Signed-off-by: Willem de Bruijn --- Changes v6 -> v7 - timer drift logic fix moved to separate commit v5 -> v6 - switch to skb_set_delivery_time to also clear SKB_CLOCK_REALTIME - add fq_offload_slack_ns v3 -> v4 - also reset tstamp_type - minor: initialize time_next_packet for more obvious correctness Sashiko, ignore pre-existing issues. In particular, effects on non-EDT packets and when queue or sk rate limit is set. Sashiko, pacing offload is an optimization. Ignore that some packets may not get offloaded, e.g., when txtime is a few usec in the future. Claude suggests to only clear delivery time in one location in fq_dequeue. Unfortunately the separate fastpath location is needed as that avoids computing now in fq_dequeue for these fastpath packets. --- net/sched/sch_fq.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/net/sched/sch_fq.c b/net/sched/sch_fq.c index 7cf7afda699f..322120760507 100644 --- a/net/sched/sch_fq.c +++ b/net/sched/sch_fq.c @@ -159,6 +159,9 @@ struct fq_sched_data { u64 stat_allocation_errors; }; +/* EDT timestamps to clear beyond now. */ +static const int fq_offload_slack_ns = 400; + /* return the i-th 2-bit value ("crumb") */ static u8 fq_prio2band(const u8 *prio2band, unsigned int prio) { @@ -399,6 +402,11 @@ static struct fq_flow *fq_classify(struct Qdisc *sch, struct sk_buff *skb, READ_ONCE(sk->sk_pacing_status) != SK_PACING_FQ) smp_store_release(&sk->sk_pacing_status, SK_PACING_FQ); + + if (q->offload_horizon && + fq_skb_cb(skb)->time_to_send <= now + fq_offload_slack_ns) + skb_set_delivery_time(skb, 0, 0); + return &q->internal; } @@ -707,6 +715,7 @@ static struct sk_buff *fq_dequeue(struct Qdisc *sch) struct fq_sched_data *q = qdisc_priv(sch); struct fq_perband_flows *pband; struct fq_flow_head *head; + u64 time_next_packet = 0; struct sk_buff *skb; struct fq_flow *f; unsigned long rate; @@ -721,7 +730,7 @@ static struct sk_buff *fq_dequeue(struct Qdisc *sch) if (skb) { q->internal.qlen--; fq_dequeue_skb(sch, &q->internal, skb); - goto out; + return skb; } now = ktime_get_ns(); @@ -758,8 +767,8 @@ static struct sk_buff *fq_dequeue(struct Qdisc *sch) skb = fq_peek(f); if (skb) { - u64 time_next_packet = max_t(u64, fq_skb_cb(skb)->time_to_send, - f->time_next_packet); + time_next_packet = max_t(u64, fq_skb_cb(skb)->time_to_send, + f->time_next_packet); if (now + q->offload_horizon < time_next_packet) { head->first = f->next; @@ -836,7 +845,12 @@ static struct sk_buff *fq_dequeue(struct Qdisc *sch) } f->time_next_packet = now + len; } + out: + if (q->offload_horizon && + time_next_packet && time_next_packet <= now + fq_offload_slack_ns) + skb_set_delivery_time(skb, 0, 0); + return skb; } -- 2.55.0.897.gb25b4bd76c-goog