Add a new YNL attribute `ignore-index` as a way to indicate that a given indexed-array attribute is really just used as an array, and hence the nested attribute-type aka. the index is unimportant. This means that the kernel never uses the index when processing received netlink messages, and that it doesn't add any additional information when sent by the kernel. For backward compatibility reasons the kernel can continue to set the index to a non-zero value in netlink messages, but it can safely be disregarded by clients when `ignore-index` is set to true. When the index is non-zero, it is often just an incremental iterator value, which provides no additional value, as the order of the array elements is already known based on the order in the netlink message. `ignore-index` is not added for the genetlink protocol, as any new families should use multi-attributes, unless they need the index. Signed-off-by: Asbjørn Sloth Tønnesen --- Documentation/netlink/genetlink-c.yaml | 6 ++++++ Documentation/netlink/genetlink-legacy.yaml | 6 ++++++ Documentation/netlink/netlink-raw.yaml | 6 ++++++ Documentation/userspace-api/netlink/genetlink-legacy.rst | 3 +++ 4 files changed, 21 insertions(+) diff --git a/Documentation/netlink/genetlink-c.yaml b/Documentation/netlink/genetlink-c.yaml index 5a234e9b5fa2e..5d022772cdb61 100644 --- a/Documentation/netlink/genetlink-c.yaml +++ b/Documentation/netlink/genetlink-c.yaml @@ -184,6 +184,12 @@ properties: nested-attributes: description: Name of the space (sub-space) used inside the attribute. type: string + ignore-index: + description: | + The indexed-array is just an array. The index, aka. the + nested attribute-type, can be disregarded, as it doesn't + contain anything interesting. + type: boolean enum: description: Name of the enum type used for the attribute. type: string diff --git a/Documentation/netlink/genetlink-legacy.yaml b/Documentation/netlink/genetlink-legacy.yaml index 66fb8653a3442..f7991a3c5e2a3 100644 --- a/Documentation/netlink/genetlink-legacy.yaml +++ b/Documentation/netlink/genetlink-legacy.yaml @@ -233,6 +233,12 @@ properties: nested-attributes: description: Name of the space (sub-space) used inside the attribute. type: string + ignore-index: + description: | + The indexed-array is just an array. The index, aka. the + nested attribute-type, can be disregarded, as it doesn't + contain anything interesting. + type: boolean enum: description: Name of the enum type used for the attribute. type: string diff --git a/Documentation/netlink/netlink-raw.yaml b/Documentation/netlink/netlink-raw.yaml index 246fa07bccf68..1d2ff5f79cada 100644 --- a/Documentation/netlink/netlink-raw.yaml +++ b/Documentation/netlink/netlink-raw.yaml @@ -251,6 +251,12 @@ properties: nested-attributes: description: Name of the space (sub-space) used inside the attribute. type: string + ignore-index: + description: | + The indexed-array is just an array. The index, aka. the + nested attribute-type, can be disregarded, as it doesn't + contain anything interesting. + type: boolean enum: description: Name of the enum type used for the attribute. type: string diff --git a/Documentation/userspace-api/netlink/genetlink-legacy.rst b/Documentation/userspace-api/netlink/genetlink-legacy.rst index fa005989193a1..839b0095c9a80 100644 --- a/Documentation/userspace-api/netlink/genetlink-legacy.rst +++ b/Documentation/userspace-api/netlink/genetlink-legacy.rst @@ -53,6 +53,9 @@ indexed-array limiting its size to 64kB). The ``ENTRY`` nests are special and have the index of the entry as their type instead of normal attribute type. +When ``ignore-index`` is set to ``true``, then the ``indexed-array`` is +just an array, and the index can be disregarded. + A ``sub-type`` is needed to describe what type in the ``ENTRY``. A ``nest`` ``sub-type`` means there are nest arrays in the ``ENTRY``, with the structure looks like:: -- 2.51.0 When decoding indexed-arrays with `ignore-index` set, then elide the index, aka. the nested attribute-type. Previously the index was always preserved for sub-type nest, but was elided for all other sub-types. I have opted to reuse the existing `subattr` variable, as renaming it will render the diff unreadable at least for this version. Output if `rates` does NOT have `ignore-index` set: $ ./tools/net/ynl/pyynl/cli.py --family nl80211 --dump get-wiphy [...] 'rates': [{0: {'rate': 60}}, {1: {'rate': 90}}, {2: {'rate': 120}}, {3: {'rate': 180}}, {4: {'rate': 240}}, [...] Output if `rates` has `ignore-index` set to true: $ ./tools/net/ynl/pyynl/cli.py --family nl80211 --dump get-wiphy [...] 'rates': [{'rate': 60}, {'rate': 90}, {'rate': 120}, {'rate': 180}, {'rate': 240}, [...] If the above example had to be passed back though --json, then it now aligns with the new output format, and would look like this: --json '{"rates":[ {"rate":60}, {"rate":90}, ... ]}' Signed-off-by: Asbjørn Sloth Tønnesen --- tools/net/ynl/pyynl/lib/ynl.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/tools/net/ynl/pyynl/lib/ynl.py b/tools/net/ynl/pyynl/lib/ynl.py index 62383c70ebb95..14c7e51db6f5c 100644 --- a/tools/net/ynl/pyynl/lib/ynl.py +++ b/tools/net/ynl/pyynl/lib/ynl.py @@ -690,23 +690,26 @@ class YnlFamily(SpecFamily): item = NlAttr(attr.raw, offset) offset += item.full_len + subattr = None if attr_spec["sub-type"] == 'nest': - subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) - decoded.append({ item.type: subattrs }) + subattr = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) elif attr_spec["sub-type"] == 'binary': subattr = item.as_bin() if attr_spec.display_hint: subattr = self._formatted_string(subattr, attr_spec.display_hint) - decoded.append(subattr) elif attr_spec["sub-type"] in NlAttr.type_formats: subattr = item.as_scalar(attr_spec['sub-type'], attr_spec.byte_order) if 'enum' in attr_spec: subattr = self._decode_enum(subattr, attr_spec) elif attr_spec.display_hint: subattr = self._formatted_string(subattr, attr_spec.display_hint) - decoded.append(subattr) else: raise Exception(f'Unknown {attr_spec["sub-type"]} with name {attr_spec["name"]}') + + if attr_spec.get('ignore-index', False): + decoded.append(subattr) + else: + decoded.append({ item.type: subattr }) return decoded def _decode_nest_type_value(self, attr, attr_spec): -- 2.51.0 When parsing indexed-array attributes to cli.py's --json, then previously the index would always be set incrementally. This patch adds support for setting arbitrary indexes, when `ignore-index` is set to false or is unspecified. When `ignore-index` is set to true, then it retains the current input format, and it's alignment with the output from cli.py. The below examples are fictive as `rates` is not used in requests. The implementation have been tested with a newer version of the previously posted wireguard spec[1]. When `rates` have `ignore-index` set to false (or unspecified): --json '{"rates":[ [0,{"rate":60}], [42,{"rate":90}] ]}' When `rates` have `ignore-index` set to true: --json '{"rates":[ {"rate":60}, {"rate":90} ]}' [1] https://lore.kernel.org/netdev/20250904-wg-ynl-rfc@fiberby.net/ Signed-off-by: Asbjørn Sloth Tønnesen --- tools/net/ynl/pyynl/lib/ynl.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/tools/net/ynl/pyynl/lib/ynl.py b/tools/net/ynl/pyynl/lib/ynl.py index 14c7e51db6f5c..a284bc2ad3440 100644 --- a/tools/net/ynl/pyynl/lib/ynl.py +++ b/tools/net/ynl/pyynl/lib/ynl.py @@ -566,8 +566,9 @@ class YnlFamily(SpecFamily): elif attr['type'] == 'indexed-array' and attr['sub-type'] == 'nest': nl_type |= Netlink.NLA_F_NESTED sub_space = attr['nested-attributes'] + ignore_idx = attr.get('ignore-index', False) attr_payload = self._encode_indexed_array(value, sub_space, - search_attrs) + search_attrs, ignore_idx) elif attr["type"] == 'flag': if not value: # If value is absent or false then skip attribute creation. @@ -635,9 +636,11 @@ class YnlFamily(SpecFamily): sub_attrs) return attr_payload - def _encode_indexed_array(self, vals, sub_space, search_attrs): + def _encode_indexed_array(self, vals, sub_space, search_attrs, ignore_idx): attr_payload = b'' for i, val in enumerate(vals): + if not ignore_idx: + i, val = val[0], val[1] idx = i | Netlink.NLA_F_NESTED val_payload = self._add_nest_attrs(val, sub_space, search_attrs) attr_payload += self._add_attr_raw(idx, val_payload) -- 2.51.0 The indexes in nl80211 indexed-arrays have no special meaning, they are just written with an iterator index, which refers to the order in which they have been packed into the netlink message. Thus this patch sets ignore-index on these attributes. Most of these are only used for dumping kernel state, and are never parsed by the kernel. ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━┳━━━━━━━━┓ ┃ ┃ out/ ┃ input/ ┃ ignore ┃ ┃ Attribute ┃ dump ┃ parsed ┃ -index ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━╇━━━━━━━━┩ │ NL80211_ATTR_SUPPORTED_COMMANDS │ 1++ │ - │ yes │ │ NL80211_ATTR_INTERFACE_COMBINATIONS │ 1++ │ - │ yes │ │ NL80211_BAND_ATTR_FREQS │ 0++ │ - │ yes │ │ NL80211_BAND_ATTR_RATES │ 0++ │ - │ yes │ │ NL80211_BAND_ATTR_IFTYPE_DATA │ 1++ │ - │ yes │ │ NL80211_FREQUENCY_ATTR_WMM │ 0++ │ - │ yes │ │ NL80211_IFACE_COMB_LIMITS │ 1++ │ - │ yes │ │ NL80211_SAR_ATTR_SPECS │ 1++ │ yes(2) │ yes │ └─────────────────────────────────────┴──────┴────────┴────────┘ Where: 0++) incrementing index starting from 0 1++) incrementing index starting from 1 2) NL80211_SAR_ATTR_SPECS is parsed in nl80211_set_sar_specs(), which doesn't use the index. Additionally it also has a NLA_POLICY_NESTED_ARRAY() policy, as defined in commit 1501d13596b9 ("netlink: add nested array policy validation"), meaning that the index has no meaning, and can be disregarded. Signed-off-by: Asbjørn Sloth Tønnesen --- Documentation/netlink/specs/nl80211.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/Documentation/netlink/specs/nl80211.yaml b/Documentation/netlink/specs/nl80211.yaml index 802097128bdae..b93c612037e26 100644 --- a/Documentation/netlink/specs/nl80211.yaml +++ b/Documentation/netlink/specs/nl80211.yaml @@ -390,6 +390,7 @@ attribute-sets: name: supported-commands type: indexed-array sub-type: u32 + ignore-index: true enum: commands - name: frame @@ -608,6 +609,7 @@ attribute-sets: name: interface-combinations type: indexed-array sub-type: nest + ignore-index: true nested-attributes: if-combination-attributes - name: software-iftypes @@ -1307,11 +1309,13 @@ attribute-sets: name: freqs type: indexed-array sub-type: nest + ignore-index: true nested-attributes: frequency-attrs - name: rates type: indexed-array sub-type: nest + ignore-index: true nested-attributes: bitrate-attrs - name: ht-mcs-set @@ -1335,6 +1339,7 @@ attribute-sets: name: iftype-data type: indexed-array sub-type: nest + ignore-index: true nested-attributes: iftype-data-attrs - name: edmg-channels @@ -1418,6 +1423,7 @@ attribute-sets: name: wmm type: indexed-array sub-type: nest + ignore-index: true nested-attributes: wmm-attrs - name: no-he @@ -1474,6 +1480,7 @@ attribute-sets: name: limits type: indexed-array sub-type: nest + ignore-index: true nested-attributes: iface-limit-attributes - name: maxnum @@ -1613,6 +1620,7 @@ attribute-sets: name: specs type: indexed-array sub-type: nest + ignore-index: true nested-attributes: sar-specs - name: sar-specs -- 2.51.0 The indexes in nlctrl indexed-arrays have no special meaning, they are just written with an iterator index, which refers to the order in which they have been packed into the netlink message. Thus this patch sets ignore-index on these attributes. ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━┳━━━━━━━━┓ ┃ ┃ out/ ┃ input/ ┃ ignore ┃ ┃ Attribute ┃ dump ┃ parsed ┃ -index ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━╇━━━━━━━━┩ │ CTRL_ATTR_OPS │ 1++ │ - │ yes │ │ CTRL_ATTR_MCAST_GROUPS │ 1++ │ - │ yes │ └─────────────────────────────────────┴──────┴────────┴────────┘ Where: 1++) incrementing index starting from 1 Signed-off-by: Asbjørn Sloth Tønnesen --- Documentation/netlink/specs/nlctrl.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/netlink/specs/nlctrl.yaml b/Documentation/netlink/specs/nlctrl.yaml index 8b4472a6aa36a..753cf1b48c252 100644 --- a/Documentation/netlink/specs/nlctrl.yaml +++ b/Documentation/netlink/specs/nlctrl.yaml @@ -67,11 +67,13 @@ attribute-sets: name: ops type: indexed-array sub-type: nest + ignore-index: true nested-attributes: op-attrs - name: mcast-groups type: indexed-array sub-type: nest + ignore-index: true nested-attributes: mcast-group-attrs - name: policy -- 2.51.0 Most of the indexes in rt-link indexed-arrays have no special meaning, they are just written with an iterator index, which refers to the order in which they have been packed into the netlink message. Thus this patch sets ignore-index on these two attributes. ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━┳━━━━━━━━┓ ┃ ┃ out/ ┃ input/ ┃ ignore ┃ ┃ Attribute ┃ dump ┃ parsed ┃ -index ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━╇━━━━━━━━┩ │ IFLA_BOND_ARP_IP_TARGET │ 0++ │ yes(1) │ yes │ │ IFLA_BOND_NS_IP6_TARGET │ 0++ │ yes(1) │ yes │ │ IFLA_OFFLOAD_XSTATS_HW_S_INFO │ (2) │ - │ no │ └─────────────────────────────────────┴──────┴────────┴────────┘ Where: 0++) incrementing index starting from 0 1) When parsed the index is unused. 2) IFLA_OFFLOAD_XSTATS_L3_STATS is used as index in rtnl_offload_xstats_fill_hw_s_info(). Signed-off-by: Asbjørn Sloth Tønnesen --- Documentation/netlink/specs/rt-link.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/netlink/specs/rt-link.yaml b/Documentation/netlink/specs/rt-link.yaml index 2a23e9699c0b6..d2aaacd29f9f9 100644 --- a/Documentation/netlink/specs/rt-link.yaml +++ b/Documentation/netlink/specs/rt-link.yaml @@ -1256,6 +1256,7 @@ attribute-sets: name: arp-ip-target type: indexed-array sub-type: u32 + ignore-index: true byte-order: big-endian display-hint: ipv4 - @@ -1330,6 +1331,7 @@ attribute-sets: name: ns-ip6-target type: indexed-array sub-type: binary + ignore-index: true display-hint: ipv6 checks: exact-len: 16 -- 2.51.0 The indexes in tc indexed-arrays are mostly used for defining the priority winin an array of actions, and when parsed by the kernel they must be unique, and not exceed TCA_ACT_MAX_PRIO (32). Therefore this patch only sets ignore-index on a single attribute TCA_CAKE_STATS_TIN_STATS, which is only used for dumping statistics, and never ingested by the kernel. ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┓ ┃ ┃ out/ ┃ input/ ┃ ignore ┃ ┃ Attribute ┃ dump ┃ parsed ┃ -index ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━┩ │ TCA_BASIC_ACT │ 1++ (2) │ yes(3) │ no │ │ TCA_BPF_ACT │ 1++ (2) │ yes(3) │ no │ │ TCA_CAKE_STATS_TIN_STATS │ 1++ (4) │ - │ yes │ │ TCA_CGROUP_ACT │ 1++ (2) │ yes(3) │ no │ │ TCA_FLOWER_ACT │ 1++ (2) │ yes(3) │ no │ │ TCA_FW_ACT │ 1++ (2) │ yes(3) │ no │ │ TCA_MATCHALL_ACT │ 1++ (2) │ yes(3) │ no │ │ TCA_ROUTE4_ACT │ 1++ (2) │ yes(3) │ no │ │ TCA_U32_ACT │ 1++ (2) │ yes(3) │ no │ └──────────────────────────────────┴─────────┴────────┴────────┘ Where: 1++) incrementing index starting from 1 2) All _ACT are dumped with tcf_exts_dump() calling tcf_action_dump(). 3) Parsed in tcf_action_init(). 4) Dumped in cake_dump_stats(). Signed-off-by: Asbjørn Sloth Tønnesen --- Documentation/netlink/specs/tc.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/netlink/specs/tc.yaml b/Documentation/netlink/specs/tc.yaml index b398f7a46dae1..24866fccb7d15 100644 --- a/Documentation/netlink/specs/tc.yaml +++ b/Documentation/netlink/specs/tc.yaml @@ -2188,6 +2188,7 @@ attribute-sets: name: tin-stats type: indexed-array sub-type: nest + ignore-index: true nested-attributes: cake-tin-stats-attrs - name: deficit -- 2.51.0