An element returned by an iterator's next() kfunc is kept alive by the iterator alone. bpf_iter_task_vma_next() returns a snapshot that lives in the iterator's own allocation and holds the vma's vm_file reference only until the next step or destroy; bpf_iter_kmem_cache_next() and bpf_iter_dmabuf_next() hand out objects whose reference the iterator drops the same way. The verifier marks every such element PTR_TRUSTED with nothing tying it to the iterator, so a pointer saved across bpf_iter__destroy() stays fully trusted after the memory or the reference behind it is gone: bpf_iter_task_vma_new(&it, task, 0); vma = bpf_iter_task_vma_next(&it); file = vma->vm_file; /* __safe_trusted_or_null */ bpf_iter_task_vma_destroy(&it); /* frees snapshot, fput() */ start = vma->vm_start; /* vma, file still trusted */ Set the element's parent_id to the iterator's reference id, as is already done for the PTR_TO_MEM elements of bpf_iter_num_next(), so that destroying the iterator invalidates the element together with every trusted field acquired from it. Elements of an RCU-protected iterator keep MEM_RCU and stay bounded by the critical section instead: they point at RCU-managed objects rather than iterator state. Fixes: 4ac454682158 ("bpf: Introduce task_vma open-coded iterator kfuncs") Cc: stable@vger.kernel.org Signed-off-by: Justin Suess --- Documentation/bpf/bpf_iterators.rst | 6 +++++- kernel/bpf/verifier.c | 13 +++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/Documentation/bpf/bpf_iterators.rst b/Documentation/bpf/bpf_iterators.rst index 189e3ec1c6c8..08c9c15b59ac 100644 --- a/Documentation/bpf/bpf_iterators.rst +++ b/Documentation/bpf/bpf_iterators.rst @@ -79,7 +79,11 @@ Constructor/next/destructor implementation contract is as follows: will eventually return NULL when elements are exhausted. Once NULL is returned, subsequent next calls *should keep returning NULL*. Next method is marked with KF_ITER_NEXT (and should also have KF_RET_NULL as - NULL-returning kfunc, of course). + NULL-returning kfunc, of course). An element is only guaranteed to be + valid until the iterator is destroyed, so the verifier invalidates it, + and every trusted pointer derived from it, at `bpf_iter__destroy()`. + Elements of an RCU-protected iterator (see KF_RCU_PROTECTED) are instead + valid until the end of the RCU critical section. - destructor, `bpf_iter__destroy()`, is always called once. Even if constructor failed or next returned nothing. Destructor frees up any diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index c85c83a6ee63..9ce5517dc1a0 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -15504,6 +15504,19 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0].btf = desc_btf; regs[BPF_REG_0].type = type; regs[BPF_REG_0].btf_id = ptr_type_id; + + /* + * A trusted element is pinned by the iterator that + * produced it and dies with it, like the memory + * returned by bpf_iter_num_next() above. An RCU + * element is bounded by the critical section instead. + */ + if (bpf_is_iter_next_kfunc(&meta) && (type & PTR_TRUSTED)) { + err = validate_ref_obj(env, &meta.ref_obj); + if (err) + return err; + regs[BPF_REG_0].parent_id = meta.ref_obj.id; + } } if (is_kfunc_ret_null(&meta)) { -- 2.55.0