Single backported AF_XDP fix on top of v0.8.1: #533

New features and APIs:

• major improvements for uprobe/uretprobe programs:
  • support auto-resolution of binaries and shared libraries from PATH, if necessary;
  • support attaching by function names (only by IP was supported before);
• support attaching to USDTs (SEC("usdt/...") and bpf_program__attach_usdt()) with initially supported architectures:
  • x86-64 (amd64);
  • x86 (i386);
  • s390x;
  • ARM64 (aarch64);
  • RISC V (riscv);
• improved BPF verifier log reporting for CO-RE relocation failures (no more obscure "invalid func unknown#195896080" errors);
• auto-adjust BPF ringbuf size according to host kernel's page size requirements;
• high-level BPF map APIs: bpf_map__lookup_elem(), bpf_map__update_elem(), etc that validate key/value buffer sizes;
• bpf_link_create() can create all bpf_link-based (including raw_tp, fentry/fexit, etc), falling back to bpf_raw_tracepoint_open() on old kernels transparently;
• support opting out from auto-loading BPF programs declaratively with SEC("?...");
• support opting out from auto-creation of declarative BPF maps with bpf_map__set_autocreate();
• support multi-kprobes (SEC("kprobe.multi/...") and bpf_program__attach_kprobe_multi_opts());
• support target-less SEC() programs (e.g., SEC("kprobe"), SEC("tp"), etc);
• support BPF sub-skeletons for "incomplete" BPF object files (requires matching bpftool to generate .subskel.h);
• BPF cookie support for fentry/fexit/fmod_ret BPF programs (bpf_program__attach_trace_opts());
• support for custom SEC() handlers (libbpf_register_prog_handler()).

BPF-side API

• BPF-side USDT APIs. See new usdt.bpf.h header:
  • BPF_USDT() program wrapper macro;
  • bpf_usdt_arg(), bpf_usdt_arg_cnt(), bpf_usdt_cookie() helpers;
• new bpf_core_field_offset() CO-RE helper and support bpf_core_field_size(type, field) forms;
• barrier() and barrier_var() macros for improving BPF code generation;
• __kptr and __kptr_ref tags added;
• ARC architecture support in bpf_tracing.h header;
• new BPF helpers:
  • bpf_skb_set_tstamp();
  • bpf_ima_file_hash();
  • bpf_kptr_xchg();
  • bpf_map_lookup_percpu_elem().

Bug fixes

• netlink bug fixes;
• libbpf.pc fixes to support patch releases properly;
• BPF_MAP_TYPE_PERF_EVENT_ARRAY map auto-pinning fix;
• minor CO-RE fixes and improvements for some corner cases;
• various other small fixes and improvements.

Full Changelog: v0.7.0...v0.8.0

Important updates towards Libbpf 1.0

• no need for explicit setrlimi(RLIMIT_MEMLOCK) when LIBBPF_STRICT_AUTO_RLIMIT_MEMLOCK is passed to libbpf_set_strict_mode(). libbpf will determine whether this is necessary automatically based on kernel's support for memcg-based memory accounting for BPF;
• legacy BPF map definitions (using struct bpf_map_def) are deprecated when LIBBPF_STRICT_MAP_DEFINITIONS is passed to libbpf_set_strict_mode(). Please use BTF-defined map definitions.
• another batch of API deprecations.

New features and APIs:

• ability to control and capture BPF verifier log output on per-object and per-program level;
• CO-RE support and other improvements for "light skeleton";
• further libbpf API documentation improvements;
• new feature-probing APIs (libbpf_probe_bpf_helper(), libbpf_probe_bpf_prog_type(), libbpf_probe_bpf_map_type());
• new streamlined low-level XDP APIs (bpf_xdp_attach(), bpf_xdp_detach(), bpf_xdp_query(), bpf_xdp_query_id());
• new SEC("xdp.frags"), SEC("xdp.frags/cpumap"), SEC("xdp.frags/devmap") section definitions;
• new SEC("iter.s/xxx") section definitions for sleepable BPF iterator programs.

BPF-side APIs and features:

• bpf_loop() helper;
• bpf_func_arg(), bpf_func_ret(), bpf_func_arg_cnt() helpers;
• added syscall-specific PT_REGS_xxx() macros for retrieving syscall arguments;
• added BPF_KPROBE_SYSCALL() helper macro for syscall kprobes;
• bpf_get_retval() and bpf_set_retval() helpers;
• bpf_xdp_get_buff_len() helper;
• bpf_copy_from_user_task() helper for sleepable BPF programs.

Bug fixes and compatibility improvements:

• fixed compilation error for C++ due to btf_dump__new() macro magic;
• improved LINUX_VERSION_CODE detection for Ubuntu;
• improved multiple kprobe support for legacy kprobe mode on old kernels;
• improved compilation when system BTF UAPI headers are outdated;
• a bunch of fixes of PT_REGS_PARMn*() macros for various architectures.

Full Changelog: v0.6.0...v0.7.0

This bugfix release contains two fixes on top of v0.6.0:

• btf__dedup_deprecated typo fix breaking symbol versioning of btf__dedup() when libbpf is used in shared library mode (7280542);
• small fix stopping libbpf from attempting to pin .rodata.* maps on old kernels, even if .rodata.* section is not used from BPF program code (56794b3).

Important updates towards Libbpf 1.0

• a first big batch of deprecated APIs; compiler will let you know or grep for "LIBBPF_DEPRECATED". Please also double-check https://github.com/libbpf/libbpf/wiki/Libbpf-1.0-migration-guide.
• documentation for a bunch of APIs added, available on https://libbpf.readthedocs.io/en/latest/api.html;
• libbpf version APIs added: compile-time LIBBPF_MAJOR_VERSION/LIBBPF_MINOR_VERSION and runtime libbpf_major_version()/libbpf_minor_version()/libbpf_version_string();
• stricter logic for SEC() definition handling (opt-in until libbpf v1.0); see https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#stricter-and-more-uniform-bpf-program-section-name-sec-handling for details.
• function name will be used when pinning if LIBBPF_STRICT_SEC_NAME strict mode flag is specified;

New features and APIs:

• support custom .rodata.* and .data.* data sections;
• bpf_program__attach_kprobe() and bpf_program__attach_uprobe() supports older kernels now (don't forget about bpf_link__destroy() when you are done!);
• BPF_MAP_TYPE_PROG_ARRAY can be initialized statically with syntax similar to map-in-map initialization (see 472c072 for an example);
• libbpf-less "light" skeleton gained new capabilities and got a bunch of fixes;
• BTF support for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG;
• new bpf_prog_load() and bpf_map_create() APIs supersede a whole zoo of to-be-deprecated APIs;
• support for writable raw tracepoints (SEC("raw_tp.w/...")) added;
• btf__add_btf() API for appending entire contents of BTF to another BTF object;
• bpf_program__insns() and bpf_program__insn_cnt() to access underlying BPF assembly instructions; can be used for inspection or BPF program cloning.
• a bunch of older APIs (perf_buffer__new(), btf__dedup(), btf_dump__new(), etc) were modernized to use OPTS infrastructure.

BPF-side APIs and features:

• unstable BPF helpers (kernel function calls) support for kernel modules;
• bpf_trace_vprintk() helper and corresponding bpf_printk() macro enhancements. Note, bpf_printk() will now attempt to use static global functions, so on very old kernels this might break existing programs. Please #define BPF_NO_GLOBAL_DATA before #include <bpf/bpf_helpers.h> if that's the case for you.
• bpf_get_branch_snapshot() helper;
• bpf_skc_to_unix_sock() helper;
• bpf_find_vma() helper;
• SEC("tc") added as a replacement for SEC("classifier").

Bug fixes and compatibility improvements:

• libbpf now guarantees that all FDs for BPF programs, maps, BTFs, and links are strictly greater than 0, which is important for some BPF UAPIs;
• no need to use __uint(key_size, ...) for special BPF maps (e.g., BPF_MAP_PERF_EVENT_ARRAY). Libbpf automatically downgrades __type(key, int) into key_size, if a map doesn't support BTF types for keys and values;
• endianness fixes in BPF_CORE_READ_BITFIELD_PROBED() macro;
• btf_dump__dump_type_data() improvements for handling unaligned data;
• various fixes and improvements found though fuzzing and sanitizers.

Full Changelog: v0.5.0...v0.6.0

New features and user-space APIs:

• libbpf_set_strict_mode() allowing to opt-in into backwards incompatible libbpf-1.0 changes. See "Libbpf: the road to 1.0" and "Libbpf 1.0 migration guide" for more details.
• streamlined error reporting for low-level APIs, high-level error-returning APIs, and pointer-returning APIs (as a libbpf-1.0 opt-in);
• "Light" BPF skeleton support;
• BPF_PROG_TYPE_SYSCALL support;
• BPF perf link support for kprobe, uprobe, tracepoint, and perf_event BPF programs;
• BPF cookie support for kprobe, uprobe, tracepoint, and perf_event BPF programs through bpf_program__attach_[ku]probe_opts() APIs;
• allow to specify ref_ctr_off for USDT semaphores through bpf_program__attach_uprobe_opts() API;
• btf_custom_path support in bpf_object_open_opts, allowing to specify custom BTF for CO-RE relocations;
• sk_reuseport/migrate program type support;
• btf_dump__dump_type_data() API, allowing to dump binary data according to BTF type description;
• btf__load_into_kernel() and btf__load_from_kernel_by_id(), and split BTF variants of them;
• btf__load_vmlinux_btf() and btf__load_module_btf() APIs;
• bpf_map__initial_value() API to get initial value of mmap-ed BPF maps;
• bpf_map_lookup_and_delete_elem_flags() API.

BPF-side APIs and features:

• support for weak typed __ksym externs;
• BPF timer helpers: bpf_timer_init(), bpf_timer_set_callback(), bpf_timer_start(), bpf_timer_cancel();
• bpf_get_attach_cookie() helper to get BPF cookie from BPF program side;
• bpf_get_func_ip() helper;
• bpf_sys_bpf() helper;
• bpf_task_pt_regs() helper;
• bpf_btf_find_by_name_kind() helper;
• usability improvements for bpf_tracing.h when target architecture is missing.

Bug fixes and compatibility improvements:

• improve BPF support detection on old Red Hat kernels with backported BPF patches;
• improvements for LTO builds with GCC 10+;
• pass NLM_F_EXCL when creating TC qdisc;
• better support of BPF map reuse on old kernels;
• fix the bug resulting in sometimes closing FD 0, which wasn't created and owned by libbpf itself.

New features/APIs:

• BPF static linker APIs;
• subprogram address relocation support (e.e., for use with bpf_for_each_map_elem());
• support for extern kernel functions (a.k.a. BPF unstable helpers);
• TC-BPF APIs;
• ksym externs support for kernel modules;
• BTF_KIND_FLOAT support;
• various AF_XDP (xsk.{c, h}) improvements and fixes;
• btf__add_type() API to copy/append BTF types generically;
• bpf_object__set_kernel_version() setter;
• bpf_map__inner_map() getter;
• __hidden attribute for global sub-program forces static BPF verifier verification;
• static BPF maps and entry-point BPF programs are explicitly rejected.

New BPF helpers:

• bpf_for_each_map_elem() helper;
• bpf_snprintf() helper and BPF_SNPRINTF() macro;
• bpf_check_mtu() helper;
• NULL and KERNEL_VERSION macros are provided by bpf_helpers.h;
• user-space BPF_CORE_READ_USER() macro variants;
• non-CO-RE BPF_PROBE_READ() and BPF_PROBE_READ_USER() macros.

Bug fixes:

• libbpf will ignore non-function pointer members in struct_ops;
• Makefile fixes for install target;
• use SOCK_CLOEXEC for netlink sockets;
• btf_dump fixes for pointer to array of struct;
• fixes for some of xxx_opts structs to work better with debug compilation modes;
• ringbuf APIs fixes and improvements for extreme cases of never ending consumption of records;
• BPF_CORE_READ_BITFIELD() macro fixes.

New features/APIs:

• kernel modules BTF support on all levels;
• ring_buffer__epoll_fd() API;
• xsk_socket__update_xskmap() and xsk_setup_xdp_prog() APIs.

New BPF helpers:

• bpf_task_storage_get() and bpf_task_storage_delete();
• get_current_task_btf();
• bpf_bprm_opts_set();
• bpf_ktime_get_coarse_ns();
• bpf_ima_inode_hash();
• bpf_sock_from_file().

Bug fixes:

• ring_buffer__poll() returns number of consumed records correctly;
• handle corner-case case with unused sub-programs;
• xsk_socket__delete() bug fixes.

New features/APIs:

• full support for BPF-to-BPF function calls, no more need for __always_inline;
• support for multiple BPF programs with the same section name;
• support for accessing in-kernel per-CPU variables;
• support for type and enum value CO-RE relocations;
• libbpf will auto-adjust CO-RE direct memory loads to adjust to 32-bit host architecture;
• BPF_PROG_BIND_MAP support, .rodata will be bound automatically if kernel supports it;
• new APIs for programmatic generation of BTF;
• support for big-endian and little-endian endianness in BTF;
• perf buffer API additions allowing better integration with polling libraries (perf_buffer__buffer_cnt(), perf_buffer__buffer_fd(), perf_buffer__consume_buffer());
• bpf_prog_test_run_opts() API;
• bpf_program__attach_freplace() API.

New BPF program types supported:

• sleepable fentry/fexit/fmod_ret/lsm BPF program.

BPF-side changes:

• libbpf will automatically fall back to bpf_probe_read[_str]() if bpf_probe_read_{kernel, user}[_str]() are not supported on older kernels, so it's always safe to use bpf_probe_read_{kernel, user}[_str]() in your application and not worry about kernel version compatibility;
• bpf_d_path() helper;
• bpf_per_cpu_ptr() and bpf_this_cpu_ptr() helpers for working with per-CPU kernel variables;
• bpf_copy_from_user() helper;
• bpf_load_hdr_opt(), bpf_store_hdr_opt(), bpf_reserve_hdr_opt() helpers;
• bpf_skb_cgroup_classid() helper;
• bpf_redirect_neigh() and bpf_redirect_peer() helpers;
• bpf_seq_printf_btf() and bpf_snprintf_btf() helpers;
• __noinline convenience macro;
• bpf_tail_call_static() wrapper.

Deprecations:

• BPF program's "title" terminology is deprecated, please use clearer "section_name". New API bpf_program__section_name() added;
• btf_ext__reloc_line_info() and btf_ext__reloc_func_info() (used by BCC) are deprecated.

This bug fix release contains a single bug fix (1107a4d)
which fixes an issue with loading XDP programs on older kernels.