Get started with eBPF using BumbleBee

What is eBPF?

eBPF stands for extended Berkeley Packet Filter. The Linux kernel has been around for a long time, however, it is not easy to modify or extend the kernel unless you know how to patch it. If you are familiar with Kubernetes’s custom resources or Envoy filters, you understand how important it is to build extensions based on your specific scenarios. What eBPF provides to the Linux kernel is the extensibility to enable developers to program the Linux kernel to quickly build intelligent or feature-rich functions based on their business needs. eBPF programs to the Linux kernel are similar to what web assembly modules are to Envoy. They allow developers to extend the kernel easily and run their eBPF code as sandboxed programs in the kernel without changing the kernel source code or loading kernel modules.

Options for building your eBPF programs

As a curious learner, I was excited to start working on my first eBPF program. Below are a few options that I explored while looking into building my first eBPF program.

BPF Compiler Collection (BCC)

BCC is a toolkit for creating efficient kernel tracing and manipulation eBPF programs, which requires Linux 4.1 and above. While BCC is designed to make BPF programs easier to write, it requires writing the kernel instrumentation in C wrapped as a plain string into your user-space program in Python or Lua. Don’t be surprised if you see both C code and python code in any eBPF python examples. If you are not a Linux user like me, it is hard to find an environment where you can compile and test your first eBPF program. I ended up using this Vagrantfile to set up my environment to build and run my hello world eBPF program.

Once you have gotten your first hello world eBPF program compiled and running, you may also want to know how portable it is. Can you take the compiled binary and run it on any kernel above 4.1? Or do you need to compile it for each kernel version that you want to run? When your eBPF program is deployed and executed, BCC invokes its embedded Clang/LLVM, pulls in local kernel headers (which you have to make sure are installed on the system from the correct kernel-devel package), and performs compilation of the plain string on the fly on the Linux kernel. While this method tailors your BPF code to the specific kernel, it can contain major drawbacks such as fat library distributed with your application, heavy resource utilization, requiring kernel headers, and more. But there are ways to address these challenges as well.

Enter BPF CO-RE and libbpf

BPF CO-RE (Compile Once Run Everywhere) is targeted to solve the above portability problem so you only need to compile once without needing to compile it across each particular kernel version. libbpf, an alternate set of tools for building BPF applications, is introduced as a BPF program loader, which knows how to tailor a BPF program code to a particular kernel on the host. It resolves and matches all the BTF (BPF Type Format) types and fields, updates necessary offsets and other relocatable data as needed to ensure that the BPF program’s logic is correctly functioning for a specific kernel on the host. If everything checks out, you will get a BPF program for the kernel on the target host as if your program was specifically compiled for it.

With libbpf, you will write plain C code along with helper marco to eliminate the mundane parts. What is really neat is that what you write is what gets executed and you no longer need to wrap your BPF code as a plain string in Python or Lua. This approach keeps overhead to the minimum, eliminates heavy dependencies, and makes BPF more practical.

Simplify BPF program with BumbleBee

In a nutshell, BumbleBee brings a Docker-like experience for eBPF to you. Through simple bee CLI commands, you can easily build, run, and distribute your eBPF programs as OCI (Open Container Initiative) images and plug the images to your existing OCI image workflows for publishing and distribution. BumbleBee is built using libbpf and allows you to focus on writing your eBPF code while taking care of the user space components automatically for you, e.g. you can write eBPF probes with zero user space code. BumbleBee automatically detects and displays maps in your program that allow the user space and kernel space programs to share data. This is accomplished through the use of special BPF conventions and keywords.

Get started with eBPF using BumbleBee

Let’s get started with creating, running, and distributing a simple hello world eBPF program using BumbleBee. Here, we’ve outlined how to get running in four steps.

1. Prepare your environment

eBPF itself is a Linux kernel technology, therefore any actual BPF programs must run in the Linux kernel. I recommend you to start with kernel 5.4 or newer. If you don’t have a Linux environment, you can use our vagrant file to quickly start a Linux VM:

git clone https://github.com/solo-io/bumblebee.git
cd bumblebee
vagrant up
vagrant ssh

From the SSH terminal, check the kernel version and verify you have CONFIG_DEBUG_INFO_BTF enabled:

uname -r
cat /boot/config-$(uname -r) | grep CONFIG_DEBUG_INFO_BTF

You’ll see output similar to this:

vagrant@impish:~$ uname -r
5.13.0-21-generic
vagrant@impish:~$ cat /boot/config-$(uname -r) | grep CONFIG_DEBUG_INFO_BTF
CONFIG_DEBUG_INFO_BTF=y
CONFIG_DEBUG_INFO_BTF_MODULES=y

Install the bee CLI:

curl -sL https://run.solo.io/bee/install | sh

Add the bee CLI to your path:

export PATH=$HOME/.bumblebee/bin:$PATH
2. Build your hello world eBPF program

Create your first hello.c eBPF program using bee init, accepting all the default options to keep it simple:

bee init

Upon successful creation of the hello.c program, you’ll see output similar as below:

INFO  Selected Language: C
INFO  Selected Program Type: Network
INFO  Selected Map Type: RingBuffer
INFO  Selected Output Type: print
INFO  Selected Output Type: BPF Program File Location hello.c

Open up hello.c using vi (or your preferred editor) to edit the auto-generated user space code. This code won’t display any useful data because there is no data in the map shared among kernel space and user space. Let us replace “// 2. Add ringbuf struct data here.” with the following so we can add pid and message to the RingBuffer map:

    u32 pid;
    char message[15];

Uncomment the following line so that we can use the the bpf helper function bpf_get_current_pid_tgid() to get the current pid:

// event->pid = bpf_get_current_pid_tgid();

Also add the following below to set the message in the event to be “Hello World!”, before submitting the event to the RingBuffer map using bpf_ringbuf_submit(event, 0):

memcpy(event->message, "Hello World!", 15);

The hello.c program should look like this. Save your change, and build the program using bee build, to create an OCI image called hello:v1:

bee build hello.c hello:v1
3. Run your hello world eBPF program

List all the images using bee list:

bee list

Verify the hello:v1 image is in the output:

vagrant@impish:~$ bee list
Name                                       | OS      | OS Version        | Arch
hello:v1                                   | Linux   | 5.13.0-21-generic | x86_64

Loading eBPF programs to the kernel (e.g. the bee run command) requires elevated privileges. Run the following command to add the required capabilities:

sudo setcap cap_sys_resource,cap_sys_admin,cap_bpf+eip $(which bee)

Run the image using bee run:

bee run hello:v1

You will not see any event initially. Curl a website from a terminal of your Linux VM to generate an event:

curl solo.io

Switch back to view the bee run UI, and you should see the “Hello World!” message along with the pid number for the curl command:

Congratulations! You just built and ran your hello world eBPF images with BumbleBee.

4. Share your hello world eBPF image

Ready to share your hello world eBPF image to an OCI compliant registry? Using bee tag and bee push commands, you can push your image to popular OCI image repositories such as Docker registry, GitHub Container Repository, or Google Container Repository, etc.

Start a test Docker registry on your local machine:

docker run --rm -p 5000:5000 registry:2

From another terminal, tag and push the hello:v1 image to the local Docker registry:

bee tag hello:v1 localhost:5000/hello:v1
bee push localhost:5000/hello:v1

You’ll get an output that indicates the image is successfully pushed to the registry. You may also use a prebuilt image from someone you trust using the bee run command:

bee run ghcr.io/solo-io/bumblebee/tcpconnect:0.0.8

Refer to our instructions for more details regarding how to collaborate with others on eBPF images.

Wrapping up on eBPF and BumbleBee

Through these simple bee init, build, run, list, and push commands, we are excited to bring the Docker-like experience to eBPF so that developers can not only easily build eBPF programs but also collaborate and share their eBPF programs with others through their favorite OCI image repositories.

To get involved with eBPF and BumbleBee, check out the open source BumbleBee repository, join the discussion in the community slack or register for an upcoming event.