Boosting Deployment Safety at GitHub with eBPF

GitHub runs its own infrastructure, including the very platform that hosts its source code. This creates a tricky situation: what if GitHub goes down while you need to deploy a fix? You can’t access the code to fix it. To avoid this, GitHub uses eBPF (extended Berkeley Packet Filter) to monitor and block deployment scripts from creating circular dependencies. In this Q&A, we explore how they do it and what you can learn from their approach.

What is the circular dependency problem GitHub faces?

GitHub hosts all its source code on github.com. If the site goes down, engineers cannot access the code needed to deploy a fix. This is a classic circular dependency: deploying GitHub requires GitHub to be working. To mitigate this, GitHub maintains a mirror of its code for emergency fixes and built assets for rollbacks. However, even with these safeguards, deployment scripts can introduce new circular dependencies—for example, by fetching a binary from GitHub or calling an internal service that itself depends on GitHub. The challenge is to prevent these scripts from creating dependencies that would block recovery during an outage.

What types of circular dependencies can occur?

GitHub identifies three main types. Direct dependencies happen when a deployment script tries to pull a tool or release from GitHub during an outage—if GitHub is down, the script fails. Hidden dependencies occur when a script uses a local tool that, during execution, checks for updates on GitHub. If GitHub is unreachable, the tool may hang or error out. Transient dependencies involve a script calling an internal API (like a migration service), which then tries to fetch a binary from GitHub, propagating the failure back. Each type can silently block a fix when time is critical.

How did GitHub traditionally handle these dependencies?

Previously, the responsibility fell on individual teams owning stateful hosts to review their deployment scripts and identify any circular dependencies. This manual process was error-prone and often missed hidden or transient dependencies. There was no systematic way to enforce that scripts avoided contacting GitHub during an outage. As the platform grew, the risk increased—new scripts could unknowingly introduce dependencies. GitHub needed a proactive approach to catch these issues before they caused problems in an incident.

Why did GitHub choose eBPF to solve this?

eBPF allows safe, efficient monitoring and filtering of system calls without modifying kernel code or restarting services. GitHub realized they could use eBPF to intercept network calls made by deployment scripts and block any that try to access GitHub or other internal services that might create a circular dependency. This gives them fine-grained control—they can allow safe traffic while blocking risky requests. eBPF runs with minimal overhead, making it suitable for production environments. It also lets them add or update rules dynamically, adapting to new dependency patterns.

How does eBPF monitor and block circular dependencies?

GitHub writes eBPF programs that hook into system calls like connect() or sendto(). When a deployment script tries to open a network connection, the eBPF program checks the destination IP or domain against a list of allowed endpoints. If the destination is GitHub or a known internal service that could cause a circular dependency, the program blocks the call and logs the event. The script receives an error, so it fails fast rather than hanging or waiting for a timeout. This gives immediate feedback to the operator, who can then adjust the script. The eBPF programs are loaded at the start of a deployment and unloaded after, keeping the attack surface minimal.

What benefits has eBPF brought to GitHub’s deployment safety?

eBPF provides a centralized, automated way to enforce deployment safety across all hosts. It eliminates the need for manual script reviews and catches hidden dependencies that humans might overlook. The system blocks calls in real time, preventing failures from cascading during an outage. Additionally, the logging from eBPF helps teams understand dependency patterns and fix scripts permanently. Since eBPF operates at the kernel level, it can’t be bypassed by a script’s logic. This has increased GitHub’s confidence that they can deploy fixes even when their primary infrastructure is unavailable, breaking the circular dependency chain.

How can others get started writing eBPF programs for similar use cases?

Start by learning the basics of eBPF through tools like bcc or bpftrace. These provide high-level languages (Python, C) to write eBPF programs without deep kernel knowledge. Focus on hooking into network-related syscalls. Define your “safe” network destinations—anything outside that list should be blocked. Test in a sandbox environment first. GitHub also recommends using cilium/ebpf for Go programs if you need more control. Remember that eBPF is powerful but requires careful security review: ensure your programs don’t open new vulnerabilities. Start small, monitor logs, and iterate.