Running a Private OCI Registry on NixOS with Zot
5 min readEvery container you push to Docker Hub is one docker pull rate limit away from ruining your Friday night deployment. You could pay for a hosted registry, but you already have perfectly good hardware sitting in a rack. What you need is something lightweight, OCI-native, and not beholden to anyone’s pricing page.
Zot fits the bill. It’s a single-binary, vendor-neutral OCI registry that speaks the distribution spec natively. No wrapping Docker’s registry in duct tape. No Java. It ships with a built-in UI, search, vulnerability scanning, and Prometheus metrics. And there’s a NixOS module that makes the whole thing declarative.
Adding the module
The Zot NixOS module lives in ijohanne/nur-packages. You can pull it in two ways.
Through the NUR overlay:
{
inputs.nur.url = "github:nix-community/NUR";
outputs = { self, nixpkgs, nur, ... }: {
nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
modules = [
nur.modules.nixos.repos.ijohanne.zot
];
};
};
}
Or as a direct flake input — useful if you don’t want the full NUR:
{
inputs.nur-packages.url = "github:ijohanne/nur-packages";
outputs = { self, nixpkgs, nur-packages, ... }: {
nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
modules = [
nur-packages.nixosModules.zot
];
};
};
}
Either way, you get the same module. Pick whichever matches how you already manage inputs.
Basic setup
The minimum viable registry is surprisingly short:
services.zot = {
enable = true;
dataDir = "/var/lib/zot";
user = "zot";
group = "zot";
};
That gets you Zot running on port 5000 with sane defaults — distribution spec 1.1.1, garbage collection on a 1-hour delay with a 6-hour interval, search, UI, scrub, metrics, and lint extensions all enabled. CVE database updates every 2 hours, scrub runs every 24 hours.
But a registry on localhost port 5000 isn’t useful to anyone. You want TLS, a real domain, and authentication. The module handles all of that.
Nginx reverse proxy
The module includes an nginx integration that does the right things:
services.zot.nginx = {
enable = true;
domain = "registry.example.com";
forceSSL = true;
acme = true;
acmeDns01 = false;
};
This generates an nginx virtual host with Let’s Encrypt certificates, sets client_max_body_size to 0 — because you don’t want nginx rejecting your 2 GB container images — disables proxy buffering, and enables chunked transfer encoding. It also blocks the /metrics endpoint with a 403, which matters later when you set up monitoring.
Users and authentication
Zot uses htpasswd for authentication. The module manages it declaratively, regenerating the htpasswd file on every service start. Passwords come from files, which makes this sops-nix friendly:
services.zot.auth.users = {
admin = {
passwordFile = config.sops.secrets.zot_admin_password.path;
admin = true;
};
ci-user = {
passwordFile = config.sops.secrets.zot_ci_password.path;
};
};
No plaintext passwords in your Nix config. The passwordFile attribute points to a file containing the raw password — sops-nix decrypts it at activation time, and the module reads it at service start. The admin = true flag gives the user elevated privileges in the access control system.
One thing to note: if metrics.enable is true — which it is by default — the module automatically creates a Prometheus scraping user. You don’t need to add one manually.
Access control
Authentication tells you who someone is. Access control tells you what they can do. The module supports fine-grained, per-repository policies using glob patterns:
services.zot.accessControl = {
adminActions = [ "read" "create" "update" "delete" ];
defaultPolicy = [];
anonymousPolicy = [];
repositories."myorg/**" = {
policies = [{
users = [ "ci-user" ];
actions = [ "read" "create" "update" "delete" ];
}];
defaultPolicy = [];
anonymousPolicy = [ "read" ];
};
};
This configuration locks things down. The defaultPolicy and anonymousPolicy at the top level are empty — no access unless explicitly granted. Admin users get full read/create/update/delete. Under myorg/**, ci-user gets full access and anonymous users can pull.
That glob pattern is doing the heavy lifting. Everything under myorg/ — myorg/frontend, myorg/api, myorg/worker — matches the same policy. You can add as many repository blocks as you need, each with its own set of users and permissions.
Retention policies
Registries accumulate images. Without retention policies, your disk fills up. The module makes garbage collection declarative:
services.zot.retention = {
dryRun = false;
delay = "24h";
policies = [
{
repositories = [ "myorg/**" ];
deleteReferrers = true;
deleteUntagged = true;
keepTags = [
{ patterns = [ "latest" ]; }
{ pushedWithin = "168h"; }
{ mostRecentlyPushedCount = 10; }
{ patterns = [ "v.*" ]; pulledWithin = "720h"; }
];
}
];
defaultPolicy = {
deleteReferrers = false;
deleteUntagged = true;
keepTags = [
{ patterns = [ ".*" ]; }
];
};
};
Read the keepTags list as a set of survival conditions. An image in myorg/** stays if it’s tagged latest, was pushed in the last 7 days, is one of the 10 most recently pushed, or matches a semver pattern and was pulled in the last 30 days. Everything else gets cleaned up after 24 hours.
The defaultPolicy at the bottom is the fallback — it keeps everything tagged and deletes untagged images. Conservative, but it stops the obvious leak.
Set dryRun = true first and check the logs before you let it actually delete anything.
Monitoring on a single server
If Prometheus and Grafana run on the same machine as your registry, the module does everything for you:
services.zot = {
metrics.enable = true;
metrics.user = "prometheus";
metrics.password = "prometheus";
enableLocalScraping = true;
grafanaDashboard = true;
};
enableLocalScraping adds a Prometheus scrape config pointed at Zot’s metrics endpoint on localhost. grafanaDashboard provisions a Grafana dashboard automatically. Metrics are enabled by default, so you technically only need the last two lines.
The auto-created Prometheus user authenticates against Zot’s htpasswd, so scraping goes through the same auth layer as everything else.
Monitoring from an external host
When Prometheus runs on a different machine, you configure the scrape config yourself:
services.prometheus.scrapeConfigs = [
{
job_name = "zot";
honor_labels = true;
metrics_path = "/metrics";
basic_auth = {
username = "prometheus";
password = "prometheus";
};
static_configs = [
{
targets = [ "10.255.101.200:5000" ];
labels = { instance = "registry"; };
}
];
}
];
The critical detail here — point your scrape target at Zot’s port directly, not at the nginx domain. The nginx config blocks /metrics with a 403. You want 10.255.101.200:5000, not registry.example.com:443. If your monitoring traffic crosses untrusted networks, put it behind a VPN or a separate authenticated tunnel.
The Grafana dashboard
The bundled dashboard is more thorough than you’d expect from a “batteries included” module. It covers download and upload counts with rates, per-repository metrics, per-prefix breakdowns, HTTP latency by method with heatmaps, scheduler queue length, and storage lock latency.
You get it for free with grafanaDashboard = true. No JSON to download, no manual import. It provisions itself through Grafana’s provisioning system.
Using the registry
Once everything is deployed, the workflow is standard Docker:
docker login registry.example.com -u ci-user
docker tag my-app:latest registry.example.com/myorg/my-app:latest
docker push registry.example.com/myorg/my-app:latest
Pulling works the same way:
docker pull registry.example.com/myorg/my-app:latest
Zot’s built-in web UI is available at https://registry.example.com. It lets you browse repositories, inspect tags, and view vulnerability scan results. Nothing to install — it ships with the binary.
Full example
Here’s a complete single-server configuration tying everything together — sops secrets, users, access control, retention, nginx with ACME, and local monitoring:
{ config, ... }:
{
sops.secrets = {
zot_admin_password = { };
zot_ci_password = { };
};
services.zot = {
enable = true;
dataDir = "/var/lib/zot";
# Nginx reverse proxy with Let's Encrypt
nginx = {
enable = true;
domain = "registry.example.com";
forceSSL = true;
acme = true;
};
# Declarative users — passwords from sops
auth.users = {
admin = {
passwordFile = config.sops.secrets.zot_admin_password.path;
admin = true;
};
ci-user = {
passwordFile = config.sops.secrets.zot_ci_password.path;
};
};
# Access control
accessControl = {
adminActions = [ "read" "create" "update" "delete" ];
defaultPolicy = [ ];
anonymousPolicy = [ ];
repositories."myorg/**" = {
policies = [
{
users = [ "ci-user" ];
actions = [ "read" "create" "update" "delete" ];
}
];
defaultPolicy = [ ];
anonymousPolicy = [ "read" ];
};
};
# Retention — keep things tidy
retention = {
dryRun = false;
delay = "24h";
policies = [
{
repositories = [ "myorg/**" ];
deleteReferrers = true;
deleteUntagged = true;
keepTags = [
{ patterns = [ "latest" ]; }
{ pushedWithin = "168h"; }
{ mostRecentlyPushedCount = 10; }
{ patterns = [ "v.*" ]; pulledWithin = "720h"; }
];
}
];
defaultPolicy = {
deleteReferrers = false;
deleteUntagged = true;
keepTags = [
{ patterns = [ ".*" ]; }
];
};
};
# Monitoring — local Prometheus and Grafana
metrics.enable = true;
enableLocalScraping = true;
grafanaDashboard = true;
};
}
The systemd service runs as a simple service type with PrivateTmp, ProtectHome, NoNewPrivileges, and a file descriptor limit of 500,000. The module handles all of that — you don’t need to think about it.
That’s a private OCI registry with authentication, per-repo access control, automatic image cleanup, TLS, and full observability. One file, no imperative setup, and nixos-rebuild switch gets you there.