Extension: ccache-remote — shared compilation cache via Redis/HTTP (#9369 )

* add hook to allow customizing before kernel make env creation
* Hook runs in docker_cli_prepare_launch() just before DOCKER_EXTRA_ARGS
is processed, allowing extensions to add Docker arguments with a more
descriptive hook name than add_host_dependencies.
* Extension: ccache-remote

Enables ccache with remote Redis storage for sharing compilation cache across build hosts.

Features:
- Auto-discovery via Avahi/mDNS (ccache.local hostname)
- Explicit Redis server configuration via CCACHE_REMOTE_STORAGE
- Build statistics display at end of build (hit/miss/error rates)
- Support for both Docker and native builds
- Hooks for kernel and u-boot compilation environments

Documentation includes server setup instructions with security warnings,
client mDNS configuration, and cache sharing requirements.


* uboot: fix ccache environment and add extension hook

U-Boot build uses `env -i` which clears all environment variables.
CCACHE_DIR and CCACHE_TEMPDIR were not explicitly passed to make,
unlike kernel build (kernel-make.sh). This caused ccache to use
default directory instead of configured Armbian one, breaking
cache statistics and shared cache functionality.

Changes:
- Add CCACHE_DIR and CCACHE_TEMPDIR to uboot_make_envs
- Add uboot_make_config hook for extensions (similar to kernel_make_config),
  allowing modification of environment variables before compilation

* add long list of allowed ccache-related env vars
* set permissions to ccache files RW for everyone if cache not private
* ccache: add ccache_post_compilation hook for extensions
* ccache-remote: use ccache_post_compilation hook instead of cleanup handler

Show remote ccache stats after each compilation (kernel, uboot) via hook,
instead of once at the end via cleanup handler. Stats now shown even on
build failure.

* ccache: show stats with safe arithmetic
* ccache/uboot: improve code comments per review feedback

- uboot.sh: clarify ARMBIAN=foe workaround for dual-compiler scenario
- ccache-remote.sh: document that CCACHE_REDIS_CONNECT_TIMEOUT must be
  set before extension loads

* ccache-remote: mask storage URLs in logs

Mask CCACHE_REMOTE_STORAGE when emitting Docker env debug logs.

* ccache-remote: extract ccache_inject_envs() helper to deduplicate passthrough loops

Extract ccache_inject_envs() helper to deduplicate identical passthrough
loops in kernel and uboot make config hooks.

ccache-remote: rename functions to follow project naming conventions

Rename get_redis_stats and mask_storage_url to ccache_get_redis_stats
and ccache_mask_storage_url to follow project naming conventions.

ccache-remote: mask credentials in debug log output for passthrough loops

Mask CCACHE_REMOTE_STORAGE value through ccache_mask_storage_url() before
logging in both Docker env and make env passthrough loops to avoid leaking
credentials into build logs.

* ccache-remote: add HTTP/WebDAV backend and DNS discovery
* ccache-remote: move extension script into directory layout
* ccache-remote: add server setup docs and config files
* ccache-remote: validate Redis credentials in URLs
* ccache-remote: document Redis auth options and safe passwords

Add separate insecure config example for trusted networks.

Recommend URL-safe hex passwords and update setup docs.

* ccache-remote: improve Docker loopback handling and IPv6 host parsing

2026-03-01 01:18:35 +01:00

4.7 KiB

Raw Permalink Blame History

ccache-remote: Server Setup Guide

Redis Server

Install packages:

apt install redis-server avahi-daemon avahi-utils

Configure Redis — merge the settings from misc/redis/redis-ccache.conf into /etc/redis/redis.conf, or add an include directive at the end of your existing config (include /etc/redis/redis-ccache.conf), then restart:
```
sudo systemctl restart redis-server
```
Authentication (recommended). Set a password in the config file — either via requirepass (Redis < 6) or ACL user entry (Redis 6+). See comments in misc/redis/redis-ccache.conf for both methods. Generate a URL-safe password:
```
openssl rand -hex 24
```
Important: Do not use openssl rand -base64 — base64 passwords contain /, +, and = which break URL parsing in redis:// connection strings. On the build host, pass the password in the Redis URL:
```
./compile.sh ENABLE_EXTENSIONS=ccache-remote \
  CCACHE_REMOTE_STORAGE="redis://default:YOUR_PASSWORD@192.168.1.65:6379" BOARD=...
```
No authentication (trusted network only). If all machines are on a fully isolated private network and access control is not needed, remove requirepass, set nopass in the ACL user entry, and set protected-mode no. See comments in misc/redis/redis-ccache.conf. No password is needed in the URL:
```
./compile.sh ENABLE_EXTENSIONS=ccache-remote \
  CCACHE_REMOTE_STORAGE="redis://192.168.1.65:6379" BOARD=...
```
For advanced security (TLS, ACL, rename-command), see: https://redis.io/docs/latest/operate/oss_and_stack/management/security/

Publish DNS-SD service — copy misc/avahi/ccache-redis.service to /etc/avahi/services/:

cp misc/avahi/ccache-redis.service /etc/avahi/services/

Avahi will pick it up automatically. Clients running avahi-browse -rpt _ccache._tcp will discover the Redis service.

Or use a systemd unit that ties the announcement to redis-server lifetime (stops advertising when Redis is down):

cp misc/systemd/ccache-avahi-redis.service /etc/systemd/system/
systemctl enable --now ccache-avahi-redis

Alternatively, publish legacy mDNS hostname:

avahi-publish-address -R ccache.local <SERVER_IP>

Or as a systemd service (/etc/systemd/system/ccache-hostname.service):

[Unit]
Description=Publish ccache.local hostname via Avahi
After=avahi-daemon.service redis-server.service
BindsTo=redis-server.service
[Service]
Type=simple
ExecStart=/usr/bin/avahi-publish-address -R ccache.local <SERVER_IP>
Restart=on-failure
[Install]
WantedBy=redis-server.service

HTTP/WebDAV Server (nginx)

Install nginx with WebDAV support:

apt install nginx-extras avahi-daemon avahi-utils

Copy misc/nginx/ccache-webdav.conf to /etc/nginx/sites-available/ccache-webdav, then enable and prepare storage:

cp misc/nginx/ccache-webdav.conf /etc/nginx/sites-available/ccache-webdav
ln -s /etc/nginx/sites-available/ccache-webdav /etc/nginx/sites-enabled/
mkdir -p /var/cache/ccache-webdav/ccache
chown -R www-data:www-data /var/cache/ccache-webdav
systemctl reload nginx

Verify:

curl -X PUT -d "test" http://localhost:8088/ccache/test.txt
curl http://localhost:8088/ccache/test.txt

WARNING: No authentication configured. Use ONLY in a fully trusted private network.

Publish DNS-SD service — copy misc/avahi/ccache-webdav.service to /etc/avahi/services/:

cp misc/avahi/ccache-webdav.service /etc/avahi/services/

Or use a systemd unit that ties the announcement to nginx lifetime:

cp misc/systemd/ccache-avahi-webdav.service /etc/systemd/system/
systemctl enable --now ccache-avahi-webdav

DNS SRV Records (for remote/hosted servers)

Set CCACHE_REMOTE_DOMAIN on the client, then create DNS records.

Redis backend:

_ccache._tcp.example.com.  SRV  0 0 6379 ccache.example.com.
_ccache._tcp.example.com.  TXT  "type=redis"

HTTP/WebDAV backend:

_ccache._tcp.example.com.  SRV  0 0 8088 ccache.example.com.
_ccache._tcp.example.com.  TXT  "type=http" "path=/ccache/"

Client Requirements for mDNS

Install one of the following for .local hostname resolution:

libnss-resolve (systemd-resolved):
```
apt install libnss-resolve
```
/etc/nsswitch.conf: hosts: files resolve [!UNAVAIL=return] dns myhostname
libnss-mdns (standalone):
```
apt install libnss-mdns
```
/etc/nsswitch.conf: hosts: files mdns4_minimal [NOTFOUND=return] dns myhostname

4.7 KiB Raw Permalink Blame History