Extension: ccache-remote — shared compilation cache via Redis/HTTP (#9369)

* add hook to allow customizing before kernel make env creation
* Hook runs in docker_cli_prepare_launch() just before DOCKER_EXTRA_ARGS
is processed, allowing extensions to add Docker arguments with a more
descriptive hook name than add_host_dependencies.
* Extension: ccache-remote

Enables ccache with remote Redis storage for sharing compilation cache across build hosts.

Features:
- Auto-discovery via Avahi/mDNS (ccache.local hostname)
- Explicit Redis server configuration via CCACHE_REMOTE_STORAGE
- Build statistics display at end of build (hit/miss/error rates)
- Support for both Docker and native builds
- Hooks for kernel and u-boot compilation environments

Documentation includes server setup instructions with security warnings,
client mDNS configuration, and cache sharing requirements.


* uboot: fix ccache environment and add extension hook

U-Boot build uses `env -i` which clears all environment variables.
CCACHE_DIR and CCACHE_TEMPDIR were not explicitly passed to make,
unlike kernel build (kernel-make.sh). This caused ccache to use
default directory instead of configured Armbian one, breaking
cache statistics and shared cache functionality.

Changes:
- Add CCACHE_DIR and CCACHE_TEMPDIR to uboot_make_envs
- Add uboot_make_config hook for extensions (similar to kernel_make_config),
  allowing modification of environment variables before compilation

* add long list of allowed ccache-related env vars
* set permissions to ccache files RW for everyone if cache not private
* ccache: add ccache_post_compilation hook for extensions
* ccache-remote: use ccache_post_compilation hook instead of cleanup handler

Show remote ccache stats after each compilation (kernel, uboot) via hook,
instead of once at the end via cleanup handler. Stats now shown even on
build failure.

* ccache: show stats with safe arithmetic
* ccache/uboot: improve code comments per review feedback

- uboot.sh: clarify ARMBIAN=foe workaround for dual-compiler scenario
- ccache-remote.sh: document that CCACHE_REDIS_CONNECT_TIMEOUT must be
  set before extension loads

* ccache-remote: mask storage URLs in logs

Mask CCACHE_REMOTE_STORAGE when emitting Docker env debug logs.

* ccache-remote: extract ccache_inject_envs() helper to deduplicate passthrough loops

Extract ccache_inject_envs() helper to deduplicate identical passthrough
loops in kernel and uboot make config hooks.

ccache-remote: rename functions to follow project naming conventions

Rename get_redis_stats and mask_storage_url to ccache_get_redis_stats
and ccache_mask_storage_url to follow project naming conventions.

ccache-remote: mask credentials in debug log output for passthrough loops

Mask CCACHE_REMOTE_STORAGE value through ccache_mask_storage_url() before
logging in both Docker env and make env passthrough loops to avoid leaking
credentials into build logs.

* ccache-remote: add HTTP/WebDAV backend and DNS discovery
* ccache-remote: move extension script into directory layout
* ccache-remote: add server setup docs and config files
* ccache-remote: validate Redis credentials in URLs
* ccache-remote: document Redis auth options and safe passwords

Add separate insecure config example for trusted networks.

Recommend URL-safe hex passwords and update setup docs.

* ccache-remote: improve Docker loopback handling and IPv6 host parsing
This commit is contained in:
Igor Velkov 2026-03-01 02:18:35 +02:00 committed by GitHub
parent 3f22500237
commit 1b74748622
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
14 changed files with 863 additions and 5 deletions

View File

@ -0,0 +1,142 @@
# ccache-remote: Server Setup Guide
## Redis Server
1. Install packages:
```bash
apt install redis-server avahi-daemon avahi-utils
```
2. Configure Redis — merge the settings from `misc/redis/redis-ccache.conf` into
`/etc/redis/redis.conf`, or add an `include` directive at the end of your
existing config (`include /etc/redis/redis-ccache.conf`), then restart:
```bash
sudo systemctl restart redis-server
```
**Authentication (recommended).** Set a password in the config file — either via
`requirepass` (Redis < 6) or ACL user entry (Redis 6+). See comments in
`misc/redis/redis-ccache.conf` for both methods. Generate a URL-safe password:
```bash
openssl rand -hex 24
```
**Important:** Do not use `openssl rand -base64` — base64 passwords contain
`/`, `+`, and `=` which break URL parsing in `redis://` connection strings.
On the build host, pass the password in the Redis URL:
```bash
./compile.sh ENABLE_EXTENSIONS=ccache-remote \
CCACHE_REMOTE_STORAGE="redis://default:YOUR_PASSWORD@192.168.1.65:6379" BOARD=...
```
**No authentication (trusted network only).** If all machines are on a fully
isolated private network and access control is not needed, remove `requirepass`,
set `nopass` in the ACL user entry, and set `protected-mode no`. See comments
in `misc/redis/redis-ccache.conf`. No password is needed in the URL:
```bash
./compile.sh ENABLE_EXTENSIONS=ccache-remote \
CCACHE_REMOTE_STORAGE="redis://192.168.1.65:6379" BOARD=...
```
For advanced security (TLS, ACL, rename-command), see:
https://redis.io/docs/latest/operate/oss_and_stack/management/security/
3. Publish DNS-SD service — copy `misc/avahi/ccache-redis.service` to `/etc/avahi/services/`:
```bash
cp misc/avahi/ccache-redis.service /etc/avahi/services/
```
Avahi will pick it up automatically. Clients running `avahi-browse -rpt _ccache._tcp`
will discover the Redis service.
Or use a systemd unit that ties the announcement to `redis-server` lifetime
(stops advertising when Redis is down):
```bash
cp misc/systemd/ccache-avahi-redis.service /etc/systemd/system/
systemctl enable --now ccache-avahi-redis
```
Alternatively, publish legacy mDNS hostname:
```bash
avahi-publish-address -R ccache.local <SERVER_IP>
```
Or as a systemd service (`/etc/systemd/system/ccache-hostname.service`):
```ini
[Unit]
Description=Publish ccache.local hostname via Avahi
After=avahi-daemon.service redis-server.service
BindsTo=redis-server.service
[Service]
Type=simple
ExecStart=/usr/bin/avahi-publish-address -R ccache.local <SERVER_IP>
Restart=on-failure
[Install]
WantedBy=redis-server.service
```
## HTTP/WebDAV Server (nginx)
1. Install nginx with WebDAV support:
```bash
apt install nginx-extras avahi-daemon avahi-utils
```
2. Copy `misc/nginx/ccache-webdav.conf` to `/etc/nginx/sites-available/ccache-webdav`,
then enable and prepare storage:
```bash
cp misc/nginx/ccache-webdav.conf /etc/nginx/sites-available/ccache-webdav
ln -s /etc/nginx/sites-available/ccache-webdav /etc/nginx/sites-enabled/
mkdir -p /var/cache/ccache-webdav/ccache
chown -R www-data:www-data /var/cache/ccache-webdav
systemctl reload nginx
```
3. Verify:
```bash
curl -X PUT -d "test" http://localhost:8088/ccache/test.txt
curl http://localhost:8088/ccache/test.txt
```
**WARNING:** No authentication configured.
Use ONLY in a fully trusted private network.
4. Publish DNS-SD service — copy `misc/avahi/ccache-webdav.service` to `/etc/avahi/services/`:
```bash
cp misc/avahi/ccache-webdav.service /etc/avahi/services/
```
Or use a systemd unit that ties the announcement to `nginx` lifetime:
```bash
cp misc/systemd/ccache-avahi-webdav.service /etc/systemd/system/
systemctl enable --now ccache-avahi-webdav
```
## DNS SRV Records (for remote/hosted servers)
Set `CCACHE_REMOTE_DOMAIN` on the client, then create DNS records.
Redis backend:
```text
_ccache._tcp.example.com. SRV 0 0 6379 ccache.example.com.
_ccache._tcp.example.com. TXT "type=redis"
```
HTTP/WebDAV backend:
```text
_ccache._tcp.example.com. SRV 0 0 8088 ccache.example.com.
_ccache._tcp.example.com. TXT "type=http" "path=/ccache/"
```
## Client Requirements for mDNS
Install one of the following for `.local` hostname resolution:
- **libnss-resolve** (systemd-resolved):
```bash
apt install libnss-resolve
```
`/etc/nsswitch.conf`: `hosts: files resolve [!UNAVAIL=return] dns myhostname`
- **libnss-mdns** (standalone):
```bash
apt install libnss-mdns
```
`/etc/nsswitch.conf`: `hosts: files mdns4_minimal [NOTFOUND=return] dns myhostname`

View File

@ -0,0 +1,492 @@
# Extension: ccache-remote
# Enables ccache with remote storage for sharing compilation cache across build hosts.
# Supports Redis and HTTP/WebDAV backends (ccache 4.4+).
#
# Documentation:
# Redis: https://ccache.dev/howto/redis-storage.html
# HTTP: https://ccache.dev/howto/http-storage.html
# General: https://ccache.dev/manual/4.10.html#config_remote_storage
#
# Usage:
# # With explicit Redis server:
# ./compile.sh ENABLE_EXTENSIONS=ccache-remote CCACHE_REMOTE_STORAGE="redis://192.168.1.65:6379" BOARD=...
#
# # With HTTP/WebDAV server:
# ./compile.sh ENABLE_EXTENSIONS=ccache-remote CCACHE_REMOTE_STORAGE="http://192.168.1.65:8088/ccache/" BOARD=...
#
# # Auto-discovery via DNS-SD (no URL needed, discovers type/host/port):
# ./compile.sh ENABLE_EXTENSIONS=ccache-remote BOARD=...
#
# # DNS SRV discovery for remote build servers:
# ./compile.sh ENABLE_EXTENSIONS=ccache-remote CCACHE_REMOTE_DOMAIN="example.com" BOARD=...
#
# # Disable local cache, use remote only (saves local disk space):
# ./compile.sh ENABLE_EXTENSIONS=ccache-remote CCACHE_REMOTE_ONLY=yes BOARD=...
#
# Automatically sets USE_CCACHE=yes
#
# Supported ccache environment variables (passed through to builds):
# See: https://ccache.dev/manual/latest.html#_configuration_options
# CCACHE_BASEDIR - base directory for path normalization (enables cache sharing)
# CCACHE_REMOTE_STORAGE - remote storage URL (redis://... or http://...)
# CCACHE_REMOTE_DOMAIN - domain for DNS SRV discovery (e.g., "example.com")
# CCACHE_REMOTE_ONLY - use only remote storage, disable local cache
# CCACHE_READONLY - read-only mode, don't update cache
# CCACHE_RECACHE - don't use cached results, but update cache
# CCACHE_RESHARE - rewrite cache entries to remote storage
# CCACHE_DISABLE - disable ccache completely
# CCACHE_MAXSIZE - maximum cache size (e.g., "10G")
# CCACHE_MAXFILES - maximum number of files in cache
# CCACHE_NAMESPACE - cache namespace for isolation
# CCACHE_SLOPPINESS - comma-separated list of sloppiness options
# CCACHE_UMASK - umask for cache files
# CCACHE_LOGFILE - path to log file
# CCACHE_DEBUGLEVEL - debug level (1-2)
# CCACHE_STATSLOG - path to stats log file
# CCACHE_PCH_EXTSUM - include PCH extension in hash
#
# CCACHE_REMOTE_STORAGE format (ccache 4.4+):
# Redis: redis://[[USERNAME:]PASSWORD@]HOST[:PORT][|attribute=value...]
# HTTP: http://HOST[:PORT]/PATH/[|attribute=value...]
# Common attributes:
# connect-timeout=N - connection timeout in milliseconds (default: 100)
# operation-timeout=N - operation timeout in milliseconds (default: 10000)
# Examples:
# "redis://default:secretpass@192.168.1.65:6379|connect-timeout=500"
# "redis://192.168.1.65:6379|connect-timeout=500"
# "http://192.168.1.65:8088/ccache/"
#
# Auto-discovery (priority order):
# 1. Explicit CCACHE_REMOTE_STORAGE - used as-is, no discovery
# 2. DNS-SD browse for _ccache._tcp on local network (avahi-browse)
# 3. DNS SRV record _ccache._tcp.DOMAIN (when CCACHE_REMOTE_DOMAIN is set)
# 4. Legacy mDNS: resolve 'ccache.local' hostname (fallback)
#
# When multiple services are found, Redis is preferred over HTTP.
#
# DNS-SD service publication (on cache server):
# # For HTTP/WebDAV:
# avahi-publish-service "ccache-webdav" _ccache._tcp 8088 type=http path=/ccache/
# # For Redis:
# avahi-publish-service "ccache-redis" _ccache._tcp 6379 type=redis
#
# DNS SRV record (for remote/hosted build servers):
# Set CCACHE_REMOTE_DOMAIN to your domain, then create DNS records:
# _ccache._tcp.example.com. SRV 0 0 8088 ccache.example.com.
# _ccache._tcp.example.com. TXT "type=http" "path=/ccache/"
# The cache server must be reachable from the build host (e.g., via port forwarding).
#
# Legacy mDNS (backward compatible):
# Publish 'ccache.local' hostname via Avahi:
# avahi-publish-address -R ccache.local <SERVER_IP>
# Or create a systemd service (see below).
#
# Server setup: see README.server-setup.md and config files in misc/
# - misc/redis/redis-ccache.conf — Redis configuration example
# - misc/nginx/ccache-webdav.conf — nginx WebDAV configuration example
# - misc/avahi/ccache-*.service — Avahi DNS-SD service files (static, always announce)
# - misc/systemd/ccache-avahi-*.service — systemd units (announce only while service runs)
#
# Fallback behavior:
# If CCACHE_REMOTE_STORAGE is not set and ccache.local is not resolvable,
# extension silently falls back to local ccache only.
#
# Cache sharing requirements:
# For cache to be shared across multiple build hosts, the Armbian project
# path must be identical on all machines (e.g., /home/build/armbian).
# This is because ccache includes the working directory in the cache key.
# Docker builds automatically use consistent paths (/armbian/...).
# Default Redis connection timeout in milliseconds (can be overridden by user)
# Note: Must be set before extension loads (e.g., via environment or command line)
declare -g -r CCACHE_REDIS_CONNECT_TIMEOUT="${CCACHE_REDIS_CONNECT_TIMEOUT:-500}"
# List of ccache environment variables to pass through to builds
declare -g -a CCACHE_PASSTHROUGH_VARS=(
CCACHE_REDIS_CONNECT_TIMEOUT
CCACHE_REMOTE_DOMAIN
CCACHE_BASEDIR
CCACHE_REMOTE_STORAGE
CCACHE_REMOTE_ONLY
CCACHE_READONLY
CCACHE_RECACHE
CCACHE_RESHARE
CCACHE_DISABLE
CCACHE_MAXSIZE
CCACHE_MAXFILES
CCACHE_NAMESPACE
CCACHE_SLOPPINESS
CCACHE_UMASK
CCACHE_LOGFILE
CCACHE_DEBUGLEVEL
CCACHE_STATSLOG
CCACHE_PCH_EXTSUM
)
# Format host:port, wrapping IPv6 addresses in brackets for URL compatibility (RFC 2732)
function ccache_format_host_port() {
local host="$1" port="$2"
if [[ "${host}" == *:* ]]; then
echo "[${host}]:${port}"
else
echo "${host}:${port}"
fi
}
# Extract hostname from CCACHE_REMOTE_STORAGE URL (strips scheme, userinfo, port, path)
function ccache_extract_url_host() {
local url="$1"
local after_scheme="${url#*://}"
# Strip userinfo if present
if [[ "${after_scheme}" == *@* ]]; then
after_scheme="${after_scheme##*@}"
fi
local host
# Handle bracketed IPv6: [addr]:port
if [[ "${after_scheme}" == \[* ]]; then
host="${after_scheme#\[}"
host="${host%%\]*}"
else
# Strip port, path, and ccache attributes
host="${after_scheme%%[:\/|]*}"
fi
echo "${host}"
}
# Discover ccache remote storage via DNS-SD (mDNS/Avahi) or DNS SRV records.
# Looks for _ccache._tcp services with TXT records: type=http|redis, path=/...
# Prefers Redis over HTTP when multiple services are found.
# Sets CCACHE_REMOTE_STORAGE on success, returns 1 if nothing found.
function ccache_discover_remote_storage() {
# Method 1: DNS-SD browse on local network (requires avahi-browse)
if command -v avahi-browse &>/dev/null; then
local browse_output
browse_output=$(timeout 5 avahi-browse -rpt _ccache._tcp 2>/dev/null || true)
if [[ -n "${browse_output}" ]]; then
# Parse resolved lines: =;IFACE;PROTO;NAME;TYPE;DOMAIN;HOSTNAME;ADDRESS;PORT;"txt"...
# Prefer IPv4 (proto=IPv4), prefer type=redis over type=http
local redis_url="" http_url=""
local redis_host="" redis_host_ip="" http_host="" http_host_ip=""
while IFS=';' read -r status iface proto name stype domain hostname address port txt_rest; do
[[ "${status}" == "=" && "${proto}" == "IPv4" ]] || continue
local svc_type="" svc_path=""
# Parse TXT records from remaining fields
if [[ "${txt_rest}" =~ \"type=([a-z]+)\" ]]; then
svc_type="${BASH_REMATCH[1]}"
fi
if [[ "${txt_rest}" =~ \"path=([^\"]+)\" ]]; then
svc_path="${BASH_REMATCH[1]}"
fi
# Use hostname for URL (Docker --add-host resolves it), fall back to address
local svc_host="${hostname%.local}"
svc_host="${svc_host%.}"
[[ -z "${svc_host}" ]] && svc_host="${address}"
if [[ "${svc_type}" == "redis" ]]; then
redis_url="redis://${svc_host}:${port}|connect-timeout=${CCACHE_REDIS_CONNECT_TIMEOUT}"
redis_host="${svc_host}"
redis_host_ip="${address}"
elif [[ "${svc_type}" == "http" ]]; then
http_url="http://${svc_host}:${port}${svc_path}"
http_host="${svc_host}"
http_host_ip="${address}"
fi
done <<< "${browse_output}"
# Redis preferred over HTTP; set hostname->IP mapping for Docker --add-host
if [[ -n "${redis_url}" ]]; then
export CCACHE_REMOTE_STORAGE="${redis_url}"
declare -g CCACHE_REMOTE_HOST="${redis_host}"
declare -g CCACHE_REMOTE_HOST_IP="${redis_host_ip}"
display_alert "DNS-SD: discovered Redis ccache" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info"
return 0
elif [[ -n "${http_url}" ]]; then
export CCACHE_REMOTE_STORAGE="${http_url}"
declare -g CCACHE_REMOTE_HOST="${http_host}"
declare -g CCACHE_REMOTE_HOST_IP="${http_host_ip}"
display_alert "DNS-SD: discovered HTTP ccache" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info"
return 0
fi
fi
fi
# Method 2: DNS SRV record for remote setups (CCACHE_REMOTE_DOMAIN must be set)
if [[ -n "${CCACHE_REMOTE_DOMAIN}" ]] && command -v dig &>/dev/null; then
local srv_output
srv_output=$(dig +short SRV "_ccache._tcp.${CCACHE_REMOTE_DOMAIN}" 2>/dev/null || true)
if [[ -n "${srv_output}" ]]; then
local srv_port srv_host
# SRV format: priority weight port target
read -r _ _ srv_port srv_host <<< "${srv_output}"
srv_host="${srv_host%.}" # strip trailing dot
if [[ -n "${srv_host}" && -n "${srv_port}" ]]; then
# Check TXT record for service type and path
local txt_output svc_type="redis" svc_path=""
txt_output=$(dig +short TXT "_ccache._tcp.${CCACHE_REMOTE_DOMAIN}" 2>/dev/null || true)
if [[ "${txt_output}" =~ type=([a-z]+) ]]; then
svc_type="${BASH_REMATCH[1]}"
fi
if [[ "${txt_output}" =~ path=([^\"[:space:]]+) ]]; then
svc_path="${BASH_REMATCH[1]}"
fi
local host_port
host_port=$(ccache_format_host_port "${srv_host}" "${srv_port}")
if [[ "${svc_type}" == "http" ]]; then
export CCACHE_REMOTE_STORAGE="http://${host_port}${svc_path}"
else
export CCACHE_REMOTE_STORAGE="redis://${host_port}|connect-timeout=${CCACHE_REDIS_CONNECT_TIMEOUT}"
fi
display_alert "DNS SRV: discovered ccache" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info"
return 0
fi
fi
fi
# Method 3: Legacy fallback - resolve ccache.local hostname
local ccache_ip
ccache_ip=$(getent hosts ccache.local 2>/dev/null | awk '{print $1; exit}' || true)
if [[ -n "${ccache_ip}" ]]; then
local host_port
host_port=$(ccache_format_host_port "${ccache_ip}" "6379")
export CCACHE_REMOTE_STORAGE="redis://${host_port}|connect-timeout=${CCACHE_REDIS_CONNECT_TIMEOUT}"
display_alert "mDNS: discovered ccache" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info"
return 0
fi
return 1
}
# Query Redis stats (keys count and memory usage)
function ccache_get_redis_stats() {
local ip="$1"
local port="${2:-6379}"
local password="$3"
local stats=""
if command -v redis-cli &>/dev/null; then
local auth_args=()
[[ -n "${password}" ]] && auth_args+=(-a "${password}" --no-auth-warning)
local keys mem
keys=$(timeout 2 redis-cli -h "$ip" -p "$port" "${auth_args[@]}" DBSIZE 2>/dev/null | grep -oE '[0-9]+' || true)
mem=$(timeout 2 redis-cli -h "$ip" -p "$port" "${auth_args[@]}" INFO memory 2>/dev/null | grep "used_memory_human" | cut -d: -f2 | tr -d '[:space:]' || true)
if [[ -n "$keys" ]]; then
stats="keys=${keys:-0}, mem=${mem:-?}"
fi
else
# Fallback: try netcat for basic connectivity check
if nc -z -w 2 "$ip" "$port" 2>/dev/null; then
stats="reachable (redis-cli not installed for detailed stats)"
fi
fi
echo "$stats"
}
# Check HTTP/WebDAV storage reachability via HEAD request
function ccache_get_http_stats() {
local url="$1"
local stats=""
local http_code
http_code=$(timeout 3 curl -s -o /dev/null -w "%{http_code}" -X HEAD "${url}" 2>/dev/null || true)
if [[ -n "${http_code}" && "${http_code}" != "000" ]]; then
stats="reachable (HTTP ${http_code})"
fi
echo "$stats"
}
# Query remote storage stats based on URL scheme (redis:// or http://)
# Parses userinfo (user:pass@) from Redis URLs to pass credentials to redis-cli
function ccache_get_remote_stats() {
local url="$1"
if [[ "${url}" =~ ^redis:// ]]; then
local password="" host="" port="6379"
# Strip scheme and attributes
local authority="${url#redis://}"
authority="${authority%%|*}"
# Extract password from userinfo (before last @)
if [[ "${authority}" =~ ^(.+)@(.+)$ ]]; then
local userinfo="${BASH_REMATCH[1]}"
authority="${BASH_REMATCH[2]}"
# password is after : in userinfo (user:pass or just :pass)
[[ "${userinfo}" == *:* ]] && password="${userinfo#*:}"
fi
# Parse host:port (IPv6 in brackets or plain)
if [[ "${authority}" =~ ^\[([^]]+)\]:?([0-9]*) ]]; then
host="${BASH_REMATCH[1]}"
[[ -n "${BASH_REMATCH[2]}" ]] && port="${BASH_REMATCH[2]}"
elif [[ "${authority}" =~ ^([^:]+):?([0-9]*) ]]; then
host="${BASH_REMATCH[1]}"
[[ -n "${BASH_REMATCH[2]}" ]] && port="${BASH_REMATCH[2]}"
fi
[[ -n "${host}" ]] && ccache_get_redis_stats "${host}" "${port}" "${password}"
elif [[ "${url}" =~ ^https?:// ]]; then
# Strip ccache attributes after | for the URL
ccache_get_http_stats "${url%%|*}"
fi
}
# Mask credentials in storage URLs to avoid leaking secrets into build logs
# Handles any URI scheme with userinfo component (e.g., redis://user:pass@host)
# Uses last @ as delimiter since userinfo may contain special characters
function ccache_mask_storage_url() {
local url="$1"
if [[ "${url}" =~ ^([a-zA-Z][a-zA-Z0-9+.-]*://)(.+)@([^@]+)$ ]]; then
echo "${BASH_REMATCH[1]}****@${BASH_REMATCH[3]}"
else
echo "${url}"
fi
}
# Validate that credentials in storage URL do not contain characters unsafe for URL parsing.
# Passwords with / + = or spaces break URL parsing in ccache and in our mask function.
# Returns 1 and displays error if invalid characters are found.
function ccache_validate_storage_url() {
local url="$1"
# Extract userinfo (part between :// and last @)
if [[ "${url}" =~ ^[a-zA-Z][a-zA-Z0-9+.-]*://(.+)@[^@]+$ ]]; then
local userinfo="${BASH_REMATCH[1]}"
if [[ "${userinfo}" =~ [/+=[:space:]] ]]; then
display_alert "Password contains URL-unsafe characters (/ + = or spaces)" \
"Generate a safe password: openssl rand -hex 24" "err"
return 1
fi
fi
return 0
}
# This runs on the HOST just before Docker container is launched.
# Resolves 'ccache.local' via mDNS (requires Avahi on server publishing this hostname
# Docker hook: resolve hostnames and handle loopback for container access.
# mDNS/local DNS may not work inside Docker, so we resolve on host and
# pass the mapping via --add-host. Loopback addresses are rewritten to
# host.docker.internal.
function host_pre_docker_launch__setup_remote_ccache() {
if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then
ccache_validate_storage_url "${CCACHE_REMOTE_STORAGE}" || return 1
display_alert "Remote ccache pre-configured" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info"
elif ! ccache_discover_remote_storage; then
display_alert "Remote ccache not found on host" "no service discovered" "debug"
fi
# Show backend stats if we have a remote storage URL
if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then
local stats
stats=$(ccache_get_remote_stats "${CCACHE_REMOTE_STORAGE}")
if [[ -n "$stats" ]]; then
display_alert "Remote ccache stats" "${stats}" "info"
fi
fi
# Ensure hostname in CCACHE_REMOTE_STORAGE is resolvable inside Docker.
# Docker containers may not have access to host mDNS/local DNS.
if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then
local _host
_host=$(ccache_extract_url_host "${CCACHE_REMOTE_STORAGE}")
if [[ -n "${_host}" ]]; then
# Loopback addresses: rewrite to host.docker.internal
if [[ "${_host}" == "localhost" || "${_host}" == "127.0.0.1" || "${_host}" == "::1" ]]; then
CCACHE_REMOTE_STORAGE="${CCACHE_REMOTE_STORAGE//localhost/host.docker.internal}"
CCACHE_REMOTE_STORAGE="${CCACHE_REMOTE_STORAGE//127.0.0.1/host.docker.internal}"
CCACHE_REMOTE_STORAGE="${CCACHE_REMOTE_STORAGE//\[::1\]/host.docker.internal}"
DOCKER_EXTRA_ARGS+=("--add-host=host.docker.internal:host-gateway")
display_alert "Rewriting loopback URL for Docker" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info"
# Hostname (not IP): resolve on host and pass via --add-host
elif [[ "${_host}" =~ [a-zA-Z] ]]; then
local _resolved_ip="${CCACHE_REMOTE_HOST_IP:-}"
# If not from discovery, resolve now; prefer IPv4 (Docker bridge often lacks IPv6)
if [[ -z "${_resolved_ip}" || "${CCACHE_REMOTE_HOST}" != "${_host}" ]]; then
_resolved_ip=$(getent ahostsv4 "${_host}" 2>/dev/null | awk '{print $1; exit}' || true)
[[ -z "${_resolved_ip}" ]] && _resolved_ip=$(getent hosts "${_host}" 2>/dev/null | awk '{print $1; exit}' || true)
fi
if [[ -n "${_resolved_ip}" ]]; then
DOCKER_EXTRA_ARGS+=("--add-host=${_host}:${_resolved_ip}")
display_alert "Docker --add-host" "${_host}:${_resolved_ip}" "info"
else
display_alert "Cannot resolve hostname for Docker" "${_host}" "wrn"
fi
fi
fi
fi
# Pass all set CCACHE_* variables to Docker
local var val
for var in "${CCACHE_PASSTHROUGH_VARS[@]}"; do
val="${!var}"
if [[ -n "${val}" ]]; then
DOCKER_EXTRA_ARGS+=("--env" "${var}=${val}")
local log_val="${val}"
[[ "${var}" == "CCACHE_REMOTE_STORAGE" ]] && log_val="$(ccache_mask_storage_url "${val}")"
display_alert "Docker env" "${var}=${log_val}" "debug"
fi
done
}
# Hook: Show ccache remote storage statistics after each compilation (kernel, uboot)
function ccache_post_compilation__show_remote_stats() {
if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then
local stats_output pct
local read_hit read_miss write error
stats_output=$(ccache --print-stats 2>&1 || true)
read_hit=$(ccache_get_stat "$stats_output" "remote_storage_read_hit")
read_miss=$(ccache_get_stat "$stats_output" "remote_storage_read_miss")
write=$(ccache_get_stat "$stats_output" "remote_storage_write")
error=$(ccache_get_stat "$stats_output" "remote_storage_error")
pct=$(ccache_hit_pct "$read_hit" "$read_miss")
display_alert "Remote ccache result" "hit=${read_hit} miss=${read_miss} write=${write} err=${error} (${pct}%)" "info"
fi
}
# This runs inside Docker (or native build) during configuration
function extension_prepare_config__setup_remote_ccache() {
# Enable ccache with a consistent cache directory ($SRC/cache/ccache).
# PRIVATE_CCACHE ensures the same CCACHE_DIR is used in native and Docker builds,
# avoiding fragmented caches in /root/.cache/ccache vs $SRC/cache/ccache.
declare -g USE_CCACHE=yes
declare -g PRIVATE_CCACHE=yes
# If CCACHE_REMOTE_STORAGE was passed from host (via Docker env), it's already set
if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then
ccache_validate_storage_url "${CCACHE_REMOTE_STORAGE}" || return 1
display_alert "Remote ccache configured" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info"
return 0
fi
# For native (non-Docker) builds, try to discover
if ccache_discover_remote_storage; then
return 0
fi
if [[ "${CCACHE_REMOTE_ONLY}" == "yes" ]]; then
display_alert "Remote ccache not available" "CCACHE_REMOTE_ONLY=yes but no remote found, ccache will be ineffective" "wrn"
else
display_alert "Remote ccache not available" "using local cache only" "debug"
fi
return 0
}
# Inject all set CCACHE_PASSTHROUGH_VARS into the given make environment array
# Uses bash nameref to write into the caller's array variable
function ccache_inject_envs() {
local -n target_array="$1"
local label="$2"
local var val
for var in "${CCACHE_PASSTHROUGH_VARS[@]}"; do
val="${!var}"
if [[ -n "${val}" ]]; then
target_array+=("${var}=${val@Q}")
local log_val="${val}"
[[ "${var}" == "CCACHE_REMOTE_STORAGE" ]] && log_val="$(ccache_mask_storage_url "${val}")"
display_alert "${label}: ${var}" "${log_val}" "debug"
fi
done
}
# This hook runs right before kernel make - add ccache env vars to make environment.
# Required because kernel build uses 'env -i' which clears all environment variables.
function kernel_make_config__add_ccache_remote_storage() {
ccache_inject_envs common_make_envs "Kernel make"
}
# This hook runs right before u-boot make - add ccache env vars to make environment.
# Required because u-boot build uses 'env -i' which clears all environment variables.
function uboot_make_config__add_ccache_remote_storage() {
ccache_inject_envs uboot_make_envs "U-boot make"
}

View File

@ -0,0 +1,10 @@
<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name>ccache-redis</name>
<service>
<type>_ccache._tcp</type>
<port>6379</port>
<txt-record>type=redis</txt-record>
</service>
</service-group>

View File

@ -0,0 +1,11 @@
<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name>ccache-webdav</name>
<service>
<type>_ccache._tcp</type>
<port>8088</port>
<txt-record>type=http</txt-record>
<txt-record>path=/ccache/</txt-record>
</service>
</service-group>

View File

@ -0,0 +1,27 @@
# nginx WebDAV configuration for ccache remote storage
# Copy to /etc/nginx/sites-available/ccache-webdav
# Then: ln -s /etc/nginx/sites-available/ccache-webdav /etc/nginx/sites-enabled/
#
# Prepare storage directory:
# mkdir -p /var/cache/ccache-webdav/ccache
# chown -R www-data:www-data /var/cache/ccache-webdav
#
# WARNING: No authentication configured.
# Use ONLY in a fully trusted private network.
# For auth, add auth_basic directives. See nginx WebDAV documentation.
# Note: ccache does not support HTTPS directly. Use a reverse proxy for TLS.
# Requires: nginx with http_dav_module (e.g. nginx-full or nginx-extras on Debian/Ubuntu)
server {
listen 8088;
server_name _;
root /var/cache/ccache-webdav;
location /ccache/ {
dav_methods PUT DELETE;
create_full_put_path on;
dav_access user:rw group:rw all:r;
client_max_body_size 100M;
autoindex on;
}
}

View File

@ -0,0 +1,13 @@
# Redis configuration for ccache remote storage
# Copy to /etc/redis/redis-ccache.conf or merge into /etc/redis/redis.conf
#
# WARNING: This configuration is INSECURE - Redis is open without authentication.
# Use ONLY in a fully trusted private network with no internet access.
# For secure setup (password, TLS, ACL), see: https://redis.io/docs/management/security/
bind 0.0.0.0 ::
protected-mode no
# Limit memory and use LRU eviction so old cache entries are purged automatically
maxmemory 4G
maxmemory-policy allkeys-lru

View File

@ -0,0 +1,35 @@
# Redis configuration for ccache remote storage
# Copy to /etc/redis/redis-ccache.conf or merge into /etc/redis/redis.conf
#
# For advanced security (TLS, ACL, rename-command), see:
# https://redis.io/docs/management/security/
bind 0.0.0.0 ::
# --- Authentication (recommended) ---
# Clients connect with: CCACHE_REMOTE_STORAGE="redis://default:YOUR_PASSWORD@host:6379"
# Generate a URL-safe password (hex only, no special characters):
# openssl rand -hex 24
# WARNING: Do not use base64 passwords (openssl rand -base64) — they contain
# / + = characters that break redis:// URL parsing.
#
# Redis 6+: use ACL user entry (preferred, overrides requirepass):
# user default on >CHANGE_ME sanitize-payload ~* &* +@all
#
# Redis < 6: use requirepass instead:
# requirepass CHANGE_ME
#
# With authentication set, protected-mode can stay enabled (default).
# It blocks unauthenticated access from non-loopback interfaces.
protected-mode yes
# --- No authentication (trusted network only) ---
# If all build hosts and the Redis server are on a fully isolated private
# network and you do not need access control, you can skip auth:
# user default on nopass sanitize-payload ~* &* +@all
# protected-mode no
# WARNING: Anyone on the network can read/write/flush the cache.
# Limit memory and use LRU eviction so old cache entries are purged automatically
maxmemory 4G
maxmemory-policy allkeys-lru

View File

@ -0,0 +1,13 @@
[Unit]
Description=Publish ccache Redis service via Avahi DNS-SD
After=avahi-daemon.service redis-server.service
BindsTo=redis-server.service
[Service]
Type=simple
ExecStart=/usr/bin/avahi-publish-service "ccache-redis" _ccache._tcp 6379 type=redis
Restart=on-failure
RestartSec=5
[Install]
WantedBy=redis-server.service

View File

@ -0,0 +1,13 @@
[Unit]
Description=Publish ccache WebDAV service via Avahi DNS-SD
After=avahi-daemon.service nginx.service
BindsTo=nginx.service
[Service]
Type=simple
ExecStart=/usr/bin/avahi-publish-service "ccache-webdav" _ccache._tcp 8088 type=http path=/ccache/
Restart=on-failure
RestartSec=5
[Install]
WantedBy=nginx.service

View File

@ -7,6 +7,43 @@
# This file is a part of the Armbian Build Framework
# https://github.com/armbian/build/
# Parse a single numeric field from "ccache --print-stats" tab-separated output
# Returns 0 if field not found or not numeric
function ccache_get_stat() {
local stats_output="$1" field="$2"
local val
val=$(echo "$stats_output" | grep "^${field}" | cut -f2 || true)
[[ "${val}" =~ ^[0-9]+$ ]] || val=0
echo "$val"
}
# Calculate hit percentage from hit and miss counts
function ccache_hit_pct() {
local hit="$1" miss="$2"
local total=$(( hit + miss ))
if [[ $total -gt 0 ]]; then
echo $(( hit * 100 / total ))
else
echo 0
fi
}
# Helper function to show ccache stats - used as cleanup handler for interruption case
function ccache_show_compilation_stats() {
local stats_output direct_hit direct_miss pct
stats_output=$(ccache --print-stats 2>&1 || true)
direct_hit=$(ccache_get_stat "$stats_output" "direct_cache_hit")
direct_miss=$(ccache_get_stat "$stats_output" "direct_cache_miss")
pct=$(ccache_hit_pct "$direct_hit" "$direct_miss")
display_alert "Ccache result" "hit=${direct_hit} miss=${direct_miss} (${pct}%)" "info"
# Hook for extensions to show additional stats (e.g., remote storage)
call_extension_method "ccache_post_compilation" <<- 'CCACHE_POST_COMPILATION'
*called after ccache-wrapped compilation completes (success or failure)*
Useful for displaying remote cache statistics or other post-build info.
CCACHE_POST_COMPILATION
}
function do_with_ccache_statistics() {
display_alert "Clearing ccache statistics" "ccache" "ccache"
@ -35,8 +72,20 @@ function do_with_ccache_statistics() {
run_host_command_logged ccache --show-config "&&" sync
fi
# Register cleanup handler to show stats even if build is interrupted
add_cleanup_handler ccache_show_compilation_stats
display_alert "Running ccache'd build..." "ccache" "ccache"
"$@"
local build_exit_code=0
"$@" || build_exit_code=$?
# Show stats and remove from cleanup handlers (so it doesn't run twice on exit)
execute_and_remove_cleanup_handler ccache_show_compilation_stats
# Re-raise the error if the build failed
if [[ ${build_exit_code} -ne 0 ]]; then
return ${build_exit_code}
fi
if [[ "${SHOW_CCACHE}" == "yes" ]]; then
display_alert "Display ccache statistics" "ccache" "ccache"

View File

@ -31,7 +31,7 @@ function run_kernel_make_internal() {
# If CCACHE_DIR is set, pass it to the kernel build; Pass the ccache dir explicitly, since we'll run under "env -i"
if [[ -n "${CCACHE_DIR}" ]]; then
common_make_envs+=("CCACHE_DIR='${CCACHE_DIR}'")
common_make_envs+=("CCACHE_DIR=${CCACHE_DIR@Q}")
fi
# Add the distcc envs, if any.
@ -74,7 +74,19 @@ function run_kernel_make_internal() {
common_make_params_quoted+=("${llvm_flag}")
fi
# Allow extensions to modify make parameters and environment variables
# Hook order: kernel_make_config runs first (generic extension config),
# then custom_kernel_make_params (user/board overrides can take precedence).
call_extension_method "kernel_make_config" <<- 'KERNEL_MAKE_CONFIG'
*Hook to customize kernel make environment and parameters*
Called right before invoking make for kernel compilation.
Available arrays to modify:
- common_make_envs[@]: environment variables passed via "env -i" (e.g., CCACHE_REMOTE_STORAGE)
- common_make_params_quoted[@]: make command parameters (e.g., custom flags)
Available read-only variables:
- KERNEL_COMPILER, ARCHITECTURE, BRANCH, LINUXFAMILY
KERNEL_MAKE_CONFIG
# Runs after kernel_make_config — allows user/board overrides to take precedence
call_extension_method "custom_kernel_make_params" <<- 'CUSTOM_KERNEL_MAKE_PARAMS'
*Customize kernel make parameters before compilation*
Called after all standard make parameters are set but before invoking make.

View File

@ -253,6 +253,27 @@ function compile_uboot_target() {
"PYTHONPATH=\"${PYTHON3_INFO[MODULES_PATH]}:${PYTHONPATH}\"" # Insert the pip modules downloaded by Armbian into PYTHONPATH (needed e.g. for pyelftools)
)
# Pass the ccache directories explicitly, since we'll run under "env -i"
if [[ -n "${CCACHE_DIR}" ]]; then
uboot_make_envs+=("CCACHE_DIR=${CCACHE_DIR@Q}")
fi
if [[ -n "${CCACHE_TEMPDIR}" ]]; then
uboot_make_envs+=("CCACHE_TEMPDIR=${CCACHE_TEMPDIR@Q}")
fi
# workaround when two compilers are needed
cross_compile="CROSS_COMPILE=\"${CCACHE:+$CCACHE }$UBOOT_COMPILER\""
# When UBOOT_TOOLCHAIN2 is set, the board's uboot_custom_postprocess handles compilers;
# pass a harmless dummy env var since empty make parameters cause errors
[[ -n $UBOOT_TOOLCHAIN2 ]] && cross_compile="ARMBIAN=foe"
call_extension_method "uboot_make_config" <<- 'UBOOT_MAKE_CONFIG'
*Hook to customize u-boot make environment*
Called right before invoking make for u-boot compilation.
Available array to modify:
- uboot_make_envs[@]: environment variables passed via "env -i" (e.g., CCACHE_REMOTE_STORAGE)
UBOOT_MAKE_CONFIG
display_alert "${uboot_prefix}Compiling u-boot" "${version} ${target_make} with gcc '${gcc_version_main}'" "info"
declare -g if_error_detail_message="${uboot_prefix}Failed to build u-boot ${version} ${target_make}"
do_with_ccache_statistics run_host_command_logged_long_running \

View File

@ -17,6 +17,11 @@ function prepare_compilation_vars() {
# private ccache directory to avoid permission issues when using build script with "sudo"
# see https://ccache.samba.org/manual.html#_sharing_a_cache for alternative solution
[[ $PRIVATE_CCACHE == yes ]] && export CCACHE_DIR=$SRC/cache/ccache # actual export
# Set default umask for ccache to allow write access for all users (enables cache sharing)
# CCACHE_UMASK=000 creates files with permissions 666 (rw-rw-rw-) and dirs with 777 (rwxrwxrwx)
# Only set this for shared cache, not for private cache
[[ -z "${CCACHE_UMASK}" && "${PRIVATE_CCACHE}" != "yes" ]] && export CCACHE_UMASK=000
else
CCACHE=""
fi

View File

@ -572,8 +572,23 @@ function docker_cli_prepare_launch() {
display_alert "Not running in a terminal" "not passing through stdin to Docker" "debug"
fi
# if DOCKER_EXTRA_ARGS is an array and has more than zero elements, add its contents to the DOCKER_ARGS array
if [[ "${DOCKER_EXTRA_ARGS[*]+isset}" == "isset" && "${#DOCKER_EXTRA_ARGS[@]}" -gt 0 ]]; then
# Preserve any pre-existing DOCKER_EXTRA_ARGS (e.g., from user environment) and let extensions append
declare -g -a DOCKER_EXTRA_ARGS=("${DOCKER_EXTRA_ARGS[@]+"${DOCKER_EXTRA_ARGS[@]}"}")
# Hook for extensions to add Docker arguments before launch
call_extension_method "host_pre_docker_launch" <<- 'HOST_PRE_DOCKER_LAUNCH'
*run on host just before Docker container is launched*
Extensions can add Docker arguments by appending to DOCKER_EXTRA_ARGS array.
Each array element should be a complete argument (e.g., "--env", "MY_VAR=value" as separate elements).
Example: DOCKER_EXTRA_ARGS+=("--env" "MY_VAR=value" "--mount" "type=bind,src=/a,dst=/b")
Available variables:
- DOCKER_ARGS[@]: current Docker arguments (do not modify directly)
- DOCKER_EXTRA_ARGS[@]: array to append extra arguments for docker run
- DOCKER_ARMBIAN_TARGET_PATH: path inside container (/armbian)
HOST_PRE_DOCKER_LAUNCH
# Add DOCKER_EXTRA_ARGS to DOCKER_ARGS if any were added by extensions
if [[ "${#DOCKER_EXTRA_ARGS[@]}" -gt 0 ]]; then
display_alert "Adding extra Docker arguments" "${DOCKER_EXTRA_ARGS[*]}" "debug"
DOCKER_ARGS+=("${DOCKER_EXTRA_ARGS[@]}")
fi