Commit Graph

151 Commits

Author SHA1 Message Date
Ricardo Pardini
c17a041637 prepare_host: handle PRE_PREPARED_HOST inside prepare_host_init() so WORKDIR etc is always available 2023-08-12 09:58:32 +02:00
Ricardo Pardini
eab306b279 tmpfs: introduce USE_TMPFS=no to disable tmpfs usage for WORKDIR and package-specific working dirs; actually use $temp_dir_id for mktemp; see #5502
- `USE_TMPFS=no` disables usage of generic tmpfs mechanism (still possibly used for rootfs/image building, which is unrelated), for last-resort cases
- use better/more descriptive `temp_dir_id`'s for kernel build than `k` (now `kernel_dest_install_dir`) and `kd` (now `kernel_debs_temp_dir`)
  - specific image/dtb/headers packaging already had decent names, same for other .deb's
- replace `mktemp -d` with `mktemp -d --tmpdir "${temp_dir_id}-XXXXX"` in `prepare_temp_dir_in_workdir_and_schedule_cleanup()`, so we know what's using what in tmpfs
2023-07-24 23:19:15 +02:00
Ricardo Pardini
e6f70a3573 docker: default COLUMNS env var to 160 if unset / not-interactive 2023-07-01 14:55:49 +02:00
Igor
1848366170 Revert "docker: use ${SRC} as DOCKER_ARMBIAN_TARGET_PATH instead of hardcoded /armbian"
This reverts commit 6dd7067c79.
2023-06-13 21:59:44 +02:00
Ricardo Pardini
6dd7067c79 docker: use ${SRC} as DOCKER_ARMBIAN_TARGET_PATH instead of hardcoded /armbian
- this way host-side and Docker-side `SRC` are now the same path
  - messages containing the path to a file will now match across Docker and actual host
2023-06-13 19:32:49 +02:00
Ricardo Pardini
70a64a0415 patching: Patching Summary and Patching Failures tables using Python's Rich
- even Rich'er patch output by colorizing certain strings green/yellow/red
- BASE_GIT_TAG now very sneakily also accepts a branch name
- IMPORTANT: this includes: BREAKING CHANGE: patches failing to apply now break the build. fixes #4958
  - also break on legacy `process_patch_file()` failure, remove `EXIT_PATCHING_ERROR`
2023-06-13 12:29:52 +02:00
Ricardo Pardini
1e3d5c3567 prepare-host: check_host_has_enough_disk_space(): don't check if dir does not exist; fixes #5246 2023-06-02 19:32:40 +02:00
Ricardo Pardini
31131fe14e bsp-cli/BETA: fixes for bsp-cli bootscript hashing; don't normalize bootscript vars; debugging for BETA=yes/no stuff
also:
- get rid of comments and `SUBREVISION` and `RC` variables
- better debugging in calculate_hash_for_variables() do_normalize_src_path=yes/no
- tag a few places where output/debs might wrong in face of BETA=yes/no
2023-05-25 19:30:40 +02:00
Ricardo Pardini
52710b3363 git/docker: bsp-cli: fix BUILD_REPOSITORY_URL and BUILD_REPOSITORY_COMMIT under Docker; fixes AR-1750 2023-05-24 19:20:57 +02:00
Ricardo Pardini
30eb5323f2 prepare-host: refactor disk space check; test "${DEST}" and "${SRC}/cache", no longer "${SRC}" 2023-05-21 16:18:35 +02:00
Ricardo Pardini
234232085d docker: remove dead code 2023-05-16 20:09:47 +02:00
Ricardo Pardini
c351936cb0 binfmts: hide output when loading qemu-arm under aarch64 (some are native, some are emulated, all are good, errors can be ignored) 2023-05-15 17:43:25 +02:00
Ricardo Pardini
f0f9159faa logging: curb CI/GHA special errors for cascading errors during build failure
- otherwise we get 4 errors for each "error" in CI
  - one for the real error
  - one for "wait for cleanups"
  - one for "Docker build failed"
  - one from GHA itself since command failed (we can't get rid of this)
2023-05-15 17:43:25 +02:00
Ricardo Pardini
25929ea6ac apt_purge_unneeded_packages_and_clean_apt_caches(): count files, don't use du: avoid WARNs when not needed; tolerate 1 file
- 'lock' might or not be there, tolerate 1 file (not 0)
2023-05-15 17:43:25 +02:00
Ricardo Pardini
83e35b1473 binfmts: don't raise warning if qemu-arm failed to load under aarch64 (some are native, some are emulated, all are good)
- reported by @igorpecovnik on Discord, during massive CI build, when armhf builds landed on arm64 builders
2023-05-09 10:04:57 +02:00
Ricardo Pardini
8f14f33f78 logging: squash undue warnings rootfs/aptcache
- rootfs is built with BOARD= so just live with it (idea about this warn was that families might hook rootfs build process)
- count, instead of du, apt caches. seems some empty dirs are reported as "1Mb" by du?
2023-05-09 10:04:57 +02:00
Ricardo Pardini
5316c6685b host-utils: more logging for when junk is found in image apt caches 2023-05-09 10:04:57 +02:00
Ricardo Pardini
73e9c89807 docker: fix: always mount .docker/config.json if it exists (in GHA) 2023-05-03 15:09:56 +02:00
Pander
0d72163868 Fixed execution rights docker.sh 2023-04-22 20:08:26 +02:00
Ricardo Pardini
d0716b65c0 docker/sudo: better control/info over CLI relaunch args; introduce CLI relaunch envs
- allow adding re-launch environments by populating `ARMBIAN_CLI_RELAUNCH_PARAMS` dict
- similar to `ARMBIAN_CLI_RELAUNCH_PARAMS`
- rename/consolidate `()`'s output globals:
  - `ARMBIAN_CLI_FINAL_RELAUNCH_ARGS` (rename)
  - `ARMBIAN_CLI_FINAL_RELAUNCH_ENVS` (new)
    - includes `ARMBIAN_HIDE_REPEAT_PARAMS` automatically
- `sudo`: include envs when relaunching, use bash explicitly
- `docker`: better logging
- `docker`: include envs when relaunching via `--env` args
2023-04-21 14:26:33 +02:00
Ricardo Pardini
2b28f29ebe rootfs/image: avoid apt junk leftover by debootstrap; add warnings and debugs over target cache and general rootfs sizes
- rootfs: rootfs-create: show a summary of the 20 biggest dirs, right before tarring the rootfs (for debugging)
- rootfs: rootfs-create: show usage of caches between first and second stages
- rootfs: rootfs-create: cleanup junk left by `debootstrap` after second stage
- rootfs: rootfs-create: _always_ clean apt stuff at the end
- rename `apt_purge_unneeded_packages()` to `apt_purge_unneeded_packages_and_clean_apt_caches()` for clarity
- image: `apt_purge_unneeded_packages_and_clean_apt_caches()`: warn if apt caches not empty; clean them off, always.
- host-utils: `local_apt_deb_cache_prepare()`: also test the target, warn if not empty
- extension: cleanup-space-final-image: do NOT clean apt stuff. done in core now
- the metric shit-ton of debugs added should help the next person who faces this in the future
2023-04-18 16:27:06 +02:00
Werner
0b90ebc54a Use native xz 2023-04-18 09:29:17 +02:00
Ricardo Pardini
eb1fe6d7ba artifacts: introduce PRE_PREPARED_HOST=yes: allow running pre-prepared host CLI's for artifacts that require aggregation 2023-04-15 22:43:06 +02:00
Ricardo Pardini
c30a907e47 python2: lunar also doesn't have python2 anymore (joins sid and bookworm) 2023-04-15 22:43:06 +02:00
Ricardo Pardini
ed0b406265 lib: fix: replace undue export statements with declare -g 2023-03-31 09:12:44 +02:00
Ricardo Pardini
fe7a24885b docker: introduce DOCKER_SKIP_UPDATE=yes to skip updating prebuilt Docker image, only when using 2023-03-31 09:12:44 +02:00
Ricardo Pardini
40c5df55ab docker: introduce ${DOCKER_ARMBIAN_BASE_COORDINATE_PREFIX:-"ghcr.io/armbian/docker-armbian-build:armbian-"} for easy override 2023-03-31 09:12:44 +02:00
Ricardo Pardini
6b965c605c completely remove any traces of KERNEL_ONLY and exit with error if set 2023-03-17 08:32:11 +01:00
Ricardo Pardini
9931350483 docker: fix: pass down OCI_TARGET_BASE to Docker instance automatically 2023-03-17 08:32:11 +01:00
Michael Beck
1223b22ffd
WSL2: Ensure the Windows Terminal is detected under WSL2 docker build. (#4915)
- by passing the WT_SESSION environment variable into the docker container  if it is set
2023-03-09 19:19:30 +01:00
Igor Pečovnik
b7b8eb7b72
Add / modify (c) in bash scripts (#4922)
* Add / modify (c) in bash scripts

Signed-off-by: Igor <igor@armbian.com>

* Add (c) to the source config files

---------

Signed-off-by: Igor <igor@armbian.com>
2023-03-09 18:30:40 +01:00
Ricardo Pardini
2f7423d0e4
shellfmt/.editorconfig: enable trim whitespaces, run shellfmt, no actual code changes (#4849)
- shellfmt: bump to 3.6.0 (run with `bash lib/tools/shellfmt.sh`)
- .editorconfig: rationalize, remove duplicated logic
2023-02-23 09:24:41 +01:00
Ricardo Pardini
aa70ad570b
armbian-next: use new/definitive pkg/image coordinates for Docker image, on Armbian org 2023-02-18 12:13:42 -03:00
Ricardo Pardini
20de5dd2c6
armbian-next: switch to using Armbian-org GH hosted git shallow trees and Docker images 2023-02-18 07:46:46 -03:00
Ricardo Pardini
a2304f28b0
armbian-next: post_build_image: run hook first, then write to CARD_DEVICE, then compress, in that order
- compress *all* present output images, not only .img
- remove 7zip compression & hostdep
- squash a few shortcircuits (yeah, this late in the game...)
2023-02-18 07:46:12 -03:00
Ricardo Pardini
e697f2e67c
armbian-next: docker: curb logging about OCI_TARGET_BASE 2023-02-18 07:46:10 -03:00
Ricardo Pardini
c73d60368a
armbian-next: remove references to stuff that doesn't exist or isn't hit by default anymore 2023-02-18 07:46:09 -03:00
Ricardo Pardini
9bffa5e749
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build
- artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes`
- artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build
- kernel_package_source() is no more
- a long dissertation about kernels, families, and the universe
- artifacts: actually use rootfs artifact for image build
- artifacts: detangle via artifact_base_dir
- artifacts: rootfs: use folders in artifact_name; include cache_type
- artifacts: some cleanups / validations
- rootfs artifact; drop old rootfs cli
- artifacts: new CLI shortcuts; remove old firmware CLI
- artifacts: full firmware & usage
- use firmware artifacts in image build and install its debs
- kernel artifact sans legacy; use tmpdir for .deb target for all packages
- legacy artifact versions is no more; pack/unpack now in common obtain;
- artifacts: uboot: cleanup legacy renaming, use artifact version directly
- artifacts: add firmware (small) artifact
- deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning
- fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash
  - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()`
  - don't use realpath for artifact_file_relative
  - curb some warnings
  - fix: only install headers & dtbs if such exist
- kernel .config hook modification hash versioning
- OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB
- artifact revolver & installing into image
  - add artifact_map_packages and artifact_map_debs dicts
  - revolver accumulates all info
  - REPOSITORY_INSTALL is no more (for uboot/kernel, later others)
  - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-18 07:46:03 -03:00
Ricardo Pardini
a75e58c320
armbian-next: docker: show $SECONDS in Docker dash logs 2023-02-18 07:46:01 -03:00
Ricardo Pardini
ca6bf9046b
armbian-next: remove all traces of Vagrant and FEL boot 2023-02-18 07:46:00 -03:00
Ricardo Pardini
843449597a
armbian-next: binfmts: suppress error message when trying to emulate armhf on arm64 2023-02-18 07:45:53 -03:00
Ricardo Pardini
399d9964dd
armbian-next: python/pip: include pip pkgs in Dockerfile; use them if available instead of downloading 2023-02-18 07:45:51 -03:00
Ricardo Pardini
bf53bd2988
armbian-next: tmpfs: use size=99% for all tmpfs's; (rootfs/WORKDIR/chroot); thanks Igor for suggestion 2023-02-18 07:45:42 -03:00
Ricardo Pardini
0e480e685e
armbian-next: docker: pass down git info via env var, since Docker doesn't get ${SRC}/.git, yet we need that info in the Docker logs 2023-02-18 07:45:40 -03:00
Ricardo Pardini
7431984025
armbian-next: docker: pass CARD_DEVICE to Docker via --device 2023-02-18 07:45:35 -03:00
Ricardo Pardini
6258ea566a
armbian-next: shellcheck: squash a bunch of 'critical' warnings 2023-02-18 07:45:29 -03:00
Ricardo Pardini
19aa7e95cf
armbian-next: introduce assert_prepared_host() and assert before using any Python, to un-mask any further fallacies 2023-02-18 07:45:19 -03:00
Ricardo Pardini
956ef9ac7d
armbian-next: fix: for the 1000th time had broken KERNEL_ONLY=yes; curb some warnings 2023-02-18 07:45:18 -03:00
Ricardo Pardini
281e3f0828
armbian-next: cli: try (and fail, probably...) to be helpful with half-working Docker installs 2023-02-18 07:45:16 -03:00
Ricardo Pardini
8329b2e970
armbian-next: docker: split docker_cli_build_dockerfile() into docker_cli_prepare_dockerfile(); introduce FAST_DOCKER
- armbian-next: docker: fix hostrelease crazy
2023-02-18 07:45:11 -03:00