enoch/armbian-build
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11,306 Commits 95 Branches 139 Tags 940 MiB
b7d5ea2321
Commit Graph

29 Commits

Author SHA1 Message Date
Ricardo Pardini
9bffa5e749
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build 
- artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes`
- artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build
- kernel_package_source() is no more
- a long dissertation about kernels, families, and the universe
- artifacts: actually use rootfs artifact for image build
- artifacts: detangle via artifact_base_dir
- artifacts: rootfs: use folders in artifact_name; include cache_type
- artifacts: some cleanups / validations
- rootfs artifact; drop old rootfs cli
- artifacts: new CLI shortcuts; remove old firmware CLI
- artifacts: full firmware & usage
- use firmware artifacts in image build and install its debs
- kernel artifact sans legacy; use tmpdir for .deb target for all packages
- legacy artifact versions is no more; pack/unpack now in common obtain;
- artifacts: uboot: cleanup legacy renaming, use artifact version directly
- artifacts: add firmware (small) artifact
- deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning
- fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash
  - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()`
  - don't use realpath for artifact_file_relative
  - curb some warnings
  - fix: only install headers & dtbs if such exist
- kernel .config hook modification hash versioning
- OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB
- artifact revolver & installing into image
  - add artifact_map_packages and artifact_map_debs dicts
  - revolver accumulates all info
  - REPOSITORY_INSTALL is no more (for uboot/kernel, later others)
  - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
 2023-02-18 07:46:03 -03:00
Ricardo Pardini
78dcf01c92
armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs 
- killed `[[ $ROOTFS_TYPE != ext4 ]] && display_alert "Assuming ${BOARD} ${BRANCH} kernel supports ${ROOTFS_TYPE}" "" "wrn"`
  - which definitely didn't belong in rootfs
- disable usage of run_host_command_logged_long_running for kernel-make
- stop lying about long_running stuff being any different from non-long_running versions
- stop lying about 'set -e' when there's still a bunch of pipes for "pv" and stuff all around
 2023-02-18 07:44:33 -03:00
Ricardo Pardini
f43732a8e9
armbian-next: use unified methods for tmp dir handling all around (except ATF, which is special) 2023-02-18 07:44:29 -03:00
Ricardo Pardini
8fedd749aa
armbian-next: kernel/logging: stop lying about LD/AR/INSTALL/SIGN/XZ make output 2023-02-18 07:44:14 -03:00
Ricardo Pardini
88ff66ae11
armbian-next: adaptive shallow/full Kernel git bare git (3gb -> 300mb download) 
- allows using pre-shallow-ed bare trees, specific to the KERNEL_MAJOR_MINOR
- uses shallow by default if storage is "slow" (mmc), or free disk space < 32g (@TODO tweak this)
- if full tree already there keeps using it
- allow forcing decision with `KERNEL_GIT=full` or `KERNEL_GIT=shallow`
- countdown to confirm if decision based on storage type/free space and user is interactive
  - this has some of the worst copy ever written
- @TODO refactor the device type / free space code, reuse across codebase (3-4 other spots detect free space)
- @TODO GHA self-hosted (full) vs hosted (shallow) runners decision
- sprinkle with `wait_for_disk_sync()` around clean/download/extract
- sprinkle git with debugs, and an info between fetch and checkout (otherwise it seems "fetch" takes a long time)
 2023-02-18 07:43:54 -03:00
Ricardo Pardini
4d43c05f7b
armbian-next: kernel: git: remove dead code, GIT_PRE_FETCH_HOOK / checked_out_revision_mtime 2023-02-18 07:43:52 -03:00
Ricardo Pardini
f2bd68afde
armbian-next: kernel: cleanup bundle after patching succeeded, not after build success 
- for extra gigabytes of space, earlier
 2023-02-18 07:43:17 -03:00
Ricardo Pardini
78fd35a1e3
armbian-next: better logging for kernel_prepare_bare_repo_from_oras_gitball(); remove dead code 
- remove dead code used to fetch tags, back from pre-bundle days
 2023-02-18 07:42:51 -03:00
Ricardo Pardini
53266feff8
armbian-next: kernel: deterministic .config mtime handling; kernel build logging sections reorg 
- .config is now compared with previous one byte-wise, diff shown, and mtime preserved directly via `cp -p`
- this does away with the previous mtime-based BS I had written during the fasthash era (and thus remove file-mtime.sh, regen lib)
- split some functions, to better control the interactive piece in the _middle_ of kernel config process
- no more `kernel_config_maybe_interactive()`, now `kernel_config()` that controls its own logging sections
- extract `kernel_determine_toolchain()` out of config into make
- introduce internal hook `armbian_kernel_config()` that runs before custom version (for .config defaults, coming soon)
 2023-02-18 07:42:39 -03:00
Ricardo Pardini
e6b1c4d875
armbian-next: CLEAN_LEVEL=make-kernel now does git clean -xfd instead, faster and 100% clean 2023-02-18 07:42:07 -03:00
Ricardo Pardini
9370c19a44
armbian-next: drop old code for patching & fasthash 
- split the modification time stuff to general
 2023-02-18 07:41:05 -03:00
Ricardo Pardini
770d508ca0
armbian-next: Python patching delusion pt 5 + EXTRAWIFI patch generator/harness + consistent patch mtime targets 
- Python patching: tune some logging all-around
- Python patching: *FINALLY* set the dates on the patched files to `max(patch_date, root_makefile_date)` thus making lighting-fast rebuilds again
- new EXTRAWIFI patch generator harness; Python patching EXTRA_PATCH_FILES_FIRST
- Python patching: use temp file for patching rejects; clean it up
- new EXTRAWIFI patch generator harness; Python: mark drivers as autogen, don't split or parse them, read as bytes, apply bytes directly
- new EXTRAWIFI patch generator harness; somewhat-works, but patches are per-family
- Python patching: add the `cache/patch` as a mountpoint
  - Darwin Docker performance is 20x with namedvolume; the cached patches are very large
 2023-02-18 07:40:55 -03:00
Ricardo Pardini
97f6836705
armbian-next: Python patching delusion, pt4: series & better markdown 
- Python patching: hopefully better Markdown: split series dir; detect more status/problems; arche only for missing Subject
  - Python patching: archeology only for missing Subject:, not description; clarify CLI pushing
  - Python patching: use `{file_base_name}.patch` for archeology as `file_name` might include a dir
- Python patching: fix: don't skip board/target patches for no reason
- Python patching: fix for series.conf patches (sunxi/sunxi64): don't sort; mark as series and show on summary
- Python patching: don't fail if archeology found no commits (but add warning)
- Python patching: CLI command `kernel-patches-to-git` with archeology summary, and git pushing of results & summary
  - Python patching: patches-to-git small fixes, auto-push if it's rpardini
  - Python patching: add `patch-kernel` CLI command
- Python patching: commit README.md & gh-pages workflow when apply_patches_to_git
  - Python patching: hopefully better markdown
- Python patching: `git add` everything all at once, for speed
 2023-02-18 07:40:54 -03:00
Ricardo Pardini
2c0e9182ed
armbian-next: introduce ORAS-based kernel git bare tree seeding/bundles 
- armbian-next: fixes to ORAS-related logging
- armbian-next: actually enable ORAS-based kernel git bare tree seeding/bundles; enable cleaning of bundle articfacts after confirmed working
- armbian-next: introduce ORAS-based kernel git bare tree seeding/bundles (although it is a .tar, not a bundle); this is 20x faster than cloning
- armbian-next: introduce `ORAS` tooling; pull and push functions & downloader/launcher
 2023-02-18 07:40:53 -03:00
Ricardo Pardini
76e276c6a9
armbian-next: Python patching delusion, pt1 & pt2 & pt3 
- WiP: Python patching delusion, pt 1: finding & parsing patches; apply & git commit with pygit2; Markdown summaries (also for aggregation); git-to-patches tool
  - Python: Markdown aggregation and patching summaries; collapsible; SummarizedMarkdownWriter
  - Python: Markdown aggregation and patching summaries
  - Python: reorg a bit into common/armbian_utils; define the `ASSET_LOG_BASE` in preparation for Markdown delusion
  - Python patching: initial apply patches & initial commit patches to git (using pygit2)
  - Python patching: add basic `series.conf` support
  - Python patching: force use of utf-8; better error handling; use realpath of dirs
  - Python patching: `git-to-patches` initial hack. not proud. half-reused some of the patches-to-git
  - Python patching: "tag" the git commits with info for extracting later; introduce REWRITE_PATCHES/rewrite_patches_in_place
  - Python patching: commented-out, recover-bad-patches hacks
  - Python patching: shorten the signature
  - Python patching: allow BASE_GIT_TAG as well as BASE_GIT_REVISION
  - Python patching: git-archeology for patches missing descriptions; avoid UTF-8 in header/desc (not diff)
  - Python patching: use modern-er email.utils.parsedate_to_datetime to parse commit date
  - Python patching: unify PatchInPatchFile; better git-commiting; re-exporting patches from Git (directly)
  - Python patching: switch to GitPython
    - GitPython is like 100x slower than pygit2, but actually allows for date & committer
    - also allows to remove untracked files before starting
  - Python aggregation: fix missing `AGGREGATED_APT_SOURCES_DICT`
  - Python patching: add `unidecode` dependency to pip3 install
  - Python patching: don't try archeology if SRC is not a Git Repo (eg, in Docker)
  - Python patching: don't try archeology if not applying patches to git
- WiP: Python patching delusion, pt2: actually use for u-boot & kernel patching
  - Python patching: much better problem handling/logging; lenient with recreations (kernel)
  - Python patching: don't force SHOW_LOG for u-boot patching
  - Python patching: don't bomb for no reason when there are no patches to apply
  - Python patching: fully (?) switch kernel patching to Python
  - Python patching: more logging fixups
  - Python patching: capture `kernel_git_revision` from `fetch_from_repo()`'s `checked_out_revision`
  - Python patching: fully switch u-boot patching to Python
  - Python aggregation/patching: colored logging; patching: always reset to git revision
  - Python aggregation/patching: better logging; introduce u-boot Python patching
- Python patching pt3: recovers and better Markdown
  - Python patching: detect, and rescue, `wrong_strip_level` problem; don't try to export patches that didn't apply, bitch instead
  - Python patching: Markdown patching summary table, complete with emoji
  - Python patching: include the problem breakdown in Markdown summary
  - Python patching: sanity check against half-bare, half-mbox patches
  - Python patching: try to recover from 1) bad utf-8 encoded patches; 2) bad unidiff patches; add a few sanity checks
  - Python patching: try, and fail, to apply badly utf-8 encoded patches directly as bytes [reverted]
  - Python patching: try to recover from patch *parse* failures; show summary; better logging
      - set `GIT_ARCHEOLOGY=yes` to do archeology, default not

- armbian-next: Python `pip` dependencies handling, similar to `hostdeps`
  - same scheme for Dockerfile caching
  - @TODO: still using global/shared environment; should move to a dir under `cache` or some kinda venv
- WiP: add `python3-pip` to hostdeps; remove `python-setuptools`
  - remove `python-setuptools` (Python2, no longer exists in Sid) from hostdeps
  - add `python3-pip` to hostdeps; part of virtualenv saga
- WiP: split `kernel.sh` a bit, into `kernel-patching.sh`, `kernel-config.sh` and `kernel-make.sh`
  - `advanced_patch()`: rename vars for clarity; no real changes
- Python patching: introduce FAST_ARCHEOLOGY; still trying for Markdown links
 2023-02-18 07:40:52 -03:00
Ricardo Pardini
84c0bf0f7d
armbian-next: kernel: use git worktree & kernel.org clone bundles for kernel working copies 
- now we've a single `.git` for all kernels with all branches; a "master" cache
- back to full stable git bundle usage from kernel.org
- bye bye, "gitballs"
- split kernel-related git stuff into `kernel-git.sh`
- introduce `ARMBIAN_MOUNTPOINTS_DEPRECATED` @TODO actually use for cleaning non-Docker
- add `axel` hostdep for multi-conn download of bundle, which comes from a CDN.
- deprecate old mountpoints for gitballs and linux-kernel
- move cleaning of old sources/kernel to mountpoints code
 2023-02-18 07:40:38 -03:00
Ricardo Pardini
b2d02071bd
armbian-next: better ccache logs/stats/Docker-support/use in u-boot/kernel 
- introduce `SHOW_CCACHE=yes` for detailed ccache statistics and logging
- fix: kernel build is done under "env -i", so pass CCACHE_DIR down to Make if CCACHE_DIR is set...
- split from kernel.sh, show stats also for u-boot targets; show compile time
- add volume definition (under `${SRC}/cache/ccache`); auto-use that in Docker builds via `CCACHE_DIR`
- better logging for `do_with_ccache_statistics()`
- there's some CCACHE_DIR code from before; unify @TODO
 2023-02-18 07:40:19 -03:00
Ricardo Pardini
da0b3080a0
armbian-next: distcc: optionally using distcc for kernel build 2023-02-18 07:40:17 -03:00
Ricardo Pardini
d502402c1a
armbian-next: show more complete gcc versions in logs 2023-02-18 07:40:04 -03:00
Ricardo Pardini
5e55c61bfc
armbian-next: kernel: add hooks fetch_sources_for_kernel_driver and patch_kernel_for_driver to allow migrating EXTRAWIFI stuff to extensions 2023-02-18 07:39:39 -03:00
Ricardo Pardini
188ef0b1af
armbian-next: kernel: run kernel's make through unbuffer as well 2023-02-18 07:39:15 -03:00
Ricardo Pardini
95e0b9d484
armbian-next: remove warm/cold bundles from fetch_from_repo, replace with 2 invocation-specific hooks and gitballs (.tar of .git) 
- drop experiment 'caches/gitbundles' and its code; all lives now in separate repo
- change LINUXSOURCEDIR **again**, cleanup the old one
 2023-02-18 07:38:56 -03:00
Ricardo Pardini
0aa57b5ef5
armbian-next: do not exclude 'CC' messages from kernel make 2023-02-18 07:38:55 -03:00
Ricardo Pardini
8511061d67
armbian-next: kernel build: always add container (series/patch-dir) mtime as minimum mtime for patch results 
- this avoids snafu's when removing patches from the directory: the file won't exist, but the parent mtime will have been updated
 2023-02-18 07:38:54 -03:00
Ricardo Pardini
b93965eca7
armbian-next: kernel: config: make olddefconfig (not oldconfig, which is interactive) when KERNEL_CONFIGURE=yes 2023-02-18 07:38:53 -03:00
Ricardo Pardini
b25b3cf499
armbian-next: lib changes - MEGASQUASH - squashed changes from c9cf3fc241cfb4c872f4aef7bbc41d5854db7ea3 to 6809de3d6063cb041205a8318e19da6a4dee68c9 ref extensions_08_10_2022_pre_v30 
- also compile.sh
- shellfmt lib
- split off shell and python tools under lib
- revert removal of stuff a-n no longer uses (ref. compilation): general packaging, mkdeb etc
- editoconfig split off
- extension changes split off
- sources and sources/families split off
- some undue stuff removed or split

armbian-next: manual merge (30) of lib changes between 882f995e21 and 31ac6383e1

armbian-next: manual merge (30) of family/board changes between 882f995e21 and 31ac6383e1

armbian-next: manual merge (29) of family/board changes between 3435c46367 and 882f995e21 (A LOT!)

armbian-next: manual merge (29) of lib changes between 3435c46367 and 882f995e21 (A LOT!)

armbian-next: manual merge (28) of lib changes between revisions af6ceee6c5 and 38df56fbf3

armbian-next: manual merge (28) of sources/families changes between revisions af6ceee6c5 and 38df56fbf3

armbian-next: manual merge (27) of `lib` changes between revisions 9c52562176 and af6ceee6c5

armbian-next: manual merge (27) of `sources/families` changes between revisions 9c52562176 and af6ceee6c5

armbian-next: move `ROOTFSCACHE_VERSION` resolution from GitHub from `main-config` down to `create-cache`

- this way config does not depend on remote...

armbian-next: move `ARMBIAN_MIRROR` selection (network) from `main-config` to `prepare-host`

- this way CONFIG_DEFS_ONLY can run without touching the network

armbian-next: manual merge (26) of MD5-checking via debsums (3955) re-imagined

- @TODO make sure

armbian-next: manual merge (26) of sources/families changes between revisions 20ee8c5450 and 9c52562176

armbian-next: manual merge (26) of lib changes between revisions 20ee8c5450 and 9c52562176

- @TODO NOT including the md5/debsums check, that needs further rewrite

armbian-next: manual merge (25) of lib changes between revisions fe972621c6 and 20ee8c5450

- @TODO hmm Igor is now going out to the network for rootfs cache version during configuration phase!!! BAD BAD BAD

armbian-next: manual merge (25) of family changes between revisions fe972621c6 and 20ee8c5450

armbian-next: manual merge (24) of families changes between revisions 9ca9120420 and 560531a635

armbian-next: manual merge (24) of lib changes between revisions 9ca9120420 and 560531a635

armbian-next: manual merge (23) of all changes between revisions 17b4fb913c and 9ca9120420

armbian-next: manual merge (22) of all changes between revisions 0eb8fe7497 and 1dddf78cd0

- @TODO EXCEPT the insanity about locales/eval/VERYSILENT in #3850, requires deep review

armbian-next: manual merge (21) of all changes between revisions e7d7dab1bb and 0eb8fe7497

armbian-next: fix: patching CREATE_PATCHES=yes

- needed to create output dir

armbian-next: add `python2-dev` dep for old uboots

- cleanup some comments

armbian-next: manual merge (20) of all changes between revisions 6b72ae3c86 and 247c4c45fd

armbian-next: fix: pass `TERM` to kernel's make, so `make menuconfig` can work

armbian-next: fix: git: read commit UNIX timestamp/local date correctly

- `checked_out_revision_ts` was correct; git outputs `%ct` as a UNIX timestamp, UTC-based
- `checked_out_revision_mtime` was incorrect: git output it without converting to local time
- manually convert using `date @xx` so it has correct local time, whatever it is.
- add debugging to `get_file_modification_time()` too

armbian-next: abstract `$QEMU_BINARY` to `qemu-static.sh`: `deploy_qemu_binary_to_chroot()`/`undeploy_qemu_binary_from_chroot()`

- add hackish logic to avoid removing binary that would be needed if image actually contains `qemu-user-static` package

armbian-next: fix `uuidgen` basic dep check; use fake bash `$RANDOM` if uuidgen not available

- not good: we need uuidgen to begin logging, but it may not be installed yet. workaround.

armbian-next: retry 3 times download-only also for `PACKAGE_LIST_BOARD`

- acng is really not helping

armbian-next: allow customizing UBUNTU_MIRROR (ports mirror) with `CUSTOM_UBUNTU_MIRROR_ARM64=host/path`

armbian-next: WiP: kernel make via `env -i` for clean env; show produced /boot tree

armbian-next: manual merge (19) of all changes between revisions b23498b949 and e621d25adc

- the ssh firstrun revert stuff mostly

armbian-next: *breaking change* remove `LIB_TAG` and `.ignore_changes` completely

- one day should be replaced with an "update checker" extension, or even "update-enforcer"
- for now this just causes chaos

armbian-next: `python2` is required for some u-boot builds

- would be "use `python-is-python2` so /usr/bin/python exists and points to Python 2.x" but Jammy does not have that anymore
- python2 is required for some u-boot builds.
- that said, python 2.x is deprecated for a while and needs work thus @TODO

armbian-next: bump Python info gatherer to RELEASE=jammy too

armbian-next: add `KERNEL_MAJOR_MINOR` info to `media` kernel (@balbes150)

- 5.18 is not yet released so might be a problem here

armbian-next: allow to skip submodules during `fetch_from_repo`; introduce hook `fetch_custom_uboot`

- via GIT_SKIP_SUBMODULES=yes, which disables all submodules everywhere
- via UBOOT_GIT_SKIP_SUBMODULES=yes, which disables fetching of submodules during uboot fetch (hidden rkbins anyone?)
- extension hook `fetch_custom_uboot` so we can fetch our own stuff if needed

armbian-next: `initrd` caching fixes (always enable hook; if cache hit, convert to uImage too)

armbian-next: introduce `initramfs`/`initrd` caching

- using hashes of (hopefully) all involved files
- cache hits are rewarded with sprinkly hearts.
  - why? this proves we got a reproducible kernel modules build!
  - also, you just saved yourself 2-10 minutes of pain

armbian-next: manual merge (18) of changes between revisions 08cf31de73 and c8855aa08d

- heh; most bash code changes are for things already done in -next, or no longer used
- some version bumps, etc

armbian-next: cleanup entrypoint and shuffle `prepare_host_basic()` into logging section

armbian-next: *breaking change* add global extlinux killswitch `ALLOW_EXTLINUX`

- unless you set `ALLOW_EXTLINUX=yes`, then `SRC_EXTLINUX` will be disabled globally.
- add a bunch of logging regarding extlinux, armbianEnv and bootscripts for clarity during build
- this is due to nand-sata-install problems with extlinux
- some boards _only work_ with extlinux; we'll have to handle it later

armbian-next: extensions: `image-output-{qcow2|ovf}`: virtual output formats

- which use `qemu-utils` for `qemu-img` conversion of the .img

armbian-next: extension: `kernel-localmodconfig`: faster/slimmer kernel builds with `make localmodconfig`

armbian-next: extension: `cleanup-space-final-image`: zerofree, slim down firmware, show used space

armbian-next: introduce `do_with_ccache_statistics` and use it for kernel compile

- some TODOs
- better logging for .config copying

armbian-next: *breaking change* really disable apt sources for non-desktop builds

armbian-next: fix: don't manage apt-cacher-ng if told NOT to, not the other way around

armbian-next: `JUST_UBOOT=yes` + hooks `build_custom_uboot()`/`post_write_uboot_platform()`

- post_write_uboot_platform()
  - only runs during build, for now (not on device)
- build_custom_uboot()
  - allow fully custom, extension driven, building of u-boot
  - also partial preparation of uboot source combined with default Armbian build
- HACK: u-boot: downgrade some errors to warnings via KCFLAGS
- fix copy of atf bins to uboot, don't do it if atf's not there

armbian-next: fix: no use testing the host for resolvconf if we're manipulating the SDCARD

armbian-next: sunxi_common: avoid shortcircuit error on family_tweaks_bsp when family_tweaks_bsp_s is not defined

armbian-next: fix: add `zstd` and `parallel` to hostdeps

armbian-next: manual merge (17) of all changes between revisions 64410fb74b and 08cf31de73

- changes about `git safe dir` ignored, I've done the same in a different way
- hash calculation changes ignored, fasthash is completely different

armbian-next: add `crossbuild-essential-armel` so `arm-linux-gnueabi-gcc` is available with system toolchains

- need to for some ATF builds, at least.

armbian-next: rockchip64_common: lotsa logging and debugging

- supposedly no practical changes

armbian-next: grub: better logging

armbian-next: fix for chaos caused by git's fix of CVE-2022-24765 otherwise "fatal: unsafe repository"

- might not be the best solution, but it's the only one I found

partitioning: fix: don't try fixing a bootscript that's not there

- this fixes a bug when "rootpart=2" without rootpart 1 being /boot

armbian-next: cleanups: umount tmpfs-based $SDCARD during cleanup too

armbian-next: indented heredoc, no functional changes

armbian-next: fix shortcircuit as last statement in case of extlinux

- yes, I wasted 3 hours on this tiny bit, so *you* don't have to!
- better logging for rootfs `mkfs` et al
- introduce `PRESERVE_SDCARD_MOUNT=yes` to preserve SDCARD, MOUNT, and LOOP for debugging

armbian-next: kernel-headers: less verbose, trimmed down tools a bit (perf and testing)

khadas-vim3l: add asound.state for Khadas VIM3L

armbian-next: introduce hook `extension_finish_config()` - late hook for ext configuration

- `extension_finish_config()` is the last thing done in config phase
    - use it for determining stuff based on kernel version details, package names, etc
- also tune down some logging which was too verbose
- CI logs with no ANSI escape codes

armbian-next: shuffle around code and logic of `add_desktop_package_sources()`

- @TODO: still needs proper asset logging for sources.list(.d)
- @TODO: tunes down adding of sources/packages to CLI builds, check with Igor

armbian-next: 4.x can't build objtool in kernel-headers; allow for handling that later

- 4.x has a lot more obtuse dependencies
- introduce KERNEL_HAS_WORKING_HEADERS calculated based on KERNEL_MAJOR_MINOR

armbian-next: downgrade `error=misleading-indentation` to warning

- some 4.x kernels patches are really messy
- newer gcc's make that an error now

armbian-next: *allow cross compilation*, even the so-called "reverse cross-compile" (amd64 on arm64)

armbian-next: add `zfs` extension, which installs headers and builds ZFS via DKMS in chroot

- similar to how `nvidia` extension does it

armbian-next: x86: enable `nvidia` extension for all releases (only desktop)

armbian-next: `headers-debian-byteshift.patch` is dead; long-live cross-compiled source-only kernel-headers

- kernel-headers package now only includes _sources_
- postinst does the compilation and preparation for DKMS compatibility
- `tools` dir is included now, which includes the byteshift utilities
- handle special scripts/module.lds case after 5.10
- tested on a 6 combinations of `x86` / `arm64` / `armhf` (3x targets, 2x hosts)
- @TODO: we might be able to reduce the size of tools a bit (perf/tests/etc)
- @TODO: still missing ARCH vs ARCHITECTURE vs SRC_ARCH clarity elsewhere

armbian-next: allow `use_clean_environment=yes` for `chroot_sdcard_apt_get()` and descendants

- this causes command to be run under `env -i`, for a clean environment

armbian-next: manual merge (16) of all changes between revisions be9b5156a4 and 2a8e1ecac1

- many `traps` ignored: we don't use them anymore

armbian-next: fix logging for apt sources/gpg keys

armbian-next: don't leak `if_error_xxx` vars across runner helper invocations; always clean then (even if no error)

- also: fix wireguard-tools install, had a double parameter there

bcm2711: rpi4b: add `pi-bluetooth` which provides working Bluetooth

armbian-next: fixes for (non-)logging when interactively configuring kernel (`KERNEL_CONFIGURE=yes`)

armbian-next: move `lz4` rootfs caches to `zstd`, multithreaded

armbian-next: customize.sh: error handling, do not mount overlay if it doesn't exist

armbian-next: extra info for runners; `if_error_detail_message` and `if_error_find_files_sdcard` globals

- those are unset after running any command
- if error occur, message and/or found files will be included in log, for clarity

armbian-next: manual merge (15) of all changes between revisions 0f7200c793 and 101eaec907

armbian-next: better logging for `rsync` calls everywhere

- make rsync verbose

armbian-next: downloads: skip download if no `ARMBIAN_MIRROR` nor `DOWNLOAD_MIRROR`; less logs

armbian-next: update rockchip.conf from master and use runners

armbian-next: update mvebu64.conf from master and use functions

armbian-next: git: fix `fetch_from_repo` with actual submodules usage

armbian-next: `armbian-next`ify the `nvidia` extension after rebase from master

- driver version is configurable via `NVIDIA_DRIVER_VERSION`
- use runner function to log/error-handle/use apt cache/etc

rpi4b: there's no legacy branch anymore, remove it from KERNEL_TARGET

armbian-next: `download_and_verify` non-error handled; logging is messy [WiP] [HACK]

armbian-next: logging: let ANSI colors pass to logfile; CALLER_PID instead of BASHPID in subshell

armbian-next: enable HTTPS CONNECT in Armbian-managed apt-cacher-ng configuration

- PPAs require it

armbian-next: don't loop forever if we can't obtain ARMBIAN_MIRROR from redirector

- also, don't even try to do it if `SKIP_ARMBIAN_REPO=yes`

armbian-next: manual merge (14) of all changes between revisions 13469fd8a9 and 09e416e31c

- also editorconfig and compile.sh (root) changes

armbian-next: *much* improved logging to HTML; log archiving; consistency

- keep only current logfile
- log to LOGFILE also if SHOW_LOG=yes
- log cmd runtime and success/error directly in runner

armbian-next: *breaking change* use `MemAvailable` (not including swap) and up requirements for tmpfs

- of course add debugging logs
- rename vars
- should really only use this if we've really a lot of completely free RAM to spare
- otherwise OOM killer comes knocking
- or swapping to disk, that is counter-productive

armbian-next: *breaking change* `DEB_COMPRESS=none` by default if not running in CI/GHA

armbian-next: *breaking change* `CLEAN_LEVEL=make` is no more; new `make-kernel`, `make-atf`, `make-uboot`

- allows individual control of what to clean
- this effectively disables `make clean` by default
- rebuilds work and timestamping works for patching, so no reason to clean everytime by default.

armbian-next: refactor `prepare_host`, give `apt-cacher-ng` some much needed attention

- library dir for host-related stuff, pull it out of "general" finally

armbian-next: hostdeps: all toolchains via `crossbuild-essential-arm64`/`armhf`/`amd64`

- trying to sort out hostdeps for Jammy [WiP]

armbian-next: remove `eatmydata` usage, leftover from failed tries to make git faster

armbian-next: fix git origin check, recreate working copy if origin does not match

- fix cold bundle https download progress reporting

armbian-next: finally consolidating logs into output/logs; colorized HTML logs

armbian-next: introduce `do_with_retries()` and use it for apt remote operations during image build

armbian-next: another round of logging tuning/fixes; log assets; git logging

- introduce `do_with_log_asset()` and `LOG_ASSET=xxx`
- separate "git" logging level
- add `trap_handler_cleanup_destimg()` to cleanup DESTIMG

armbian-next: kernel: use parallel compressors; reproducible kernel builds

- also remove leftover mkdebian/builddeb parameters in make invocation
- add pbzip2 to hostdeps

armbian-next: tuning logging for timestamp/fasthash related stuff which is very verbose

- idea is to not completely overwhelm `SHOW_DEBUG=yes` case
- make patching quieter and use file instead of stdin
- set checked_out_revision_ts during git checkout (timestamp version of _mtime)
- timestamp | fasthash logging level (via `SHOW_FASTHASH=yes`)

armbian-next: completely remove mkdebian/builddeb/general-packaging kernel packaging stuff

armbian-next: manual merge (12) of all changes between revisions 34d4be6b7b and 5fe0f36aa8

armbian-next: introduce `PRESERVE_WORKDIR=yes` for heavy debugging

armbian-next: packaging linux-headers again

- do NOT use any output from `make headers_install` - that's for libc headers
- grabs "headers" (and binary tools) directly from the kernel build tree, not install target
- does not produce headers if cross compiling, for now
- produces kernel-headers package for the architecture on which it was built
- doing a single make invocation with build and install for packaging
  - using 'make all' in place of vmlinuz/bzImage/image/zImage

armbian-next: apt download-only retried 3 times before installing main packages

armbian-next: fix `VER=` read from kernel-image package, also add `linux` default

armbian-next: some logging for atf compilation

armbian-next: rewrite hostdeps as array, add armhf toolchains

armbian-next: distro-agnostic: cleanups

armbian-next: armbianEnv hooks/debugs (bsp / image)

armbian-next: rpi: completely remove dtb hacks, allowing flash-kernel to work again

armbian-next: refactor new kernel packaging; add linux-dtb package back in finally, upgrades untested

armbian-next: refactor new kernel packaging; extract hook helper, fix kernel symlink

armbian-next: refactor new kernel packaging; add linux-dtb package back in finally, all hooks untested

flash-kernel: fix short-circuits as last statement in functions

armbian-next: do not force `SHOW_LOG=yes` if `CI=true`; let's _trust_ logging and error handling works

armbian-next: back out of setting mtime to the revision time during git checkout.

- of course this causes huge recompiles when wanted revision moves forward

armbian-next: sync 'config' dir from master revision ed589b248a

- this is _also_ getting out of hand... gotta merge soon

armbian-next: sync 'packages' dir from master revision ed589b248a

armbian-next: manual merge (11) of all lib/*.sh changes between revisions 3305d45b81 and ed589b248a

armbian-next: more refactorings, general logging; fixes; split image stuff

- logging flowing correct to LOGDIR, still needs packaging

armbian-next: complete removal of usages of `LOG_SUBPATH`; 100% error handled

- loose ends, use new LOGDIR
- remove the last shortcircuit in extensions execution, now it's 100% error handled
- many logging fixes
- still missing: final log consolidation/cleanup

logging: blue groups if `SHOW_DEBUG=yes` or `SHOW_GROUPS=yes` (console equivalent of CI's avocado)

armbian-next: shut down some too-verbose logging: logo building and update-initramfs

armbian-next: git/patching, kernel: use date from git as mtime minimum for patched files

- use revision's date from git log as mtime for all fetch_from_repo
- fix patched files date at least checkout date, otherwise some patches never build

armbian-next: first attempt at kernel packaging rework; just linux-image pkg, no dtbs yet

- correctly predict KERNELRELEASE, put image-dtbs in the right spot for flash-kernel
- remove dpkg-gencontrol, do it all directly

armbian-next: rework kernel source packaging, now exporting from git, to .tar.zst

- compress with zstdmt (multi-thread zstd), remove pv indicator, it's much faster anyway
- export from git (soon will have all patches et al too)
- better logging, show pkg name
- much, much faster due to zstdmt and deb with none compression

armbian-next: a bit atrocious, nameref loaded, `get_list_of_all_buildable_boards()`

- in the process, add support for userpatches/config structure mirroring core, for easy upstreaming

armbian-next: make `SKIP_EXTERNAL_TOOLCHAINS=yes` default. lets see what breaks [WiP]

armbian-next: keeping stdout clean, use display_alert() for cleanup logging

armbian-next: library cleanups; remove `build-all.sh` for good; bring `patching.sh` back

armbian-next: `interactive_desktop_main_configuration()` and stderr'ed + error handl'ed dialog

- use redirection to fd 3 for dialog, now cleanly on stderr
- `show_menu()` -> `dialog_menu()` et al
- interactive configuration now works again!

armbian-next: logging: `SHOW_PIDS=yes`

armbian-next: refactor and error-handle most of desktop configuration, incl menus/dialog

- `dialog_if_terminal_set_vars()` in place of `dialog_if_terminal()`

[WiP] ddk stuff, allow if not in `$KERNEL_TARGET`

armbian-next: split `compile_kernel()` function into smaller functions (+logging)

- `do_with_logging()` galore, much better error reporting for kernel
- `do_with_hooks()` is for the future, just a placeholder for now

armbian-next: `do_with_hooks()` placeholder for future ideas

armbian-next: logging: small refactor and `do_with_logging` admit it does not do error control

armbian-next: fix: traps: `trap_manager_error_handled` is integer (`-i`) not array (`-a`)

armbian-next: sunxi-tools: fix logging for sunxi-tools compilation

armbian-next: runners now run bash with `-o pipefail` in addition to `-e`

- attention, only affects stuff run through the functions in runners.sh

armbian-next: kernel: reduce logging clutter (CC,LD,AR)

- hide fasthash_debug under `SHOW_FASTHASH`

armbian-next: `armhf` should make `zImage` -- or should it?

armbian-next: show logs through ccze; avoid ANSI escapes in file; `SHOW_xxx` control

- `SHOW_DEBUG` shows the debug level
- `SHOW_COMMAND` shows all invoked commands through the runners
- `SHOW_TRAPS` to show 'cleanup' and 'trap' level
- `SHOW_TIMING` to show $SECONDS but pretty
- replace hardcoded traps/cleanups logging

armbian-next: add `KERNEL_MAJOR_MINOR=x.z` to every family, manually from the `KERNELBRANCH`

armbian-next: cold/warm bundles import/download/export for fetch_from_repo

- warm remote, if present, can be exported shallow
- if warm remote bundle is present, can be imported shallow too
- fallback to cold bundle if warm not present
- export (from cold, if exists + warm) shallow bundle
- use temp clone and DATE (not rev or tag) for shallowing, WORKS!
- info JSON/CSV, include "config_ok" true/false, kernel/uboot info
  - include logs for failed configs too
  - core reports ARMBIAN_WILL_BUILD_KERNEL and ARMBIAN_WILL_BUILD_UBOOT now with package names

armbian-next: `KERNELDIR` is out, `KERNEL_MAJOR_MINOR` is in for all `meson64`, `rpi4b` and `uefi`

armbian-next: new kernel workdir layout: cache/sources/kernel/<ARCH>-<KERNEL_MAJOR_MINOR>-<LINUXFAMILY>

- `GIT_FIXED_WORKDIR` is used to ignore 2nd param and use a specific dir
- this now REQUIRES `KERNEL_MAJOR_MINOR` to be set.
- prepare some `WARM_REMOTE_NAME` and related, based on it

armbian-next: JUST_KERNEL=yes (opposed to KERNEL_ONLY=yes) is really just the kernel build

armbian-next: fetch_from_repos now done when actually compiling atf/uboot/kernel, not before

- lib regen after removing empty files (sources.sh and errors.sh are now gone)

armbian-next: linux: back to Torvalds bundle, no tags; reminder about export

armbian-next: full cached kernel build; refactor all make's in a single place, even for packaging

- 2nd+ runs build in less than a minute
- kernel: compile and package in a single step, more efficient?
- KERNEL_BUILD_DTBS yes/no to build or not dtbs, replaces KERNEL_EXTRA_TARGETS
- dtbs_install, modules_install and headers_install now called by Armbian, not packaging
- kernel with split, but identical, build and install makes for modules/headers/dtbs
- make mkdebian and builddeb idempotent as possible
- keep a lot more cache, specially 'debian' folder
- filtering logging of install stuff
- might be a few leftovers, revisit gains with packaging later
  - keeping the arm64 makefile Image.gz vs Image hack
  - fix order of packaging patch byteshift, but still there
  - cleaning of scripts tools on cross compile removed (!)

armbian-next: minor stylistic changes that I end up doing while working on other stuff

- I am `OFFLINE_WORK`'ing, I don't wanna waste 3 seconds, thanks
- OCD-fix of double `local` declarations

[giga-wip] rework patching, introducing fasthash

[wip] git: experiment with stable kernel bundle, and all tags. nice, but for what?

- also: never delete working copy, exit with error instead.

[wip] disable make clean during packaging. I wanna rebuild fast, always [NO PR?]

armbian-next: export CHOSEN_KERNEL_WITH_ARCH for reporting info

- fix info gathering, parse all boards first, and stop if some failed
- fix KERNEL_TARGET regex by including optional "export "
- add export from info to CSV, very basic stuff, but works

[squash] remove ddk bullshit from KERNEL_TARGET

armbian-next: remove file logging of aggregation stuff. config phase can't touch disk anymore.

[WiP] git cold bundle; no: shallow clones/fetched; yes: locally packed repo

armbian-next: reorder functions in file, they have a ~logical call-tree order

armbian-next: move `fingerprint_image()` out of `git.sh` into its own file

logging: fix for double stderr redirect during `fakeroot_dpkg_deb_build`

logging: subdued "command" logging for debugging low level cmd invocations ("frog")

armbian-next: when showing log, emit all host-side invocations as `debug` too.

[WiP] trap handler abstraction, somewhat works!

armbian-next: manual merge (10) of all lib/*.sh changes between revisions a4ae3a2270 and 3305d45b81 - but NOT the git unshallow stuff, that will be rewritten

armbian-next: trapmanager pt1, identifying spots for trap manager intervention

armbian-next: `install_pkg_deb` -> `install_host_side_packages` which is completely rewritten version

- much simplified; compare installed packages vs wanted, and only update if some missing

armbian-next: force u-boot and kernel's gcc to output colors to make easy to spot warnings and errors

docker: pass the `CI` env var through Docker invocation, for GitHub Actions

armbian-next: avoid warning if "file" utility not installed

- should not happen, but better safe than sorry

armbian-next: disable long_running `pv` progress bar for custom case too

- will rework later, for now pipe causes subshell and caos

armbian-next: if `CI=true` then `SHOW_LOG=yes` always

docker: add arm64 toolchain to Dockerfile; warn, but don't break, on modprobe failure

armbian-next: docker: use ubuntu:rolling, fix deps, use `$SRC/cache` as container's cache dir

armbian-next: logging fixes (padding, don't show error more than once, don't remove trap)

armbian-next: fixes for early error handling and logging

- split stacktrace-related functions into their own lib file
- simplify the traps
- some stacktrace formatting for readability

armbian-next: fix: don't `trap` `ERR` twice, it causes bash to go bananas regarding `caller`

armbian-next: `UPSTEM_VER` -> `UBOOT_REPO_VERSION` and related fixes

armbian-next: oops, fix some non-lib changes I missed, up to revision ff4346c468

armbian-next: manual merge (9) of all lib/*.sh changes between revisions 3b7f5b1f34 and ff4346c468

armbian-next: more error handling fixes. avoid shortcircuits.

- store previous error message in `MSG_IF_ERROR` (still to be actually shown somewhere during error)

armbian-next: more error handling fixes. avoid subshells, shortcircuits, and pipes

- add `CFLAGS=-fdiagnostics-color=always` to kernel compile; would need also for u-boot soon

WiP: indexing JSON into OpenSearch, all-hardcoded version

rpi: add DTB symlink in Debian/Ubuntu standard location /lib/firmware/$version/device-tree; remove build-time-only hacks

- this allows us to remove the most horrible hack
- should allow for correctly working DTB upgrades
- should NOT impact other families, although a new symlink will be created, nothing uses it.

rpi: fix: flash-kernel fix to ignore kernel 'flavour' for all raspi's

armbian-next: don't try to remove packages that are not installed to begin with

- much faster
- new chroot_sdcard_with_stdout() runner, without bash or any escaping.

armbian-next: don't try to enable systemd services for units that don't exist

- those might be removed by a bsp extension, so check for existence before trying to enable

armbian-next: don't error/warn on failure to enable bootsplash when systemd units missing

armbian-next: use indented HEREDOCS for all call_extension_method's

armbian-next: manual merge (8) of all lib/*.sh changes between revisions 1d499d9ac2 and 3b7f5b1f34

armbian-next: manual merge (7) of all lib/*.sh changes between revisions d885bfc97d and 1d499d9ac2

armbian-next: manual merge (6) of all lib/*.sh changes between revisions c7f3c239fe and d885bfc97d

armbian-next: avoid writing to disk during configuration; `ANSI_COLOR=none` logging; make CONFIG_DEFS_ONLY=yes runnable without sudo

- when `CONFIG_DEFS_ONLY=yes`, avoid writing the config summary output.log file.
  - refactor that into a function as to be easy to if-out-of
  - don't write to disk during aggregate_content() if `CONFIG_DEFS_ONLY=yes`
  - don't write to disk during show_checklist_variables() if `CONFIG_DEFS_ONLY=yes`
  - don't write to disk during write_deboostrap_list_debug_log() if `CONFIG_DEFS_ONLY=yes`
  - don't compress and rotate logs if `CONFIG_DEFS_ONLY=yes`
- don't pretend to be handling errors we can't handle during var capture
- I foresee a world we can build all .debs without sudo
- and a some kind of split of codebase entrypoint due to that future feature
- some python info.py enhancements, not ready yet

armbian-next: shellfmt and regen library (after rebase from master n.5)

tools/shellfmt.sh: exclude "cache" and ".tmp" from formatting, for obvious reasons

tools/gen-library.sh: sort function files, so it does not keep changing between runs on different machines.

- order should not be important, since files only contain functions, but avoid git churn

armbian-next: manual merge (5) of all lib/*.sh changes between revisions 1b18df3c83 and e7962bb2b5

- most PKG_PREFIX work was already done

armbian-next: `TMPDIR` for all, many logging fixes, error handling: leave-no-garbage-behind without needing traps.

- set `MOUNT_UUID` and `WORKDIR`/`MOUNT`/`SDCARD`/`EXTENSION_MANAGER_TMP_DIR`/`DESTIMG` early in do_main_configuration()
  - but, they're just _set_ there, dirs are not created early, but on demand later
  - still @TODO: actually clean those during error trap. (unhappy path leaves garbage still)
  - but does not leave garbage behind during "successful" runs at least (happy path works)
- actually export `TMPDIR` (== `WORKDIR`) during start of build (not config!), so all `mktemp` are subject to it
  - `runners.sh` has helpers to avoid passing `TMPDIR` to chroot. Use the helpers! don't call `chroot` directly.
  - don't trap/cleanup individual `mktemp` temp dirs during .deb packaging's, all is handled at once now.
  - kernel packaging, for example, automatically picks up `TMPDIR` too. So now hosts `/tmp` is mostly left alone.
- fix some "infodumps" that are done into `.log` files directly.
- don't use sudo if `CONFIG_DEFS_ONLY=yes`; we'll only be collecting info, not doing anything.
- simpler logging for `rsync` operations (just dump to stdout, logging will handle it!)
- use padded counter for section logfiles, so we know which order things ran. exported as `CURRENT_LOGGING_COUNTER`
- no reason to use `apt-get` with `-yqq` anymore, since all logging is handled, so now `-y` by default
- desktop: using runners helpers for rootfs-desktop.sh, which should help a lot with acng caching and finding of problems
- extensions: correctly cleanup temp stuff, extensions has its own tmp/workdir now, and is always cleaned up at end of build.

armbian-next: bye `PKG_PREFIX`, hello `run_host_x86_binary_logged()` wrapper function; better error handling

- we've x86-only utilities that might need to be run on non-x86 build machines
- previously duplicated logic in PKG_PREFIX variable refactored into logged function
- added centralized debug logging
- replace all PKG_PREFIX usage with the new wrapper function, which already handles logging and errors.
  - mostly FIP tooling invocations
  - but also the boot_logo image builder
  - wrapper function delegates to common `run_host_command_logged`
- wrap other FIP invocations with `run_host_command_logged` too, for tidy logging
- avoid using conditionals when invoking functions; that completely disables error handling inside the called function
- use explicit bash opts instead of shortcuts like `set -e`
- a _lot_ of debug logging added

armbian-next: always use UPPERCASE labels for FAT32 UEFI filesystems (rpi4b, uefi-*)

armbian-next: shellfmt after rebase onto master

armbian-next: manual merge (4) of all lib/*.sh changes between revisions 23afccf56e and e610f00bc7

- plus ooops

atf: fix for `set -e` mode; fix CROSS_COMPILE quoting

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: predict the future location of .img file

- otherwise it's really unhelpful

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

uefi: alias `BRANCH=ddk` to `current`'s `DISTRO_GENERIC_KERNEL=yes`

- no real change, just to match rpi4b's BRANCH=style
- opens space for Armbian-built `current` soon

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

rpi: `legacy`->`ddk` (distro default kernel), remove overclock

- common vars in bcm2711.conf moved to top
- removed overclock/overvolt that was leftover my old setup
- confirmed: works with rpi3b too, should work with CM4/CM3 and others
- use valid UPPERCASE FAT label for RPICFG (in place of `rpicfg`)

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

armbian-next: shellfmt again after rebase

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

armbian-next: manual merge (3) of all lib/*.sh changes between revisions 1035905760 and e4e4ab0791

- missed non lib change on Several improvements for RPi builds (#3391)
- I just realized I will have to drop all non lib changes

rockship: fixes for `set -e` mode in rockship armhf family and bsp tweaks

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

armhf: enable building armhf targets on amd64 using system toolchains

- SKIP_EXTERNAL_TOOLCHAINS=yes on amd64 should use the same system toolchains as an arm64 build

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: better logging about family_tweaks and family_tweaks_bsp

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

kernel: unblock cross compilation, warn about headers package

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: fixes for sunxi/megous stuff with `set -e`

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: fix shellcheck references generation

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: manual merge (2) of all lib/*.sh changes between revisions 117633687e and 3083038855

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: renaming function files a bit more consistently

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: removing leftover empty file after all the moving around

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: really insist on set -e during library loading

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: shellfmt again after rebasing master

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: manual merge of all lib/*.sh changes between revisions f6143eff67 and f3388b9aee

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: generic do_capturing_defs wrapper; Python parser

- enabled by passing CONFIG_DEFS_ONLY=yes; in this case does not build anything
- [WiP] Python3 info reader / matrix expander
  - multithreaded version

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: move some interactive parts of config into its own functions

- mostly from config-prepare;
- there is still a lot of others in main-config

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: use chroot_custom for grub and flash-kernel extension logging

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: use line buffering, fix runner output color for GHA

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: wrap dpkg-deb; set TMPDIR (not in chroot); refactor kernel make

- And a huge amount of @TODO's added
- Add "debug" and "deprecation" `display_alert()` levels
- insist that `install_common` is now `install_distribution_agnostic`
- unrelated: realtek 8822CS is EXTRAWIFI=yes only now, sorry.
- many debug statements for desktop

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: don't bail out on patching error

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: bunch of fixes; no-stdin; traps; better stacks

- mostly no-stdin dialog handling (desktop et al)
- let ERR trap run together with unmount trap (EXIT etc)

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

logging: trap ERR very early, pass-in caller info

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

armbian-next: huge refactor, shellfmt, codesplit, logging/error handling

- *this commit changes most/all the lines of bash code in armbian/build*
- *author is set to IgorPec for historical reasons, rpardini is to blame for the actual changes*
- logging: refactorings, pt.4: autogen lib, shellfmt tool, extract cli
  - shellfmt: auto-downloader and runner of shellfmt
    - darwin/linux
    - amd64/arm64
    - find ~correct files to format
    - run formatting
    - check formatting soon
  - refactor compile's CLI stuff out into function/cli
  - gen, and use genned library with tool
- logging: refactoring pt3: HUGE split of everything into everything else
  - plus rebasing fixes
- logging: refactorings, pt. 2: error handling
  - fix: no errors during umount_chroot()
  - no progress for CI=true builds
  - WiP disable kernel hashing. too crazy.
  - a few builds now working with "set -e"
  - wtf. or something
  - kernel logging and long_running stuff - a mess - needs moving around in files
  - rewrite uboot compile loop without using subshells. remove ALL traps. refactor host command
  - better logging for u-boot
  - more fixes, u-boot
  - more fixes for logging et al
  - git stuff
  - many (many) fixes
  - new color scheme
  - a monster. make sure unmounted at the end. remove set -e's, to-be-readded.
  - remove set -e "for safety reasons"
  - more alerts. we gotta know whats failing
  - some more logging stuff and fixes for error checking
  - more logging and error handling stuff
  - fixes; some set -e's
  - more logging stuff
- logging: refactoring codebase, pt.1: functions-only
  - Refactor the hell out of everything into functions
  - rename build-all-ng to build-multi; other fixes, extensions init
  - slight nudge
  - some were already good, like this one.
  - syntax fixes
  - some need a little nudge
  - another clean one
  - some just need a better name (and splitting later)
  - syntax fixes
  - some were already good, like this desktop one
  - some were already good, like this other one
  - some were already good, like this one.
  - debootstrap is gone.
  - extract functions from compile.sh
  - add logging to main_default_build
  - more stuff
  - cleanups and refactors of main.sh
- logging: first steps
- logging: pt. 0: shellfmt everything

- add riscv64 do SRC_ARCH/ARCH/ARCHITECTURE mess; add warn
 2023-02-18 07:38:46 -03:00
Ash
408bc67619
Add shebangs for shellcheck (#4493) 
* Add shebangs for shellcheck

See #AR-1406

* Add shebangs for shellcheck

Also for `extensions` scripts
 2022-11-27 21:44:50 +01:00
Ricardo Pardini
85c8cbf15d run shellfmt on all of lib 2022-10-08 14:37:22 +02:00
hzy
445f2ddcb3 Split /lib/compilation.sh 2022-10-08 14:37:21 +02:00