Commit Graph

70 Commits

Author SHA1 Message Date
Ricardo Pardini
f3c3fe8e82 ubuntu: add mantic (23.10), symlinks to lunar 2023-10-14 09:40:45 +00:00
Ricardo Pardini
5ed338fb60 dpkg-deb: don't use fakeroot, we're already root; don't add fakeroot dep either 2023-10-13 18:04:56 +02:00
Ricardo Pardini
11df81403d config/lib/exts: run shellfmt
- `bash lib/tools/shellfmt.sh`
- mostly spacing changes that accumulated over time
2023-08-22 14:28:14 +02:00
Ricardo Pardini
64b5220853
debian: add Trixie (Debian 13, testing) csc target & build host (#5621)
- Trixie, like Bookworm and Sid, does not have python2 anymore
2023-08-22 08:14:55 +02:00
Igor
2a8cb5793f Removing exceptions needed to support riscv64 withing Debian
Signed-off-by: Igor <igor@armbian.com>
2023-08-17 22:09:05 +02:00
Ricardo Pardini
1e3d5c3567 prepare-host: check_host_has_enough_disk_space(): don't check if dir does not exist; fixes #5246 2023-06-02 19:32:40 +02:00
Ricardo Pardini
30eb5323f2 prepare-host: refactor disk space check; test "${DEST}" and "${SRC}/cache", no longer "${SRC}" 2023-05-21 16:18:35 +02:00
Ricardo Pardini
c351936cb0 binfmts: hide output when loading qemu-arm under aarch64 (some are native, some are emulated, all are good, errors can be ignored) 2023-05-15 17:43:25 +02:00
Ricardo Pardini
83e35b1473 binfmts: don't raise warning if qemu-arm failed to load under aarch64 (some are native, some are emulated, all are good)
- reported by @igorpecovnik on Discord, during massive CI build, when armhf builds landed on arm64 builders
2023-05-09 10:04:57 +02:00
Werner
0b90ebc54a Use native xz 2023-04-18 09:29:17 +02:00
Ricardo Pardini
eb1fe6d7ba artifacts: introduce PRE_PREPARED_HOST=yes: allow running pre-prepared host CLI's for artifacts that require aggregation 2023-04-15 22:43:06 +02:00
Ricardo Pardini
c30a907e47 python2: lunar also doesn't have python2 anymore (joins sid and bookworm) 2023-04-15 22:43:06 +02:00
Ricardo Pardini
ed0b406265 lib: fix: replace undue export statements with declare -g 2023-03-31 09:12:44 +02:00
Ricardo Pardini
6b965c605c completely remove any traces of KERNEL_ONLY and exit with error if set 2023-03-17 08:32:11 +01:00
Igor Pečovnik
b7b8eb7b72
Add / modify (c) in bash scripts (#4922)
* Add / modify (c) in bash scripts

Signed-off-by: Igor <igor@armbian.com>

* Add (c) to the source config files

---------

Signed-off-by: Igor <igor@armbian.com>
2023-03-09 18:30:40 +01:00
Ricardo Pardini
a2304f28b0
armbian-next: post_build_image: run hook first, then write to CARD_DEVICE, then compress, in that order
- compress *all* present output images, not only .img
- remove 7zip compression & hostdep
- squash a few shortcircuits (yeah, this late in the game...)
2023-02-18 07:46:12 -03:00
Ricardo Pardini
c73d60368a
armbian-next: remove references to stuff that doesn't exist or isn't hit by default anymore 2023-02-18 07:46:09 -03:00
Ricardo Pardini
843449597a
armbian-next: binfmts: suppress error message when trying to emulate armhf on arm64 2023-02-18 07:45:53 -03:00
Ricardo Pardini
19aa7e95cf
armbian-next: introduce assert_prepared_host() and assert before using any Python, to un-mask any further fallacies 2023-02-18 07:45:19 -03:00
Ricardo Pardini
fa666aa3d4
armbian-next: interactive: review -t 0 instead of -t 1; move non-interactive check_basic_host() from prepare-host to config-prepare, for consistency 2023-02-18 07:45:09 -03:00
Ricardo Pardini
b91d752aa6
armbian-next: prepare-host: don't manage ACNG in prepare if not asked to 2023-02-18 07:45:07 -03:00
Ricardo Pardini
6177b94268
armbian-next: logging: pure ASCII .log; included in Markdown for GHA 2023-02-18 07:44:45 -03:00
Ricardo Pardini
041297c563
armbian-next: binfmt: make sure we have all binfmt's setup & enabled for rootfs build (v2) 2023-02-18 07:44:39 -03:00
Ricardo Pardini
78dcf01c92
armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs
- killed `[[ $ROOTFS_TYPE != ext4 ]] && display_alert "Assuming ${BOARD} ${BRANCH} kernel supports ${ROOTFS_TYPE}" "" "wrn"`
  - which definitely didn't belong in rootfs
- disable usage of run_host_command_logged_long_running for kernel-make
- stop lying about long_running stuff being any different from non-long_running versions
- stop lying about 'set -e' when there's still a bunch of pipes for "pv" and stuff all around
2023-02-18 07:44:33 -03:00
Ricardo Pardini
3cac8f44a4
armbian-next: we don't really need target RELEASE defined to install host deps (for now), don't freak out about it 2023-02-18 07:44:31 -03:00
Ricardo Pardini
44ae761927
armbian-next: apt: expand local caching to include package lists, as well as .deb's
- `/var/cache/apt` in chroot was already cached;
- add `/var/lib/apt/lists`
- remove usage of namerefs, replace with global dict
2023-02-18 07:44:19 -03:00
Ricardo Pardini
2c9582f9ba
armbian-next: add Debian Ports keyring hostdep to strap Debian riscv64 from ports
- rpardini: actually use ports keyring for Debian Risc-V
- `risc-v`: correctly include `debian-ports-archive-keyring` in Risc-V CLI packages
  - include in debootstrap list, via `config/optional/architectures/riscv64/_config/cli/_all_distributions/main/packages`
  - can't have two `--include` in debootstrap call
- riscv64: move `debian-ports-archive-keyring` from main to debootstrap; also only for `sid`
2023-02-18 07:43:49 -03:00
Ricardo Pardini
da1cf2b475
armbian-next: rename too-long main function to prep_conf_main_build_single()
- this commit is proof of OCD (old name didn't fit stacktrace space)
2023-02-18 07:43:42 -03:00
Ricardo Pardini
c1100fa461
armbian-next: still fighting tee leaking under duress, turns out I _hadn't_ really won
- last resort, use lazy umount.
- include `lsof` in hostdeps, useful to debug these situations
2023-02-18 07:43:39 -03:00
Ricardo Pardini
4d59ad7ac4
armbian-next: try harder to get en_US.UTF-8 logging 2023-02-18 07:43:22 -03:00
Ricardo Pardini
8444e7e308
armbian-next: bring back swig hostdep -- needed for some u-boot's 2023-02-18 07:43:15 -03:00
Ricardo Pardini
ff94b79e60
armbian-next: do basic host checks before config if interactive, or during prepare-host if not
- group basic checks at `check_basic_host()`
2023-02-18 07:43:12 -03:00
Tony
f549ddb273
armbian-next: officially support WSL2; pester user for UTF-8 terminal
- new `wsl2.sh` file with WSL2-related functions; checking/validating/pestering user; allow WSL2 (not WSL1)

Co-authored-by: Ricardo Pardini <ricardo@pardini.net>
2023-02-18 07:43:11 -03:00
Ricardo Pardini
bf891d54d9
armbian-next: split prepare_host(), fix .tmp reset trap
- `prepare_host()`: split; do checks earlier and allow them to be interactive
- introduce `exit_if_countdown_not_aborted()` for "Low Disk Space" and other critical conditions
- split `prepare_host()` into interactive & non-interactive parts
- split off `clean_deprecated_mountpoints()` from prepare into `cleaning.sh`
- introduce and use `reset_uid_owner_non_recursive()` for `.tmp` reset in trap, to avoid disasters
- add more logging sections to default-build.sh, avoid unlogged parts
2023-02-18 07:43:10 -03:00
Ricardo Pardini
7879895049
armbian-next: core extensions for xfs / f2fs / brtfs / LUKS/cryptroot hostdeps; enable in main-config 2023-02-18 07:42:46 -03:00
Ricardo Pardini
47af820f5d
armbian-oleg: junk: drastically reduce host-side dependencies / "remove junk"
- remove a lot of hopefully / hardly confirmed, unused dependencies
- don't use crossbuild-essential-xxx; avoid the c++ compiler that comes with it, install gcc-only instead
- hostdeps: use `libc6-dev make dpkg-dev gcc` (without `g++`) instead of `build-essential`
- drop `btrfs-progs` and `f2fs-tools`  (@TODO add in extension when/whereused)
- more: drop `cryptsetup` (@TODO add in extension when/whereused)
- don't be too quiet when doing apt-update for hostdeps
2023-02-18 07:42:14 -03:00
Ricardo Pardini
93c8ce99af
armbian-next: apt-cacher-ng is now optional and activated via MANAGE_ACNG=yes; drop NO_APT_CACHER
- acng is constant source of headaches
- we don't need it anymore, since we have local caching now
- You can also have acng running on a different machine/container and set `APT_PROXY_ADDR=xxx:3142` if you're running a build farm
2023-02-18 07:42:12 -03:00
Ricardo Pardini
d4d3c1346c
armbian-oleg: split hostdeps again, full of ifs, and reduce the minimum set of pkgs for Oleg-conditions while trying to keep Docker full
- probably breaks a lot of stuff; c++ compilers, distcc, possibly others
- split into late_prepare_host_dependencies / early_prepare_host_dependencies calling adaptative_prepare_host_dependencies() full of if's

also:
- calc deps out of configuration
- reorg checks into obtain_and_check_host_release_and_arch()
2023-02-18 07:42:11 -03:00
Ricardo Pardini
1e9bd386c5
armbian-next: SYNC_CLOCK: don't attempt if ntpd is running; don't fail if ntpdate fails for any reason 2023-02-18 07:42:04 -03:00
Ricardo Pardini
6cb12a3397
armbian-next: [focal-host] try to support focal-like host OS's again, via python3.9 (and drop buster completely)
- completely remove support for building under `buster` -- that's way too old, sorry.
- de-hardcode `python3` invocations, instead use `python3_binary_path` set by `prepare_python3_binary_for_python_tools()`
- juggle `$HOSTRELEASE`: read from actual host, or determined from Docker image name (during Dockerfile build)
- TL;DR: include and use `python3.9` for focal-like host OS's
2023-02-18 07:41:47 -03:00
Ricardo Pardini
67108e31c0
armbian-next: move var HOSTRELEASE from config to prepare_host() 2023-02-18 07:41:36 -03:00
Ricardo Pardini
690ca4df7c
armbian-next: add (trap) cleanup handler to reset ./tmp and ./output to original pre-sudo UID before exiting, even with failure/abort 2023-02-18 07:41:17 -03:00
Ricardo Pardini
76e276c6a9
armbian-next: Python patching delusion, pt1 & pt2 & pt3
- WiP: Python patching delusion, pt 1: finding & parsing patches; apply & git commit with pygit2; Markdown summaries (also for aggregation); git-to-patches tool
  - Python: Markdown aggregation and patching summaries; collapsible; SummarizedMarkdownWriter
  - Python: Markdown aggregation and patching summaries
  - Python: reorg a bit into common/armbian_utils; define the `ASSET_LOG_BASE` in preparation for Markdown delusion
  - Python patching: initial apply patches & initial commit patches to git (using pygit2)
  - Python patching: add basic `series.conf` support
  - Python patching: force use of utf-8; better error handling; use realpath of dirs
  - Python patching: `git-to-patches` initial hack. not proud. half-reused some of the patches-to-git
  - Python patching: "tag" the git commits with info for extracting later; introduce REWRITE_PATCHES/rewrite_patches_in_place
  - Python patching: commented-out, recover-bad-patches hacks
  - Python patching: shorten the signature
  - Python patching: allow BASE_GIT_TAG as well as BASE_GIT_REVISION
  - Python patching: git-archeology for patches missing descriptions; avoid UTF-8 in header/desc (not diff)
  - Python patching: use modern-er email.utils.parsedate_to_datetime to parse commit date
  - Python patching: unify PatchInPatchFile; better git-commiting; re-exporting patches from Git (directly)
  - Python patching: switch to GitPython
    - GitPython is like 100x slower than pygit2, but actually allows for date & committer
    - also allows to remove untracked files before starting
  - Python aggregation: fix missing `AGGREGATED_APT_SOURCES_DICT`
  - Python patching: add `unidecode` dependency to pip3 install
  - Python patching: don't try archeology if SRC is not a Git Repo (eg, in Docker)
  - Python patching: don't try archeology if not applying patches to git
- WiP: Python patching delusion, pt2: actually use for u-boot & kernel patching
  - Python patching: much better problem handling/logging; lenient with recreations (kernel)
  - Python patching: don't force SHOW_LOG for u-boot patching
  - Python patching: don't bomb for no reason when there are no patches to apply
  - Python patching: fully (?) switch kernel patching to Python
  - Python patching: more logging fixups
  - Python patching: capture `kernel_git_revision` from `fetch_from_repo()`'s `checked_out_revision`
  - Python patching: fully switch u-boot patching to Python
  - Python aggregation/patching: colored logging; patching: always reset to git revision
  - Python aggregation/patching: better logging; introduce u-boot Python patching
- Python patching pt3: recovers and better Markdown
  - Python patching: detect, and rescue, `wrong_strip_level` problem; don't try to export patches that didn't apply, bitch instead
  - Python patching: Markdown patching summary table, complete with emoji
  - Python patching: include the problem breakdown in Markdown summary
  - Python patching: sanity check against half-bare, half-mbox patches
  - Python patching: try to recover from 1) bad utf-8 encoded patches; 2) bad unidiff patches; add a few sanity checks
  - Python patching: try, and fail, to apply badly utf-8 encoded patches directly as bytes [reverted]
  - Python patching: try to recover from patch *parse* failures; show summary; better logging
      - set `GIT_ARCHEOLOGY=yes` to do archeology, default not

- armbian-next: Python `pip` dependencies handling, similar to `hostdeps`
  - same scheme for Dockerfile caching
  - @TODO: still using global/shared environment; should move to a dir under `cache` or some kinda venv
- WiP: add `python3-pip` to hostdeps; remove `python-setuptools`
  - remove `python-setuptools` (Python2, no longer exists in Sid) from hostdeps
  - add `python3-pip` to hostdeps; part of virtualenv saga
- WiP: split `kernel.sh` a bit, into `kernel-patching.sh`, `kernel-config.sh` and `kernel-make.sh`
  - `advanced_patch()`: rename vars for clarity; no real changes
- Python patching: introduce FAST_ARCHEOLOGY; still trying for Markdown links
2023-02-18 07:40:52 -03:00
Ricardo Pardini
0c5b10a77b
armbian-next: introduce SKIP_ARMBIAN_ROOTFS_CACHE which is =yes by default
- that is in addition to `SKIP_ARMBIAN_REPO` which is not set by default
2023-02-18 07:40:50 -03:00
Ricardo Pardini
fd043d707a
armbian-next: squash some warn loggings that were undue; remove $SUDO_USER handling in prepare-host.sh
- squash warning about `NO_APT_CACHER` when in Docker
2023-02-18 07:40:43 -03:00
Ricardo Pardini
84c0bf0f7d
armbian-next: kernel: use git worktree & kernel.org clone bundles for kernel working copies
- now we've a single `.git` for all kernels with all branches; a "master" cache
- back to full stable git bundle usage from kernel.org
- bye bye, "gitballs"
- split kernel-related git stuff into `kernel-git.sh`
- introduce `ARMBIAN_MOUNTPOINTS_DEPRECATED` @TODO actually use for cleaning non-Docker
- add `axel` hostdep for multi-conn download of bundle, which comes from a CDN.
- deprecate old mountpoints for gitballs and linux-kernel
- move cleaning of old sources/kernel to mountpoints code
2023-02-18 07:40:38 -03:00
Ricardo Pardini
42e26e9ade
armbian-next: hostdeps: python2/3: add setuptools (required by some u-boot configs) 2023-02-18 07:40:31 -03:00
Ricardo Pardini
a752ac8899
armbian-next: REVERT: remove the BUILD_ONLY build-tasks.sh which makes no sense in armbian-next 2023-02-18 07:40:29 -03:00
Ricardo Pardini
093336e914
armbian-next: cli: distccd launcher for build farm
- relaunched under docker
- possibly much nicer if zeroconf enabled?
- this thing is ignoring signals. can't ctrl-c out of it. why? @TODO
- pt 1, subpt 3: distccd - distcc-pump - avahi
- pt 1, subpt 3: distccd + distcc-pump
- pt 1, subpt 2: distccd + avahi
- pt 1, subpt 1: distccd
2023-02-18 07:40:18 -03:00
Ricardo Pardini
75de1c2d82
armbian-next: remove usage of REQUIREMENTS_DEFS_ONLY; initialize extensions for requirements cli command & docker pre-game deps 2023-02-18 07:40:07 -03:00