Commit Graph

63 Commits

Author SHA1 Message Date
Ricardo Pardini
a4c703dfb0 partitioning: declare global readonly ROOT_PART_UUID for later hooks
- re-use local `root_part_uuid` instead of running blkid multiple times
2023-07-18 14:21:01 +02:00
Ricardo Pardini
c5fe25aa17 partitioning: move hook call image_specific_armbian_env_ready() after the end of related manipulations
- depending on rootpart/extlinux/boot.ini it might not have run or had any effect
2023-07-04 03:09:07 +05:30
Ricardo Pardini
0a5740040e rootfs-to-image: find, warn, and remove dangling symlinks in FAT32 /boot 2023-05-24 19:23:37 +02:00
Ricardo Pardini
2e570b9473 initrd: when cache hit, don't preserve perms during deploy, otherwise FAT32 /boot gags 2023-05-21 13:59:26 +02:00
Igor Velkov
9b68c4e42a Add support for nilfs2 fs based image
Signed-off-by: Igor Velkov <iav@iav.lv>
2023-05-16 08:51:05 +02:00
Ricardo Pardini
b18cd8aaf4 logging: curb some warnings into info's 2023-05-09 10:04:57 +02:00
Ricardo Pardini
c692676da7 kernel-debs: postinst/etc: fix: unify is_boot_dev_vfat() function; make work during image build too via env var
- determinining if `/boot` is vfat is more complex than it looks;
  - use a common function to unify across all scripts
- during image build, the kernel install is always done with a non-fat, non-mounted /boot
  - use a special variable passed through the env so unified function can know ahead of time
  - introduce `extra_apt_envs` param to `chroot_sdcard_apt_get()`; array with extra environment vars
- if vfat /boot:
  - `linux-update-symlinks` is not called (Debian would just choke on this)
  - don't even try to symlink Armbian stuff, move instead
- more/better logging board-side (unfortunately mixed in all the set -x debugs)
2023-05-01 22:46:43 +02:00
Werner
945bb6dff9 Use native xz 2023-04-18 09:29:17 +02:00
Ricardo Pardini
7ea8e040d0 cli: flash: introduce flash CLI command; introduce hook post_build_image_write 2023-04-15 22:43:06 +02:00
Ricardo Pardini
d59271f903 config: refactor calculate_image_version() out of create_image_from_sdcard_rootfs(), make more flexible, predict IMAGE_FILE_ID during config
- default `IMAGE_INSTALLED_KERNEL_VERSION="generic"` instead of `"linux"`
2023-04-15 22:43:06 +02:00
Ricardo Pardini
ac25ac18be config/extensions: consolidate EXTRA_IMAGE_SUFFIXES array as _the way_ to add to image filename, during configuration
- can add to `EXTRA_IMAGE_SUFFIXES` array during `user_config` or `extension_prepare_config`
- in the end goes into a `EXTRA_IMAGE_SUFFIX` global readonly
- this  simplifies extension code
- we still can't "predict" image names during configuration since it includes the kernel version (6.2.11/generic) which only comes later
2023-04-15 22:43:06 +02:00
Ricardo Pardini
ed0b406265 lib: fix: replace undue export statements with declare -g 2023-03-31 09:12:44 +02:00
Ricardo Pardini
c8412c8791 compress-checksum: don't compress files ending in .txt, thanks 2023-03-17 08:32:11 +01:00
Ricardo Pardini
91b7b35fa3 rootfs-to-image/distro-agnostic: introduce IMAGE_VERSION, can be used to override VERSION file ($REVISION) just for image filename/issue/os-release 2023-03-17 08:32:11 +01:00
Igor Pečovnik
b7b8eb7b72
Add / modify (c) in bash scripts (#4922)
* Add / modify (c) in bash scripts

Signed-off-by: Igor <igor@armbian.com>

* Add (c) to the source config files

---------

Signed-off-by: Igor <igor@armbian.com>
2023-03-09 18:30:40 +01:00
Vyacheslav Bocharov
fad48ac670
Fix "No space left on device" while rsync root files to image 2023-02-27 22:34:17 +03:00
Jianfeng Liu
fdcabaad3f
not rsync /boot when copying root partition (#4859) 2023-02-24 23:43:32 +08:00
Ricardo Pardini
2f7423d0e4
shellfmt/.editorconfig: enable trim whitespaces, run shellfmt, no actual code changes (#4849)
- shellfmt: bump to 3.6.0 (run with `bash lib/tools/shellfmt.sh`)
- .editorconfig: rationalize, remove duplicated logic
2023-02-23 09:24:41 +01:00
Ricardo Pardini
4f36681818
armbian-next: compression: fast xz compression for images 2023-02-18 07:46:13 -03:00
Ricardo Pardini
a2304f28b0
armbian-next: post_build_image: run hook first, then write to CARD_DEVICE, then compress, in that order
- compress *all* present output images, not only .img
- remove 7zip compression & hostdep
- squash a few shortcircuits (yeah, this late in the game...)
2023-02-18 07:46:12 -03:00
Ricardo Pardini
9bffa5e749
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build
- artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes`
- artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build
- kernel_package_source() is no more
- a long dissertation about kernels, families, and the universe
- artifacts: actually use rootfs artifact for image build
- artifacts: detangle via artifact_base_dir
- artifacts: rootfs: use folders in artifact_name; include cache_type
- artifacts: some cleanups / validations
- rootfs artifact; drop old rootfs cli
- artifacts: new CLI shortcuts; remove old firmware CLI
- artifacts: full firmware & usage
- use firmware artifacts in image build and install its debs
- kernel artifact sans legacy; use tmpdir for .deb target for all packages
- legacy artifact versions is no more; pack/unpack now in common obtain;
- artifacts: uboot: cleanup legacy renaming, use artifact version directly
- artifacts: add firmware (small) artifact
- deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning
- fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash
  - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()`
  - don't use realpath for artifact_file_relative
  - curb some warnings
  - fix: only install headers & dtbs if such exist
- kernel .config hook modification hash versioning
- OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB
- artifact revolver & installing into image
  - add artifact_map_packages and artifact_map_debs dicts
  - revolver accumulates all info
  - REPOSITORY_INSTALL is no more (for uboot/kernel, later others)
  - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-18 07:46:03 -03:00
Ricardo Pardini
68d76d6f3a
armbian-next: squash some shellcheck warnings; don't local and assign subshell value in same statement, that masks errors (declare/local almost-always works) 2023-02-18 07:45:31 -03:00
Ricardo Pardini
6258ea566a
armbian-next: shellcheck: squash a bunch of 'critical' warnings 2023-02-18 07:45:29 -03:00
Ricardo Pardini
39208d1007
armbian-next: compress-checksum: move config to config-prepare; drop weight; kill warnings 2023-02-18 07:44:57 -03:00
Ricardo Pardini
f43732a8e9
armbian-next: use unified methods for tmp dir handling all around (except ATF, which is special) 2023-02-18 07:44:29 -03:00
Ricardo Pardini
72c61f2f68
armbian-next: replace bare sync invocations by introducing wait_for_disk_sync(); sync early at start of build
- wait 10s for sync call; otherwise warn user to be patient
- wait_for_sync "reason for sync" everywhere
- add @TODO for /run/user/0 etc and XDG_RUNTIME_DIR in chroot
2023-02-18 07:43:53 -03:00
Ricardo Pardini
35fbb7ed6c
armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code 2023-02-18 07:43:28 -03:00
Ricardo Pardini
af7ace7e57
armbian-next: don't leak .raw image if failure before move; show rootfs-tmpfs info
- add escapings and TODO's around the area...
2023-02-18 07:43:26 -03:00
Ricardo Pardini
1636a2afea
armbian-next: tune logging: unmounting, u-boot prep/source/install; traps; extensions 2023-02-18 07:43:25 -03:00
Ricardo Pardini
ff4b5069ce
armbian-next: extract move_images_to_final_destination() and optimize for fast-move when src/dst on same filesystem 2023-02-18 07:43:24 -03:00
Ricardo Pardini
d6200a30e6
armbian-next: make superglobals readonly (DEST/WORKDIR/LOGDIR/SDCARD/MOUNT and others) 2023-02-18 07:43:08 -03:00
Ricardo Pardini
e633ff584a
armbian-next: debug note about fstab with tmpfs in chroot 2023-02-18 07:43:03 -03:00
Ricardo Pardini
75997b0e2c
armbian-oleg: drastically curb update-initramfs logging, unless SHOW_DEBUG=yes
- also, don't let the logging filter be the reason for failures
2023-02-18 07:42:26 -03:00
Ricardo Pardini
12ffcd6319
armbian-oleg: curb rsync logging during bsp 2023-02-18 07:42:25 -03:00
Ricardo Pardini
5b024f64ad
armbian-next: curb free_loop_device_retried()'s logging on first try 2023-02-18 07:42:18 -03:00
Ricardo Pardini
8aa0ee640f
armbian-next: rootfs: don't stop if df -h fails
- still needs work
2023-02-18 07:42:05 -03:00
Ricardo Pardini
690ca4df7c
armbian-next: add (trap) cleanup handler to reset ./tmp and ./output to original pre-sudo UID before exiting, even with failure/abort 2023-02-18 07:41:17 -03:00
Ricardo Pardini
3bc8afd593
armbian-next: refactor losetup -d ... into free_loop_device_insistent() which retries 10 times 2023-02-18 07:41:14 -03:00
Ricardo Pardini
277ce40e6f
armbian-next: fix for grub's DISTRO_GENERIC_KERNEL on Debian (not Ubuntu) during update-initramfs
- `initrd`: when kernel not found, list the installed kernels
- armbian-next: small fix for `grub` extension naming
2023-02-18 07:40:34 -03:00
Ricardo Pardini
4a9529dd34
armbian-next: qemu-uboot-arm64 and qemu-uboot-x86 new boards
- sharing most UEFI code, will replace the `virtual` one soon
- x86: patch uboot defconfig to use the `q35` machine type, not `i440fx`
  - separate x86 bootscript, due to non-uInitrd-ness of it
  - hack ramdisk load address both in u-boot source and bootscript
  - use 32-bit u-boot, not 64-bit
- grub: introduce `UEFI_GRUB=skip`, does not deploy GRUB (but does the kernel packages, etc)
- auto-enable qcow2 output for these
- works with both distro's and Armbian's kernels
2023-02-18 07:40:33 -03:00
Ricardo Pardini
3e9c123dd8
armbian-next: docker: try support Rancher Desktop (in dockerd mode), experimental.
- Changes how Docker Desktop works too, there's a `case` for switching back if needed.
- Now the linux-only CONTAINER_COMPAT hacks are used also for Darwin (`DOCKER_SERVER_REQUIRES_LOOP_HACKS=yes`)
  - `/dev/loop*` is spelled out since they don't exist on Darwin (`DOCKER_SERVER_USE_STATIC_LOOPS=yes`)
2023-02-18 07:39:52 -03:00
Ricardo Pardini
a612a03678
armbian-next: retry 3 times for loop device's size, then give up; introduce RETRY_RUNS in do_with_retries() 2023-02-18 07:39:47 -03:00
Ricardo Pardini
d24d3327a8
armbian-next: the great cli entrypoint (+docker) rewrite; introduce USE_LOCAL_APT_DEB_CACHE replacing apt-cacher-ng
- armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker)
  - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for
      - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages
      - debootstrap, by passing it `--cache-dir`
  - utility function to help understand what is happening to cache during usage
  - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_
      - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s
      - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot
  - clean up `install_deb_chroot()` a little, find an unrelated bug there
- WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile
  - it's `drastic red` on non-buildx dockers
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x
  - source configs in a logging section.
  - Docker: silent, fast retries to make sure `docker system df` works
  - shut-up `chown` (no `-v`) output related to  `SET_OWNER_TO_UID`
  - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow
  - use green apple for Mac logging, instead of red apple which might imply error...
- WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker
  - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol
  - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root.
    - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID`
    - use it for userpatches files created, logs, and output files, including images and debs.
  - @TODOs ref. `$SUDO_USER` which I think the old version of this?
  - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command.
    - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3
- WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented
- WiP: the great cli entrypoint (+docker) rewrite, Phase 1
- armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs.
  - using image in my private repo.
  - this has significant speedup to "start building time" on the 1st run
  - move some Linux specific stuff to its own if
  - add comments and todo
- armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences
- armbian-next: WiP: Docker, configure `BUILDKIT_COLORS`
- armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers
- armbian-next: `logging`: add whale indicator if build running under Docker
- armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only
- armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy
- armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots
- WiP: try with debian:bullseye -- can't detect docker at all
- armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working
  - gen .dockerignore together with Dockerfile
  - split in funcs
  - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes
  - mac still works, Linux stuff would break it but I if'fed
- armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing
  - this fails for the right reasons now, causing retries, which are then retried and work ;-)
  - this is related to building under Docker on Linux, using docker.io package (not docker-ce)
- armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated
- armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build
  - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal
-  armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore
-  armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY
  - uses REQUIREMENTS_DEFS_ONLY
  - works on Docker Desktop on Mac;
  - linux TBA
- armbian-next: don't error out if `.git` not present; other small fixes
- armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance"
  - can't assume things, for example:
  - that `sudo` will be available; it might not, and might be already root, no reason to fail
  - that `/etc/timezone` will exist
  - that `systemd-detect-virt` will be available
  - that `git` will be available
  - that `locale-gen` will be available
2023-02-18 07:39:43 -03:00
Ricardo Pardini
2c6751f584
armbian-next: show mkfs ext2/ext4 output (remove -q), it's being run under the logging manager already 2023-02-18 07:39:42 -03:00
Ricardo Pardini
72cf555bce
armbian-next: bye bye $FAST_CREATE_IMAGE: always use truncate for blank image 2023-02-18 07:39:31 -03:00
Ricardo Pardini
f921c8cd74
armbian-next: let all unmounting of ${MOUNT} be done by umount_chroot_recursive 2023-02-18 07:39:29 -03:00
Ricardo Pardini
d85bf2a155
armbian-next: many fixes after v29 rebase; mostly involving umount
- remove grub's --verbose, it's really too verbose
- trust in TMPDIR, do not use trap at all in compile_plymouth-theme-armbian - settings traps adhoc is forbidden in armbian-next as well
- be more verbose in umount_chroot(), do not try to unmount tmpfs-mounted dir itself, only xx/tmp
- try to be smarter about /tmp being mounted in rootfs-to-image -- in the end the recursive saves us
- run post_debootstrap_tweaks when the SDCARD is still mounted, not after
- don't try to download anything from any mirrors if SKIP_ARMBIAN_REPO==yes
2023-02-18 07:39:28 -03:00
Ricardo Pardini
833a7d9ab6
armbian-next: fix "create list of installed packages for debug" 2023-02-18 07:39:27 -03:00
Ricardo Pardini
379b7d6f19
armbian-next: write_image_to_device: if SKIP_VERIFY=no, don't waste time checksumming the image, it won't be used 2023-02-18 07:39:12 -03:00
Ricardo Pardini
7643c2901d
armbian-next: check_loop_device() now uses do_with_retries 5 ... to give the OS time
- prevents spurious fails I get sometimes...
2023-02-18 07:39:04 -03:00