Commit Graph

103 Commits

Author SHA1 Message Date
Ricardo Pardini
8b0fb1537b meson-{sm1|g12a|g12b}: default, but don't override, ASOUND_STATE; this way board-specific asound.state's are honored
- add logging during actual deployment so we know what is happening
2023-06-27 11:39:36 +02:00
Ricardo Pardini
040158b004 distro-agnostic: don't assume armbian.list exists when seding for beta 2023-06-02 19:32:57 +02:00
Ricardo Pardini
aca22a53c3 repo: split image apt .list deployment into image-early and image-late
- split `image` apt .list deployment into `image-early` (after rootfs extract, before building) and `image-late` (after building)
- rationale: having the repo enabled during the image build can cause 'apt upgrade' and others to do inconsistent stuff depending on the (possibly random) state of the repo
2023-06-02 19:32:28 +02:00
Ricardo Pardini
1fd5d519d4 introduce armbian-base-files artifact, which downloads & repacks base-files from upstream distro
- bsp-cli: now depends on `base-files (>= ${REVISION})`, this way upgrading the bsp-cli causes our base-files to be installed
  - bsp-cli no longer does gymnastics with /etc/os-release et al, all done in armbian-base-files now
- general/apt-utils.sh: introduce `apt_find_upstream_package_version_and_download_url()`
- base-files: add release to version, in order to comply with repo restrictions (valid repos can't have two different debs with same name and version, md5 must match)
2023-05-18 20:49:06 +02:00
Ricardo Pardini
f26a41ff6c rootfs/image: introduce new hook custom_apt_repo() (hashed into rootfs version); deploy different repo components/custom repos depending on rootfs or image
- rationale: we don't want an eternal chicken-egg problem with rootfs vs repo.
  - but, desktop rootfs require some parts of repo. case in point: `system-monitoring-center`
    - so only add certain components of repo (-desktop, -utils) to rootfs so that is honored
- introduce `custom_apt_repo()` hook for extensions to add their repos as well
  - same chicken-egg-avoiding is possible via param `CUSTOM_REPO_WHEN`
2023-05-16 20:09:47 +02:00
Ricardo Pardini
25929ea6ac apt_purge_unneeded_packages_and_clean_apt_caches(): count files, don't use du: avoid WARNs when not needed; tolerate 1 file
- 'lock' might or not be there, tolerate 1 file (not 0)
2023-05-15 17:43:25 +02:00
Ricardo Pardini
b18cd8aaf4 logging: curb some warnings into info's 2023-05-09 10:04:57 +02:00
Ricardo Pardini
ab696d0776 rootfs: bump non-desktop tmpfs_estimated_size from 2000mb to 2300mb (determined empirically, after building all images) 2023-05-09 10:04:57 +02:00
Ricardo Pardini
637c4fec01 include (clean/real) apt lists in built images; update xapian index during image build, not rootfs; don't leave junk in the rootfs cache
- revert #5139 Revert "Apt lists is needed for synpatic index generation"
  This reverts commit 63e9bd3baa
  - this actually only left junk in rootfs cache for the future to find
2023-05-09 10:04:50 +02:00
Igor
63e9bd3baa Apt lists is needed for synpatic index generation 2023-05-05 19:37:19 +02:00
Ricardo Pardini
3767729dc5 rootfs: add artifact name prefix and EXTRA_ROOTFS_NAME suffix (for extensions), yyyymm one-per-month auto-version in place of REVISION for artifact version
- rootfs is not a deb package, and thus not subject to the same strict ordering rules of `artifact_prefix_version`
  - replace with `yyyymm` (`202305`) in version so we get one-per-month automatically
2023-05-01 22:47:12 +02:00
Ricardo Pardini
c692676da7 kernel-debs: postinst/etc: fix: unify is_boot_dev_vfat() function; make work during image build too via env var
- determinining if `/boot` is vfat is more complex than it looks;
  - use a common function to unify across all scripts
- during image build, the kernel install is always done with a non-fat, non-mounted /boot
  - use a special variable passed through the env so unified function can know ahead of time
  - introduce `extra_apt_envs` param to `chroot_sdcard_apt_get()`; array with extra environment vars
- if vfat /boot:
  - `linux-update-symlinks` is not called (Debian would just choke on this)
  - don't even try to symlink Armbian stuff, move instead
- more/better logging board-side (unfortunately mixed in all the set -x debugs)
2023-05-01 22:46:43 +02:00
Ricardo Pardini
2b28f29ebe rootfs/image: avoid apt junk leftover by debootstrap; add warnings and debugs over target cache and general rootfs sizes
- rootfs: rootfs-create: show a summary of the 20 biggest dirs, right before tarring the rootfs (for debugging)
- rootfs: rootfs-create: show usage of caches between first and second stages
- rootfs: rootfs-create: cleanup junk left by `debootstrap` after second stage
- rootfs: rootfs-create: _always_ clean apt stuff at the end
- rename `apt_purge_unneeded_packages()` to `apt_purge_unneeded_packages_and_clean_apt_caches()` for clarity
- image: `apt_purge_unneeded_packages_and_clean_apt_caches()`: warn if apt caches not empty; clean them off, always.
- host-utils: `local_apt_deb_cache_prepare()`: also test the target, warn if not empty
- extension: cleanup-space-final-image: do NOT clean apt stuff. done in core now
- the metric shit-ton of debugs added should help the next person who faces this in the future
2023-04-18 16:27:06 +02:00
Igor
a6897e85a0 Revert "remove apt cache in the rootfs"
This reverts commit 790e673096.
2023-04-17 16:42:36 +02:00
amazingfate
790e673096 remove apt cache in the rootfs 2023-04-17 13:42:00 +02:00
Ricardo Pardini
d59271f903 config: refactor calculate_image_version() out of create_image_from_sdcard_rootfs(), make more flexible, predict IMAGE_FILE_ID during config
- default `IMAGE_INSTALLED_KERNEL_VERSION="generic"` instead of `"linux"`
2023-04-15 22:43:06 +02:00
Ricardo Pardini
0d276c93e9 bsp-cli/distro-agnostic: introduce KEEP_ORIGINAL_OS_RELEASE=yes so Debian/Ubuntu PRETTY_NAME is left unchanged in /etc/os-release
- useful when downstream software wants exact Debian OS
2023-04-15 22:43:06 +02:00
Igor
b2be0e5a45 Enable non-free-firmware repository for Debian Sid
Addressing:

  [🐳|💥] Error context msg [ Installation of Armbian main packages for sid 3dsupport xfce no failed ]
  Error:  Error context msg Installation of Armbian main packages for sid 3dsupport xfce no failed
  [🐳|🔨]   Reading package lists...
  [🐳|🔨]   Building dependency tree...
  [🐳|🔨]   E: Unable to locate package firmware-sof-signed
2023-03-31 13:21:35 +02:00
Igor Pečovnik
f51a99f1fa distro-agnostic.sh: fix, install armbian-zsh correctly (not armbian-config twice) 2023-03-31 09:12:44 +02:00
Igor Pecovnik
bf801d194b distro-agnostic.sh: bring back BSPFREEZE=yes which now works (somewhat) 2023-03-31 09:12:44 +02:00
Ricardo Pardini
f969a5e30f lib: completely remove CHOSEN_xxx, BSP_xxx_NAME, BSPFREEZE=yes 2023-03-31 09:12:44 +02:00
Ricardo Pardini
ed0b406265 lib: fix: replace undue export statements with declare -g 2023-03-31 09:12:44 +02:00
Ricardo Pardini
77921725fb lib: run shellfmt; remove commented-out; no actual changes 2023-03-31 09:12:44 +02:00
Ricardo Pardini
f12015954a distro-agnostic: emit 'info' message before setting root password, otherwise logs are confusing 2023-03-31 09:12:44 +02:00
Ricardo Pardini
d868d76182 artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec
- bsp-cli: does not require aggregation
- bsp-desktop / desktop: requires aggregation
- 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp
- make consistent naming/folders
- extract .sh generation workflow into 'declare -f' magic functions
- `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen
- artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD
- artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore
- cli: commands: fix artifact aliases, reorder
- desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name
- artifact bsps: require aggregation to build, not to list what needs to be built
- distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`)
- armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check
- distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-31 09:12:44 +02:00
Ricardo Pardini
b59447c96e apt-install: install_deb_chroot(): useful logging, when trying to install invalid stuff 2023-03-31 09:12:44 +02:00
Igor
00f849ac71 [AR-1620 + AR-1639 squashed]:: @igorpec's armbian-config/zsh/bsp/desktop/ubuntu-advantage artifacts 2023-03-31 09:12:44 +02:00
Ricardo Pardini
91b7b35fa3 rootfs-to-image/distro-agnostic: introduce IMAGE_VERSION, can be used to override VERSION file ($REVISION) just for image filename/issue/os-release 2023-03-17 08:32:11 +01:00
Igor Pečovnik
b7b8eb7b72
Add / modify (c) in bash scripts (#4922)
* Add / modify (c) in bash scripts

Signed-off-by: Igor <igor@armbian.com>

* Add (c) to the source config files

---------

Signed-off-by: Igor <igor@armbian.com>
2023-03-09 18:30:40 +01:00
Igor Pečovnik
e71d8dba10
Clenup minimal and server lists (#4893) 2023-03-02 22:44:47 +01:00
Ricardo Pardini
d453c873fd
distro-agnostic.sh: fix: check if chroot (not host!) has timesyncd.conf as condition to enable systemd-timesyncd.service (#4870) 2023-02-26 09:36:32 +01:00
amazingfate
e6915eabc4 add shebang to rootfs-create.sh 2023-02-24 18:59:53 +08:00
amazingfate
3d2cfbc620 use chroot_sdcard_apt_get_install for packages 2023-02-24 18:49:18 +08:00
Ricardo Pardini
2f7423d0e4
shellfmt/.editorconfig: enable trim whitespaces, run shellfmt, no actual code changes (#4849)
- shellfmt: bump to 3.6.0 (run with `bash lib/tools/shellfmt.sh`)
- .editorconfig: rationalize, remove duplicated logic
2023-02-23 09:24:41 +01:00
Igor
27cdc2f7ff
Armbian next CI/pre switch adjustements
- bump version to 23.05
- adjust OCI repositories, changed hash lenght to 12 and removed rootfs- prefix to shorten names (costmetic adjustement)
- build configs were moved to https://github.com/armbian/os
- remove CI that is not going to work anymore. This means also master is out of this CI
- small changed to Readme.txt. Just adding some note, removing obvious
- add propery - supported arhitectures for distribution. Needed for CI optimisation
- remove README.armbian-next.md (need to be somewhere else)
- shrink CODEOWNERS due to messages overload

Signed-off-by: Igor <igor@armbian.com>
2023-02-21 18:51:41 +01:00
Rich
7f11a59c3a
sid: add unstable repo 2023-02-18 07:46:42 -03:00
Kreyren
b7d5ea2321
distro-agnostic: dropbear-initramfs: make sure directory exists [cherry-pick] 2023-02-18 07:46:37 -03:00
Kreyren
ec88aaf149
bookworm: has new/separate repos for non-free-firmware (#4801) [cherry-pick]
- "Debian moved the non-free firmware into it's own sub-repository"
2023-02-18 07:46:35 -03:00
Ricardo Pardini
9bffa5e749
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build
- artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes`
- artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build
- kernel_package_source() is no more
- a long dissertation about kernels, families, and the universe
- artifacts: actually use rootfs artifact for image build
- artifacts: detangle via artifact_base_dir
- artifacts: rootfs: use folders in artifact_name; include cache_type
- artifacts: some cleanups / validations
- rootfs artifact; drop old rootfs cli
- artifacts: new CLI shortcuts; remove old firmware CLI
- artifacts: full firmware & usage
- use firmware artifacts in image build and install its debs
- kernel artifact sans legacy; use tmpdir for .deb target for all packages
- legacy artifact versions is no more; pack/unpack now in common obtain;
- artifacts: uboot: cleanup legacy renaming, use artifact version directly
- artifacts: add firmware (small) artifact
- deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning
- fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash
  - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()`
  - don't use realpath for artifact_file_relative
  - curb some warnings
  - fix: only install headers & dtbs if such exist
- kernel .config hook modification hash versioning
- OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB
- artifact revolver & installing into image
  - add artifact_map_packages and artifact_map_debs dicts
  - revolver accumulates all info
  - REPOSITORY_INSTALL is no more (for uboot/kernel, later others)
  - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-18 07:46:03 -03:00
Ricardo Pardini
9f6ef8deeb
armbian-next: rootfs: fix: too many bash files included in rootfs B hash, reduce to only what is used to produce rootfs tarball
- otherwise we get rootfs cache misses when working distro-agnostic and others, which are not part of rootfs code
2023-02-18 07:45:52 -03:00
Ricardo Pardini
5d7924ebe5
armbian-next: rootfs: include hash of rootfs bash code in the packages_hash/rootfs_cache_id
- don't use dash separator for `B`ash hash in `packages_hash`/`rootfs_cache_id`
2023-02-18 07:45:49 -03:00
Ricardo Pardini
bf53bd2988
armbian-next: tmpfs: use size=99% for all tmpfs's; (rootfs/WORKDIR/chroot); thanks Igor for suggestion 2023-02-18 07:45:42 -03:00
Igor
00bc1b4a1e
armbian-next: rootfs: introduce ROOTFS_COMPRESSION_RATIO, default 5 for general use, default 15 for rootfs CLI; fix pv | tar bug; remove GPG signing
- Remove PGP signing from the script as its done in GHA
- Also remove generating .current control file as its obsolete in this design

Co-authored-by: Ricardo Pardini <ricardo@pardini.net>
2023-02-18 07:44:42 -03:00
Ricardo Pardini
1cec42392e
armbian-next: GHA & rootfs adventures, pt 1
- rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version)
- add "oras-upload" CLI command (dumb, no retries, single target/single source)
- docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker
- introduce `github-actions.sh::github_actions_add_output()`
- during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available
2023-02-18 07:44:41 -03:00
Ricardo Pardini
78dcf01c92
armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs
- killed `[[ $ROOTFS_TYPE != ext4 ]] && display_alert "Assuming ${BOARD} ${BRANCH} kernel supports ${ROOTFS_TYPE}" "" "wrn"`
  - which definitely didn't belong in rootfs
- disable usage of run_host_command_logged_long_running for kernel-make
- stop lying about long_running stuff being any different from non-long_running versions
- stop lying about 'set -e' when there's still a bunch of pipes for "pv" and stuff all around
2023-02-18 07:44:33 -03:00
Ricardo Pardini
695bfb387b
armbian-next: rootfs: reset UID of produced rootfs tarball; announce what is doing and what has done 2023-02-18 07:44:27 -03:00
Ricardo Pardini
6077d40f9b
armbian-next: rootfs: build rootfs from CLI, sans-BOARD, sans-FAMILY, etc -- rootfs are meant to be shared
- require ARCH/RELEASE for rootfs cli
- disallow BOARD/LINUXFAMILY/BOARDFAMILY being set for rootfs cli
- split `prep_conf_main_only_rootfs()` instead of `prep_conf_main_build_single()` for rootfs-only stuff
- run both conf and build without stdin for rootfs CLI (no interactive allowed)
- don't blindly set `ARCH=armhf` for no reason in main-config
- allow skipping family stuff in main-config, via `allow_no_family=yes` (used only in `prep_conf_main_only_rootfs()`)
- kill double loading of userpatches family, not needed/old code
- allow skipping kernel stuff via `skip_kernel=yes` in `config_pre_main()`  (used only in `prep_conf_main_only_rootfs()`)
2023-02-18 07:44:26 -03:00
Ricardo Pardini
091ae1d27f
armbian-next: rootfs: adapt to new reality after huge refactor
- make `tmpfs_estimated_size` and `available_physical_memory_mib` readonly and global and only ever set once
- remove subshell from `get_rootfs_cache_list()`, new version `get_rootfs_cache_list_into_array_variable()` for sanity
- warns if `ROOT_FS_CREATE_ONLY=yes` is set during cached build path
- add a `wait_for_disk_sync()` after restoring rootfs cache
- fix logging sections for `build_rootfs_only()` and `prepare_rootfs_build_params_and_trap()` in `build_rootfs_and_image()`
- remove the logging section from `get_or_create_rootfs_cache_chroot_sdcard()` call in `build_rootfs_only()` (which is now itself logged)
- @TODO: `deploy_qemu_binary_to_chroot()` is leaking a qemu bin in rootfs cache!
2023-02-18 07:44:24 -03:00
Ricardo Pardini
28d89e2c91
armbian-next: rootfs: new CLI command for rootfs cache building; further refactorings
- _told ya it would all make sense eventually..._
- introduce new, real, non-alias `rootfs` command in `cli-rootfs` -- this _only_ builds rootfs, nothing else
- no more `main_default_build_single()`; instead `full_build_packages_rootfs_and_image()`
- introduce `do_with_default_build()` wrapper, for things that need host prepared + aggregation to run
- CLI-specific parts of `main_default_build_single()` moved to cli-build
- put prepare/cleanup pair of `prepare_rootfs_build_params_and_trap()` and `trap_handler_cleanup_rootfs_and_image()` into new `trap-rootfs.sh` (this needs further splitting for the image part)
- refactor `get_or_create_rootfs_cache_chroot_sdcard()` into `build_rootfs_only()`, `calculate_rootfs_cache_id()`
- update library with new files
2023-02-18 07:44:22 -03:00
Ricardo Pardini
6176dc3702
armbian-next: rootfs: split create-cache into rootfs-create
- bear with me, this will make sense in the end
- a `mount_chroot()` was ejected out of rootfs building code into main, for reuse's sake
2023-02-18 07:44:21 -03:00