- split `image` apt .list deployment into `image-early` (after rootfs extract, before building) and `image-late` (after building)
- rationale: having the repo enabled during the image build can cause 'apt upgrade' and others to do inconsistent stuff depending on the (possibly random) state of the repo
- this way it's consistent across all artifacts and images
- debs-to-repo-download: fix: use DEB_STORAGE instead of hardcoding output/debs (so BETA=yes works)
also:
- fix `Duplicate oci_target` message in mapper-oci-uptodate
also:
- get rid of comments and `SUBREVISION` and `RC` variables
- better debugging in calculate_hash_for_variables() do_normalize_src_path=yes/no
- tag a few places where output/debs might wrong in face of BETA=yes/no
- some hashed variables might contain "${SRC}", so hashes never match, unless built in Docker
- this strips away SRC from all vars and adds debugging so we can detect more later.
- `artifact-uboot`: include more variables into hash, for ATF & rk stuff
- `artifact-armbian-desktop`:
- hashed vars actually contain /armbian in different context, skip normalization in this case
- include results of relevant aggregation into artifact_input_variables
- otherwise: desktops with different appgroups/configs lead to build failures in pipeline
- will cause warnings in JSON preparation step, if more than one appgroups/config combo is targeted, since repo can only have one
- bsp-cli: now depends on `base-files (>= ${REVISION})`, this way upgrading the bsp-cli causes our base-files to be installed
- bsp-cli no longer does gymnastics with /etc/os-release et al, all done in armbian-base-files now
- general/apt-utils.sh: introduce `apt_find_upstream_package_version_and_download_url()`
- base-files: add release to version, in order to comply with repo restrictions (valid repos can't have two different debs with same name and version, md5 must match)
- rationale: we don't want an eternal chicken-egg problem with rootfs vs repo.
- but, desktop rootfs require some parts of repo. case in point: `system-monitoring-center`
- so only add certain components of repo (-desktop, -utils) to rootfs so that is honored
- introduce `custom_apt_repo()` hook for extensions to add their repos as well
- same chicken-egg-avoiding is possible via param `CUSTOM_REPO_WHEN`
- otherwise we get 4 errors for each "error" in CI
- one for the real error
- one for "wait for cleanups"
- one for "Docker build failed"
- one from GHA itself since command failed (we can't get rid of this)
- `odroidxu4`'s firmware hook for vendor kernel. such is life, no use warning.
- `sun50iw9`: vendor kernel with extension, proven working, stop warning
- old kernels don't have working headers, it's a fact of life, remove warning
- `PACKAGE_LIST_BOARD_REMOVE` is used only in `helios64` board
- the fact is `PACKAGE_LIST_BOARD_REMOVE` removes the packages _from the rootfs_ and not _from the image_
- this only accounts for that fact, but in the future we should actually change how this works
- partially revert 6ef394d95d (thus bring back the TUI/dialog for selecting KERNEL_CONFIGURE=yes/no)
- partially revert d890b418c7 (thus bring back the capacity to config & build image in one go)
- stop after configuring kernel, but only if command is `kernel-config`, not regular image-build KERNEL_CONFIGURE=yes
- rootfs is built with BOARD= so just live with it (idea about this warn was that families might hook rootfs build process)
- count, instead of du, apt caches. seems some empty dirs are reported as "1Mb" by du?
- revert #5139 Revert "Apt lists is needed for synpatic index generation"
This reverts commit 63e9bd3baa
- this actually only left junk in rootfs cache for the future to find
- stop at the correct places for `inventory` and `targets` non-GHA-specific commands
- handle `gha-template` first (and return), since it does not really depend on any info/targets/etc
- batcat the output for show-off
- python-tools: add Jinja2. Incredible how we made it this far without it.
- output-gha-workflow-template: a double-templater, first runs jinja with a custom syntax, then "moar magic" to be useful for GHA
* Move packages to new location
* Adjust build runners
* Switch to OS
Signed-off-by: Igor <igor@armbian.com>
---------
Signed-off-by: Igor <igor@armbian.com>
Co-authored-by: Ricardo Pardini <ricardo@pardini.net>
- rootfs is not a deb package, and thus not subject to the same strict ordering rules of `artifact_prefix_version`
- replace with `yyyymm` (`202305`) in version so we get one-per-month automatically
- all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends
- user is required to put the produced patches in the right place and build again, for full consistency
- split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot
- non-interactive artifact builds are now 100% sans-stdin
- introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch
- reworked `userpatch_create()` to be (hopefully) more useful:
- detects a previous patch and offers to apply it before continuing
- enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch
- produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters
- uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers)
- don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
- determinining if `/boot` is vfat is more complex than it looks;
- use a common function to unify across all scripts
- during image build, the kernel install is always done with a non-fat, non-mounted /boot
- use a special variable passed through the env so unified function can know ahead of time
- introduce `extra_apt_envs` param to `chroot_sdcard_apt_get()`; array with extra environment vars
- if vfat /boot:
- `linux-update-symlinks` is not called (Debian would just choke on this)
- don't even try to symlink Armbian stuff, move instead
- more/better logging board-side (unfortunately mixed in all the set -x debugs)
- pipeline: add `pipeline` context object to targets; use it to filter artifacts and images to build; warn about oci-name with multiple oci-tags
- pipeline: better error messages when info's fail; show some (out-of-order) progress messages during parallel info gathering
- pipeline: targets-compositor: add `not-eos` inventory
- TARGETS_FILENAME, log all OCI lookups
- SKIP_IMAGES
- IMAGES_ONLY_OUTDATED_ARTIFACTS
- no dash in chunk id in JSON
- pipeline: very initial chunking, using the same outputs
- pipeline: template targets, `items-from-inventory:` inventory expansion, CHECK_OCI=yes, CLEAN_MATRIX=yes, CLEAN_INFO=yes, many fixes
- cli: `inventory` / `targets` / `matrix` / `workflow`
- pipeline: workflow beginnings
- pipeline: general log cleanup + OCI stats / better miss handling
- pipeline: fixes/reorg
- pipeline: catch & log JSON parsing errors
- pipeline: gha matrix: use IMAGE_FILE_ID as job description
- pipeline (delusion): gha workflow output, based on old matrix code
- pipeline: better parsing and reporting of stderr log lines (under `ANSI_COLOR=none`)
- pipeline: mapper-oci-uptodate: use separate positive/negative cache dirs (GHA will only cache positives); cache negs for 5 minutes locally
- pipeline: output-gha-matrix artifacts + images
- pipeline: output-gha-matrix artifacts + images: "really" and fake 1-item matrix if empty
- pipeline: move files into subdir; update copyright & cleanup
- pipeline: refactor bash jsoninfo driver a bit
- pipeline: outdated-artifact-image-reducer
- pipeline: introduce `target_id` at the compositor, aggregate it at the reducer, carry it over in the artifact info mapper
- pipeline: mapper-oci-uptodate
- pipeline: info-gatherer-artifact, with PRE_PREPARED_HOST
- pipeline: refactor/rename info-gatherer-image.py
- pipeline: beginnings
- git-ref2info: when resolving SHA1's for 'xxx' tag or branch, try first `refs/heads/xxx`, then just 'xxx'; validate SHA1 with 40-char regexp
- this works-around Gerrit-style repositories that have a 'refs/for/xxx' ref that gets in the way
- case in point: https://github.com/hardkernel/u-boot/tree/odroidc-v2011.03 (has both refs/heads/odroidc-v2011.03 and refs/for/odroidc-v2011.03)
- git-ref2info: when processing tags for a SHA1, try both dereference and non-dereferenced, in that order, to get a SHA1
- this solves old problem reported in #4916: GitHub incorrectly processes dereference requests for tags that are not actually annotated
- case in point: https://github.com/ayufan-rock64/linux-kernel/releases/tag/4.4.202-1237-rockchip-ayufan
- `codeaurora` is no more, https://bye.codeaurora.org/
- allow adding re-launch environments by populating `ARMBIAN_CLI_RELAUNCH_PARAMS` dict
- similar to `ARMBIAN_CLI_RELAUNCH_PARAMS`
- rename/consolidate `()`'s output globals:
- `ARMBIAN_CLI_FINAL_RELAUNCH_ARGS` (rename)
- `ARMBIAN_CLI_FINAL_RELAUNCH_ENVS` (new)
- includes `ARMBIAN_HIDE_REPEAT_PARAMS` automatically
- `sudo`: include envs when relaunching, use bash explicitly
- `docker`: better logging
- `docker`: include envs when relaunching via `--env` args
Add interactive configurations to Repeat Build Options
Simplify function produce_repeat_args_array()
- make use of $ARMBIAN_NON_PARAM_ARGS and $ARMBIAN_PARSED_CMDLINE_PARAMS (associated array)
- quote parameter values to be on the safe side
Move "Repeat Build Options" logs to start-end.sh
- cli-build.sh, start-end.sh:
- move function function produce_repeat_args_array() to start-end.sh
- add $WHAT as first arg to produce_repeat_args_array() if
$WHAT is not empty
- move early display log of "Repeat Build Options" to
function main_default_start_build()
- move last log of "Repeat Build Options" to
function main_default_end_build()
- add last log of "Repeat Build Options" to
LOG_SECTION="repeat_build_options"
- rootfs: rootfs-create: show a summary of the 20 biggest dirs, right before tarring the rootfs (for debugging)
- rootfs: rootfs-create: show usage of caches between first and second stages
- rootfs: rootfs-create: cleanup junk left by `debootstrap` after second stage
- rootfs: rootfs-create: _always_ clean apt stuff at the end
- rename `apt_purge_unneeded_packages()` to `apt_purge_unneeded_packages_and_clean_apt_caches()` for clarity
- image: `apt_purge_unneeded_packages_and_clean_apt_caches()`: warn if apt caches not empty; clean them off, always.
- host-utils: `local_apt_deb_cache_prepare()`: also test the target, warn if not empty
- extension: cleanup-space-final-image: do NOT clean apt stuff. done in core now
- the metric shit-ton of debugs added should help the next person who faces this in the future
- can add to `EXTRA_IMAGE_SUFFIXES` array during `user_config` or `extension_prepare_config`
- in the end goes into a `EXTRA_IMAGE_SUFFIX` global readonly
- this simplifies extension code
- we still can't "predict" image names during configuration since it includes the kernel version (6.2.11/generic) which only comes later
- if CONFIG_DEFS_ONLY during `obtain_complete_artifact()`, just dump JSON to stdout after calculating version/coordinates
- don't prepare ORAS tooling if CONFIG_DEFS_ONLY
- don't run in standard build (and thus don't prepare_host()) if under CONFIG_DEFS_ONLY
- if CONFIG_DEFS_ONLY, don't run `check_basic_host()`
Addressing:
[🐳|💥] Error context msg [ Installation of Armbian main packages for sid 3dsupport xfce no failed ]
Error: Error context msg Installation of Armbian main packages for sid 3dsupport xfce no failed
[🐳|🔨] Reading package lists...
[🐳|🔨] Building dependency tree...
[🐳|🔨] E: Unable to locate package firmware-sof-signed
- fix: indent of generated sources
- fix: no reason to source the generated file in a loop. do it once. 20x faster?
- `dump_extension_method_sources_functions "name_of_hook"`: get function definitions (inside braces)
- `dump_extension_method_sources_body "name_of_hook"`: get the function bodies (sans-braces)
- bsp-cli: does not require aggregation
- bsp-desktop / desktop: requires aggregation
- 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp
- make consistent naming/folders
- extract .sh generation workflow into 'declare -f' magic functions
- `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen
- artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD
- artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore
- cli: commands: fix artifact aliases, reorder
- desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name
- artifact bsps: require aggregation to build, not to list what needs to be built
- distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`)
- armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check
- distro-specific: fix installation of `fake-ubuntu-advantage-tools`
- include rudimentary artifact dependency information for config-dump-json: fixes/refactor/cleanup
- artifact-rootfs.sh: include rudimentary artifact dependency information for config-dump-json
- artifact-armbian-(config/plymouth-theme/zsh).sh: include rudimentary artifact dependency information for config-dump-json
- artifact-fake-ubuntu-advantage-tools.sh: include rudimentary artifact dependency information for config-dump-json
- use new capture'd vars scheme
- so `./compile.sh BOARD=xxx BRANCH=yyyy config-dump-json | jq .` now works and is consistent/newline tolerant
- introduce internal `skip_host_config=yes` for `prep_conf_main_minimal_ni()` to skip calling `check_basic_host()`
- doesn't show nor clear vars `if_error_find_files_sdcard` and `if_error_detail_message`
- used by `chroot_sdcard_apt_get()` which uses `run_host_command_logged()` multiple times, and the first would clear the important/last one
- giving up on the "bright black" == "gray" thing. There's enough bad default themes out there, and not the first time people (rightfully) complained...
Changing export-logs.sh script from posting to armbian.next.armbian.com to posting to the primary paste server which has now been upgraded at paste.armbian.com.
* Add / modify (c) in bash scripts
Signed-off-by: Igor <igor@armbian.com>
* Add (c) to the source config files
---------
Signed-off-by: Igor <igor@armbian.com>
* main source for driver and patches are sunxi64 megous patches
* move into patch/misc/wireless-rtl8723cs directory
* integrate with minor patches from rk322x/rockchip64 families
* update drivers_network.sh to apply patches in same order as before
* remove references from patches.megous and series.conf in sunxi64
* remove patches from rk322x and rockchip64 family patches
* do not touch kernel archives older than 6.1
