ddbb91653f
30 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Ricardo Pardini
|
5384a7dcbc
|
armbian-next: ignore git errors due to missing 'origin' remote and/or revisions of ${SRC}/.git during main-config
|
||
|
Ricardo Pardini
|
76e276c6a9
|
armbian-next: Python patching delusion, pt1 & pt2 & pt3
- WiP: Python patching delusion, pt 1: finding & parsing patches; apply & git commit with pygit2; Markdown summaries (also for aggregation); git-to-patches tool
- Python: Markdown aggregation and patching summaries; collapsible; SummarizedMarkdownWriter
- Python: Markdown aggregation and patching summaries
- Python: reorg a bit into common/armbian_utils; define the `ASSET_LOG_BASE` in preparation for Markdown delusion
- Python patching: initial apply patches & initial commit patches to git (using pygit2)
- Python patching: add basic `series.conf` support
- Python patching: force use of utf-8; better error handling; use realpath of dirs
- Python patching: `git-to-patches` initial hack. not proud. half-reused some of the patches-to-git
- Python patching: "tag" the git commits with info for extracting later; introduce REWRITE_PATCHES/rewrite_patches_in_place
- Python patching: commented-out, recover-bad-patches hacks
- Python patching: shorten the signature
- Python patching: allow BASE_GIT_TAG as well as BASE_GIT_REVISION
- Python patching: git-archeology for patches missing descriptions; avoid UTF-8 in header/desc (not diff)
- Python patching: use modern-er email.utils.parsedate_to_datetime to parse commit date
- Python patching: unify PatchInPatchFile; better git-commiting; re-exporting patches from Git (directly)
- Python patching: switch to GitPython
- GitPython is like 100x slower than pygit2, but actually allows for date & committer
- also allows to remove untracked files before starting
- Python aggregation: fix missing `AGGREGATED_APT_SOURCES_DICT`
- Python patching: add `unidecode` dependency to pip3 install
- Python patching: don't try archeology if SRC is not a Git Repo (eg, in Docker)
- Python patching: don't try archeology if not applying patches to git
- WiP: Python patching delusion, pt2: actually use for u-boot & kernel patching
- Python patching: much better problem handling/logging; lenient with recreations (kernel)
- Python patching: don't force SHOW_LOG for u-boot patching
- Python patching: don't bomb for no reason when there are no patches to apply
- Python patching: fully (?) switch kernel patching to Python
- Python patching: more logging fixups
- Python patching: capture `kernel_git_revision` from `fetch_from_repo()`'s `checked_out_revision`
- Python patching: fully switch u-boot patching to Python
- Python aggregation/patching: colored logging; patching: always reset to git revision
- Python aggregation/patching: better logging; introduce u-boot Python patching
- Python patching pt3: recovers and better Markdown
- Python patching: detect, and rescue, `wrong_strip_level` problem; don't try to export patches that didn't apply, bitch instead
- Python patching: Markdown patching summary table, complete with emoji
- Python patching: include the problem breakdown in Markdown summary
- Python patching: sanity check against half-bare, half-mbox patches
- Python patching: try to recover from 1) bad utf-8 encoded patches; 2) bad unidiff patches; add a few sanity checks
- Python patching: try, and fail, to apply badly utf-8 encoded patches directly as bytes [reverted]
- Python patching: try to recover from patch *parse* failures; show summary; better logging
- set `GIT_ARCHEOLOGY=yes` to do archeology, default not
- armbian-next: Python `pip` dependencies handling, similar to `hostdeps`
- same scheme for Dockerfile caching
- @TODO: still using global/shared environment; should move to a dir under `cache` or some kinda venv
- WiP: add `python3-pip` to hostdeps; remove `python-setuptools`
- remove `python-setuptools` (Python2, no longer exists in Sid) from hostdeps
- add `python3-pip` to hostdeps; part of virtualenv saga
- WiP: split `kernel.sh` a bit, into `kernel-patching.sh`, `kernel-config.sh` and `kernel-make.sh`
- `advanced_patch()`: rename vars for clarity; no real changes
- Python patching: introduce FAST_ARCHEOLOGY; still trying for Markdown links
|
||
|
Ricardo Pardini
|
b4da0c869c
|
armbian-next: a few fixes (shortcircuits...) and enhancements for the GUI building; thanks NicoD | ||
|
Ricardo Pardini
|
2d563312bd
|
armbian-next: fixes for interactive config; change wording of stuff and sort the list of boards in "expert" mode | ||
|
Ricardo Pardini
|
adce39d3b7
|
armbian-next: aggregation.py rewrite; introduce package-lists.sh; usage in core lib
|
||
|
Ricardo Pardini
|
996aa14f77
|
armbian-next: include packages.external and apt sources only in DESKTOP builds (pre-aggregation.py)
- cli builds won't have PPAs and other external repos - packages from such repos should be listed in `packages.external` now, so they're not mismatched - `PACKAGE_LIST_EXTERNAL` (packages.external) only in DESKTOP builds - this logic may change later, example: - we might _also_ want pure-Ubuntu desktops, without external repos - we might _instead_ want external repos in the cli builds - this way is easy to change, since everything is kept separate and we can change in a single place. |
||
|
Ricardo Pardini
|
93f0545519
|
armbian-next: the great aggregation rewrite: pt 0: SHOW_AGGREGATION debug logging
|
||
|
Ricardo Pardini
|
94d451b812
|
armbian-next: introduce hook post_family_config_branch_<branch>() as convenience after post_family_config() hook
|
||
|
Ricardo Pardini
|
d24d3327a8
|
armbian-next: the great cli entrypoint (+docker) rewrite; introduce USE_LOCAL_APT_DEB_CACHE replacing apt-cacher-ng
- armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker)
- this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for
- apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages
- debootstrap, by passing it `--cache-dir`
- utility function to help understand what is happening to cache during usage
- apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_
- introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s
- don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot
- clean up `install_deb_chroot()` a little, find an unrelated bug there
- WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile
- it's `drastic red` on non-buildx dockers
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x
- source configs in a logging section.
- Docker: silent, fast retries to make sure `docker system df` works
- shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID`
- ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow
- use green apple for Mac logging, instead of red apple which might imply error...
- WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker
- introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol
- introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root.
- introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID`
- use it for userpatches files created, logs, and output files, including images and debs.
- @TODOs ref. `$SUDO_USER` which I think the old version of this?
- add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command.
- initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3
- WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented
- WiP: the great cli entrypoint (+docker) rewrite, Phase 1
- armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs.
- using image in my private repo.
- this has significant speedup to "start building time" on the 1st run
- move some Linux specific stuff to its own if
- add comments and todo
- armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences
- armbian-next: WiP: Docker, configure `BUILDKIT_COLORS`
- armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers
- armbian-next: `logging`: add whale indicator if build running under Docker
- armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only
- armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy
- armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots
- WiP: try with debian:bullseye -- can't detect docker at all
- armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working
- gen .dockerignore together with Dockerfile
- split in funcs
- hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes
- mac still works, Linux stuff would break it but I if'fed
- armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing
- this fails for the right reasons now, causing retries, which are then retried and work ;-)
- this is related to building under Docker on Linux, using docker.io package (not docker-ce)
- armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated
- armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build
- IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal
- armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore
- armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY
- uses REQUIREMENTS_DEFS_ONLY
- works on Docker Desktop on Mac;
- linux TBA
- armbian-next: don't error out if `.git` not present; other small fixes
- armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance"
- can't assume things, for example:
- that `sudo` will be available; it might not, and might be already root, no reason to fail
- that `/etc/timezone` will exist
- that `systemd-detect-virt` will be available
- that `git` will be available
- that `locale-gen` will be available
|
||
|
Ricardo Pardini
|
38e2e50328
|
armbian-next: split early_prepare_host_dependencies() (run early, during config, before aggregation) from prepare_host() (run after aggregation)
- add hook `host_dependencies_known` (different from `host_dependencies_ready` from before) |
||
|
Ricardo Pardini
|
95e0b9d484
|
armbian-next: remove warm/cold bundles from fetch_from_repo, replace with 2 invocation-specific hooks and gitballs (.tar of .git)
- drop experiment 'caches/gitbundles' and its code; all lives now in separate repo - change LINUXSOURCEDIR **again**, cleanup the old one |
||
|
Ricardo Pardini
|
3723cc1b7b
|
armbian-next: source arch common.conf before the actual arch configs in main-config.sh
|
||
|
Ricardo Pardini
|
b25b3cf499
|
armbian-next: lib changes - MEGASQUASH - squashed changes from c9cf3fc241cfb4c872f4aef7bbc41d5854db7ea3 to 6809de3d6063cb041205a8318e19da6a4dee68c9 ref extensions_08_10_2022_pre_v30
- also compile.sh - shellfmt lib - split off shell and python tools under lib - revert removal of stuff a-n no longer uses (ref. compilation): general packaging, mkdeb etc - editoconfig split off - extension changes split off - sources and sources/families split off - some undue stuff removed or split armbian-next: manual merge (30) of lib changes between |
||
Markus Hoffrogge
|
fba71fa703
|
Extend BUILD_ONLY value to single value "default" (#4519)
* Extend BUILD_ONLY value to single value "any"
- build-tasks.sh:
- build_task_is_enabled():
- handle value "any" adequate to logic for empty string
- build_validate_buildOnly():
- handle value "any" adequate to logic for empty string
- add specific error message in case "any" appears in a list of task names
- add function build_only_value_for_kernel_only_build():
- provide a constant list of task names for kernel only package build
supposed to be used unique at several places
- backward_compatibility_build_only():
- propagate KERNEL_ONLY="no" to BUILD_ONLY="any"
- propagate KERNEL_ONLY="yes" to BUILD_ONLY="$(build_only_value_for_kernel_only_build)"
- update log messages accordingly
- interactive.sh:
- re-factor interactive_config_ask_kernel_only()
-> interactive_config_ask_build_only()
- interactive_config_ask_build_only():
- provide three pre-defined choices:
- "u-boot" - "U-boot package only"
- "$(build_only_value_for_kernel_only_build)" - "U-boot and kernel packages"
- "any" - "Full OS image for flashing"
- un-comment call of interactive_config_ask_build_only
in interactive_config_ask_kernel()
* Extend BUILD_ONLY value to single value "default"
- build-tasks.sh:
- change BUILD_ONLY setting of "any" to "default"
to be closer to parallel WIP of #4526
- interactive.sh:
- interactive_config_ask_build_only():
- improve menu selection texts for unique quick selection first characters:
- "u-boot" - "U-boot package only"
- "$(build_only_value_for_kernel_only_build)" - "Kernel and U-boot packages only"
- "default" - "Full OS image for flashing"
* Update sequence of BUILD_ONLY selection
- start with default to "Kernel and U-Boot packages only"
Co-authored-by: Igor <igor@armbian.com>
|
||
hzyitc
|
0b437bfd75
|
buildscript: Bugfix: handle over-mounted mountpoint (#4660)
Adjust the parameters when calculating the mount point |
||
Igor Pečovnik
|
ff6c515e3f
|
Add next Ubuntu 23.04 / Lunar lobster (#4625) | ||
|
Igor Pečovnik
|
424ac69206
|
Change text on interactive menu (#4522)
* Change text on interactive menu |
||
Ash
|
408bc67619
|
Add shebangs for shellcheck (#4493)
* Add shebangs for shellcheck See #AR-1406 * Add shebangs for shellcheck Also for `extensions` scripts |
||
|
Markus Hoffrogge
|
950c659a4a
|
Add option BUILD_ONLY to the build compile scripts (#4455)
* Rename default-build.sh -> build-tasks.sh This commit is just a file rename without any further code change before this script will be refactored to improve build task structure for partial building. Refers to #4421 * Prepare build-tasks.sh for moving each build step to a dedicated function This commit is an intermediate update to reflect NO functional code changes of the particular build steps before moving the unmodified code into the appropriate function by copy/paste. Only build_get_boot_sources() has slightly changed to an appropriate if / fi structure. Note: This .sh file is bash syntactically not correct. The purpose of this commit is to leverage code review. Refers to #4421 * Update build-tasks.sh function do_default() for filtered function calls Function do_default(): - Add $1 function parameter for filtering build tasks and assign this to local variable $_buildOnly - move "pseudo inline"-functions to the beginning of the script - replace the "inline functions" by filtered function calls - add BUILD_ONLY variable to the build epilog print Refers to #4421 * Complete refactoring of build-tasks.sh for BUILD_ONLY support - update indentation of functions build_get_boot_sources() and build_get_kernel_sources() to meet code style - rename original function do_default() -> build_main() - create new do_default() for backward compatibility and comment this one as deprecated - call build_main "" from this deprecated do_default() - cli-entrypoint.sh: - update call of do_default by build_main "${BUILD_ONLY}" closes #4421 * Print final runtime as min:seconds As with filtered build runtimes may get below a minute, we would like to get the runtime printed with seconds. * Delete duplicate messages * Fix the confusion of the choice of building the kernel * Fix a typo in comment on do_default() * Backward compatibility of KERNEL_ONLY and new BUILD_ONLY keys. * interactive_config: replace KERNEL_ONLY on BUILD_ONLY * Fix: Correct verification when an image is created * Abort with error if BUILD_ONLY contains invalid build task names - add function build_validate_buildOnly() - call this function from build_main - fix position of KERNEL_ONLY / BUILD_ONLY contradiction message (did log empty _buildOnly content all time) - improve local variable names * Improve final error message to list valid BUILD_ONLY task names * Improve error message logging - collect multiple invalid BUILD_ONLY task names in a single error message - log final message for valid BUILD_ONLY task names on ext level - simple exit with exit code 1 in case of error * The RELEASE variable cannot be empty. Additional conditions for checking RELEASE are unacceptable. * Fix build_main() to use BUILD_ONLY inside instead of local var set from parameter $1 - build-tasks.sh: - build_main(): - don't take parameter $1 - replace usage of $_buildOnly by $BUILD_ONLY - added info messages on KERNEL_ONLY cases in case BUILD_ONLY was propagated - build_validate_buildOnly(): - remove parameter $2 - assign _buildOnly from global BUILD_ONLY accordingly - cli-entrypoint.sh: - call build_main without parameter * Choosing interactive_config if the goal is bootstrap * fix: order of selection backward_compatibility_build_only * Update function comments to match current state - removed obsolete parameter of build_main call do_default() - replaced :space: by :comma: in _kernel_buildOnly to leverage copy / paste for final repeatable command line printed after build * Improve logic coding to filter build tasks - build-tasks.sh: - added functions: - build_task_is_enabled() - build_task_one_of_is_enabled() - build_task_each_of_is_enabled() - updated existing build task filter logic to use function build_task_is_enabled - config-prepare.sh, prepare-host.sh: - replaced existing build task filter logic to use function build_task_is_enabled * Remove unused functions - remove function build_task_one_of_is_enabled() and build_task_each_of_is_enabled() according to code review Co-authored-by: The-going <48602507+The-going@users.noreply.github.com> |
||
|
Igor Pečovnik
|
3e701a857a
|
Add initial RiscV support (#4387)
* Add initial RISCV support * Add UEFI edge and current kernel configs * Cleanup and adjust packages * Those files were in wrong path * Fix locations * Ubuntu images built * Remove Debian support as its too fragile * Remove Pinta as it doesn't work under riscv * Limitation is only for building images * Remove obsolete switch * Cleanup major arch config files * Support for Docker building |
||
David Manouchehri
|
b5ba3c493f
|
Rename SERIAL_AUTOLOGIN to CONSOLE_AUTOLOGIN. | ||
|
David Manouchehri
|
f037b773b9
|
Enable auto-login by default. | ||
|
Igor Pečovnik
|
5d7744c880
|
Add Ubuntu 22.10 Kinetic (#4326)
* Add Ubuntu Kinetic * Remove missing package * Keep only working desktops in kinetic * Optimisation * Cleanup |
||
|
hzy
|
0c72faf421 |
Split out functions from /lib/main.sh
|
||
|
hzy
|
58051b04b7 |
Split interactive_config_ask_* from /lib/main.sh
|
||
|
hzy
|
a4a4cc385f |
Wrap /lib/configuration.sh as function
|
||
|
hzy
|
da15acb0c3 |
Split interactive_desktop_main_configuration from /lib/configuration.sh
|
||
|
hzy
|
cda53306a6 |
Split out functions from /lib/configuration.sh
|
||
|
Ricardo Pardini
|
85c8cbf15d |
run shellfmt on all of lib
|
||
|
hzy
|
d4c8dff4f0 |
Split lib/general.sh
|