Commit Graph

10997 Commits

Author SHA1 Message Date
Ricardo Pardini
cc7972df0d
armbian-next: grub: make /usr/share/desktop-base/grub_background.sh executable, otherwise does nothing 2023-02-18 07:41:26 -03:00
Ricardo Pardini
ec2e6c42be
armbian-next: happy new year, bump Igor's copyright 2023-02-18 07:41:25 -03:00
Ricardo Pardini
43f268fdcb
armbian-next: remove very old extlinux killswitch 2023-02-18 07:41:24 -03:00
Ricardo Pardini
c61b985417
armbian-next: better logging in distro-agnostic.sh
- still _a lot_ to fix
2023-02-18 07:41:23 -03:00
Ricardo Pardini
8f3fd355eb
armbian-next: tune logging in multiple places, there was a lot of redundant/un-needed info's 2023-02-18 07:41:22 -03:00
Ricardo Pardini
7987dad34d
armbian-next: Python tools (aggregation/patching): use separate pycache dir in ./cache; use unbuffered stdout/stderr 2023-02-18 07:41:21 -03:00
Ricardo Pardini
59031be6b3
armbian-next: only put systemd-resolved's symlink in place if _not_ using NetworkManager
- why is it even installed? dunno, but fix anyway.
2023-02-18 07:41:20 -03:00
Ricardo Pardini
f8b141cdf0
armbian-next: fix: only aggregate packages if building images (AR-1471) 2023-02-18 07:41:19 -03:00
Ricardo Pardini
c4a85c4771
armbian-next: json-info: do not replace ${SRC} 2023-02-18 07:41:18 -03:00
Ricardo Pardini
690ca4df7c
armbian-next: add (trap) cleanup handler to reset ./tmp and ./output to original pre-sudo UID before exiting, even with failure/abort 2023-02-18 07:41:17 -03:00
Ricardo Pardini
9973010972
armbian-next: Python patching: pass $HOME/$PATH to Python, so git can find .gitconfig and Python can find git
- add sanity check before handling the Git repo in Python (call `status()` to make sure access is OK)
2023-02-18 07:41:16 -03:00
Ricardo Pardini
12b31afa51
armbian-next: git: mark submodule's dirs as safe before recursive fetch_from_repo() 2023-02-18 07:41:15 -03:00
Ricardo Pardini
3bc8afd593
armbian-next: refactor losetup -d ... into free_loop_device_insistent() which retries 10 times 2023-02-18 07:41:14 -03:00
Ricardo Pardini
f679b81b10
armbian-next: git: use --recurse-submodules=no for fetch; fetch_from_repo() handles submodules separately
- also remove `du` working copy size debugging
2023-02-18 07:41:13 -03:00
Ricardo Pardini
65da2a697f
armbian-next: remove @TODO for stuff already done 2023-02-18 07:41:12 -03:00
Ricardo Pardini
ddbb91653f
armbian-next: reset_uid_owner for the markdown log file too 2023-02-18 07:41:11 -03:00
Ricardo Pardini
2f9a4470c7
armbian-next: validate cmdline params against malformed
- eg: `./compile.sh $ERROR=here` is caught now
2023-02-18 07:41:10 -03:00
Ricardo Pardini
093e60d111
armbian-next: exec into sudo when relaunching (AR-1452)
- with `exec`, `sudo` replaces the current shell, and thus never returns
2023-02-18 07:41:09 -03:00
Ricardo Pardini
a3c64bb6e7
armbian-next: aggregation: fix PACKAGE_LIST_BOARD_REMOVE/PACKAGE_LIST_FAMILY_REMOVE; also remove pkgs from DESKTOP aggregation
- case in point: `hostapd`, again.
2023-02-18 07:41:08 -03:00
Ricardo Pardini
8399c5d72b
armbian-next: **regen library** after some 13 squashes (and repo.sh removal) 2023-02-18 07:41:07 -03:00
Ricardo Pardini
a512edbaa9
armbian-next: better logging for jethub's tweaks 2023-02-18 07:41:06 -03:00
Ricardo Pardini
9370c19a44
armbian-next: drop old code for patching & fasthash
- split the modification time stuff to general
2023-02-18 07:41:05 -03:00
Ricardo Pardini
143ba996ca
armbian-next: git: setup the worktree first before trying to ensure safe 2023-02-18 07:41:04 -03:00
Ricardo Pardini
5384a7dcbc
armbian-next: ignore git errors due to missing 'origin' remote and/or revisions of ${SRC}/.git during main-config 2023-02-18 07:41:03 -03:00
Ricardo Pardini
85009a6431
armbian-next: git worktree: forcefully fix the worktree's reference to the bare repo, and vice-versa 2023-02-18 07:41:02 -03:00
Ricardo Pardini
e607e9e54e
armbian-next: remove spurious output.log redirections in downloads.sh; warn that download_and_verify() is unhandled in armbian-next 2023-02-18 07:41:01 -03:00
Ricardo Pardini
bd04f71a05
armbian-next: Revert "Fix issues 4589 (#4614)" (installing HEADERS dependencies manually)
- This reverts commit 36d9420d
- this makes no sense
- we don't do adhoc installs in armbian-next
2023-02-18 07:41:00 -03:00
Ricardo Pardini
eb6081678f
armbian-next: fix sunxi/sunxi64 using wrong KERNELPATCHDIR and incurring in old symlinks 2023-02-18 07:40:59 -03:00
Ricardo Pardini
9200a2ce0c
armbian-next: bring find_toolchain() back from the dead; force SKIP_EXTERNAL_TOOLCHAINS=no to force use external toolchains
- handle a lot of corner cases, some toolchains are i386-only and won't run on modern x64 systems
2023-02-18 07:40:58 -03:00
Ricardo Pardini
39ac9e00b1
armbian-next: docker: don't try to double init when recovering from failed pull 2023-02-18 07:40:57 -03:00
Ricardo Pardini
7842113c1d
armbian-next: make sure at least one board config file is sourced, otherwise misspell BOARD= is hard to find 2023-02-18 07:40:56 -03:00
Ricardo Pardini
770d508ca0
armbian-next: Python patching delusion pt 5 + EXTRAWIFI patch generator/harness + consistent patch mtime targets
- Python patching: tune some logging all-around
- Python patching: *FINALLY* set the dates on the patched files to `max(patch_date, root_makefile_date)` thus making lighting-fast rebuilds again
- new EXTRAWIFI patch generator harness; Python patching EXTRA_PATCH_FILES_FIRST
- Python patching: use temp file for patching rejects; clean it up
- new EXTRAWIFI patch generator harness; Python: mark drivers as autogen, don't split or parse them, read as bytes, apply bytes directly
- new EXTRAWIFI patch generator harness; somewhat-works, but patches are per-family
- Python patching: add the `cache/patch` as a mountpoint
  - Darwin Docker performance is 20x with namedvolume; the cached patches are very large
2023-02-18 07:40:55 -03:00
Ricardo Pardini
97f6836705
armbian-next: Python patching delusion, pt4: series & better markdown
- Python patching: hopefully better Markdown: split series dir; detect more status/problems; arche only for missing Subject
  - Python patching: archeology only for missing Subject:, not description; clarify CLI pushing
  - Python patching: use `{file_base_name}.patch` for archeology as `file_name` might include a dir
- Python patching: fix: don't skip board/target patches for no reason
- Python patching: fix for series.conf patches (sunxi/sunxi64): don't sort; mark as series and show on summary
- Python patching: don't fail if archeology found no commits (but add warning)
- Python patching: CLI command `kernel-patches-to-git` with archeology summary, and git pushing of results & summary
  - Python patching: patches-to-git small fixes, auto-push if it's rpardini
  - Python patching: add `patch-kernel` CLI command
- Python patching: commit README.md & gh-pages workflow when apply_patches_to_git
  - Python patching: hopefully better markdown
- Python patching: `git add` everything all at once, for speed
2023-02-18 07:40:54 -03:00
Ricardo Pardini
2c0e9182ed
armbian-next: introduce ORAS-based kernel git bare tree seeding/bundles
- armbian-next: fixes to ORAS-related logging
- armbian-next: actually enable ORAS-based kernel git bare tree seeding/bundles; enable cleaning of bundle articfacts after confirmed working
- armbian-next: introduce ORAS-based kernel git bare tree seeding/bundles (although it is a .tar, not a bundle); this is 20x faster than cloning
- armbian-next: introduce `ORAS` tooling; pull and push functions & downloader/launcher
2023-02-18 07:40:53 -03:00
Ricardo Pardini
76e276c6a9
armbian-next: Python patching delusion, pt1 & pt2 & pt3
- WiP: Python patching delusion, pt 1: finding & parsing patches; apply & git commit with pygit2; Markdown summaries (also for aggregation); git-to-patches tool
  - Python: Markdown aggregation and patching summaries; collapsible; SummarizedMarkdownWriter
  - Python: Markdown aggregation and patching summaries
  - Python: reorg a bit into common/armbian_utils; define the `ASSET_LOG_BASE` in preparation for Markdown delusion
  - Python patching: initial apply patches & initial commit patches to git (using pygit2)
  - Python patching: add basic `series.conf` support
  - Python patching: force use of utf-8; better error handling; use realpath of dirs
  - Python patching: `git-to-patches` initial hack. not proud. half-reused some of the patches-to-git
  - Python patching: "tag" the git commits with info for extracting later; introduce REWRITE_PATCHES/rewrite_patches_in_place
  - Python patching: commented-out, recover-bad-patches hacks
  - Python patching: shorten the signature
  - Python patching: allow BASE_GIT_TAG as well as BASE_GIT_REVISION
  - Python patching: git-archeology for patches missing descriptions; avoid UTF-8 in header/desc (not diff)
  - Python patching: use modern-er email.utils.parsedate_to_datetime to parse commit date
  - Python patching: unify PatchInPatchFile; better git-commiting; re-exporting patches from Git (directly)
  - Python patching: switch to GitPython
    - GitPython is like 100x slower than pygit2, but actually allows for date & committer
    - also allows to remove untracked files before starting
  - Python aggregation: fix missing `AGGREGATED_APT_SOURCES_DICT`
  - Python patching: add `unidecode` dependency to pip3 install
  - Python patching: don't try archeology if SRC is not a Git Repo (eg, in Docker)
  - Python patching: don't try archeology if not applying patches to git
- WiP: Python patching delusion, pt2: actually use for u-boot & kernel patching
  - Python patching: much better problem handling/logging; lenient with recreations (kernel)
  - Python patching: don't force SHOW_LOG for u-boot patching
  - Python patching: don't bomb for no reason when there are no patches to apply
  - Python patching: fully (?) switch kernel patching to Python
  - Python patching: more logging fixups
  - Python patching: capture `kernel_git_revision` from `fetch_from_repo()`'s `checked_out_revision`
  - Python patching: fully switch u-boot patching to Python
  - Python aggregation/patching: colored logging; patching: always reset to git revision
  - Python aggregation/patching: better logging; introduce u-boot Python patching
- Python patching pt3: recovers and better Markdown
  - Python patching: detect, and rescue, `wrong_strip_level` problem; don't try to export patches that didn't apply, bitch instead
  - Python patching: Markdown patching summary table, complete with emoji
  - Python patching: include the problem breakdown in Markdown summary
  - Python patching: sanity check against half-bare, half-mbox patches
  - Python patching: try to recover from 1) bad utf-8 encoded patches; 2) bad unidiff patches; add a few sanity checks
  - Python patching: try, and fail, to apply badly utf-8 encoded patches directly as bytes [reverted]
  - Python patching: try to recover from patch *parse* failures; show summary; better logging
      - set `GIT_ARCHEOLOGY=yes` to do archeology, default not

- armbian-next: Python `pip` dependencies handling, similar to `hostdeps`
  - same scheme for Dockerfile caching
  - @TODO: still using global/shared environment; should move to a dir under `cache` or some kinda venv
- WiP: add `python3-pip` to hostdeps; remove `python-setuptools`
  - remove `python-setuptools` (Python2, no longer exists in Sid) from hostdeps
  - add `python3-pip` to hostdeps; part of virtualenv saga
- WiP: split `kernel.sh` a bit, into `kernel-patching.sh`, `kernel-config.sh` and `kernel-make.sh`
  - `advanced_patch()`: rename vars for clarity; no real changes
- Python patching: introduce FAST_ARCHEOLOGY; still trying for Markdown links
2023-02-18 07:40:52 -03:00
Ricardo Pardini
c251073b78
armbian-next: introduce internal base tmp dir; check it early for sanity ref noexec and nodev mount options 2023-02-18 07:40:51 -03:00
Ricardo Pardini
0c5b10a77b
armbian-next: introduce SKIP_ARMBIAN_ROOTFS_CACHE which is =yes by default
- that is in addition to `SKIP_ARMBIAN_REPO` which is not set by default
2023-02-18 07:40:50 -03:00
Ricardo Pardini
b4da0c869c
armbian-next: a few fixes (shortcircuits...) and enhancements for the GUI building; thanks NicoD 2023-02-18 07:40:49 -03:00
Ricardo Pardini
b606ae9715
armbian-next: docker: fix when using buildx, always pass --load so produced image is actually available to Docker proper 2023-02-18 07:40:48 -03:00
Ricardo Pardini
2d563312bd
armbian-next: fixes for interactive config; change wording of stuff and sort the list of boards in "expert" mode 2023-02-18 07:40:47 -03:00
Ricardo Pardini
66747a12e1
armbian-next: disable cpufrequtils by default; to enable set CPUFREQUTILS_ENABLE=true [HIGHLY CONTENTIOUS CHANGE] 2023-02-18 07:40:46 -03:00
Ricardo Pardini
a9a9428bf7
armbian-next: shellcheck downloader/runner; only for lib/ 2023-02-18 07:40:45 -03:00
Ricardo Pardini
c2af130a94
armbian-next: fix all shellcheck errors in lib/ 2023-02-18 07:40:44 -03:00
Ricardo Pardini
fd043d707a
armbian-next: squash some warn loggings that were undue; remove $SUDO_USER handling in prepare-host.sh
- squash warning about `NO_APT_CACHER` when in Docker
2023-02-18 07:40:43 -03:00
Ricardo Pardini
b386e4615e
armbian-next: introduce INSTALL_HOST_TOOLS=no to skip rkbin, fips, etc. for fast(er) development 2023-02-18 07:40:42 -03:00
Ricardo Pardini
50b6b232d9
armbian-next: git: fetch_from_repo(): determine, use for checkout, and set the SHA1 rev in checked_out_revision var 2023-02-18 07:40:41 -03:00
Ricardo Pardini
52d89e132b
armbian-next: logging: add Markdown logging output
- armbian-next: logging: add Markdown logging output; collapsible
2023-02-18 07:40:40 -03:00
Ricardo Pardini
074857ede3
armbian-next: u-boot: use git worktree for u-boot; full mainline clone for bare
- share a single `.git` for all u-boots
- all fetches are done against it, maximizing cache hit ratio drastically
- also reduces the size of each working copy by more than 70%
- split uboot's git stuff into uboot-git.sh
- split `uboot_prepare_git()` from `compile_uboot()`
2023-02-18 07:40:39 -03:00
Ricardo Pardini
84c0bf0f7d
armbian-next: kernel: use git worktree & kernel.org clone bundles for kernel working copies
- now we've a single `.git` for all kernels with all branches; a "master" cache
- back to full stable git bundle usage from kernel.org
- bye bye, "gitballs"
- split kernel-related git stuff into `kernel-git.sh`
- introduce `ARMBIAN_MOUNTPOINTS_DEPRECATED` @TODO actually use for cleaning non-Docker
- add `axel` hostdep for multi-conn download of bundle, which comes from a CDN.
- deprecate old mountpoints for gitballs and linux-kernel
- move cleaning of old sources/kernel to mountpoints code
2023-02-18 07:40:38 -03:00
Ricardo Pardini
a51448a1da
armbian-next: non-Docker cleaning of deprecated mountpoints (before general_cleaning) 2023-02-18 07:40:37 -03:00