* extensions framework (née "fragments")
- this should actually change nothing at this point, just add capabilities
- the framework is implemented in lib/extensions.sh
- the "if function x exists then call x" replaced with call_extension_method()
- +inline documentation
- +compatibility names
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* extensions framework; meta-extensions: auto-docs and sample extension gen
- 2 extensions dealing with extensibility itself
- detect-unused-extensions: shows which extensions are enabled, but never called.
- gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* new extension methods and features via config variables in core Armbian
- `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones
- `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file
- `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP
- `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images
- `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs"
- `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al
- `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware
- `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64
- `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image
- define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al
- initial support for targeting x86/amd64 UEFI and BIOS
- some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default
- GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends
up at `$uefipart`=15)
- GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14)
- `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac
- hard-requires gdisk package host-side
- add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains
download
- add pre_prepare_partitions() extension method, for custom partition size calculations
- add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes`
- add post_create_partitions() extension method, mostly for easy debugging
- add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification
- add post_install_kernel_debs() extension method.
- multiple fixes to bsp to avoid spurious errors when files are not where it expects
- v4: detect `update-initramfs` failure and abort build with useful message if it does
- v4: show useful stacktrace in `exit_with_error`
- if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong
- v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand
- v4: preserve kernel .config's dates when copying
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* extensions framework; refactor tool fetching/building into extensions
- a few examples of core refactoring using extensions
- sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example)
- marvel-tools extension, enabled by 2 different mvebu family includes
- rkbin-tools extension, enabled by rockship64_common family include
- amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example)
- removed the 'testings' fetch_from_repo completely since not used anywhere.
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension
- v3: added `growroot`-awareness to `armbian-resize-filesystem`
- the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout
- `growroot` is installed on UEFI images by default, that handles growing partition during initramfs
- now `armbian-resize-filesystem` handles `resize2fs` only, and works.
- v4: reworked UEFI board/family/include structure:
- use Distro's `linux-generic` kernel only for `current`
- `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64
- `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al
- v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb`
- to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it
- v5: many fixes
- v7: more small fixes.
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* .wip for the RaspberryPi 4B via new flash-kernel extension
- this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu
- flash-kernel is not really a bootloader
- it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader
- flash-kernel is standard Debian package, but has only been tested on Ubuntu releases
- it is really only known-working since Hirsute release.
- Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels.
- v3: fixed focal rootfs build. untested.
- v3: better variable names, preparing for source-built kernel.
- v5: new edge build with pure mainline kernel.
- v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's)
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* Added first patch to edge x86 related to wifi drivers
* extensions: leave hostapd alone; remove hackish ext; block reentrancy
- package-list-utils does not belong in this PR
- grub or bcm2711 is not the place to remove hostapd
- block recursive enable_extension() calls, for now.
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* gen-sample-extension-docs: fix: avoid counter in generated sample
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* extensions: dependencies: enable_extension() in extensions with a stack
- and better stacktraces, I hope
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* Remove code from package list since we don't have it in repository
Adjust kernel config to disable driver that needs further polishing.
* Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work
* amd64: allow building amd64 on aarch64 with system toolchain
- conditionally add gcc-x86-64-linux-gnu to hostdeps
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* add libelf-dev directly to hostdeps (and Dockerfile), remove extension
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause
- which was the missing $image_name for non-arm64 & non-arm, so: x86 for example
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* grub: really obliterate u-boot stuff from BSP
- for now. soon we'll refactor u-boot so not have to do this
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* flash-kernel: really obliterate u-boot stuff from BSP
- for now. soon we'll refactor u-boot so not have to do this
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* extensions: add host_dependencies_ready() hook
- this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* Add verification functions for correct selection.
* If UEFI Skip symlink creation
* Do not create dtb package for amd64
* Skip scripts folder cleaning if build process native.
Skip creating postinst prerm scripts for headers.
* Skip applying headers-debian-byteshift.patch if build native
* Fix architecture syntax as x86_64
* Revert "amd64: allow building amd64 on aarch64 with system toolchain"
This reverts commit 0c5ee20bb1.
* Compare architectures before starting compilation.
Signed-off-by: The-going <48602507+The-going@users.noreply.github.com>
* extensions: cleanups after fixes by the-Going
- packaging:
- there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below
- it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks!
- Explicitly disallow "reverse cross compile" in amd64.conf.
- whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today.
- fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision.
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* grub: remove debug
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* firmware: allow installing `armbian-firmware-full`; make it really full
- can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI.
- don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes`
- rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts.
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all()
- to reset/unset everything done by the the initializer, so build can run again
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* extensions: remove 'global' logging, for use with build_all_ng
- enable_extensions() will have to live on without logging to file. it's just too early.
- now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager()
- now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager()
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* extensions: build-all-ng.sh bugfix due to extension's debug to stdout
- extensions (among other things) can produce output to stdout when activated
- fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger
- debugging output got mixed with "idential", rendering hash cache void for families that used extensions
- eg: sunxi, others
- fix is to send stdout to the bitbucket when sourcing the board & arch config files
- proper fix would be stop using stdout in this case and use return code for check_hash()
- one day soon
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
* Add CI build targets
Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com>
Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
236 lines
6.3 KiB
Bash
236 lines
6.3 KiB
Bash
#!/bin/bash
|
|
#
|
|
# Copyright (c) 2013-2021 Igor Pecovnik, igor.pecovnik@gma**.com
|
|
#
|
|
# This file is licensed under the terms of the GNU General Public
|
|
# License version 2. This program is licensed "as is" without any
|
|
# warranty of any kind, whether express or implied.
|
|
#
|
|
# This file is a part of the Armbian build script
|
|
# https://github.com/armbian/build/
|
|
|
|
# Functions:
|
|
|
|
# mount_chroot
|
|
# umount_chroot
|
|
# unmount_on_exit
|
|
# check_loop_device
|
|
# install_external_applications
|
|
# write_uboot
|
|
# copy_all_packages_files_for
|
|
# customize_image
|
|
# install_deb_chroot
|
|
# run_on_sdcard
|
|
|
|
|
|
|
|
|
|
# mount_chroot <target>
|
|
#
|
|
# helper to reduce code duplication
|
|
#
|
|
mount_chroot()
|
|
{
|
|
|
|
local target=$1
|
|
mount -t proc chproc "${target}"/proc
|
|
mount -t sysfs chsys "${target}"/sys
|
|
mount -t devtmpfs chdev "${target}"/dev || mount --bind /dev "${target}"/dev
|
|
mount -t devpts chpts "${target}"/dev/pts
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# umount_chroot <target>
|
|
#
|
|
# helper to reduce code duplication
|
|
#
|
|
umount_chroot()
|
|
{
|
|
|
|
local target=$1
|
|
display_alert "Unmounting" "$target" "info"
|
|
while grep -Eq "${target}.*(dev|proc|sys)" /proc/mounts
|
|
do
|
|
umount -l --recursive "${target}"/dev >/dev/null 2>&1
|
|
umount -l "${target}"/proc >/dev/null 2>&1
|
|
umount -l "${target}"/sys >/dev/null 2>&1
|
|
sleep 5
|
|
done
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# unmount_on_exit
|
|
#
|
|
unmount_on_exit()
|
|
{
|
|
|
|
trap - INT TERM EXIT
|
|
local stacktrace="$(get_extension_hook_stracktrace "${BASH_SOURCE[*]}" "${BASH_LINENO[*]}")"
|
|
display_alert "unmount_on_exit() called!" "$stacktrace" "err"
|
|
if [[ "${ERROR_DEBUG_SHELL}" == "yes" ]]; then
|
|
ERROR_DEBUG_SHELL=no # dont do it twice
|
|
display_alert "MOUNT" "${MOUNT}" "err"
|
|
display_alert "SDCARD" "${SDCARD}" "err"
|
|
display_alert "ERROR_DEBUG_SHELL=yes, starting a shell." "ERROR_DEBUG_SHELL" "err"
|
|
bash < /dev/tty || true
|
|
fi
|
|
|
|
umount_chroot "${SDCARD}/"
|
|
umount -l "${SDCARD}"/tmp >/dev/null 2>&1
|
|
umount -l "${SDCARD}" >/dev/null 2>&1
|
|
umount -l "${MOUNT}"/boot >/dev/null 2>&1
|
|
umount -l "${MOUNT}" >/dev/null 2>&1
|
|
[[ $CRYPTROOT_ENABLE == yes ]] && cryptsetup luksClose "${ROOT_MAPPER}"
|
|
losetup -d "${LOOP}" >/dev/null 2>&1
|
|
rm -rf --one-file-system "${SDCARD}"
|
|
exit_with_error "debootstrap-ng was interrupted" || true # don't trigger again
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# check_loop_device <device_node>
|
|
#
|
|
check_loop_device()
|
|
{
|
|
|
|
local device=$1
|
|
if [[ ! -b $device ]]; then
|
|
if [[ $CONTAINER_COMPAT == yes && -b /tmp/$device ]]; then
|
|
display_alert "Creating device node" "$device"
|
|
mknod -m0660 "${device}" b "0x$(stat -c '%t' "/tmp/$device")" "0x$(stat -c '%T' "/tmp/$device")"
|
|
else
|
|
exit_with_error "Device node $device does not exist"
|
|
fi
|
|
fi
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# write_uboot <loopdev>
|
|
#
|
|
write_uboot()
|
|
{
|
|
|
|
local loop=$1 revision
|
|
display_alert "Writing U-boot bootloader" "$loop" "info"
|
|
TEMP_DIR=$(mktemp -d || exit 1)
|
|
chmod 700 ${TEMP_DIR}
|
|
revision=${REVISION}
|
|
if [[ -n $UPSTREM_VER ]]; then
|
|
revision=${UPSTREM_VER}
|
|
dpkg -x "${DEB_STORAGE}/linux-u-boot-${BOARD}-${BRANCH}_${revision}_${ARCH}.deb" ${TEMP_DIR}/
|
|
else
|
|
dpkg -x "${DEB_STORAGE}/${CHOSEN_UBOOT}_${revision}_${ARCH}.deb" ${TEMP_DIR}/
|
|
fi
|
|
|
|
# source platform install to read $DIR
|
|
source ${TEMP_DIR}/usr/lib/u-boot/platform_install.sh
|
|
write_uboot_platform "${TEMP_DIR}${DIR}" "$loop"
|
|
[[ $? -ne 0 ]] && exit_with_error "U-boot bootloader failed to install" "@host"
|
|
rm -rf ${TEMP_DIR}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# copy_all_packages_files_for <folder> to package
|
|
#
|
|
copy_all_packages_files_for()
|
|
{
|
|
local package_name="${1}"
|
|
for package_src_dir in ${PACKAGES_SEARCH_ROOT_ABSOLUTE_DIRS};
|
|
do
|
|
local package_dirpath="${package_src_dir}/${package_name}"
|
|
if [ -d "${package_dirpath}" ];
|
|
then
|
|
cp -r "${package_dirpath}/"* "${destination}/" 2> /dev/null
|
|
display_alert "Adding files from" "${package_dirpath}"
|
|
fi
|
|
done
|
|
}
|
|
|
|
|
|
|
|
|
|
customize_image()
|
|
{
|
|
|
|
# for users that need to prepare files at host
|
|
[[ -f $USERPATCHES_PATH/customize-image-host.sh ]] && source "$USERPATCHES_PATH"/customize-image-host.sh
|
|
|
|
call_extension_method "pre_customize_image" "image_tweaks_pre_customize" << 'PRE_CUSTOMIZE_IMAGE'
|
|
*run before customize-image.sh*
|
|
This hook is called after `customize-image-host.sh` is called, but before the overlay is mounted.
|
|
It thus can be used for the same purposes as `customize-image-host.sh`.
|
|
PRE_CUSTOMIZE_IMAGE
|
|
|
|
cp "$USERPATCHES_PATH"/customize-image.sh "${SDCARD}"/tmp/customize-image.sh
|
|
chmod +x "${SDCARD}"/tmp/customize-image.sh
|
|
mkdir -p "${SDCARD}"/tmp/overlay
|
|
# util-linux >= 2.27 required
|
|
mount -o bind,ro "$USERPATCHES_PATH"/overlay "${SDCARD}"/tmp/overlay
|
|
display_alert "Calling image customization script" "customize-image.sh" "info"
|
|
chroot "${SDCARD}" /bin/bash -c "/tmp/customize-image.sh $RELEASE $LINUXFAMILY $BOARD $BUILD_DESKTOP $ARCH"
|
|
CUSTOMIZE_IMAGE_RC=$?
|
|
umount -i "${SDCARD}"/tmp/overlay >/dev/null 2>&1
|
|
mountpoint -q "${SDCARD}"/tmp/overlay || rm -r "${SDCARD}"/tmp/overlay
|
|
if [[ $CUSTOMIZE_IMAGE_RC != 0 ]]; then
|
|
exit_with_error "customize-image.sh exited with error (rc: $CUSTOMIZE_IMAGE_RC)"
|
|
fi
|
|
|
|
call_extension_method "post_customize_image" "image_tweaks_post_customize" << 'POST_CUSTOMIZE_IMAGE'
|
|
*post customize-image.sh hook*
|
|
Run after the customize-image.sh script is run, and the overlay is unmounted.
|
|
POST_CUSTOMIZE_IMAGE
|
|
}
|
|
|
|
|
|
|
|
|
|
install_deb_chroot()
|
|
{
|
|
|
|
local package=$1
|
|
local variant=$2
|
|
local transfer=$3
|
|
local name
|
|
local desc
|
|
if [[ ${variant} != remote ]]; then
|
|
name="/root/"$(basename "${package}")
|
|
[[ ! -f "${SDCARD}${name}" ]] && cp "${package}" "${SDCARD}${name}"
|
|
desc=""
|
|
else
|
|
name=$1
|
|
desc=" from repository"
|
|
fi
|
|
|
|
display_alert "Installing${desc}" "${name/\/root\//}"
|
|
[[ $NO_APT_CACHER != yes ]] && local apt_extra="-o Acquire::http::Proxy=\"http://${APT_PROXY_ADDR:-localhost:3142}\" -o Acquire::http::Proxy::localhost=\"DIRECT\""
|
|
# when building in bulk from remote, lets make sure we have up2date index
|
|
[[ $BUILD_ALL == yes && ${variant} == remote ]] && chroot "${SDCARD}" /bin/bash -c "DEBIAN_FRONTEND=noninteractive apt-get $apt_extra -yqq update"
|
|
chroot "${SDCARD}" /bin/bash -c "DEBIAN_FRONTEND=noninteractive apt-get -yqq $apt_extra --no-install-recommends install $name" >> "${DEST}"/${LOG_SUBPATH}/install.log 2>&1
|
|
[[ $? -ne 0 ]] && exit_with_error "Installation of $name failed" "${BOARD} ${RELEASE} ${BUILD_DESKTOP} ${LINUXFAMILY}"
|
|
[[ ${variant} == remote && ${transfer} == yes ]] && rsync -rq "${SDCARD}"/var/cache/apt/archives/*.deb ${DEB_STORAGE}/
|
|
|
|
}
|
|
|
|
|
|
run_on_sdcard()
|
|
{
|
|
|
|
# Lack of quotes allows for redirections and pipes easily.
|
|
chroot "${SDCARD}" /bin/bash -c "${@}" >> "${DEST}"/${LOG_SUBPATH}/install.log
|
|
|
|
}
|