This commit implements a complete parallel repository management system
that allows building and publishing Debian repositories in parallel,
significantly reducing build time for multiple distributions.
- `update-main`: Builds common/main component once for all releases
- `update -R <release>`: Builds release-specific components in isolated DBs
- `merge`: Combines common + release-specific components into final repos
- Isolated databases (aptly-isolated-<release>) avoid locking during parallel builds
- Common component built once, not duplicated per release
- Release-specific components (utils, desktop) built independently
- Final merge combines all components with proper GPG signing
- Fixed GPG signing to target top-level Release files (dists/{release}/Release)
- Pool cleanup before publishing avoids "file already exists" errors
- Smart package import skips duplicates during merge
- Proper handling of empty repositories and missing components
- Improved error handling and logging throughout
1. update-main: Build common component (once)
2. update -R <release>: Parallel workers build release-specific components
3. merge: Combine all components and publish with GPG signatures
This enables GitHub Actions to run multiple release builders in parallel,
reducing total repository build time from hours to minutes.
Signed-off-by: Igor Pecovnik <igor@armbian.com>
* tools/repository/extract-repo.sh: simplify extraction by copying directly from pool
Remove dependency on Packages index files. Instead of parsing package metadata
to find file locations, directly scan the pool/ directory structure and copy all
.deb files found in each component subdirectory.
This simplifies the code and makes it more robust since it doesn't rely on
index files being present or correctly formatted.
Signed-off-by: Igor Pecovnik <igor@armbian.com>
* Add helper script: recursively clean Armbian Debian package artifacts
* Update tools/repository/cleanup-debs.sh
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
---------
Signed-off-by: Igor Pecovnik <igor@armbian.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
- Remove redundant run_cmd function, use run_aptly consistently
- Include EOS releases in repository processing (remove -ve 'eos' filter)
- Fix showall command to auto-discover repos from isolated databases
- Fix source file preservation in parallel mode (keep sources for workers)
- Fix snapshot management to handle published snapshots correctly
- Fix drop_unsupported_releases to only drop unsupported, not all repos
- Remove isolated DB cleanup that caused data loss on each run
- Remove inappropriate sudo usage from html and date commands
- Enable FORCE_PUBLISH by default for better workflow
- Improve common snapshot creation in isolated mode
- Add isolated database support to showall function
Signed-off-by: Igor Pecovnik <igor@armbian.com>
**Repository tools: enable parallel repo generation and refactor repo.sh**
Add support for parallel repository generation using GitHub Actions, allowing
multiple workers to build different releases concurrently without database
locking conflicts.
### Highlights
- Add `-R/--single-release`, `update-main`, and `merge` for CI-level parallelism
- Use isolated per-release databases; remove local `-j/--parallel-jobs`
- Sign all Release files; optimize signing order
- Add `extract-repo.sh` for extracting and reorganizing packages from repositories
- Replace file-based logging with syslog (`logger`), removing sudo requirement
- Improve error handling, signing logic, and robustness
- Refactor repo.sh for better readability and documentation
- Apply CodeRabbit-recommended fixes
Signed-off-by: Igor Pecovnik <igor@armbian.com>
* Added Debian 12 Bookworm.
* Link identical
* Make minimal truly minimal again
Move netplan.io to CLI image only
* Add netplan.io to Bullseye standard CLI
Co-authored-by: Konstantin Litvinov <klitvinov@piesoft.us>
Co-authored-by: Igor <igor@armbian.com>
* Moving repository handling from main script
- cleaning
- working
TBD: handling of parameters
* Not needed anymore
* Paramter handling, edit config in mktemp
* Typo
* Improve string handling
* Cleanining
* Compacting
* Change to backslash compatible sed replacement
* Publish all, update selected to avoid having erros at update
* Add support for html index generation
* Bugfix
* More arch
* Don't run publish if there is nothing in the incoming
* Add repo sync control file creation
* Fix hardcoded value
