Fixes an issue where subsequent repository runs would fail with 404 errors
for Release files when no new packages were added. The problem was that
snapshots were only created conditionally, leading to missing components
in the published repository.
Changes:
- update_main: always drop and recreate common snapshot (remove check that
prevented updates if snapshot was already published)
- process_release: always create utils/desktop snapshots even if repos are
empty, ensuring all components are included in publish
- merge_repos: always create snapshots for all repos and create repos if
they don't exist, preventing missing components on merge
This ensures the repository structure is complete on every run, regardless
of whether new packages are added.
Signed-off-by: Igor Pecovnik <igor@armbian.com>
This fixes the case where repositories like debs-beta only have packages
in the main/common component (e.g., sid with only kernel packages).
Previously, the merge command would skip publishing if both utils and
desktop repos were empty, resulting in an incomplete repository.
Now we always publish at minimum the main/common component, ensuring all
distributions with any packages get properly published.
This commit implements a complete parallel repository management system
that allows building and publishing Debian repositories in parallel,
significantly reducing build time for multiple distributions.
- `update-main`: Builds common/main component once for all releases
- `update -R <release>`: Builds release-specific components in isolated DBs
- `merge`: Combines common + release-specific components into final repos
- Isolated databases (aptly-isolated-<release>) avoid locking during parallel builds
- Common component built once, not duplicated per release
- Release-specific components (utils, desktop) built independently
- Final merge combines all components with proper GPG signing
- Fixed GPG signing to target top-level Release files (dists/{release}/Release)
- Pool cleanup before publishing avoids "file already exists" errors
- Smart package import skips duplicates during merge
- Proper handling of empty repositories and missing components
- Improved error handling and logging throughout
1. update-main: Build common component (once)
2. update -R <release>: Parallel workers build release-specific components
3. merge: Combine all components and publish with GPG signatures
This enables GitHub Actions to run multiple release builders in parallel,
reducing total repository build time from hours to minutes.
Signed-off-by: Igor Pecovnik <igor@armbian.com>
* tools/repository/extract-repo.sh: simplify extraction by copying directly from pool
Remove dependency on Packages index files. Instead of parsing package metadata
to find file locations, directly scan the pool/ directory structure and copy all
.deb files found in each component subdirectory.
This simplifies the code and makes it more robust since it doesn't rely on
index files being present or correctly formatted.
Signed-off-by: Igor Pecovnik <igor@armbian.com>
* Add helper script: recursively clean Armbian Debian package artifacts
* Update tools/repository/cleanup-debs.sh
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
---------
Signed-off-by: Igor Pecovnik <igor@armbian.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
- Remove redundant run_cmd function, use run_aptly consistently
- Include EOS releases in repository processing (remove -ve 'eos' filter)
- Fix showall command to auto-discover repos from isolated databases
- Fix source file preservation in parallel mode (keep sources for workers)
- Fix snapshot management to handle published snapshots correctly
- Fix drop_unsupported_releases to only drop unsupported, not all repos
- Remove isolated DB cleanup that caused data loss on each run
- Remove inappropriate sudo usage from html and date commands
- Enable FORCE_PUBLISH by default for better workflow
- Improve common snapshot creation in isolated mode
- Add isolated database support to showall function
Signed-off-by: Igor Pecovnik <igor@armbian.com>
**Repository tools: enable parallel repo generation and refactor repo.sh**
Add support for parallel repository generation using GitHub Actions, allowing
multiple workers to build different releases concurrently without database
locking conflicts.
### Highlights
- Add `-R/--single-release`, `update-main`, and `merge` for CI-level parallelism
- Use isolated per-release databases; remove local `-j/--parallel-jobs`
- Sign all Release files; optimize signing order
- Add `extract-repo.sh` for extracting and reorganizing packages from repositories
- Replace file-based logging with syslog (`logger`), removing sudo requirement
- Improve error handling, signing logic, and robustness
- Refactor repo.sh for better readability and documentation
- Apply CodeRabbit-recommended fixes
Signed-off-by: Igor Pecovnik <igor@armbian.com>
* Added Debian 12 Bookworm.
* Link identical
* Make minimal truly minimal again
Move netplan.io to CLI image only
* Add netplan.io to Bullseye standard CLI
Co-authored-by: Konstantin Litvinov <klitvinov@piesoft.us>
Co-authored-by: Igor <igor@armbian.com>
* Moving repository handling from main script
- cleaning
- working
TBD: handling of parameters
* Not needed anymore
* Paramter handling, edit config in mktemp
* Typo
* Improve string handling
* Cleanining
* Compacting
* Change to backslash compatible sed replacement
* Publish all, update selected to avoid having erros at update
* Add support for html index generation
* Bugfix
* More arch
* Don't run publish if there is nothing in the incoming
* Add repo sync control file creation
* Fix hardcoded value
