Description
Describe the problem
Metadata for the libraries is stored persistently by the sync
command in a JSON file (referred to as "DB" in the codebase).
The file consists of two arrays:
Libraries
: list of the librariesReleases
: list of all releases of all libraries
The sync
command only generates an entry in the Libraries
array when there is at least one valid release for the library.
This is problematic because it means that maintenance commands for the library will fail with an ambiguous and confusing error message:
$ libraries-repository-engine modify --repo-url https://example.com FooLib
2025/07/04 07:55:35 Loaded 8141 libraries from DB
error: Library of name FooLib not found
$ libraries-repository-engine modify --types Contributed FooLib
2025/07/04 07:56:49 Loaded 8141 libraries from DB
error: Library of name FooLib not found
$ libraries-repository-engine remove FooLib
2025/07/04 07:53:18 Loaded 8141 libraries from DB
error: Library name FooLib not found
🐛 There is no way to know whether the command failed due to an error, or was expected to fail due to a lack of a valid release in the library repository.
To reproduce
Setup
$ DEMO_FOLDER=\"/tmp/libraries-repository-engine-demo\"
$ mkdir \"$DEMO_FOLDER\"
$ curl -fsSL https://raw.githubusercontent.com/arduino/arduino-lint/main/etc/install.sh | BINDIR=\"$DEMO_FOLDER\" sh
$ echo "
{
\"BaseDownloadUrl\": \"http://www.example.com/\",
\"LibrariesFolder\": \"$DEMO_FOLDER/libraries/\",
\"LogsFolder\": \"$DEMO_FOLDER/logs/\",
\"LibrariesDB\": \"$DEMO_FOLDER/libraries_db.json\",
\"LibrariesIndex\": \"$DEMO_FOLDER/library_index.json\",
\"GitClonesFolder\": \"$DEMO_FOLDER/gitclones\",
\"DoNotRunClamav\": true,
\"ArduinoLintPath\": \"$DEMO_FOLDER/arduino-lint\"
}
" > "$DEMO_FOLDER/config.json"
$ echo "
https://github.com/arduino-libraries/Servo.git|Arduino|Servo
https://github.com/arduino-libraries/LuckyShield.git|Arduino|Arduino_LuckyShield
" > "$DEMO_FOLDER/registry.txt"
$ touch "$DEMO_FOLDER/libraries_db.json"
Demo
$ libraries-repository-engine sync --config-file "$DEMO_FOLDER/config.json" "$DEMO_FOLDER/registry.txt"
[...]
2025/07/04 15:22:19 Scraping https://github.com/arduino-libraries/LuckyShield.git
2025/07/04 15:22:21 Checking out tag: v1.0.0
2025/07/04 15:22:21 Release Lucky Shield:1.0.0 has wrong library name, should be Arduino_LuckyShield
2025/07/04 15:22:21 Checking out tag: v1.0.2
2025/07/04 15:22:21 Release Lucky Shield:1.0.2 has wrong library name, should be Arduino_LuckyShield
2025/07/04 15:22:21 Checking out tag: v1.0.3
2025/07/04 15:22:21 Release Lucky Shield:1.0.3 has wrong library name, should be Arduino_LuckyShield
2025/07/04 08:22:21 Completed worker!
2025/07/04 15:22:19 Scraping https://github.com/arduino-libraries/Servo.git
[...]
2025/07/04 08:22:30 Completed worker!
2025/07/04 08:22:30 ...DONE
$ cat "$DEMO_FOLDER/libraries_db.json"
{
"Libraries": [
{
"Name": "Servo",
"Repository": "https://github.com/arduino-libraries/Servo.git",
"LatestCategory": "Device Control"
}
],
"Releases": [
[...]
]
}
$ libraries-repository-engine modify --config-file "$DEMO_FOLDER/config.json" --repo-url https://example.com Arduino_LuckyShield
2025/07/04 08:30:13 Loaded 1 libraries from DB
error: Library of name Arduino_LuckyShield not found
$ libraries-repository-engine modify --config-file "$DEMO_FOLDER/config.json" --types Contributed Arduino_LuckyShield
2025/07/04 08:31:12 Loaded 1 libraries from DB
error: Library of name Arduino_LuckyShield not found
$ libraries-repository-engine remove --config-file "$DEMO_FOLDER/config.json" Arduino_LuckyShield
2025/07/04 08:28:46 Loaded 1 libraries from DB
error: Library name Arduino_LuckyShield not found
Expected behavior
Correctly configured modify
and remove
commands are successful even when the library does not have any valid releases.
This can be achieved by changing the behavior of the sync
command so that it always adds an entry to the Libraries
array, even in the case where the library doesn't have any valid releases.
libraries-repository-engine version
Additional context
Example of the current behavior causing confusion:
arduino/library-registry#4978 (review)
Issue checklist
- I searched for previous reports in the issue tracker
- My report contains all necessary details