8356
Wx
www-xonecas-io / files
www-xonecas-io / tickets / 1003-import-all-public-history-into-a-local-bare.md
2.4 KB
1--- 2id: 1003 3title: Import all public history into a local bare repo store 4type: feature 5status: done 6priority: p1 7area: data 8created: 2026-11-07T19:30:00Z 9updated: 2026-11-08T03:58:19Z 10source: agent 11moves: 2026-11-07T19:30:00Z inbox todo agent; 2026-11-08T01:09:36Z todo doing drag; 2026-11-08T01:23:43Z doing review agent; 2026-11-08T03:58:19Z review done drag 12--- 13 14Mirror every public repo from github.com/xonecas (44), github.com/sacenox (44) and gitlab.com/xonecas (17) into bare repos, idempotently, with the source list in one file. 15 16Codeberg stays out. Checked: 0 of its commits are missing from GitHub and GitLab. 17 18Facts from discovery: plain https clones work without a token. The GitHub API allows 60 unauthenticated requests per hour, so use it for metadata only. A blobless clone of all 182 remotes was 32 MB, full clones will be larger (not measured). Forks are large, such as pi, solarized, Font-Awesome and BrowserQuest. 19 20Done when: re-running fetches only what changed, and the store holds all 105 GitHub and GitLab remotes. 21 22## Notes 23 24- 2026-11-08 01:23Z (agent): Done: 'make import' mirrors all 105 public remotes into repos/ as bare repos, and a re-run only fetches what changed. 25 26 WHAT IS THERE: 27 - sources.txt: the one source list, with 105 clone URLs (44 github.com/xonecas, 44 github.com/sacenox, 17 gitlab.com/xonecas). I generated it from the GitHub and GitLab APIs, 3 requests in all. Codeberg is left out. 28 - 'site import' (internal/store): each remote goes to repos/<host>/<owner>/<name>.git, 8 at a time. The first run creates the bare repo and later runs 'git fetch --prune'. Only branches and tags are kept, so GitHub's refs/pull/* (other people's PR commits) are not fetched. HEAD follows the remote's default branch. A clone that fails is logged and makes the command exit non-zero, without stopping the others. 29 - repos/ is gitignored. repo.List now finds *.git at any depth. 30 31 RESULT: 105 new, 0 failed, in 25 s. The store is 351 MB on disk (full history, forks included). The second run printed '0 new, 0 updated, 105 same, 0 failed' in 16 s. 32 CHECKED: unit tests for new, same and updated against a local remote, the default branch, and parsing sources.txt. Spot checks: ffmpeg-node HEAD is master, mini-coder HEAD is main, and neither has pull refs. 33 NOT CHECKED: what happens when a remote is deleted upstream. That remote would fail and be reported, and its mirror stays.