What it is
Tensorseed is a tool that mirrors Hugging Face models and datasets and
verifies them. Bytes move over BitTorrent v2 and hybrid torrents, plus
HTTP and S3-compatible mirrors. A signed catalog sits on top, so you
can find a revision, check its signature, and fetch the same files from
more than one place.
Catalog backends are pluggable. The first backends are a chain-free
signed file tree (the default for this site) and Chia DataLayer. The
chain can change later without changing how clients verify content.
Signed catalog
Each revision publishes a signed head, an append-only delta, and
periodic snapshots. Clients pin a publisher key, walk a hash chain,
and reject rollback, forks, and stale heads. The catalog for this
project will be served from
catalog.tensorseed.dev.
BitTorrent v2
Model and dataset bytes are published as BitTorrent v2 or hybrid
torrents, with HTTP webseeds so a pull still works when the swarm is
quiet. Operators can seed what they already verified.
HTTP and S3 mirrors
Named, repeatable provider blocks describe R2, B2, S3, and other
HTTP mirrors. A signed mirrors.toml is published with
the catalog so clients can discover sources without a chain.
Verify the catalog
Pin this publisher public key (minisign, key id CE1E0E3D543DFFB8) in
catalog.publisher_pubkey. Clients reject any catalog
file that is not signed by it.
RWS4/z1UPQ4ezmjrcpKqDYpqyn5h1WUS2oDs1B73Dh91kVzzNIZd03xd
Set catalog.backend = "file",
catalog.name = "tensorseed", and
file.base_url = "https://catalog.tensorseed.dev/",
then run tensorseed catalog info, list,
show, pull, and verify.
Source and license
Tensorseed is licensed under
AGPL-3.0-or-later.
Copyright Tuned Chaos.
The GitHub repository is not public yet. When it is, it will be linked
from here:
https://github.com/TunedChaos/TensorSeed
(placeholder, not public).