Software designed to make data distribution easy.
Pelican federates data repositories across the country and delivers their objects to researchers — open, fast, and at national scale.
The Pelican Software Suite
Four components, one federation.
Watch a request travel the federation — or pick a component to see the part it plays.
Storage Services
Central Services
Live object request
The client sends a request — it first reaches the Director, the federation’s routing service.
Clients & Tools
Get your data, anywhere.
Pelican meets your data where it lives — from the command line to your Python code, your batch jobs, or a browser.
Meet the Components
What each part does.
Two storage services run by data owners and sites, and two central services run by the federation — together they make data findable, trustworthy, and fast.
Origin
Storage Service
An Origin is how data enters a federation. It sits in front of an existing repository — a POSIX filesystem, an S3 bucket, and more — and exports its objects without moving or copying them. At startup it registers its namespace and public key with the Registry, then advertises the prefixes it serves to the Director. From that moment on, the data it backs is reachable from anywhere in the federation.
Cache
Storage Service
A Cache keeps copies of objects close to where researchers and compute live, on fast, high-bandwidth, low-latency storage. When a client requests an object the Director points it at the best Cache: if that Cache already holds the object it serves it instantly, and if not it fetches the object once from the Origin, keeps a copy, and streams it to the client. Popular data arrives quickly — and the Origin is never overwhelmed.
Director
Central Service
The Director is the federation’s traffic controller. It keeps a live map of every Origin and Cache and the namespaces they serve, and when a client asks for an object it returns a ranked list of Caches to try — ordered by availability, load, and proximity rather than geography alone. It also hosts the federation’s discovery endpoint, so clients and servers can locate the other central services. Notably, data never passes through the Director itself; it only does the routing.
Registry
Central Service
The Registry is the federation’s root of trust. It records every namespace prefix together with the public key that owns it, so the federation can verify that a server is genuinely authorized to serve — or write to — a given path. Origins and Caches register automatically when they start, and federation operators can require manual approval before a new server is allowed to join. Because authorization rests on time-sensitive tokens, the Registry quietly underpins secure access across every other component.
From the Community

