Pelican Logo
Pelican Platform

Release

v7.27.0

Download

Pelican v7.27.0

v7.27.0 adds a user/group ownership and admin model to all servers, a rewritten local cache, per-namespace cache storage reservations, a third-party-copy mode for the client, and a web client in the Origin. It includes all fixes from the v7.26.x line, including four security advisories.

Breaking Changes & Upgrade Notes for Admins

  • [Servers]: New user/group ownership and admin model for all Pelican servers: collections with invite links, user- and collection-level admin permissions, user/group CLI commands, and collection pages in the web UI. All Pelican servers apply new database schema migrations on first startup after upgrade #3299
  • [Servers]: Usernames are now globally unique, serving as the authorization handle for Server.UIAdminUsers, Server.UserAdminUsers, and Server.CollectionAdminUsers. Entries that are OIDC subjects no longer grant admin and are reported at startup; use Pelican usernames or Server.AdminGroups. The upgrade migration fails if one username exists under two issuers. #3618
  • [Servers]: Revamped collection ownership, ACLs, and API-key attribution models. They are keyed on immutable user/group IDs, not names. ACL grants to deleted users are dropped and API keys of deleted creators are detached. Password accounts no longer get groups from Issuer.GroupFile. #3759
  • [Cache/Director]: Director health-test files written to caches are now organized into per-director, daily-nested paths and cleaned up by two parallel mechanisms, preventing unbounded growth of the monitoring namespace #3457
  • [Registry]: The Registry now automatically removes stale pending server registration requests based on observed server activity, controlled by the new parameters #3327
  • [Director/Cache/Origin]: Removed the legacy "v1" server advertisement code, which was only used by Pelican v7.8 servers and earlier. Servers on v7.8 or older can no longer join a federation with a v7.27 Director; no server in the OSDF federation still uses v1 ads #3507

New Features and Enhancements

  • [Client/Origin]: Added a third-party-copy (HTTP COPY) mode to the Client, allowing it to orchestrate direct server-to-server transfers #3141
  • [Client]: Improved handling of client-configured caches: Director-supplied caches that duplicate a user-supplied one are now skipped, and the "try 3 caches" cap applies only to Director-supplied caches, meaning the user-provided list is tried in full #3528
  • [Client]: Overhauled the pelican config tooling and made configuration defaults generate directly from parameters.yaml, eliminating drift between documented and actual defaults #3367
  • [Client]: Cleaned up the pelican token tooling: users are warned when requesting scopes a namespace doesn't advertise, and key-ID/issuer mismatches now produce clear errors instead of a generic "no issuer found" #3470
  • [Client]: Logging.DisableProgressBars has moved to Logging.Client.DisableProgressBars #3206
  • [Client]: Added a pelican object du command that reports directory sizes, backed by a new Walk-shaped streaming API in the Go client #3541
  • [Client/Cache/Origin]: Added a new top-level GeoLocation parameter that lets Clients, Caches, and Origins declare their own physical locations to the federation; declared locations take precedence over IP-based geolocation when the Director makes routing decisions #3461
  • [Servers]: The one-time web-UI activation-code flow is now skipped when admins are pre-configured via Server.UIAdminUsers or Server.AdminGroups, since those admins can already log in via OAuth2 #3405
  • [Origin]: Web client in the Origin UI; logged-in users browse and transfer objects directly against the Origin. #3532
  • [Origin]: The embedded OAuth2 issuer's token lifespans (access, refresh, authorization/device code, and ID tokens) are now configurable under the Issuer.* namespace, with defaults matching the previous hardcoded values; this change also fixes a security bug in refresh-token tombstone cleanup #3395
  • [Origin]: Improved how ETags and checksums are computed for objects served by Origins #3477
  • [Origin]: Turns off http.tlsclientauth in Xrootd Origin by default #3562
  • [Cache]: When Lotman is enabled, the cache file-usage purge bands (Cache.FilesBaseSize, Cache.FilesNominalSize, Cache.FilesMaxSize) now accept percentages of total disk and default to percentages, so they scale with disk size instead of requiring hand-tuning #3518
  • [Cache]: Introduced "Kingfisher" per-namespace reservations, which turn the Lotman integration into a full reservation system for cache storage — caches build a lot tree from the federation namespace hierarchy, continuously renew per-namespace lots, and expose a REST API (/api/v1.0/lots/*) for namespace owners to inspect and adjust their reservations #3465
  • [Cache]: Rewrote the local cache component ("Cache V2"), with follow-ups adding a chaos/fault-injection API and pelican cache chaos CLI, a "verify once" data-scan mode for integrity-checked storage, XRootD monitoring-packet emission, and a cache-specific transfer worker count #3121 and #3525
  • [Cache/Origin]: Caches and Origins now check at startup that the XRootD plugins they require are actually installed, failing fast with a clear error instead of misbehaving later #3026

Bugs Fixed

  • [Client]: Improved the error message shown when a pelican:// URL contains a triple slash (///) #3467
  • [Client]: Transfers that fail after all download attempts succeed (e.g. a checksum mismatch) now populate the structured TransferErrorData in transfer ads, so such failures carry an ErrorType/Pelican error code instead of only a bare message #3535
  • [Director]: Fixed high-availability cascade-failure bugs in Director peer discovery and self-advertisement, where the loss of one Director could leave a surviving Director redirecting all traffic to its offline peer and returning "No sources found" to clients #3452
  • [Director]: User-supplied input (e.g. request paths) is now sanitized before being used in Prometheus labels, preventing a panic that malformed UTF-8 paths could trigger remotely #3444
  • [Director]: Server lists now use a unique name+URL key, fixing rendering issues when a Topology server duplicates a Pelican server's name #3486
  • [Registry]: The Registry now starts successfully when the external institutions API is down, degrading gracefully instead of refusing to start #3462
  • [Origin]: Made shutdown routines more robust by cancelling tickers and adding nil-checks, fixing a potential segfault during shutdown #3384
  • [Cache]: Initialize the database before the web engine serves; fixes 500s on /api/v1.0/auth/whoami during startup. #3715
  • [Cache/Origin]: Director discovery runs concurrently with a per-request timeout, so one unreachable Director no longer delays startup. #3759
  • [Cache/Origin]: A failure to fetch metadata from the Registry no longer aborts the origin/cache advertisement cycle #3484
  • [Cache/Origin]: Register all issuer public keys with the Registry at startup, not just the active one. #3504
  • [Servers]: Non-YAML files (editor backups, .rpmsave/.rpmnew, etc.) in ConfigLocations directories are now ignored instead of being silently merged into the configuration #3406

Dependencies

  • [All]: Go 1.26. #3650
  • [Cache/Origin]: XRootD 5.9.7 (Pelican fork). #3712
  • [Cache/Origin]: xrdcl-pelican 1.8.2. #3713
  • [Cache/Origin]: xrootd-s3-http 0.6.9, installed from OSG RPMs. #3612
  • [Cache]: lotman and xrootd-lotman v0.1.0, installed from OSG RPMs; runtime requires ≥ 0.1.0 and < 0.2.0. #3464 #3619

Documentation

  • [Servers]: Users, Groups, Collections, and Shares guide. #3705
  • [Director]: Director health-test mechanisms. #3675

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.26.0...v7.27.0

Changelog

  • 4b6e562f7d77d31d8901ca3e8acb8bdc374a7087 Bump version to v7.27.0-rc.7
  • c42ca75ac40800b922e8031d8ab8652aa438a7f2 Patch PR #3675: Document the mechanisms of Director Test
  • fb1cea4bdb3cbb942d8b1ac30f34eb88d2f3670c Patch PR #3705: Add Groups/Users/Collections/Shares Documentation documentation
  • b4e41e1f37ec7f3477397c82142b95be36551c33 client, plugin: refuse a remote path whose base name is not a usable file name
  • bacea49dcebdccf0a98b93cd4f340d8c819db22d client: add federation test for a tarball that writes through a symlink
  • 8ab47234036eff212291e7ce8b190f80cb6f043d client: close the archive unpacker after a download and report its errors
  • 58552bd64c258fc816a6f964e19d8f7d8854fabc client: confine archive unpacking to the destination directory with os.Root
  • fe321c7a5aafe69c2418b7a628e3a2d32fc536d5 client: keep the sanitizer's reason in the hard-link error
  • d63f9f93cc1ba6cf292274df64a246882e1ad4c2 client: refuse absolute traversal names in the unpacker instead of collapsing them
  • b8ffada43f097fb61906a65e360444bff0e9b597 client: reject collection listing entries that are not a plain path component
  • 63245594a8f4eab22028af178801b6208222785b server_utils: confine AferoFileSystem paths to the configured prefix
  • 8bbd29d998709906e3841448cd6aeb10922c9cd4 xrootd: clean the cache self-test path before checking its prefix

v7.27.0-rc.6 backports PR #3759 (fixes #3752, #3753) onto v7.27.x: collection ownership, collection ACLs, and API-key attribution are now keyed on immutable user/group IDs instead of names, so renaming or deleting a user or group can no longer leave authority behind for whoever takes the name next. It also records provider(e.g. CILogon)-asserted groups and memberships in the database and adds a guard that stops a user-admin from acting on an account that might hold administrator privileges.

Upgrade notes

  • Back up pelican.sqlite before upgrading. This release adds five database migrations (20260916120000 through 20260919120000). They are one-way: rollback is restore-from-backup, and downgrading to rc.5 or earlier is not possible on a migrated database.
  • The migration drops collection ACL grants whose user-<username> target no longer resolves to a live account, and detaches API keys whose creator was deleted (they fall closed on every user-grantable scope). To preview: SELECT group_id FROM collection_acls WHERE group_id LIKE 'user-%'; on the pre-upgrade database.
  • Two configurations that used to start now fail at startup: an unrecognized Issuer.GroupSource value, and a Server.AdminGroups / Server.UserAdminGroups / Server.CollectionAdminGroups entry Pelican cannot reserve as a group name (leading @, user- prefix, or over 255 characters). Check site configs before rolling out.
  • Issuer.GroupSource is now single-valued. On an oidc or github server, password logins no longer read Issuer.GroupFile; accounts whose groups or admin authority came only from that file lose them at their next login. Local administrators should be listed in Server.UIAdminUsers, or granted server.admin directly (user scope) or through a Pelican-created group they belong to.
  • After the upgrade every account's group-admin-status field is unknown until it logs in once; until then a server.user_admin cannot act on it. Where no group confers admin, nothing changes.
  • New parameters: Issuer.DisableGroupAutoCreation(default: false), Issuer.AssertedGroupMembershipTTL (default 168h), Issuer.GroupFileRefreshInterval (default 15m).

Changelog

  • c20e17aac8c133781f67268315166b8bdd7f7861 Bump version to v7.27.0-rc.6
  • 0c26925f9c84b9cffaccbd2a0a2c02c1eea214af config: refuse to start on an unrecognized Issuer.GroupSource
  • 221ac90c72ba3b2f3a4a4897193d0b10b2bd5c31 database: build test schemas from the migrations, not from struct tags
  • f2c2825d58f92b24d74ca1be2f1799f9fba1b47c database: carry the identifier space in the type; latch admin observations
  • 5d3bca6659e543a31d700ab669a8f30b0f050fb4 database: don't let the api_keys migration bind a key to a reused username
  • bdcc6e16d8a1635298a9e9753af3c81564bdc610 database: give the collection write paths one load-and-authorize
  • f3e8d1919ecd932ed0d3047dc4411621af9200d1 database: keep the name space and the ID space disjoint
  • 080ce3abe7efddf846c737ab1d5f4356deae8d25 database: key collection ownership and ACLs on IDs, not names
  • 6338d4e4dc452342a823f7f16a3f62f333478e8d database: make the group source single-valued and reserve admin group names
  • 458846387cb594dca42200bf5e44cb3c648f7c75 database: mirror provider-asserted group memberships
  • afdfc4a248551e41427381dade66ec007c7fb062 database: pin the durability and restart behavior of mirrored memberships
  • e92ae3b12cd543a0401058cc611b6e5e4770beb5 database: reserve admin group names as unknown, and hold them
  • 86e162bad801cfe4f2a049173cf23de3427f2255 database: retract mirrored memberships from a source no longer configured
  • c0b40bae14588a4223d6902dea32d870275edb41 database: soft-delete groups and key API tokens on the creator's ID
  • 21953c4bef62d84dbe9f9716dab751c87f6cf3e4 database: split the api_keys migration out of the groups rebuild
  • ee8c21720b1596bc17b5d6dbb3998740f86c08b5 database: stop an assertion claiming a group this server owns
  • 2f5e85c7ab9e150310ab2325da5ce9b697f0a492 database: stop banning identifiers that look like IDs
  • 2c2e700ea7e59e91d522d044e488e73aba19ac81 database: stop the slug-shape rule from rejecting an IdP's username
  • 521b8af9a8319b01ca2162eafa524f871e32c61a database: stop tombstoned groups and stale assertions from conferring authority
  • e06c99773cfb0813af72fa809b5772d1fe22faa5 images: don't fail the test image build when condor pulls in no Pelican
  • 14a8f5f2e7682e00cc59e43f7ebe3bde874caafd server_utils: condense the comments on concurrent director discovery
  • e20f8f56dc0357864ab9f5e511bc9493ecd4165c server_utils: don't let one unreachable director stall server startup
  • aa5225c61f4b5422ccf2b250a61dad40d7d7b4a8 web_ui: constrain the new tests to !windows; make the group-file refresh ctx-aware
  • 6931b87da363cacbbc2f1d74b07e3c79b28b25d5 web_ui: fix the Windows build, and back out the Pelican-group ACL filter
  • da31af6edbcf73ce86c532e1417577e94642e76c web_ui: let a user-admin act on accounts nothing will ever observe

v7.27.0-rc.5 contains the commits in rc.3 and rc.4. There is no need to run tests on two previous versions.

Changelog

v7.27.0-rc.5

  • ed51f25099b07ecf9c454c08cbe497000a410772 Patch PR #3715: Initialize cache server database before the web engine starts serving

v7.27.0-rc.4

  • 4ce7cf92f2a3320746c351cf3e0d3c497837b836 Patch PR #3712: Bump xrootd version to 5.9.7 in images and OSX
  • a2319e837dff86bd2402719ee0bab3f72903c67c Patch PR #3713: Bump xrdcl-pelican version to 1.8.2 in images and OSX

v7.27.0-rc.3

  • bc277479e17354f6155353883da828d18e70b337 Address review: drop server build tag, trim comments, remove fragile assertion
  • 44d5c73ada74bb9f315f1e163878c39fd3e86e30 launchers: skip issuer health check on an all-public embedded-issuer origin

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.27.0-rc.2...v7.27.0-rc.5

This is an intermediate build and should be skipped

Changelog

  • 4ce7cf92f2a3320746c351cf3e0d3c497837b836 Patch PR #3712: Bump xrootd version to 5.9.7 in images and OSX
  • a2319e837dff86bd2402719ee0bab3f72903c67c Patch PR #3713: Bump xrdcl-pelican version to 1.8.2 in images and OSX

This is an intermediate build and should be skipped

Changelog

  • bc277479e17354f6155353883da828d18e70b337 Address review: drop server build tag, trim comments, remove fragile assertion
  • 44d5c73ada74bb9f315f1e163878c39fd3e86e30 launchers: skip issuer health check on an all-public embedded-issuer origin

Changelog

Dependency:

  • 3224eb2c30975038cf4ebf415d0c2753a83095c8 Upgrade to Go 1.26
  • 2018dd639aa15cae8b3bc7d8cafce32932e985fa Patch PR #3637: CI/CD: Stop building the dev image for release tags
  • f928e0c0732aeeb5fcc8f6f291d1cb3f50de9c4f Patch PR #3639: Install the requirements for the dev image from the osg-testing
  • 6f6ecfd6c0d613696165ace9d652ba75e44ff4c1 Patch PR #3648: Cache: log director-test cleanup successes at debug level
  • 5fc115c5ba36693c7edf99e29f841f02cef043fa Patch PR #3693: Enable osg-testing for the dev containers
  • 3b2bf2500c1d36f0a5cf479fe237379dccb822e8 Patch PR #3612: Update xrootd-s3-http dependency in images to 0.6.9
  • f3156c6dfc32ba50a5ad7bf780499481c80047c1 Patch PR #3613: Update xrdcl-pelican dependency to 1.8.1
  • a7ddf1b7dac8e4ce1decc4c7bac4937a621cdd5c Patch PR #3619: Install lotman and xrootd-lotman from RPMs into the image

Bugfix

  • 565c5d85934d140007edb7f7c06a6dca7a56b92d Patch PR #3633: Move cookie to same-site lax
  • 1d8e06a0372fa55b90e574bb12bc951417505b69 Add mising nextra callout in Pelican Docs
  • 647640060d7bdbb534e8e56c9b567783fa086d2b Fix nil-database panic in API token verification
  • a2f8839eb008ba0f8cdb6314eff2c747ebe1f90c Initialize config before gin engine build
  • 031e5ec693e0b0f9f55a029417a97fbf5c1b43a9 Make usernames a truly global authorization handle

Misc

  • e4c56673d2758ef927acd3eea55badb0047b93b6 Remove standalone-origin feature in the backported commit
  • 4b70fd14fedab3b4a4184ed60ee6c334a9c4cd65 Remove unit tests from the future
  • 636d491bb192936973706f367f03ee1365d62655 Remove unused database.DirectorDB
  • b896ecab14f7496eeddc583bd95bc8b231399d0b Add regression coverage for the API token database handle
  • 2df58ebbdfc1443e8347bd3ca40099c9cc339447 Bump version to 7.27.0-rc.2
  • d9d41d2d582700cf2fe2ffd1963bace5805b60cb Fix linter issue

7.26 Security Patches

  • f9fd9341214ac658c7e3ac4ecdeb61ddd87dee64 Merge commit from fork
  • f6ecd88d94bde541200602a2174d8ccd283d923c Merge commit from fork
  • 4a8c350c9152f19ccae1673e3fb853b298dd140e Merge commit from fork

v7.27.x built with xrootd 6

Changelog

  • 78ce679bbdef6322d31b6bec2fcc83f1d3b27c15 Bump version to 7.27.0-rc.1
  • 006e6139d21a7aaa25113ea036c392a6e91ee0c1 Patch PR #3504: Register all issuer public keys at startup
  • a4d2795a590ff441d1ac19167bd53fc67ea3f56c Patch PR #3532: Add Web Client to Origin
  • 5e89e6b88d3d027d41ebe13d69a86db1b683e026 Patch PR #3562: Turns off http.tlsclientauth in Xrootd Origin by default
  • 3e2f00249e4d736f7fa226bd175370c80b43cf02 Patch PR #3581: Remove stale pre-split CLI doc trees that break the MDX build

Changelog

  • 1d04b402c72f2365a01122c9267842078cdb1fdf Register OIDC routes when the origin and director are not co-located
  • 6e6dd446f6cb51153806df0c3697f3bf75a09d4a Set version to 7.27.0-experimental.3

Changelog

  • b5e1ca5a9e1994e073999e5b7dd8d84476d1d639 Accept opaque-blob object metadata via multipart upload
  • 71d52fd1b581647f180b1cc5da8bbb34580fc4ed Add POSC + object-metadata tracking and publisher to V2 origin
  • 90f74b8afa25125ab47d72996ca3dde8b6be3ab9 Initial draft of metadata upload functionality.
  • c59923a1d9eaedf3ac145bcda3a514a2d7de50e7 Refine object header metadata
  • 0eb1cc728cceb0fea1e6d204cba8b83e681692c9 Set version to 7.27.0-experimental.2
  • 6a091eed2cd1003111efd706d3a944c646f923e7 Switch to immediate transaction locks to avoid SQLITE_BUSY

Changelog

  • c02ec8b45e5efb3f37d2368d84828e2aea5ea86c Update actions to run on experimental builds as well.