Pelican Logo
Pelican Platform

Pelican Releases


V7.9

Highlights

Congratulations to @patrickbrophy for his first contribution to Pelican!

For Users
  • [Client] Clients can now read the PELICAN_NEAREST_CACHE environment variable for determining which cache to use #1234
  • [Client] Clients can now list a collection of objects and get statistics of an object #1393 #1400
  • [Plugin] The plugin now advertises the ability to move pelican:// objects #1265
  • [Client] Additional error handling has been introduced that helps users build custom clients on top of the Pelican golang libraries #1263
  • [Client] Clients can now send a timeout value in their object requests, indicating to Pelican services the maximum time the client is willing to wait for requests #1299
  • [Cache/Client] Caches now report to clients whether the data they returned came from the disk on the cache or from the origin #1301
  • [All] Pelican now warns users about unrecognized configuration input #1325
For Admins
  • [Cache] Caches no longer shutdown on startup if advertisements fail, allowing admins to fix registrations #1244
  • [Director/Cache] The director can now filter caches marked as down in Topology #1260
  • [Origin] Pelican can now create "XRoot" backends for adding xroot-only protocol servers to the federation #1285
  • [Origin] HTTP backend export information has been added to Origin webpages #1291
  • [Origin] Origins can now export Globus collections as one of their backends #1336
  • [Director] The Director only responds to object requests with up to six Origin/Cache candidates #1304
  • [All] Pelican images are now built on top of EL9 from the OSG23 repositories #1288
  • [All] Servers now have a more permissive 5s timeout when checking for the existence of required remote resources #1328
  • [Director] The Director now checks object availability among Origins for object requests sent to Origins #1283
  • [All] Pelican now supports layered configuration through the specification of extra config directories in the root config using the ConfigLocations key #1313

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.8.0...v7.9.0

Changelog
  • [Director]: Bug fixes in stat code so that the Director stats against the correct URL for authenticated Topology origins.

Highlights

Congratulations to @patrickbrophy for his first contribution to Pelican!

For Users
  • [Client] Clients can now read the PELICAN_NEAREST_CACHE environment variable for determining which cache to use #1234
  • [Client] Clients can now list a collection of objects and get statistics of an object #1393 #1400
  • [Plugin] The plugin now advertises the ability to move pelican:// objects #1265
  • [Client] Additional error handling has been introduced that helps users build custom clients on top of the Pelican golang libraries #1263
  • [Client] Clients can now send a timeout value in their object requests, indicating to Pelican services the maximum time the client is willing to wait for requests #1299
  • [Cache/Client] Caches now report to clients whether the data they returned came from the disk on the cache or from the origin #1301
  • [All] Pelican now warns users about unrecognized configuration input #1325
For Admins
  • [Cache] Caches no longer shutdown on startup if advertisements fail, allowing admins to fix registrations #1244
  • [Director/Cache] The director can now filter caches marked as down in Topology #1260
  • [Origin] Pelican can now create "XRoot" backends for adding xroot-only protocol servers to the federation #1285
  • [Origin] HTTP backend export information has been added to Origin webpages #1291
  • [Origin] Origins can now export Globus collections as one of their backends #1336
  • [Director] The Director only responds to object requests with up to six Origin/Cache candidates #1304
  • [All] Pelican images are now built on top of EL9 from the OSG23 repositories #1288
  • [All] Servers now have a more permissive 5s timeout when checking for the existence of required remote resources #1328
  • [Director] The Director now checks object availability among Origins for object requests sent to Origins #1283
  • [All] Pelican now supports layered configuration through the specification of extra config directories in the root config using the ConfigLocations key #1313
  • [All] Fixed a bug that prevented some PELICAN_* environment variables from being read correctly #1451

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.8.0...v7.9.2

Changelog
  • d18dd643 Add config knob to enable pprof endpoints
  • 2b1a1c2f Add config knob to turn off stat for the director
  • c8982a6d Add pprof endpoints
  • 3ca0f14d Add two Prometheus metrics for directorstat queries
  • 26f8e39f Change to use EnableStat instead
  • 559f64e9 Fix deadlock in recordAd and LaunchTTLCache
  • aca7a484 Pre Release: Update npm version
  • f49108fe Remove stat utils for caches as well

V7.8

Highlights

For Users
  • [Client] Stopped file transfers are now retryable errors #1101
  • [Client] General improvements to error messages within the client and the plugin, ensuring more accurate errors and removing redundancies
  • [Client] The client can bypass the cache and download directly from the origin if a directread query parameter is set #1166
For Admins
  • [Documentation] Added documentation on how to get the public key of a running origin #1078
  • [Documentation] Added documentation on how to run pelican servers via docker containers #1089
  • [Origin] Pelican origin token creation now respects the issuer flag #1135
  • [Cache] Cache UI will now redirect to a login page for an unauthorized user #1103
  • [Origin/Cache] Improved “site name” handling #1176
  • [Director] Director JWKS cache no longer becomes invalid when the registry goes down #1119
  • [Director] Added a prometheus metric keep track of the total number of advertisements to the director #1133
  • [Director] Enable director login via OAuth #1150
  • [Cache] Allow the cache xrootd to have multiple data/meta mount locations #1191
  • [Director/Registry/Origin/Cache[ Read environment variables with PELICAN prefix for Pelican binaries with alias (stashcp and osdf) and deprecate alias-specific prefixes #1162
Changelog
  • Correct typo in map name to fix ad sorting/filtering and add unit test with real topology data
Changelog
  • Fix issue with filtering/consolidating multi-export topology origins
Changelog
  • Fix panic in Director
  • Improve Director's error response to clients
Changelog

Disable ZTN authentication on Pelican origins

Changelog

Bug fixes to restore missing collections-url when the Director is redirecting to a topology-only origin.

Changelog
  • bb74abb7 Adjust timeout test to reflect new timeout value
  • e0dd8794 Build xrootd-multiuser from source in dev container
  • 37beb73f Cleanup unstopped ticker in WaitUntilWorking
  • 0dead744 Fix bug w/ client not working for WithToken option
  • 06865b24 Increase WaitUntilWorking transfer timeout and adjust error message
  • ece8c4c7 Pre Release: Update npm version
Changelog
  • a92d4954 Add an XRoot storage backend
  • 9c09ab14 Aggregate gin routes for Prometheus metrics
  • 19d1c86c Cleanup missed linter item
  • 6364cea7 Incorporate review feedback
  • 4a026005 Pre Release: Update npm version
Changelog
  • 3d129f6e Concatenate multiple issuer names for the same issuer url
  • f1a7028c Improve error reporting for cache self-test
  • b0e78303 Improve scitoken generation error handling
  • f10f7cad Pre Release: Update npm version
  • d11c32d7 Pre Release: Update npm version
  • 66d2dcee Pull sefl-test folder out from the director tests
  • 8f5102c8 Use Cache.LocalRoot internally
  • 3a8652c2 Use and to connect two issuer names
  • 694b3524 Use externalWebUrl as the token issuer for self-tests
Changelog

Fixed Cache shutting down if advertisement fails #1244 Plugin advertises PluginVersion with -classad #1252

V7.7

Highlights

For Users
  • [Plugin] stash-plugin now supports recursive uploads and downloads #1011
For Admins
  • [General] Fixed administrator authentication check failure for all servers #1013
  • [Documentation] Major revisions to the documentation with new documentation for serving a cache #1028 #904 #912
  • [General] Globus was added as the second supported OAuth provider #963
  • [General] Formatted discovery and federation endpoints to be standard https URLs #974
  • [Origin/Cache] Pelican Origin and Cache now support OAuth authentication for the admin website #963
  • [Origin] Origins that export S3 can now export multiple buckets with unique access/secret keys #1045
  • [Origin] Origin now can check a sentinel file for the mounted POSIX storage #1003
  • [Origin] Origins now support HTTPS servers as a storage backend #958
  • [Cache] Admin website is now available for the Cache #1039
  • [Cache] New Prometheus metric to monitor cache access statistics #1007
  • [Director] Director admin can now disable traffic to cache and origin servers #933
  • [Director] HEAD request is now supported when the client request a file from the federation #1000
What's Changed
  • Upgrade golangci GHA version and turn off caching by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/935
  • Fix log color loss and LogLocation not working by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/939
  • Fix pelican plugin bug no valid classads... by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/942
  • Embed serve as default command to run the server container by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/947
  • Improve installation docs and include running Pelican server with docker image by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/904
  • Update documentation to serve an origin by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/912
  • Correct the key for docker doc page by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/955
  • Moved failing macos tests into a linux only test file so as to stop c… by @turetske in https://github.com/PelicanPlatform/pelican/pull/959
  • Remove generate package when releasing by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/937
  • Create tmp cinfo file for cache self test under the same folder as the final file by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/952
  • Updating GitHub Actions to deal with size issues by @turetske in https://github.com/PelicanPlatform/pelican/pull/950
  • Fix bug with version being "dev" by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/962
  • API support for registry admin to delete a namespace registration by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/915
  • Allow RSA key for TLS by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/965
  • Revert changes that remove generate package by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/967
  • Airplane mode by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/926
  • Update Pre Release by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/982
  • Consolidate two namespaceExists DB function by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/924
  • Allow admin to configure filtering servers for redirecting traffic at the director by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/933
  • Director based cache health tests by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/960
  • Import parameters.json into website by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/972
  • Fix director default response and add HEAD as allowed verb by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/1000
  • Pelican client advertises timeout to cache by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/1002
  • Overhauling discovery endpoint by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/974
  • Unit test for statHttp within Pelican Client by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/890
  • Redirect logging location reminder to stderr by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/989
  • Fix bug where project name not set in user-agent by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/977
  • Authenticated self-tests for cache by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/978
  • More verbose error message at director registration failure by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/992
  • Bump github.com/lestrrat-go/jwx/v2 from 2.0.20 to 2.0.21 by @dependabot in https://github.com/PelicanPlatform/pelican/pull/1014
  • Periodic cleanup of origin test files by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/987
  • Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 by @dependabot in https://github.com/PelicanPlatform/pelican/pull/1015
  • Output pelican config to stderr at server start by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/994
  • More detailed error message at WaitUntilWorking not working by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1016
  • Update Prometheus doc to introduce counting server number by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1009
  • Cache hit/miss bytes Prometheus metric by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1007
  • Fix invalid error message type as server response by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1023
  • Pin external c++ dependencies at a specific commit by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1022
  • Add LotMan configuration to the cache by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/889
  • getNamespaceByPrefix now checks topology for namespace existence by @turetske in https://github.com/PelicanPlatform/pelican/pull/1021
  • List namespace topology through UI by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1031
  • Remove Testing Code by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1036
  • Create serving_a_cache.mdx by @turetske in https://github.com/PelicanPlatform/pelican/pull/1028
  • Add server urls to error messages for WaitUntilWorking by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1034
  • Print pelican version at server startup to stderr by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1027
  • Remove use of global fed metadata within client by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/878
  • Add the option for a sentinel file for the origin by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1003
  • Do not log "director test timeout" warning if director can't be reached out by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/991
  • Improved plugin error messaging by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/999
  • Add "login required" endpoints behind "admin" check and fix bug in admin auth handler by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1013
  • Fixes to pelican plugin staging by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/882
  • Allow configurable cache watermarks by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/973
  • Add Cache Webpage by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1039
  • Add support for director/federation contact info as a configuration by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1012
  • Protocol version added to plugin -classad output by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/1051
  • Fix error responses for local cache by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/1041
  • Removed the setting of an AuthURL for Pelican serverAds by @turetske in https://github.com/PelicanPlatform/pelican/pull/1043
  • Refactor registry namespace check to be compatible with OSDF topology by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1038
  • More error tolerant web ui check by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1040
  • Add support for custom xrootd config file by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/968
  • Config fixups by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1057
  • Add topology timeout by @turetske in https://github.com/PelicanPlatform/pelican/pull/1058
  • Plug S3 plugin changes into multi-export S3 origins by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/1045
  • Bump the revision of xrdcl-pelican for libcurl testing by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/1065
  • Correctly return the transfer attempt information by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/1063
  • Update Director by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1068
  • Move default S3 url style from defaults.yaml to config.go by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/1070
  • Update Origin UI to use API for the new Origin.Exports scheme by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1042
  • Allow recursive uploads/downloads in plugin by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/1011
  • Make the local cache location unique for each test run by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/1066
  • Enabling OAuth for the origin and cache, adding support for Globus as another auth server by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/963
  • Update login UI by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1030
  • Only push the production container when the release was cut by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1077
  • Cache Updates by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1083
  • Fix broken URLs of namespaces from the topology at the director by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1071
  • Add support for HTTP backend by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/958

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.6.5...v7.7.0

Changelog

  • Fix director memory leak (https://github.com/PelicanPlatform/pelican/pull/1237)

Changelog

  • Update S3 plugin in containers to fix segfault (#1207)

Note: This release only contains changes to Pelican Docker containers. For users who use binary or packaged version of Pelican, this release is the same as v7.7.5 release.

Changelog

  • Fix cache dashboard page skips auth check and wrong health metric name (#1103)
  • Fix infinite redirect of UI init page (#1193)

Changelog

  • Fix --version flag output #1171

Changelog

  • Plugin returns last attempted source & filters \n (#1115)
  • Replace jwks.cache by TTL cache for namespace jwks at the director (#1119)

Changelog

  • Fix deadlock in client #1098

Changelog

  • Turn off LotMan while we investigate dlopen issues #1087

V7.6

Highlights

For Users
  • [Breaking Change] – Removed the deprecated Federation.NamespaceURL parameter. This will cause failures if a user is still using that value as opposed to Federation.RegistryURL
  • [Experimental] – Adds a local whole-file-only cache which works over a Unix domain socket
  • [Client] Multiple uploads and downloads are always done in parallel
  • Federation discovery download failures are marked as retryable
For Admins
  • [Breaking Change] – Removed the redacted Federation.NamespaceURL parameter. This will cause failures if a user is still using that value as opposed to Federation.RegistryURL
  • [Documentation] – Added documentation for serving a registry and director to the website
  • [Documentation] – Added documentation for setting up Grafana to visualize metrics collected on Pelican servers
  • [General] – We now build separate images for each Pelican server module
  • [Origin] – S3 origins no longer require a bucket to be specified
  • [General] – Fixed a bug where Prometheus token didn’t renew and caused 403 errors at the origin and cache servers
  • [General] – We now allow edits of Pelican server configurations via the web UI
  • [Registry] – Improved web UI for namespace registration and management
  • [Cache] – Added a Cache.Concurrency parameter to limit the number of connections to a cache
  • [General] – Fixed a bug where XrootD process leaks when Pelican server encounters error at its start
  • [Cache] – Cache’s can now start with a list a accepted namespaces they will accept origin data from
  • [Origin] – Origins now support multiple exports with different authorization configurations
Changelog
  • 27be834e Address PR comments
  • 037c0fb2 Adjusted call to topology to include a transport with a timeout
  • 5a009413 Adjusted patch to work with 7.6
  • a74d9ec1 Allow registration with existing topology ns, update tests to pass
  • f02fb6b4 Better error message when registry status check failed and director attempt to verify a reg
  • 004c0ff3 Changed some namespaceExists to namespaceExistsByPrefix calls that were missed in the cherry-pick
  • 64545bf4 Fix description
  • 2f93a17f Fixed failing tests from patched commits
  • d3543165 Improv to registry endpoint and err msg with new registry health metric
  • f6476a22 Pre Release: Update npm version
  • cade41f3 Reduced topology reload times to 4.5 minutes
  • 694e5b47 Revert previous changes
  • da0593e8 Revise error message to include topo ns
  • 45643eb4 Update tests
  • 575dc3e2 getNamespaceByPrefix now checks topology for namespace existence
Changelog
  • eda90ad2 Fixed missing space after input authfile namespaces
  • ce63a85e Pre Release: Update npm version

Notes

Fixed an error with an extra newline in the cache authfile generation

Changelog
  • e8c7cccf Allow RSA key for TLS
  • 9d587ccb Fix bug with version being "dev"
  • f92e83fe Moved failing macos tests into a linux only test file so as to stop constantly failing test runs until we get a fix
  • d447abdc Pre Release: Update npm version
  • d360e742 Pre Release: Update npm version
  • fe2d0687 Removed extra newline in authfile cache generation
  • 39a5c822 Rename tests to be more clear
  • 93c99582 Revert changes that remove generate package
  • 47cbbad3 Update Pre Release

Highlights

For Users
  • [Breaking Change] – Removed the deprecated Federation.NamespaceURL parameter. This will cause failures if a user is still using that value as opposed to Federation.RegistryURL
  • [Experimental] – Adds a local whole-file-only cache which works over a Unix domain socket
  • [Client] Multiple uploads and downloads are always done in parallel
  • Federation discovery download failures are marked as retryable
  • [BugFix] Fixed “No Valid ClassAds …” error with a return code of 0
For Admins
  • [Breaking Change] – Removed the redacted Federation.NamespaceURL parameter. This will cause failures if a user is still using that value as opposed to Federation.RegistryURL
  • [Documentation] – Added documentation for serving a registry and director to the website
  • [Documentation] – Added documentation for setting up Grafana to visualize metrics collected on Pelican servers
  • [General] – We now build separate images for each Pelican server module
  • [Origin] – S3 origins no longer require a bucket to be specified
  • [General] – Fixed a bug where Prometheus token didn’t renew and caused 403 errors at the origin and cache servers
  • [General] – We now allow edits of Pelican server configurations via the web UI
  • [Registry] – Improved web UI for namespace registration and management
  • [Cache] – Added a Cache.Concurrency parameter to limit the number of connections to a cache
  • [General] – Fixed a bug where XrootD process leaks when Pelican server encounters error at its start
  • [Cache] – Cache’s can now start with a list a accepted namespaces they will accept origin data from
  • [Origin] – Origins now support multiple exports with different authorization configurations
Changelog
  • 6ce3c95 Clean up dockerfile
  • 4a9acaa Embed serve as default command to run the server container
  • e52e8e6 Fix log color loss and LogLocation not working
  • c8afe0a Fix pelican plugin bug no valid classads...
  • 9e4669f Fix the bug where logging level doesn't work
  • d360e74 Pre Release: Update npm version
  • d34a5ec Set default command while allowing users to change

V7.5

Changelog

For Admins
  • [Origin] -- Configured public origins to use XRootD directive sec.protbind to allow backwards compatibility with legacy OSDF caches.
Changelog
  • 1ee7db3 Add missing xrootd key to osx_install test
  • 10c963d Add virtual-style url test
  • 860f979 Allow Pelican to use path-style URLs for S3 origins
  • b22440e Empty commit to force sync
  • d260a07 Pre Release: Update npm version
  • ea65a38 Update S3 test to use path-style URL
CHTC Logo

Branding

This project is supported by National Science Foundation under Cooperative Agreement OAC-2331480. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.