Pelican Logo
Pelican Platform

Pelican Releases


V7.15

Pelican v7.15.0

New Features and Enhancements

  • [Origin] Added config parameter Issuer.OIDCPreferClaimsFromIDToken which causes Pelican’s user authentication to look for claims in the OIDC ID token before falling back to the OIDC Userinfo endpoint. #2100
  • [Origin] Pelican origins can now specify multiple issuers per namespace #2103
  • [Director] Director can now report cache redirection reasoning when client commands are run with -d #2054
  • [Director] Added support for a Federation to have multiple Directors #2023
  • [Origin, Cache] Integrate XRootD OSS statistics #2092
  • [Director] Add API Token Listing endpoint #2086

Bugs Fixed

  • [Client] Fixed a regression in pelican generate keygen that caused it to generate invalid public keys #2085
  • [All] When key files are deleted, the private keys are also now removed from Pelican’s in memory processes #2087
  • [Client] The client no longer panics when doing a put or a share with just the base namespace prefix #2097
  • [Client] Fixed client bug where schemaless Pelican URLs don’t work with ‘-f’ fed infor #2116
  • [All] Assorted issues that prevented using an authentication provider other than CILogon or Globus #2091 #2046
  • [Client] Now correctly reports the start time for uploads [#2144]

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.14.2...v7.15.0

Pelican v7.15.1

Bugs Fixed

  • [Origin] Made Origin throttling configurable #2186
  • [Origin] Enabled multiuser origins to cache public keys for validating tokens [#2184](https://github.com/PelicanPlatform/pelican/pull/2184
  • [Client] Now sanity checks size for client GETs #2177

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.15.0...v7.15.1

V7.14

Pelican v7.14.0

New Features and Enhancements

Client

  • N/A

Cache, Director, Origin, Registry

  • Add /version endpoint to expose version and build info #2038
  • Log Pelican config on server startup #1970

Registry, Director

  • Drop privileges to user pelican if requested #1850

Origin, Cache

  • Fix xrootd log level mapping bug and inherit from Pelican log level #1988
  • Allow for Shoveler messages to be understood by monitoring packet handler #1981
  • Prometheus Metric For When a Service Crashed #2015

Origin

  • Refactor & Cleanup Origin Exports code #1919
  • Deprecate Lotman.DbLocation in favor of Lotman.LotHome #2001

Cache

  • Remove tls authentication from caches #1994

Director

  • Fix Director Capabilities Display #1980
  • Add Cache and Server Info Cards to Map #1947
  • Geoip table #1893
  • Add Server Outage information #1982
  • Add federation token endpoint in Director, and implement routine for caches to fetch one #1985
  • Grafana Token API Access #1917
  • Enable PROPFIND of Director health test API #2002
  • Support using IP address to query the geo location of a server in Director container #2059

Condor-Pelican

  • K8SPhysicalHostName Error Message #1891

Document Updates

  • Simplify Linux install instructions #1921
  • Remove jargon from the credential management page #1922
  • Document restricted characters in fed prefixes #2050
  • Update Server.XYZ docs to include all server components #1997

Operations & Bugs Fixed

  • Don't build dev images on git tags, tag image with latest-itb and sha256 #1946
  • Clean up the container image build and testing process #1972
  • Run tests in more appropriate container images #2000
  • Update xrootd release version to 1.5.purge.osg #1983
  • Pinned the goreleaser version to 2 #1991
  • Attempted speedups for GHA PR runs #2030
  • Rebase on OSG 24 (#1858) #2016
  • Install new plugin: xrdhttp-pelican v0.0.3 #2013
  • Update OS X install of xrootd-s3-http to v0.2.1 #2026
  • Reduce memory hoarding in the unit test #2028
  • Pin xrootd log levels in fed test and use typed params throughout #2019
  • Fix container building process: Properly install pelican-server into /usr/local/sbin #2053
  • Resolve the excessive use of xrootd user #2057 #2062

Full Changelog

Compare v7.13.0...v7.14.0

Bugfix

  • Install sssd-client into both origin container images #2102

Operation & Bugfix

  • Fix index out of bound error in origin servers #2082
  • Update XRootD version from 5.7.2 to 5.7.3 c555c47b

V7.13

Pelican v7.13.0

Breaking Changes

Registry Upgrade Requirement

Pelican v7.13 Director is not compatible with older versions of the Registry. The Registry must be upgraded before upgrading the Director to ensure proper functionality.

New Features and Enhancements

Client

  • Improved detection and retry logic for clients that start transfers while the Director is down #1890
  • Added new prestage functionality in the client #1923

Cache, Director, Origin, Registry

  • Added pelican user and group along with some default directories for RPM installations #1856
  • Added a "restart" button in each server's web UI #1869
  • Added configuration knobs to increase server memory performance by limiting Prometheus cardinality #1887
  • Created a new config parameter, Server.HealthMonitoringPublic, to make the server's health monitoring endpoint public #1936

Cache

  • Disabled redirects for HTTP caches #1915
  • Added experimental patches that modify a cache's ability to monitor resources and keep track of purgeable objects #1926

Director

  • Created a Prometheus metric in the Director to track rejected cache/origin advertisements #1885

Origin

  • Enabled server-side key refreshing so that origins/caches can update their keys in the Registry #1748

Bugs Fixed

Director

  • Fixed issue with director tests failing due to the token being created using the director URL instead of the discovery URL #1936

Full Changelog

Compare v7.12.4...v7.13.0

Bug fixes

  • [Cache, Origin] - Rebuilt containers to include newer version of xrootd: xrootd-5.7.3-1.3.purge.osg24 which contains additional bug fixes.

V7.12

Pelican v7.12.0

Breaking Changes

[Cache] Corrected previously unsafe creation of namespace, meta and data directories in caches in #1709. This change may result in cache data invalidation, and will likely require manual intervention during upgrades. Failure to adjust the data hierarchy of existing caches could result in unintended data leaks or zombie data that can't be cleaned by the cache. If this affects you, please make sure you understand update requirements before beginning your upgrades.

New Features and Enhancements

[All] The config parameter Server.TLSCertificate has been deprecated in favor of Server.TLSCertificateChain to more clearly convey the contents of the file pointed to. The deprecated parameter should continue to function for the time being, but may removed in a future release. [All] Added a CLI tool pelican config for displaying and interacting with Pelican configuration in #1693. [All] Standardized and improved RPM packaging in #1666. [Director] Gave Directors the ability to collect Prometheus metrics for GeoIP resolution failures in #1772. [Director] Servers now advertise their version to the Director in #1753. [Director] Started displaying namespace capabilities in the Director in #1737. [Director] Improved Prometheus metrics for tracking the number of currently-advertising servers in #1684. [Director] Gave the ability for the Director to persist origin/cache downtime labels between restarts in #1673. [Origin] Gave S3 origins the ability to specify sentinel files in #1791. [Plugin] Allowed the plugin to detect when it is named pelican_plugin in addition to pelican_xfer_plugin in #1733.

Bugs Fixed

[Client] Cleaned up display of progress bars for object gets in the client in #1814. [Client] Fixed bug where object ls against namespaces that do not support listings caused nil pointer panics in #1710. [Client] Added additional client checks to prevent creation of empty objects with directory uploads in #1707. [Director] Fixed over-collection of server redirects in the Director's Prometheus database in #1743. [Director] Fixed OSDF Director startup bug that caused erroneous warnings/errors mentioning the proj label in #1708.

New Contributors

@alexandertuna made their first contribution in #1730 @mwestphall made their first contribution in #1791 @georgezzeng made their first contribution in #1727

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.11.8...v7.12.0

Bugs Fixed

  • [Caches] - Rebuilt containers to ingest xrdcl-pelican v1.0.5, which comes with additional segfault bugfixes.
  • [Origins] - Rebuilt containers to ingest upstream bugfixes for S3/HTTP origins.

Bugs Fixed

  • [Cache Containers] - Grabbed patched version of xrootd 5.7.2 to mitigate further segfault bugs related to caches that are handling many concurrent requests for non-existent objects.

Bugs Fixed

[Cache] - Fixed a broken dependency for our cache containers that prevented caches from fetching non-cached objects from origins. [Cache] - Updated an underlying library version to prevent caches from segfaulting when non-existent objects are requested. [Client] - Added extra handling for client PUTs to accept XRootD's new HTTP 201 response.

Known Bugs

[Director/Cache] - This release contains a known bug where the Director's test utility fails for caches. This shouldn't affect the cache's ability to function, but will result in the cache showing errors in the Director's Web interface.

NOTE

Containerized versions of this release were dead on arrival due to a broken dependency for xrdcl-pelican.

Bugs Fixed

  • [Servers] Restored the functionality of various tickers that had been erroneously stopped, preventing XRootD servers from refreshing some files like scitokens/auth config and preventing some metrics from being scraped by Prometheus.
  • [Caches] Updated caches' underlying version of xrdcl-pelican from 0.9.4 to 1.0.2, which prevents a segfault in caches whenever a non-existent object is requested.

Bugs Introduced

  • [Caches] The directorTest functionality will fail in caches due to the new xrdcl-pelican version until updates can be made in Directors. These test failures are limited in scope and should not affect the rest of the cache's functionality, but may generate excess noise in logs.

V7.11

Highlights

For Users

  • [Client] The client now supports object sync commands, which recursively syncs source and destinations #1586 #1691
  • [Client] ls -l and the -L flag for logging have switched behavior – ls -l now enables listings and -L is now the logging flag #1598

For Admins

  • [Director] The Director will now only enable x509 authentication for given selected prefixes #1563 #1624
  • [Origin] Issuer config has now been added to Origin webpages #1584
  • [Origin] Tokens are now auto generated if the issuer is available 1591
  • [Origin] Now allow HTTP backends for an Origin #1597
  • [Origin] Bug fixes for S3 origins with multiple virtual-style bucket exports #1601
  • [Director] GeoIP resolution failures are now recorded as Prometheus metrics #1608
  • [Director] Clients whose GeoIP resolution fails are now temporarily assigned a location within the continental US to improve followup cache selection. #1620
  • [Director/Origin] Prometheus metrics are now displayed on the Origin and Director Web UI #1623
  • [Registry] Admins can now delete a namespace from the Registry using the webpage #1626
  • [Director] There is now a director redirect metric displayed which keeps track of the number of redirects to an origin or a cache #1632
  • [All] Length of Prometheus metric retention is now configurable #1640
  • [Origin] S3 Origin performance/reliability enhancements #1681

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.11.0...v7.10.0

Bugs Fixed

Restored the functionality of various tickers that had been erroneously stopped, preventing XRootD servers from refreshing some files like scitokens/auth config.

Changelog

  • 03a49337 Updating fronend version number

Changelog

  • 95c363ca Fixed missing newline
  • f5e30f6d Fixing linter error that was added during rebase
  • 15b75c79 Removed calling the x509 test script until we can fix the functionality
  • 14634ae5 Removed the tsl defer functionality from the cache until we can fix it on the xrootd end
  • ebc49ff8 Update version frontend to 7.11.8

Changelog

  • 9810cbdf Bump frontend version to 7.11.7
  • d7857861 Fixed macos failing tests:
  • 62b413db Pinning the xrootd build in macos to a specific tag
  • e6771978 Revert "Removed new metrics that seem to causing Prometheus to use a very lar…"
  • 8353dd2a Revert "Removed the "network" labels from the metrics"

Changelog

  • 2dce0176 Removed new metrics that seem to causing Prometheus to use a very large amount of memory
  • b935faca Updated frontend version number

Changelog

  • be21bd24 Adjust ownership/perms of /usr/local ahead of time for MacOS test runner
  • a6be79ea Fixed linter error in director/sort.go
  • 61a30f29 Format
  • 15729b6d Prettier Format
  • e052a6fd Redirect app to root on success password
  • a5b49958 Remove the use of index.d.ts files
  • fa7cf4db Removed the "network" labels from the metrics
  • 7503842f Uniform Error Handling
  • 3d66971f Uniform Error Handling
  • 1a8af8f1 Uniform Error Handling
  • 24bad1be Update Error Handling
  • 31cbad2f Update MacOS GHA test env to fix deprecation error
  • 593323b8 Updated the web-ui package version

Changelog

  • c9cd02d5 Fixes an issue where a misplaced defer ticker.Stop() stopped the re-advertisement of caches/origins
  • 4ee13c06 Updated front-end version

Changelog

  • 5d333660 Revert "Switch to use new Caps struct but still handle old JSON from Director"
  • 26238933 Revert "Unmarshal Capabilities correctly in all structs that use them"
  • b3df1c87 Update the version number

Changelog

  • 7a096d57 Unmarshal Capabilities correctly in all structs that use them
  • 8629adaa Update the version number

Changelog

  • bc54370a Comment-out all origin export config
  • fa87b928 Drop old default origin port
  • 74888b40 First pass at default config.d based configuration
  • 50db1e25 Fix nil pointer bug caused by trying to list namespaces with no coll URL
  • 826da111 Move Pelican federation-facing config to an example file
  • c514ab57 Move Topology specific configuration to example file
  • 43d9b0d0 Move common OSDF config to the federation-level config dir
  • 9f9a09d0 Move common debugging configs to editable example file
  • b3861d18 Move multiuser origin config to an example file
  • 8dad09d8 Move origin export config to an example file
  • d83b4c4c Move web UI config to example file
  • a91a4335 Remove prescriptive cache / origin locations
  • 3beae08d Updated version to 7.11.1

V7.10

Highlights

For Users

  • [Documentation] Updated documentation structure #1423
  • [Client] Stopped file transfers are now retryable errors #1101
  • [Client] General improvements to error messages within the client and the plugin, ensuring more accurate errors and removing redundancies
  • [Client] Put requests using the pelican client work now with Origins using OA4MP Issuers #1490
  • [Client] Retries metadata lookup failures #1514
  • [Client] Fixed naming issue with queries on get/copy #1541

For Admins

  • [Director] Director now supports intelligent server sorting method to consider distance, server load, and object availability #1470
  • [Director] Add object availability test for cache access #1454
  • [Director] Allow caches to fetch objects from other caches when there’s a cache miss #1457
  • [Origin/Cache/Registry] Site name is now populated in the Origin/Cache auto-registration process #1460
  • [Origin/Cache/Director] Origins and Caches now advertise their storage backend type to the director and are able to control if they want to receive director-based object transfer tests #1462

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.9.0...v7.10.0

Changelog

  • a116a69f Adding back in poller fix to make macosx more reliable
  • 7b2ca77b Adjusted pelican_protocol.patch
  • ecd6906a Update version to 7.10.12

Changelog

  • 5f47b175 Update s3 backend to include new multi_curl handler

Changelog

  • ca3d0abf Move sssd-client installation to final stage builder

Changelog

  • a7b05d8c Install sssd-client in dev/prod containers

Changelog

  • 156743fd Don't construct Pelican URLs for stashcp copies

[plugin] Fix bug where transfer plugin fails if there is no $HOME directory set

Director: Selectively enable client X509 authentication

Changelog

  • 842c2a57 Update xrdcl-pelican to v0.9.4

Changelog

  • [Director] e40950e3 Fix topology downtime integration by using downtime URL and not ns JSON

Changelog

  • d29e1d2b Origins now have directreads on by default
  • 6f5341f3 Pre Release: Update npm version
  • a8189e09 Update S3 backend to v0.1.6

Changelog

  • aebe31a6 Pre Release: Update npm version
  • [Origin] Origins now have directress enabled by default

V7.9

Highlights

Congratulations to @patrickbrophy for his first contribution to Pelican!

For Users

  • [Client] Clients can now read the PELICAN_NEAREST_CACHE environment variable for determining which cache to use #1234
  • [Client] Clients can now list a collection of objects and get statistics of an object #1393 #1400
  • [Plugin] The plugin now advertises the ability to move pelican:// objects #1265
  • [Client] Additional error handling has been introduced that helps users build custom clients on top of the Pelican golang libraries #1263
  • [Client] Clients can now send a timeout value in their object requests, indicating to Pelican services the maximum time the client is willing to wait for requests #1299
  • [Cache/Client] Caches now report to clients whether the data they returned came from the disk on the cache or from the origin #1301
  • [All] Pelican now warns users about unrecognized configuration input #1325

For Admins

  • [Cache] Caches no longer shutdown on startup if advertisements fail, allowing admins to fix registrations #1244
  • [Director/Cache] The director can now filter caches marked as down in Topology #1260
  • [Origin] Pelican can now create "XRoot" backends for adding xroot-only protocol servers to the federation #1285
  • [Origin] HTTP backend export information has been added to Origin webpages #1291
  • [Origin] Origins can now export Globus collections as one of their backends #1336
  • [Director] The Director only responds to object requests with up to six Origin/Cache candidates #1304
  • [All] Pelican images are now built on top of EL9 from the OSG23 repositories #1288
  • [All] Servers now have a more permissive 5s timeout when checking for the existence of required remote resources #1328
  • [Director] The Director now checks object availability among Origins for object requests sent to Origins #1283
  • [All] Pelican now supports layered configuration through the specification of extra config directories in the root config using the ConfigLocations key #1313

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.8.0...v7.9.0

Changelog

  • [Director]: Fix a memory explosion in the Director caused by unbound Prometheus logging.

Changelog

  • [Director]: Bug fixes to stat capabilities in the Director
  • [Director]: Increase stat timeouts from 300ms to 1s.

Changelog

  • [Client]: Fixed a bug that could cause high-concurrency downloads to stall
  • [Client]: Fixed destination filename for 'unpack' mode
  • [Client]: Populate director information for stash:// URLs

Changelog

  • [Origins]: Update S3 backend version to correctly handle S3 resource URLs with a configured path component

Changelog

  • [Director/Client]: Add X-Pelican-[Authorization/Token-Generation] headers to origin redirects triggered by client requests. This fixes cases where pelican object put is unable to generate a token due to the missing headers.

Changelog

  • [Servers] Make server startup timeout configurable

Changelog

  • [Director]: Bug fixes in stat code so that the Director stats against the correct URL for authenticated Topology origins.

Highlights

Congratulations to @patrickbrophy for his first contribution to Pelican!

For Users

  • [Client] Clients can now read the PELICAN_NEAREST_CACHE environment variable for determining which cache to use #1234
  • [Client] Clients can now list a collection of objects and get statistics of an object #1393 #1400
  • [Plugin] The plugin now advertises the ability to move pelican:// objects #1265
  • [Client] Additional error handling has been introduced that helps users build custom clients on top of the Pelican golang libraries #1263
  • [Client] Clients can now send a timeout value in their object requests, indicating to Pelican services the maximum time the client is willing to wait for requests #1299
  • [Cache/Client] Caches now report to clients whether the data they returned came from the disk on the cache or from the origin #1301
  • [All] Pelican now warns users about unrecognized configuration input #1325

For Admins

  • [Cache] Caches no longer shutdown on startup if advertisements fail, allowing admins to fix registrations #1244
  • [Director/Cache] The director can now filter caches marked as down in Topology #1260
  • [Origin] Pelican can now create "XRoot" backends for adding xroot-only protocol servers to the federation #1285
  • [Origin] HTTP backend export information has been added to Origin webpages #1291
  • [Origin] Origins can now export Globus collections as one of their backends #1336
  • [Director] The Director only responds to object requests with up to six Origin/Cache candidates #1304
  • [All] Pelican images are now built on top of EL9 from the OSG23 repositories #1288
  • [All] Servers now have a more permissive 5s timeout when checking for the existence of required remote resources #1328
  • [Director] The Director now checks object availability among Origins for object requests sent to Origins #1283
  • [All] Pelican now supports layered configuration through the specification of extra config directories in the root config using the ConfigLocations key #1313
  • [All] Fixed a bug that prevented some PELICAN_* environment variables from being read correctly #1451

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.8.0...v7.9.2

Changelog

  • d18dd643 Add config knob to enable pprof endpoints
  • 2b1a1c2f Add config knob to turn off stat for the director
  • c8982a6d Add pprof endpoints
  • 3ca0f14d Add two Prometheus metrics for directorstat queries
  • 26f8e39f Change to use EnableStat instead
  • 559f64e9 Fix deadlock in recordAd and LaunchTTLCache
  • aca7a484 Pre Release: Update npm version
  • f49108fe Remove stat utils for caches as well

V7.8

Highlights

For Users

  • [Client] Stopped file transfers are now retryable errors #1101
  • [Client] General improvements to error messages within the client and the plugin, ensuring more accurate errors and removing redundancies
  • [Client] The client can bypass the cache and download directly from the origin if a directread query parameter is set #1166

For Admins

  • [Documentation] Added documentation on how to get the public key of a running origin #1078
  • [Documentation] Added documentation on how to run pelican servers via docker containers #1089
  • [Origin] Pelican origin token creation now respects the issuer flag #1135
  • [Cache] Cache UI will now redirect to a login page for an unauthorized user #1103
  • [Origin/Cache] Improved “site name” handling #1176
  • [Director] Director JWKS cache no longer becomes invalid when the registry goes down #1119
  • [Director] Added a prometheus metric keep track of the total number of advertisements to the director #1133
  • [Director] Enable director login via OAuth #1150
  • [Cache] Allow the cache xrootd to have multiple data/meta mount locations #1191
  • [Director/Registry/Origin/Cache[ Read environment variables with PELICAN prefix for Pelican binaries with alias (stashcp and osdf) and deprecate alias-specific prefixes #1162

Changelog

  • Correct typo in map name to fix ad sorting/filtering and add unit test with real topology data

Changelog

  • Fix issue with filtering/consolidating multi-export topology origins

Changelog

  • Fix panic in Director
  • Improve Director's error response to clients

Changelog

Disable ZTN authentication on Pelican origins

Changelog

Bug fixes to restore missing collections-url when the Director is redirecting to a topology-only origin.

Changelog

  • bb74abb7 Adjust timeout test to reflect new timeout value
  • e0dd8794 Build xrootd-multiuser from source in dev container
  • 37beb73f Cleanup unstopped ticker in WaitUntilWorking
  • 0dead744 Fix bug w/ client not working for WithToken option
  • 06865b24 Increase WaitUntilWorking transfer timeout and adjust error message
  • ece8c4c7 Pre Release: Update npm version

Changelog

  • a92d4954 Add an XRoot storage backend
  • 9c09ab14 Aggregate gin routes for Prometheus metrics
  • 19d1c86c Cleanup missed linter item
  • 6364cea7 Incorporate review feedback
  • 4a026005 Pre Release: Update npm version

Changelog

  • 3d129f6e Concatenate multiple issuer names for the same issuer url
  • f1a7028c Improve error reporting for cache self-test
  • b0e78303 Improve scitoken generation error handling
  • f10f7cad Pre Release: Update npm version
  • d11c32d7 Pre Release: Update npm version
  • 66d2dcee Pull sefl-test folder out from the director tests
  • 8f5102c8 Use Cache.LocalRoot internally
  • 3a8652c2 Use and to connect two issuer names
  • 694b3524 Use externalWebUrl as the token issuer for self-tests

Changelog

Fixed Cache shutting down if advertisement fails #1244 Plugin advertises PluginVersion with -classad #1252

V7.7

Highlights

For Users

  • [Plugin] stash-plugin now supports recursive uploads and downloads #1011

For Admins

  • [General] Fixed administrator authentication check failure for all servers #1013
  • [Documentation] Major revisions to the documentation with new documentation for serving a cache #1028 #904 #912
  • [General] Globus was added as the second supported OAuth provider #963
  • [General] Formatted discovery and federation endpoints to be standard https URLs #974
  • [Origin/Cache] Pelican Origin and Cache now support OAuth authentication for the admin website #963
  • [Origin] Origins that export S3 can now export multiple buckets with unique access/secret keys #1045
  • [Origin] Origin now can check a sentinel file for the mounted POSIX storage #1003
  • [Origin] Origins now support HTTPS servers as a storage backend #958
  • [Cache] Admin website is now available for the Cache #1039
  • [Cache] New Prometheus metric to monitor cache access statistics #1007
  • [Director] Director admin can now disable traffic to cache and origin servers #933
  • [Director] HEAD request is now supported when the client request a file from the federation #1000

What's Changed

  • Upgrade golangci GHA version and turn off caching by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/935
  • Fix log color loss and LogLocation not working by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/939
  • Fix pelican plugin bug no valid classads... by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/942
  • Embed serve as default command to run the server container by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/947
  • Improve installation docs and include running Pelican server with docker image by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/904
  • Update documentation to serve an origin by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/912
  • Correct the key for docker doc page by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/955
  • Moved failing macos tests into a linux only test file so as to stop c… by @turetske in https://github.com/PelicanPlatform/pelican/pull/959
  • Remove generate package when releasing by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/937
  • Create tmp cinfo file for cache self test under the same folder as the final file by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/952
  • Updating GitHub Actions to deal with size issues by @turetske in https://github.com/PelicanPlatform/pelican/pull/950
  • Fix bug with version being "dev" by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/962
  • API support for registry admin to delete a namespace registration by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/915
  • Allow RSA key for TLS by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/965
  • Revert changes that remove generate package by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/967
  • Airplane mode by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/926
  • Update Pre Release by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/982
  • Consolidate two namespaceExists DB function by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/924
  • Allow admin to configure filtering servers for redirecting traffic at the director by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/933
  • Director based cache health tests by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/960
  • Import parameters.json into website by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/972
  • Fix director default response and add HEAD as allowed verb by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/1000
  • Pelican client advertises timeout to cache by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/1002
  • Overhauling discovery endpoint by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/974
  • Unit test for statHttp within Pelican Client by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/890
  • Redirect logging location reminder to stderr by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/989
  • Fix bug where project name not set in user-agent by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/977
  • Authenticated self-tests for cache by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/978
  • More verbose error message at director registration failure by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/992
  • Bump github.com/lestrrat-go/jwx/v2 from 2.0.20 to 2.0.21 by @dependabot in https://github.com/PelicanPlatform/pelican/pull/1014
  • Periodic cleanup of origin test files by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/987
  • Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 by @dependabot in https://github.com/PelicanPlatform/pelican/pull/1015
  • Output pelican config to stderr at server start by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/994
  • More detailed error message at WaitUntilWorking not working by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1016
  • Update Prometheus doc to introduce counting server number by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1009
  • Cache hit/miss bytes Prometheus metric by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1007
  • Fix invalid error message type as server response by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1023
  • Pin external c++ dependencies at a specific commit by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1022
  • Add LotMan configuration to the cache by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/889
  • getNamespaceByPrefix now checks topology for namespace existence by @turetske in https://github.com/PelicanPlatform/pelican/pull/1021
  • List namespace topology through UI by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1031
  • Remove Testing Code by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1036
  • Create serving_a_cache.mdx by @turetske in https://github.com/PelicanPlatform/pelican/pull/1028
  • Add server urls to error messages for WaitUntilWorking by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1034
  • Print pelican version at server startup to stderr by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1027
  • Remove use of global fed metadata within client by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/878
  • Add the option for a sentinel file for the origin by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1003
  • Do not log "director test timeout" warning if director can't be reached out by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/991
  • Improved plugin error messaging by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/999
  • Add "login required" endpoints behind "admin" check and fix bug in admin auth handler by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1013
  • Fixes to pelican plugin staging by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/882
  • Allow configurable cache watermarks by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/973
  • Add Cache Webpage by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1039
  • Add support for director/federation contact info as a configuration by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1012
  • Protocol version added to plugin -classad output by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/1051
  • Fix error responses for local cache by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/1041
  • Removed the setting of an AuthURL for Pelican serverAds by @turetske in https://github.com/PelicanPlatform/pelican/pull/1043
  • Refactor registry namespace check to be compatible with OSDF topology by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1038
  • More error tolerant web ui check by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1040
  • Add support for custom xrootd config file by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/968
  • Config fixups by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1057
  • Add topology timeout by @turetske in https://github.com/PelicanPlatform/pelican/pull/1058
  • Plug S3 plugin changes into multi-export S3 origins by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/1045
  • Bump the revision of xrdcl-pelican for libcurl testing by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/1065
  • Correctly return the transfer attempt information by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/1063
  • Update Director by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1068
  • Move default S3 url style from defaults.yaml to config.go by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/1070
  • Update Origin UI to use API for the new Origin.Exports scheme by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1042
  • Allow recursive uploads/downloads in plugin by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/1011
  • Make the local cache location unique for each test run by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/1066
  • Enabling OAuth for the origin and cache, adding support for Globus as another auth server by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/963
  • Update login UI by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1030
  • Only push the production container when the release was cut by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1077
  • Cache Updates by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/1083
  • Fix broken URLs of namespaces from the topology at the director by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/1071
  • Add support for HTTP backend by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/958

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.6.5...v7.7.0

Changelog

  • Fix director memory leak (https://github.com/PelicanPlatform/pelican/pull/1237)

Changelog

  • Update S3 plugin in containers to fix segfault (#1207)

Note: This release only contains changes to Pelican Docker containers. For users who use binary or packaged version of Pelican, this release is the same as v7.7.5 release.

Changelog

  • Fix cache dashboard page skips auth check and wrong health metric name (#1103)
  • Fix infinite redirect of UI init page (#1193)

Changelog

  • Fix --version flag output #1171

Changelog

  • Plugin returns last attempted source & filters \n (#1115)
  • Replace jwks.cache by TTL cache for namespace jwks at the director (#1119)

Changelog

  • Fix deadlock in client #1098

Changelog

  • Turn off LotMan while we investigate dlopen issues #1087

V7.6

Highlights

For Users

  • [Breaking Change] – Removed the deprecated Federation.NamespaceURL parameter. This will cause failures if a user is still using that value as opposed to Federation.RegistryURL
  • [Experimental] – Adds a local whole-file-only cache which works over a Unix domain socket
  • [Client] Multiple uploads and downloads are always done in parallel
  • Federation discovery download failures are marked as retryable

For Admins

  • [Breaking Change] – Removed the redacted Federation.NamespaceURL parameter. This will cause failures if a user is still using that value as opposed to Federation.RegistryURL
  • [Documentation] – Added documentation for serving a registry and director to the website
  • [Documentation] – Added documentation for setting up Grafana to visualize metrics collected on Pelican servers
  • [General] – We now build separate images for each Pelican server module
  • [Origin] – S3 origins no longer require a bucket to be specified
  • [General] – Fixed a bug where Prometheus token didn’t renew and caused 403 errors at the origin and cache servers
  • [General] – We now allow edits of Pelican server configurations via the web UI
  • [Registry] – Improved web UI for namespace registration and management
  • [Cache] – Added a Cache.Concurrency parameter to limit the number of connections to a cache
  • [General] – Fixed a bug where XrootD process leaks when Pelican server encounters error at its start
  • [Cache] – Cache’s can now start with a list a accepted namespaces they will accept origin data from
  • [Origin] – Origins now support multiple exports with different authorization configurations

Changelog

  • 27be834e Address PR comments
  • 037c0fb2 Adjusted call to topology to include a transport with a timeout
  • 5a009413 Adjusted patch to work with 7.6
  • a74d9ec1 Allow registration with existing topology ns, update tests to pass
  • f02fb6b4 Better error message when registry status check failed and director attempt to verify a reg
  • 004c0ff3 Changed some namespaceExists to namespaceExistsByPrefix calls that were missed in the cherry-pick
  • 64545bf4 Fix description
  • 2f93a17f Fixed failing tests from patched commits
  • d3543165 Improv to registry endpoint and err msg with new registry health metric
  • f6476a22 Pre Release: Update npm version
  • cade41f3 Reduced topology reload times to 4.5 minutes
  • 694e5b47 Revert previous changes
  • da0593e8 Revise error message to include topo ns
  • 45643eb4 Update tests
  • 575dc3e2 getNamespaceByPrefix now checks topology for namespace existence

Changelog

  • eda90ad2 Fixed missing space after input authfile namespaces
  • ce63a85e Pre Release: Update npm version

Notes

Fixed an error with an extra newline in the cache authfile generation

Changelog

  • e8c7cccf Allow RSA key for TLS
  • 9d587ccb Fix bug with version being "dev"
  • f92e83fe Moved failing macos tests into a linux only test file so as to stop constantly failing test runs until we get a fix
  • d447abdc Pre Release: Update npm version
  • d360e742 Pre Release: Update npm version
  • fe2d0687 Removed extra newline in authfile cache generation
  • 39a5c822 Rename tests to be more clear
  • 93c99582 Revert changes that remove generate package
  • 47cbbad3 Update Pre Release

Highlights

For Users

  • [Breaking Change] – Removed the deprecated Federation.NamespaceURL parameter. This will cause failures if a user is still using that value as opposed to Federation.RegistryURL
  • [Experimental] – Adds a local whole-file-only cache which works over a Unix domain socket
  • [Client] Multiple uploads and downloads are always done in parallel
  • Federation discovery download failures are marked as retryable
  • [BugFix] Fixed “No Valid ClassAds …” error with a return code of 0

For Admins

  • [Breaking Change] – Removed the redacted Federation.NamespaceURL parameter. This will cause failures if a user is still using that value as opposed to Federation.RegistryURL
  • [Documentation] – Added documentation for serving a registry and director to the website
  • [Documentation] – Added documentation for setting up Grafana to visualize metrics collected on Pelican servers
  • [General] – We now build separate images for each Pelican server module
  • [Origin] – S3 origins no longer require a bucket to be specified
  • [General] – Fixed a bug where Prometheus token didn’t renew and caused 403 errors at the origin and cache servers
  • [General] – We now allow edits of Pelican server configurations via the web UI
  • [Registry] – Improved web UI for namespace registration and management
  • [Cache] – Added a Cache.Concurrency parameter to limit the number of connections to a cache
  • [General] – Fixed a bug where XrootD process leaks when Pelican server encounters error at its start
  • [Cache] – Cache’s can now start with a list a accepted namespaces they will accept origin data from
  • [Origin] – Origins now support multiple exports with different authorization configurations

Changelog

  • 6ce3c95 Clean up dockerfile
  • 4a9acaa Embed serve as default command to run the server container
  • e52e8e6 Fix log color loss and LogLocation not working
  • c8afe0a Fix pelican plugin bug no valid classads...
  • 9e4669f Fix the bug where logging level doesn't work
  • d360e74 Pre Release: Update npm version
  • d34a5ec Set default command while allowing users to change

V7.5

Updates

Congratulations @williamnswanson for your first contribution in #684!

Highlights

For Users

  • [General] – Improved logging doesn't report as many false errors
  • [Client] – Improved file unpacking to handle more use cases
  • [Client] – The plugin now downloads files in parallel

For Admins

  • [Origin] – Pelican origins now support public namespaces
  • [General] – Federation admins can now fine tune whether origins and caches require admin approval before joining the federation
  • [Origin/Cache] – XRootD log levels are now configurable
  • [Director] – The director can now report whether a specific origin has a requested object via the new director stat feature
  • [Origin] – Fixed rate graph data loss issues in the origin
  • [Registry] – Added the ability to configure a URL for fetching the registry's institution list

Changelog

For Admins

  • [Origin] -- Configured public origins to use XRootD directive sec.protbind to allow backwards compatibility with legacy OSDF caches.

Changelog

  • 1ee7db3 Add missing xrootd key to osx_install test
  • 10c963d Add virtual-style url test
  • 860f979 Allow Pelican to use path-style URLs for S3 origins
  • b22440e Empty commit to force sync
  • d260a07 Pre Release: Update npm version
  • ea65a38 Update S3 test to use path-style URL

What's new

For Admins

  • [Cache]: Caches now fetch new namespace ads every time they advertise with the director. This is necessary for discovering new origins if they join the federation after the cache is served.

What's new

For Admins

  • [Origin]: Origins configured to export S3 backends are no longer explicitly tied to a single bucket.

What's new

For Admins

  • [Cache]: Fixed bug that prevented the cache's pss.origin directive from being set correctly with pelican://-style URLs.

What's new

For Admins

  • [Cache]: Fixed bug that prevented debug output from XRootD processes.

What's new

For Admins

  • [Registry]: Fixes bugs related to the registry's topology table that introduced race conditions and caused excess load on the topology endpoint.

What's new

For Users

  • [Client]: Bug fixes for pelican object put with pelican:// urls

For Admins

  • [Director]: The director can now display more federation information without requiring authentication

Changelog

  • a47a13b Fixed issue with added new line in cache authfile
  • 3cc2452 Pre Release: Update npm version

V7.4

Highlights

For Users

  • [Client] Introduced pelican object get and pelican object put to specify a get or put rather than just running pelican object copy. #259
  • [Client] Fixed issue where the stash_plugin file-transfer plugin was not reporting proper upload bytes. #510
  • [Client] Fixed issue with progress bars not outputting properly with debug logs for recursive uploads/downloads #402
  • [Client] Fixed issue where stash_plugin was not working with environment variable-based configuration #508
  • [Client] pelican object copy now doesn’t rely on topology for PUTs, it gets the endpoint by querying the director #504

For Admins

  • [Director] Introduced GeoIP Overrides that allow a director to pin IP ranges to a particular lat/long. #496
  • [Registry] Introduced web-ui for registry that allows users to register their namespace through web interface. #264
  • [Director] Added federation discovery endpoint to director. #417
  • [General] Added topology components to Prometheus health metrics. #494
  • [Cache] Added Admin approval to caches before they can advertise to director #439
  • [Registry] Added support to use CILogon to login to registry UI #355
  • [Registry] Deprecated Federation.NamespaceUrl which will be removed in the next major release #388
  • [General] Web-ui now outputs more detailed error messages if login fails #424
  • [Origin] Basic S3 origin functionality has been restored #592
  • [Cache] Fixed issue where caches were logging sensitive information #604
  • [Origin] Boolean flag added to starting origins indicating if they are writable or not (default is true) #507

Changelog

  • Fix duplicated director file transfer tests
  • Added three Prometheus metrics recording number of test file transfers the director issues:
    • pelican_director_total_ftx_test_suite: total number of test suite the director issued, which means how many times the director started a new test go routine to an origin
    • pelican_director_active_ftx_test_suite: the number of active test suite running per origin, which should be 1, meaning only one active test go routine is running per origin. This will be a good indicator of whether the director is doing wild things.
    • pelican_director_total_ftx_test_runs: the total number of test file transfer runs per origin. i.e. How many tests in total were ran per origin. We run A test per 15s (by default) per origin, so this is also a good indicator of whether the origin is up or whether the director is going crazy. It comes with label status and report_status with value Success and Failure indicating if the transfer itself was successful and the reporting to the origin was successful.

Changelog

  • Fix the issue where Pelican won't exit main process if error group goroutine returns error, unless an error returns in the main goroutine
  • Fix the panic at the director when a cache advertise itself to the director, as we didn't have the case for cache when doing version compatibility check
  • Improve logging at program exit to be consistent.
  • Improve cache error group, remove duplicated exit handler that root.go already had.

V7.3

Highlights

For Users

  • Users can now recursively upload and download files. #260, #261
  • Users can upload a local directory to a data federation as a tarball and can unpack a tarball into a directory as it downloads.
  • For improved debugging, writing a debug file into a known location is now supported. #22
  • Introduced Javascript client allowing browsers to perform file transfers. Installation of the Javascript client enables users via their browser to download a file from a data federation supported by Pelican. #344

For Admins

  • [Origin] Added support for S3 file backend so that admins may expose or configure the S3 backend. #169
  • [Cache] Added authentication to cache service. #311
  • [Issuer] Introduced new OAuth2 issuer built into the origin packaging. #262
  • [General] Introduced web pages for the director and registry components. #265, #354, #355
  • New feature to allow the director to aggregate metrics from registered origins and to allow the director to periodically test origins #291 #175

What's Changed

  • Get rid of superfluous symlink by @timtheisen in https://github.com/PelicanPlatform/pelican/pull/348
  • Code cleanup in serveNamespaceRegistry by @turetske in https://github.com/PelicanPlatform/pelican/pull/341
  • Made Pelican logging able to output to file by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/346
  • Fix problems with config_test.go by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/363
  • Revert "Code cleanup in serveNamespaceRegistry" by @turetske in https://github.com/PelicanPlatform/pelican/pull/364
  • Namespace registry serve cleanup by @turetske in https://github.com/PelicanPlatform/pelican/pull/365
  • Renamed getOriginJWK to getServerJWK by @turetske in https://github.com/PelicanPlatform/pelican/pull/339
  • Use Server.Hostname than os.Hostname() in generating CA by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/356
  • Standardize server init and cleanup a few other small items by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/362
  • Fix issue with progress bar panic on download by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/369
  • Always generate a CA if no host certificate present by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/375
  • Add authorization to Prometheus /metrics endpoint by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/373
  • Set correct log level for Prometheus logger by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/358
  • Unit test for namespace-registry by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/337
  • Introduce enum for health status and component names for SetComponentHealthStatus by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/376
  • Fix Config Bug by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/381
  • Refactor init_server_cred to be more clear on their usage by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/382
  • Adjusted dev.Dockerfile to run make web-build properly by @turetske in https://github.com/PelicanPlatform/pelican/pull/386
  • Git test action updates by @turetske in https://github.com/PelicanPlatform/pelican/pull/385
  • Fix small bugs in DiscoverFederation() by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/400
  • Fixed scitokens generation bug by @turetske in https://github.com/PelicanPlatform/pelican/pull/392
  • Add swagger spec and include auth APIs and common APIs by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/401
  • Replace ComputeExternalAddress by param Server.ExternalWebUrl and rename Server.Port and Server.Address by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/378
  • End-to-end test for XRootD by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/328
  • Add unit tests to XRootD monitoring handler by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/347
  • Hook up monitoring paths functionality by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/407
  • Refactor origin_ui to web_ui to reuse basic logic for server web UI/API by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/398
  • If no auth token is present, do not emit verbose error message by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/404
  • Director-based origin health tests by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/305
  • Always unmarshall config when /config API is called by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/395
  • Update integration test file by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/422
  • Rename namespace-registry to conform to go standards and move registry db test by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/425
  • Update integration test file - update stashcp source by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/426
  • Ensure TTL caches evict expired item by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/416
  • Federation Overview #221 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/411
  • Reimplimented recursive downloads by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/377
  • Fixed non running e2e test by @turetske in https://github.com/PelicanPlatform/pelican/pull/434
  • Remove content length from upload to enable chunked by @djw8605 in https://github.com/PelicanPlatform/pelican/pull/208
  • Add authorization policies for OA4MP by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/393
  • Add supervisor script to make origins servable by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/428
  • Fix bad error message from director by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/433
  • Fix racing conditions in various director-related code and tests by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/441
  • Periodically prepare a combined custom + system CA bundle by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/406
  • Add cache supervisord process by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/443
  • Add test dependencies to dev.Dockerfile for PR #419 by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/450
  • Pelican cache with auth by @turetske in https://github.com/PelicanPlatform/pelican/pull/423
  • Automatically unpack tarballs when downloaded by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/408
  • Web UI Move by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/449
  • Fix doc missing space for client-usage.mdx by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/432
  • Update Web UI src code location by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/455
  • Upgrade NextJS to ^14 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/454
  • Refactor director SD tests and add test for duplicated origins by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/430
  • Refactor generate package by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/442
  • Provide ConfigureOriginAPI a cancel context that can be cancelled on shutdown by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/431
  • Implement recursive uploads by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/415
  • Add listServer API to director Web UI by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/458
  • Refactor file transfer tests into a utility class by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/447
  • Swagger docs by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/456
  • Provide better message (with no error) when we register an already-registered namespace prefix by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/461
  • Arm-compatible Pelican container and fix 404 minio RPM by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/469
  • Adding dir listing capability to Pelican Origins by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/468
  • Enable key chaining to protect super/sub namespaces by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/460
  • Fix progress bar for recursive downloads by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/471
  • Director UI by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/466
  • Build containers but don't push on PR by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/467
  • Data Exports Overview Table #220 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/410
  • Lock NextJS to 14.0.3 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/475
  • Fix progress bars w/ recursive uploads and downloads by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/473
  • Add S3 backend to Pelican by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/419
  • Refactor checkAPIToken to be more generic and less verbose by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/429
  • Make the registry aware of topology by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/470
  • Add draft of SECURITY.md by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/324
  • Placeholder namespace UI 354 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/478

New Contributors

  • @timtheisen made their first contribution in https://github.com/PelicanPlatform/pelican/pull/348
  • @djw8605 made their first contribution in https://github.com/PelicanPlatform/pelican/pull/208

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.2.0...v7.3.0

Changelog

  • 4e93152 Address code review feedback * Update minio package version as the existing one was removed from archive * Remove redundant code * Preserve dev environment for dev container
  • 8904cde Replace xrd-http plugin by xrdcl-pelican plugin in dev-container
  • d1e3b82 Replace xrd-http plugin by xrdcl-pelican plugin in prod container

Changelog

  • 85cf48c Remove chmod on user-provided xrootd config files

Changelog

  • ef6d348 Truncate the plugin output file as soon as possible

V7.2

What's Changed

  • fixed typos within plugin_stage.go by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/186
  • Testing common transport timeouts by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/195
  • Add --version flag to pelican-client by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/199
  • Ship OA4MP in the Pelican docker image by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/181
  • Update goreleaser command in test CI and enable --single-target by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/218
  • Add cache for Go modules and npm packages by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/215
  • Strip "Bearer " from authz query and header if they're present by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/198
  • Refactored xrootd specific code out of pelican/cmd by @turetske in https://github.com/PelicanPlatform/pelican/pull/216
  • Start the documentation folder by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/244
  • Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from 0.42.0 to 0.44.0 by @dependabot in https://github.com/PelicanPlatform/pelican/pull/248
  • Moved transport config options to parameters.yaml by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/250
  • Federation component health status by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/217
  • Json generation by @turetske in https://github.com/PelicanPlatform/pelican/pull/266
  • Cleanup of pelican config files by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/243
  • Fixed mispelled filename in .gitignore file by @turetske in https://github.com/PelicanPlatform/pelican/pull/278
  • Documentation for user installation of the Pelican Client by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/242
  • Add first draft of client usage docs by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/249
  • Expose director's public key via Open ID style URLs by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/254
  • Replaced some remaining viper.Get* calls with the appropriate param.GetString() call by @turetske in https://github.com/PelicanPlatform/pelican/pull/269
  • Origin Installation Documentation by @turetske in https://github.com/PelicanPlatform/pelican/pull/245
  • Fix broken Director auth headers and add regression test by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/274
  • Added the StagePlugin.* params to the param table by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/270
  • Make default log level part of pelican config by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/236
  • Improve test of stash_plugin to be more like HTCSS environment by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/252
  • Embed OA4MP as an issuer by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/240
  • Fix dev container build failure by specify base image platform by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/286
  • Update Dockerfile by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/282
  • Fix some issues with Pelican RPMs by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/279
  • Move npm path placeholder creation from GHA to Go by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/273
  • Add some documentation on developing the website by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/267
  • Bump golang.org/x/net from 0.12.0 to 0.17.0 by @dependabot in https://github.com/PelicanPlatform/pelican/pull/292
  • Auto-generate config struct from parameters.yaml and provide REST API by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/268
  • Implement creation of a sharing URL by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/257
  • Web UI improvements by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/253
  • Fixed issues with Progress Bars by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/302
  • Add Web URL in origin's advertisement message by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/288
  • Upgrade status component by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/316
  • Updated the docker images to include xrdcl-http by @turetske in https://github.com/PelicanPlatform/pelican/pull/319
  • Auto-configure origin issuer access by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/299
  • Bump postcss and next in /origin_ui/src by @dependabot in https://github.com/PelicanPlatform/pelican/pull/296
  • Closes #275 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/317
  • Fix bypassed authentication for /config API by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/321
  • Add pelican-osdf-compat package by @matyasselmeci in https://github.com/PelicanPlatform/pelican/pull/327
  • Fix null dereference bug in Director by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/326
  • Graph Robustness #276 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/318
  • Director Prometheus scraper of origins' metrics by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/289
  • Api view by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/284
  • Add go generate step to make file by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/334
  • Pelican cache noauth by @turetske in https://github.com/PelicanPlatform/pelican/pull/306
  • Automatically register origin with the namespace daemon by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/295
  • Add Parameters page and order/name docs by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/336
  • Make director service host-aware by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/313

New Contributors

  • @dependabot made their first contribution in https://github.com/PelicanPlatform/pelican/pull/248
  • @matyasselmeci made their first contribution in https://github.com/PelicanPlatform/pelican/pull/327

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.1.0...v7.2.0

Changelog

  • 8515fb4 Add cache supervisord process
  • a2938c0 Add default environment var to origin supervisord script
  • 4173537 Add supervisor script to make origins servable
  • 0c5b1a0 Fix process name

Changelog

  • fc84142 Fix small bugs in DiscoverFederation()

V7.1

What's Changed

  • Makefile updates by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/128
  • Added redirect_test.go by @turetske in https://github.com/PelicanPlatform/pelican/pull/126
  • Add pre-commit hooks for the repo by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/93
  • Bump linter timeout to 3m by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/138
  • Switched over remaining usage of http.Client to use common transport by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/137
  • Added code coverage to github actions workflow by @turetske in https://github.com/PelicanPlatform/pelican/pull/134
  • Run pre-commit against entire repo by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/139
  • Remove commenting code that causes errors when using a fork by @turetske in https://github.com/PelicanPlatform/pelican/pull/145
  • Incorporate PR #78 from old osdf-client by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/144
  • Fixed Generate Private Key Problems in Windows by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/152
  • Revive codeql by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/151
  • Updated test.yml to attach code coverage to the action summary by @turetske in https://github.com/PelicanPlatform/pelican/pull/156
  • Updated mpb bar progress bars by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/132
  • Add a variety of configurable parameters to docs by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/133
  • Match ns prefix by longest match, not first match by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/142
  • Remove errant stdout message by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/158
  • First step toward self-testing: configure authorization by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/149
  • Test Origin Login APIs by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/154
  • Remove redundant viper call by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/162
  • Fix federation auto-discovery bug by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/161
  • Removed isDir from handle_http.go by @turetske in https://github.com/PelicanPlatform/pelican/pull/165
  • Parameter generation by @turetske in https://github.com/PelicanPlatform/pelican/pull/164
  • Expose CLI for creating tokens by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/157
  • Added tests for the three functions in origin_api.go by @turetske in https://github.com/PelicanPlatform/pelican/pull/160
  • Refactor the client to be in a sub-package by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/177
  • Add development Dockerfile by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/163
  • Pelican transport fixes and generated after configs by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/183
  • Origin self-test by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/167
  • Query director for available namespaces by @Techming in https://github.com/PelicanPlatform/pelican/pull/185
  • Put prometheus metrics behind federation/origin token by @turetske in https://github.com/PelicanPlatform/pelican/pull/176
  • Put the token acquisition workflow behind a plugin check by @turetske in https://github.com/PelicanPlatform/pelican/pull/180
  • Give the Director the ability to detect version mismatches by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/184
  • Overhaul config names by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/182
  • Fix template rendering failure by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/191
  • Added check of cookie login for prometheus by @turetske in https://github.com/PelicanPlatform/pelican/pull/193
  • Create Login Page for Post Initialization Login by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/196
  • Port OSDF-Client PR #87 to Pelican by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/197
  • Create new graph components by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/189
  • Add Status Box Component by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/192

New Contributors

  • @Techming made their first contribution in https://github.com/PelicanPlatform/pelican/pull/185

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.0.0...v7.1.0

This release modifies the creation of Pelican sub-RPMs to allow compatibility with osdf-client.

This patch fixes authentication headers in the director so that pelican object copy can determine issuer metadata for some namespaces dynamically.

What's Changed

  • Allow Pelican to run in some settings where $HOME is not defined @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/238

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.1.1...v7.1.2

What's Changed

  • Fix director authz by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/223

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.1.0...v7.1.1

V7.0

The first release of the new Pelican codebase!

Pelican currently includes the functionality of the prior osdf-client / stashcp; it emulates the CLI of those clients if the binary is renamed to stashcp or stash_plugin.

Two new components have been developed, the director and the namespace registry. The director is responsible for routing clients to an appropriate cache or origin to serve up their request (and has similar functionality to the redirector for the OSDF). The namespace registry adds an authorization layer to the data federation, associating a prefix with the public key of an origin.

This includes experimental support for launching an origin via the Pelican command line. Over the next few monthly releases, this support will mature to the point of replacing currently running origin containers in the OSDF.

Release Notes:

  • Merge OSDF Client by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/2
  • Cleanup linter failures in origin_ui by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/11
  • Finish implementation of the init login flow by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/13
  • Simple landing page for Origin UI by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/15
  • Director cli hookup by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/12
  • Add Director capability into the Pelican CLI by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/7
  • Have UI always use TLS by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/16
  • Various Director cleanups by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/19
  • Do not create key with group read permissions by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/26
  • Dockerize by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/29
  • Allow passing key directly via CL by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/28
  • Add Namespace Management CLI Commands by @CohenQU in https://github.com/PelicanPlatform/pelican/pull/14
  • Have origin advertise to the director service by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/18
  • Dockerize by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/31
  • Fix director tls by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/33
  • Turn off autostart for director daemon by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/35
  • Add cl flag to make default endpoint serve caches or origins by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/36
  • Add xrootd, xrootd-client, xrootd-server to container by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/37
  • Move groupadd and useradd before xrootd installation by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/38
  • Namespace client cleanup by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/30
  • Start adding documentation for existing parameters. by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/20
  • Fix small bug in director's cache sorting by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/40
  • Fix bad error message (bad content length) by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/10
  • Additional user interfaces and APIs by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/17
  • Embed of Prometheus monitoring by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/27
  • Add support for running as root and 'multiuser' mode by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/51
  • Draft integration of namespace registry app in Pelican by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/39
  • Bump go version in the publish-container action by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/57
  • Fix YAML error in go version for GHA by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/58
  • Improve the use of the X-Pelican-* responses by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/48
  • Reformat code base with go fmt by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/59
  • Enable go fmt as one of the linters by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/49
  • Clean UI by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/43
  • Create (and explain) the need of a placeholder for golintci by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/60
  • Create a single --port flag object by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/65
  • Upgrade remainder of the code to lestrrat-go v2 by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/66
  • Remove CVMFS shortcut within Pelican client by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/78
  • Fix small bug in ns-registry auto creation by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/75
  • Add copyright notices to Pelican source by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/80
  • Use lestrrat jwks instead of custom structs in NS Registry by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/76
  • Add deletion authorization by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/89
  • Fix director responses by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/92
  • Remove default user by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/95
  • Add registry to container by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/102
  • Create a privkey for registry by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/104
  • Add simple verification to namespace registration by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/94
  • UI updates by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/105
  • Include token in redirect by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/79
  • Copy all supervisord confs into image by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/107
  • Fix naming of oidc config vars by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/108
  • Add default port to supervisord daemons by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/109
  • Remove redundant env prefix by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/110
  • Using common transport for https based writes by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/103
  • UI changes by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/106
  • Fix bug with common transport TLSSkipVerify by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/115
  • Update Releaser by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/116
  • Send token and namespace ad to director by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/121
  • Remove hardcoded originUrl by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/124

New Contributors

  • @bbockelm made their first contribution in https://github.com/PelicanPlatform/pelican/pull/2
  • @jhiemstrawisc made their first contribution in https://github.com/PelicanPlatform/pelican/pull/12
  • @CohenQU made their first contribution in https://github.com/PelicanPlatform/pelican/pull/14
  • @joereuss12 made their first contribution in https://github.com/PelicanPlatform/pelican/pull/10
  • @CannonLock made their first contribution in https://github.com/PelicanPlatform/pelican/pull/43

Full Changelog: https://github.com/PelicanPlatform/pelican/commits/v7.0.0

CHTC Logo
Morgridge Logo

Branding

This project is supported by National Science Foundation under Cooperative Agreement OAC-2331480. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.