Pelican Releases

Pelican v7.25.0

This release introduces an embedded OAuth2 issuer as the OA4MP alternative third-party copy enabled by default for origins, a web read-only mode for servers, and periodic encrypted database backups. It also includes a broad set of improvements to the client, UI, director reliability, security, and federation operations.

Breaking Changes & Upgrade Notes for Admins

• [Cache]: [Admin Action Required] Override the new storage-health defaults on caches: set Monitoring.StorageWarningThreshold: 92 and Monitoring.StorageCriticalThreshold: 97. The defaults added in #3241 (80/90) sit below typical cache purge watermarks, causing the Director to filter the cache before it can purge. The knobs were unhidden in #3453; defaults will be fixed in a follow-up release.
• [Origin]: Hardened the POSC plugin to prevent startup across filesystem boundaries in #3269; Added POSC bypass filter for monitoring paths in #3297

New Features and Enhancements

• [Origin]: POSIX origins now validate filesystem permissions against namespace capabilities at startup in #2881
• [Origin]: Added an embedded OAuth2 issuer in #3151
• [Origin]: Third-party copies are now enabled in origins out-of-the-box in #3148
• [Origin]: Integrated the N2N plugin to automatically map FederationPrefix to StoragePrefix for XRootD storage backends in #3111
• [Servers]: Added periodic SQLite database backups with compression and encryption in #3154
• [Servers]: Added a web read-only mode (Server.WebReadOnly) to restrict mutating operations via the web UI in #3205
• [Client]: Improved object ls output with sorted entries, terminal-aware column formatting, and clear directory distinction in #3250
• [Director]: Director redirect information is now probabilistically added to plugin output class ads in #3177
• [Registry]: Displayed registration type explicitly in Registry’s WebUI in #3218
• [Origin/Cache]: Presented the registered server name provided in Origin and Cache’ WebUI #3220

Bugs Fixed

• [Client]: The client now accurately reports when a token is not sent in #3223
• [Client]: Preserved PreferredCaches ordering against responsiveness-based reordering in sortAttempts in #3253
• [Client]: Fixed token discovery logic in #2846
• [Client]: Fixed checksum verification for mismatched algorithm types in #3174
• [Director]: The Director no longer stats more caches than it intends to provide for sorting in #3179
• [Cache]: Caches now always use the federation host for the metadata discovery endpoint in #3268
• [Cache]: Disabled client x509 by default, as most CAs will no longer issue certificates with both client/server auth in the EKU #3202
• [Origin]: Fixed Globus origin to correctly handle non-existent directories in #3186, rejected multiple exports #3170, and OAuth flow requires Server.EnableUI = true #3144
• [Origin]: StoragePrefix and FederationPrefix are no longer validated the same way in #3267
• [Servers]: Configuration values marked as hidden are now hidden in the config editor webUI in #3222
• [Servers]: Downtime scheduling now prevents an end time before the start time in #3243

New Contributors

• @Macman1234 made their first contribution in #3098

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.24.3...v7.25.0

Pelican v7.24.3

Contains bugfixes for Caches and Origins

Bugs Fixed

• [Cache]: Patched instantaneous client timeout bug in Cache middleware (xrdcl-pelican) in #3217
• [Origin]: Restored missing sssd dependency in Origin containers in #3312
• [Origin]: Patched Origin multiuser plugin for containers to fix segfault bug in #3303

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.24.2…v7.24.3

This patch release resolves a known vulnerability.

Pelican v7.24.0

This release introduces a substantial set of new features requested by users and server administrators, with development accelerated through AI-assisted engineering workflows.

Breaking Changes & Upgrade Notes for Admins

• [Cache/Origin]: Caches and Origins have a new dependency requirement on xrdhttp-pelican >= v0.0.10

New Features and Enhancements

• [Client]: Made significant updates to error classifications in Clients to provide users with more actionable/intelligible errors in #3089
• [Client]: Introduced a Client API Server for Pelican that exposes Client functionality through a RESTful API over Unix domain sockets, enabling programmatic access to Pelican transfer capabilities without directly invoking the CLI in #2780
• [Client]: Improved performance of the Client's object prestage command when using Caches that support the pre-stage API in #3014
• [Client]: Introduced a --dry-run option for the Client's object commands that lets users preview what actions the Client will take before moving data in #2865
• [Servers]: Added CLI tooling to create API keys for interacting with Pelican servers in #2958
• [Director]: Made discrepancies between federation metadata hosted at the Discovery URL and federation Directors more obvious and less prone to drift in #3009 and #3008
• [Servers]: Added the ability for Pelican servers to analyze/report on various storage metrics, including disk consumption by Origin exports and health monitoring for service log files in #2957 and #2967
• [Cache/Origin]: Added support for PKCS#11 to protect TLS credentials from unanticipated XRootD security vulnerabilities in #2948, #3055, and #3072
• [Cache/Origin]: Caches and Origins now check for installation of a minimum required XRootD version on startup and cause an error if no such version is found in #3092

Bugs Fixed

• [Client]: Fixed a progress bar bug that caused degraded performance with recursive Client actions (e.g. pelican object get --recursive) on namespaces containing ~100k+ objects in #2975
• [Client]: Made it more explicit in Client checksum operations when a checksum type other than what was requested is returned by a server in #2847
• [Director]: Fixed a bug that prevented the Director from logging errors or recording failure metrics when some Cache Director tests failed in #3099

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.23.0...v7.24.0

This patch release resolves a known vulnerability.

Pelican v7.23.0

This release brings new features and enhancements - such as write support for Globus origins and a golang io.FS interface. It also includes some behind the scenes reliability improvements.

New Features and Enhancements

• [All]: Non-root users who do not provide configuration in ~/.config/pelican will now fallback to look at system configuration in /etc/pelican #2990
• [Client]: Added --direct flag to allow for object retrieval directly from an origin #2864
• [Client]: Implemented a golang io.FS interface in the client #2962
• [Plugin]: The plugin can now retrieve Pelican configuration variables from HTCondor ClassAds #2891
• [Plugin]: There is now a CacheHit field in transfer output ClassAd #2952
• [Origin]: Full support for Globus origins - now allows for ls and writes #2769
• [Cache/Origin]: Now allows for admin groups #2863
• [Servers]: Added the ability to temporarily change the log level on running servers #2896
• [Servers]: Added support for GitHub OAuth2 authentication #2901

Bugs Fixed

• [Client]: Implemented retries for some types of client ls failures #2910
• [Client]: Pelican sync no longer creates directories with incorrect permissions
• [Plugin]: PELICAN_CLIENT_PREFERREDCACHES environment variable no longer ignored by the plugin #2828
• [Director]: Fixed race condition in director advertise shutdown #2922
• [Director]: Fixed frequent panics and restarts due to two Prometheus endpoints #2936
• [Cache/Origin]: Origins and Caches no longer need Registry acknowledgement to go into downtime #2942

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.22.1...v7.23.0

This patch release resolves a known vulnerability.

Pelican v7.22.1

This release fixes critical bugs in the multi-user plugin configuration and prevents potential deadlocks in stat routines.

Breaking Changes & Upgrade Notes for Admins

• [Origin]: For admins using the Multi-User feature, checksums will now be written to the xattrs of uploaded files

New Features and Enhancements

• [Origin]: Enhanced authorization template form with new multi-select fields for improved value selection in #2968

Bugs Fixed

• [Origin]: Fixed multi-user plugin configuration that was preventing proper initialization with checksumming in #2978 and #3041
• [Director]: Fix director's stat feature to no longer "leak" goroutines #2935
• [All]: Updated Docker image build process to use stock AlmaLinux base image for cleaner builds in #2992
• [All]: Added bsdtar to work around ARM64 emulation bug with standard tar command in #3069

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.22.0...v7.22.1

Pelican v7.22.0

This release brings a broad set of improvements to reliability, observability, and authorization, offering meaningful quality-of-life upgrades for operators and users.

Breaking Changes & Upgrade Notes for Admins

Several config parameters (e.g., Debug, various DbLocation keys) have been formally deprecated or removed. The central services (Registry & Director) must be upgraded before Origins and Caches. The ordering between Registry and Director upgrade doesn't matter.

New Features and Enhancements

• [**Client/Origin **] New collection management CLI commands and revamped collections authorization #2971
• [All] A new web interface for managing groups and user membership #2740
• [Cache] A new Site-Local cache mode #2798
• [All] Prometheus data retention policy is now configurable #2779
• [Cache/Origin] New health metrics for config update and XRootD self-tests #2666
• [Client] More robust error classifications in client #2825
• [Director/Origin] Director can now scrape metrics from brokered Origins #2824

Bugs Fixed

• [Client] Fixed authentication loop issue in the client #2790
• [Director] Fixed namespace TTL cache behavior in the Director #2821

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.21.1..v7.22.0

This patch release resolves a known vulnerability.

Pelican v7.21.2

Bugs Fixed

• [Cache/Origin] Trigger degraded state with a configurable ratio of server concurrency #2801
• [Client] Make tls.AlertError retryable #2813
• [All] Fixed 404 error when switching to other Pelican Docs versions from v7.21 #2804
• [Director] Fixed a TTL cache bug that prevents the eviction of stale JWKS #2821
• [Origin] Restore the ability to view an Origin's directory listing directly in the browser #2842

Pelican v7.21.1

Pelican v7.21.1 addresses a build infrastructure issue that was preventing Docker images from being built successfully due to insufficient disk space in the GitHub Actions runner. No changes were made on Pelican client and server.

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.21.0…v7.21.1

Pelican v7.21.0

This release comes with UX improvements on Pelican Client, HTCondor Plugin, and Servers. It also provides a short circuit mechanism for Caches/Origins with outdated XRootD config files, and other useful updates.

Breaking Changes & Upgrade Notes for Admins

This release further standardizes using a Federation's "Discovery URL" as the root of trust when the Federation central services generate authZ tokens – this may cause issues if the Director's private signing key does NOT match one of the keys exported by the Discovery URL's issuer metadata. To check for metadata alignment, visit the /.well-known/pelican-configuration path at your Discovery URL and follow the jwks_uri link. Compare the keys here with the keys from the jwks_uri link at your Director's /.well-known/openid-configuration path. The Director's keys should be in the list you accessed via the Discovery URL.

• [Cache/Origin] Enabled auto shutdown a Cache/Origin server if its scitokens.cfg file and/or Authfile are not updating for a given time (1h by default) #2666
• [All] Deprecated Debug param and modified default client/server log levels #2750
• [Servers] Switched to installing the pelican-server binary in all server images #2725
• [Servers] Updated Goreleaser config to retain symbols in release builds #2667
• [Registry] Added cleanup mechanism for stale entries in servers and services tables upon upgrade #2745; Additionally, the creation of new stale entries will be prevented #2748, #2739

New Features and Enhancements

• [All] Supported Dark Mode in Pelican Docs #2651
• [All] Added versions to Pelican Docs #2749
• [Client] Improved transfer error handling #2742, #2733, #2757
• [Client] Added Permission Denied error to Client/Plugin #2669
• [Client] Logged checksum error only when checksum is required but failing to get #2693
• [Servers] Improved downtimes UI #2636
• [Registry] Added Deletion Confirm Button #2670
• [Registry] Supported Autofill Security Contact if the registration is from Origin #2630
• [Client] Supported depth limited recursive listings in pelican object ls command #2720
• [Client] Checked local filesystem before contacting director on upload #2677
• [Servers] Made OIDC scopes configurable #2726

Bugs Fixed

• [Client]: If the download fails for some reason (ex 404), no half-done file remains on disk #2701
• [Cache/Origin] Fixed a bug preventing Origins with Logging.Level: Warn from starting #2712
• [Director] Fixed a JSON marshalling bug that causes Origin/Cache-set downtimes stuck in Director when no active downtime #2684
• [Cache] Fixed repetitive error log message for the XrdCurlStats monitoring #2775
• [Origin]: Fixed a bug that prevented Origins from parsing some custom authfile attributes in #2778

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.20.2...v7.21.0

Pelican v7.20.3

This release comes with a bugfix for custom XRootD authfiles at Origins.

Bugs Fixed

• [Origin]: Fixed a bug that prevented Origins from parsing some custom authfile attributes in #2778

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.20.2…v7.20.3

Pelican v7.20.2

This release comes with general Cache/Origin stability bugfixes, tunes the Director's adaptive sorting algorithm and provides an important security update for Caches/Origins.

Bugs Fixed

• [Origin]: Bumped the containerized xrootd-s3-http version to fix a bug in S3 Origins that could cause server crashes in #2718
• [Cache/Origin]: Changed the way authfile entries are sorted to prevent applying access policies from public namespaces to private namespaces when the public namespace is a prefix of the private (e.g. /foo is public and /foo-auth is private) in #2705
• [Cache/Origin]: Fixed a bug that prevented Caches and Origins from starting when Pelican's global logging level was set to Warn in #2712
• [Director]: Made small changes to the Director's adaptive sort algorithm to help stabilize load distribution around closer caches in #2716

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.20.1…v7.20.2

Pelican v7.20.1

Pelican v7.20.1 fixes a Cache bug that results in unresponsive Caches, as well as a Director bug that prevents some server downtimes from clearing.

Bugs Fixed

• [Cache]: Upgraded the containerized xrdcl-pelican dependency to v1.5.6 to fix a Cache thread deadlock in #2688
• [Director]: Fixed a bug that prevented expired downtimes set by Origin/Cache admins from clearing in #2684

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.20.0…v7.20.1

Pelican v7.20.0

Pelican v7.20 brings a variety of Client usability improvements and bugfixes, server UI upgrades and Cache stability/debugging enhancements.

Breaking Changes & Upgrade Notes for Admins

No special upgrade considerations.

New Features and Enhancements

• [Client]: Modified the Client command pelican object ls to produce single column outputs for better piping with grep in #2621
• [Plugin]: Enabled the HTCondor plugin to run condor_reconfig automatically whenever it's updated in #2624
• [Origin]: Gave Origin admins the ability to configure the issuer URL set in the Director's X-Pelican-Token-Generation response header (used by Clients for bootstrapping automatic token generation) in #2598
• [Cache]: Made a variety of new XRootD metrics available via Prometheus to aid in debugging server performance in #2597 and #2588
• [Director]: Changed the Director's UI to distinguish between a Cache/Origin's XRootD health tests and its overall server health status in #2625

Bugs Fixed

• [Client]: Fixed various authorization errors related to listings and recursive deletes in #2499 and #2579
• [Client]: Fixed a bug that prevented pelican origin token create from using custom paths for private keys in #2641
• [Client]: Fixed a Client deadlock that occurred in some transfers in #2655
• [Servers]: Fixed an RPM packaging bug that overrode the OSDF's discovery URL in #2612
• [Cache]: Updated containerized Cache dependencies to fix a variety of deadlocks/segfault in #2611 and #2615
• [Director]: Fixed a bug in the Director's "adaptive sort" algorithm that resulted in sending Clients to some broken Caches in #2647

New Contributors

• @fallow64 made their first contribution to Pelican documentation in #2511

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.19.0...v7.20.0

Pelican v7.19.5

Pelican v7.19.5 fixes a Cache bug that results in unresponsive Caches.

Bugs Fixed

• [Cache]: Upgraded the containerized xrdcl-pelican dependency to v1.5.6 to fix a Cache thread deadlock in #2688

Changelog

• c3a376bb89d145d0192d5998fe9943c5589ff9e1 Changed version to 7.19.5

Pelican v7.19.3 Release Notes

Bugs Fixed

• [Client] Fixed a deadlock bug #2655
• [Cache] Increased the monitoring eviction interval from 60ns to 60s #2662

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.19.2...v7.19.3

Pelican v7.19.2 Release Notes

New Features and Enhancements

• [Client] Prevent overwriting of objects on upload #2480
• [Logging] Logs now don’t include ANSI color codes when going to non-terminals #2529
• [Client] New error classification for slow origins #2559
• [Origin] Added in a collections API to create mutable groupings of data #2550
• [Client] Added --pack flag to get and put in the CLI #2549

Bugs Fixed

• [Client] Fixed the race condition causing the failed to verify size bug #2533
• [Client] Fix deadlock where results channel is not read after cancellation #2536
• [Client] Fixed an issue where checksum requirements on put failed when not required #2546
• [Client] Fix segfault in object get with pack=auto when destination directory doesn't exist #2561

Notes for Admins

• Bumped xrdcl-pelican version to 1.5.4
• Bumped XRootD to v5.8.4-1.2
• [Federation Admin] Before upgrading Registry, finish the cleanup for site name, as explained in the “for Registry admin” section in #2558

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.19.1...v7.19.2

Do Not Use This Release

This version was improperly packaged due to an issue with GoReleaser. Please use the latest version in Pelican v7.19 series instead.

Bugs Fixed

• [Cache] Use xrdcl-pelican 1.5.4 to fix deadlocks in Cache

Bugs Fixed

• [Client] pelican object put now only fails during the checksum process if we required a checksum in the transfer
• [CLI] pelican config tool now takes environment variables into account
• [Cache] DropPrivileges is now ready to used in Cache
• [Client] Client doesn't stall during shutdown

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.18.0...v7.18.1

Pelican v7.18.0 Release Notes

New Features and Enhancements

• [Client] Introduced client transfer statistics reporting. #2436
• [Origin] S3 tracing can now be enabled via Logging.Origin.Oss configuration #2410
• [Broker] Allow connection broker to be used for director-to-cache communications #2378
• [Origin] Added S3 file cache Prometheus metrics #2433
• [Client] Added checksum support to pelican object put #2415
• [Client] Keyring errors are now set to the Debug log level #2420
• [Client] Marked certain transient errors as retryable, including invalid chunk headers, unexpected EOFs, and TLS connection failures during uploads. #2442
• [Client] Improved handling of deprecated/hidden parameter handling in pelican config get #2513
• [Director,Origin,Cache] Enabled web client to GET authenticated files, PUT all files, and interactively list objects #2468 #2467
• [Origin] Add support for regular expressions to the built-in token issuer's authorization templates #2462
• [Packaging] Update OA4MP #2444
• [Packaging] ​​Bump XRootD release to 5.8.3-1.3 #2474

Bugs Fixed

• [Client] Fixed how tokens are deemed as acceptable #2455
• [Client] Fixed “failed to verify size” bug #2533
• [LocalCache] Fixed local cache cleanup bugs. #2432
• [Director] Fix not-found logic when querying against caches and origins #2392
• [Director] Fix link depth header for listings #2437
• [Director] High Availability Director Bug Fixes #2400
• [Packaging] Fix pelican-server binary panic and upgrade xrdhttp-pelican to v0.0.7 #2476

Notes for Admins

• Bumped XRootD to v5.8.4-1.2 and upgraded xrdhttp-pelican to v0.0.7.
• Native packages now require XRootD ≥ 5.8.2.

Full Changelog: v7.17.2...v7.18.0

Bugs Fixed

• [Director]: Prevent Director's web UI crash from non-semantic versions
• [Origin]: Actually make use of Origin.ScitokensNameMapFile in SciTokens configuration generated for XRootD

Bugs Fixed

• [Client]: Fix token scope for specific object operation https://github.com/PelicanPlatform/pelican/pull/2407
• [Cache]: Trim trailing '/' from federation-derived lot paths https://github.com/PelicanPlatform/pelican/pull/2435

Pelican v7.17.0

New Features and Enhancements

• [Client]: New pelican token create command for on-demand JWT generation #2314
• [Client]: The new pelican key create command generates a private-public key pair. If a private key already exists at the specified location, the command uses it to derive the corresponding public key. This new command retires pelican generate keygen. https://github.com/PelicanPlatform/pelican/pull/2380
• [Client]: Client uses token issuer as selection criteria during token selection process, also displays which token it chooses #2219, #2348
• [Client]: SciTokens-profile JWTs are now accepted in addition to wlcg profile #2223
• [Client]: Friendlier message when pelican get is run without a destination #2351
• [Origin/Cache]: Graceful shutdown a server, waiting some time for in-flight transfers before exit #2318
• [Origin/Cache]: Persist an Origin/Cache’s registered name #2300
• [Origin/Cache]: Server ads now carry a status field containing degraded state when concurrency limits are exceeded #2338
• [Origin/Cache]: Collect XRootD CPU-utilisation metric in monitoring #2225
• [Origin/Cache]: Added XRootD logging on the open, close, and transfer events #2277
• [Origin]: Concurrency limits can now be enforced in OSS throttle mode #2276
• [Servers]: Login cookies (and generated tokens) now last 16 hours #2255, #2396
• [Registry]: Automatically adds the current user as security contact when creating a new namespace #2132
• [Registry]: Namespace list refreshes automatically after edits #2269
• [Director]: Service-version graphs added to the Director dashboard #2402
• [Documentation]: Origin documentation overhaul #2298
• [Documentation]: Update FAQ section in Pelican Docs #2089, #2261
• [All]: Startup warning if the deprecated IssuerKey file is still in use #2228
• [Packaging]: New pelican-client-debug RPM for easier troubleshooting #2317
• [Packaging]: Container images rebased on XRootD 5.8.3-1.2, xrdcl-pelican 1.2.3 and xrootd-s3-http 0.4.1 #2283, #2340, #2342, #2368, #2399
• [Plugin]: Improved error propagation when servers return HTTP error codes #2344

Notes for Admins

• Deprecated key path – A clear warning is now emitted if IssuerKey is still set; please migrate to the new directory-based key layout (put private keys into IssuerKeysDirectory, and ensure they have .pem extension) #2228
• Name – When a new origin/cache registers, Pelican will use os.Hostname as sitename, unless the admin specifies its name by setting Xrootd_Sitename in the config. For all servers in the federation, if their name in the local config is different from the registered one, Pelican will still use the registered one and throw a warning in the logs. #2300
• Container images – Note that Pelican are now based on XRootD 5.8.3-1.2 #2399
• Graceful shutdown timeout – The default Pelican Origin/Cache graceful shutdown timeout is 1m (set by config param Xrootd_ShutdownTimeout). If the Origin/Cache is running in a k8s pod, the pod should be configured to have a slightly longer termination timeout than Pelican itself.

Bugs Fixed

• [Client]: Fixed Windows client’s ability to request and save tokens #2267
• [Client]: Fixed checksum validation and corrected downloaded-size comparisons #2259, #2275
• [Client]: Repaired rate/progress calculations #2322
• [Client]: Aligned runPut proxy logic with downloadHttp behaviour #2372
• [Director]: Downtime toggles now work reliably and downtime history tests are no longer flaky #2280
• [Director]: Stopped unintentionally extending TTL on every serverAds.Get() call #2290
• [Director]: Geo-location widget hidden from non-admin users #2272
• [Registry]: Properly handle trailing slashes in namespace registration and advertisement #2270
• [Origin/Cache]: Fixed self-test failures and permission-denied errors in drop-privileges mode, though it is not ready for production due to Kubernetes limitations #2120, #2291
• [Origin/Cache]: Restored debug logging from XRootD framework itself #2387
• [Origin/Cache]: Uses the proper name when requesting federation tokens #2343

New Contributors

• @whwjiang made their first contribution in https://github.com/PelicanPlatform/pelican/pull/2345

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.16.6...v7.17.0

Pelican v7.16.7

Bugs Fixed

• [Cache]: Fixed various bugs related to Lotman storage accounting in Lotman-enabled caches

Pelican v7.16.6

Bugs Fixed:

• [Client]: Fixed part of an issue related to pelican object puts being proxied through an HTTPs proxy by allowing the client to ignore those proxies.
• [Cache]: Fixed a bug that prevented Caches from obtaining a valid Federation Token when the service was registered using the Xrootd.Sitename configuration parameter.
• [Origin/Cache]: Updated the containerized XRootD version to 5.8.3-1.2, which among other things comes with additional segfault fixes.
• [Origin]: Bumped containerized xrootd-s3-http plugin to v0.4.1, which comes with bug fixes related to data corruption coming from the Origin and deadlocks that blocked transfers from completing.

Pelican v7.16.5

Bugs Fixed

• [Caches]: Updated containerized dependencies (xrdcl-pelican v1.2.3 and xrootd v5.8.2-1.5) to pull in fixes for additional segfaults triggered by high-concurrency cache failures.

Pelican v7.16.4

Bugs Fixed

• [Client]: Corrected an issue that caused calculated download rates to tend toward 0, causing long transfers to terminate
• [Client]: Fixed incorrect calculation in the client's EWMA download progress bars
• [Client]: Fixed a bug that prevented Client.MinimumDownloadSpeed from taking effect unless a deprecated config option MinimumDownloadSpeed was also set

New Features and Enhancements

• [Client]: Added the ability for clients to log their transfer progress at a defined interval using the Logging.Client.ProgressInterval config option (default 1m)

Pelican v7.16.3

Bugs Fixed

• [Origin/Cache]: Updated Pelican container images to use XRootD 5.8.2-1.3, which contains fixes for a potential segfault related to the "Close does not refer to an open file" error message.

Pelican v7.16.2

Note: This release failed to generate container artifacts. The issue has been corrected and this bugfix will be released as v7.16.3

Bugs Fixed

• [Origin/Cache]: Updated XRootD to 5.8.2-1.3 to fix potential segfault related to "Close does not refer to an open file" error message.

Pelican v7.16.1

Bugs Fixed

• [Director]: Corrected an issue that prevented the Director from forgetting about some Origins/Caches in #2290
• [Director]: Fixed a bug that impacted some types of server downtime filtering and prevented the Director from receiving the proper updates to these configurations in #2280
• [All]: Fixed RPM packaging issues involving unavailable dependencies for some platforms in #2286

Pelican v7.16.0

New Features and Enhancements

• [Client]: Added end-to-end checksumming in the client for GETs in #2205
• [Servers]: Created user interface for creating/managing API tokens in #2151
• [Director]: Added Origin/Cache version information to the Director's web display cards in #2135
• [Cache]: Added Cache metrics page to the Director's user interface in #1788
• [Origin]: Added Origin metrics page to the Director's user interface in #1990
• [Client]: Clients now perform a sanity check to verify that the size of a downloaded object is the same size as reported by the source in #2177
• [Origin]: Gave Origin admins the ability to configure connection throttling to act as a circuit breaker between the Origin and the object store in #2186
• [LocalCache]: Added local cache API that lets admins control some aspects of the cache's purge queue by denoting which objects should be purged first in #2118
• [Servers]: Added flags to selectively disable Topology integration (OSDF Only) in #2189
• [Origin]: Finished "CacheAuthz" features that let Origins specify they're only willing to send data to federation-authorized caches in #2175, #2234
• [Registry,Origin,Cache]: Added new downtime CLI and Web UIs so administrators can configure service downtimes with full history in #2197, #2218, #2210
• [Cache,Origin]: Added fair-share throttling to XRootD that uses token IDs to throttle individuals based on the demand they generate in #2280

Bugs Fixed

• [Client]: Fixed bug that prevented schemeless Pelican URLs from working with federation info passed through -f flag in #2116
• [Plugin]: Corrected transfer start time in the plugin's transfer classads in #2144
• [Servers]: Standardized Prometheus metric names to follow a common pattern in #2105
• [Cache]: Fixed processing of XRootD throttle plugin metrics in #2133
• [Origin]: Corrected issue preventing multi-user origins from caching SciTokens issuer information in #2184
• [LocalCache]: Fixed bug that prevented server restarts from re-binding to the cache socket in #2224
• [Registry]: Fixed bug that prevented Registry admins from updating namespace keys via the Web UI in #2173
• [Client]: Corrected issues generating tokens for Windows clients in #2267 https://github.com/PelicanPlatform/pelican/pull/2267

Notes for Admins

None

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.15.2...v7.16.0

Changelog

• 4d103c12a50778980332ec3d7d5aa223a9e52d52 Build and package a version of pelican that includes debugging symbols
• e294ba6af579487109f02591240e5094670228ce Install 'delve' into the dev container image
• d8dc5c8851c15e9bd0b9cbc719a09757faed7662 Updated version is ".goreleaser.yml" for pelican-debug
• 2d8132bd2543461634bc3275f5e2a6c5a4f717b6 Updated version to 7.15.4

Pelican v7.15.3

Bugs Fixed

[Cache] Fixed throttling in the cache [Cache] Updated xldcl-pelican to 5.8.1 to prevent caches from seg faulting, this is an urgent fix for caches. [Cache/Origin] Fixed bug in metric collections

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.15.2...v7.15.3

Pelican v7.15.2

Bugs Fixed

• [Cache] XRootD now uses xldcl-pelican v1.2.0 which contains bug fixes for the cache

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.15.1...v7.15.2

Pelican v7.15.1

Bugs Fixed

• [Origin] Made Origin throttling configurable #2186
• [Origin] Enabled multiuser origins to cache public keys for validating tokens #2184
• [Client] Now sanity checks size for client GETs #2177

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.15.0...v7.15.1

Pelican v7.15.0

New Features and Enhancements

• [Origin] Added config parameter Issuer.OIDCPreferClaimsFromIDToken which causes Pelican’s user authentication to look for claims in the OIDC ID token before falling back to the OIDC Userinfo endpoint. #2100
• [Origin] Pelican origins can now specify multiple issuers per namespace #2103
• [Director] Director can now report cache redirection reasoning when client commands are run with -d #2054
• [Director] Added support for a Federation to have multiple Directors #2023
• [Origin, Cache] Integrate XRootD OSS statistics #2092
• [Director] Add API Token Listing endpoint #2086

Bugs Fixed

• [Client] Fixed a regression in pelican generate keygen that caused it to generate invalid public keys #2085
• [All] When key files are deleted, the private keys are also now removed from Pelican’s in memory processes #2087
• [Client] The client no longer panics when doing a put or a share with just the base namespace prefix #2097
• [Client] Fixed client bug where schemaless Pelican URLs don’t work with ‘-f’ fed infor #2116
• [All] Assorted issues that prevented using an authentication provider other than CILogon or Globus #2091 #2046
• [Client] Now correctly reports the start time for uploads [#2144]

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.14.2...v7.15.0

Bugfix

• Install sssd-client into both origin container images #2102

Operation & Bugfix

• Fix index out of bound error in origin servers #2082
• Update XRootD version from 5.7.2 to 5.7.3 c555c47b

Pelican v7.14.0

New Features and Enhancements

Client

• N/A

Cache, Director, Origin, Registry

• Add /version endpoint to expose version and build info #2038
• Log Pelican config on server startup #1970

Registry, Director

• Drop privileges to user pelican if requested #1850

Origin, Cache

• Fix xrootd log level mapping bug and inherit from Pelican log level #1988
• Allow for Shoveler messages to be understood by monitoring packet handler #1981
• Prometheus Metric For When a Service Crashed #2015

Origin

• Refactor & Cleanup Origin Exports code #1919
• Deprecate Lotman.DbLocation in favor of Lotman.LotHome #2001

Cache

• Remove tls authentication from caches #1994

Director

• Fix Director Capabilities Display #1980
• Add Cache and Server Info Cards to Map #1947
• Geoip table #1893
• Add Server Outage information #1982
• Add federation token endpoint in Director, and implement routine for caches to fetch one #1985
• Grafana Token API Access #1917
• Enable PROPFIND of Director health test API #2002
• Support using IP address to query the geo location of a server in Director container #2059

Condor-Pelican

• K8SPhysicalHostName Error Message #1891

Document Updates

• Simplify Linux install instructions #1921
• Remove jargon from the credential management page #1922
• Document restricted characters in fed prefixes #2050
• Update Server.XYZ docs to include all server components #1997

Operations & Bugs Fixed

• Don't build dev images on git tags, tag image with latest-itb and sha256 #1946
• Clean up the container image build and testing process #1972
• Run tests in more appropriate container images #2000
• Update xrootd release version to 1.5.purge.osg #1983
• Pinned the goreleaser version to 2 #1991
• Attempted speedups for GHA PR runs #2030
• Rebase on OSG 24 (#1858) #2016
• Install new plugin: xrdhttp-pelican v0.0.3 #2013
• Update OS X install of xrootd-s3-http to v0.2.1 #2026
• Reduce memory hoarding in the unit test #2028
• Pin xrootd log levels in fed test and use typed params throughout #2019
• Fix container building process: Properly install pelican-server into /usr/local/sbin #2053
• Resolve the excessive use of xrootd user #2057 #2062

Full Changelog

Compare v7.13.0...v7.14.0

Bug fixes

• [Cache, Origin] - Rebuilt containers to include newer version of xrootd: xrootd-5.7.3-1.3.purge.osg24 which contains additional bug fixes.

Pelican v7.13.0

Breaking Changes

Registry Upgrade Requirement

Pelican v7.13 Director is not compatible with older versions of the Registry. The Registry must be upgraded before upgrading the Director to ensure proper functionality.

New Features and Enhancements

Client

• Improved detection and retry logic for clients that start transfers while the Director is down #1890
• Added new prestage functionality in the client #1923

Cache, Director, Origin, Registry

• Added pelican user and group along with some default directories for RPM installations #1856
• Added a "restart" button in each server's web UI #1869
• Added configuration knobs to increase server memory performance by limiting Prometheus cardinality #1887
• Created a new config parameter, Server.HealthMonitoringPublic, to make the server's health monitoring endpoint public #1936

Cache

• Disabled redirects for HTTP caches #1915
• Added experimental patches that modify a cache's ability to monitor resources and keep track of purgeable objects #1926

Director

• Created a Prometheus metric in the Director to track rejected cache/origin advertisements #1885

Origin

• Enabled server-side key refreshing so that origins/caches can update their keys in the Registry #1748

Bugs Fixed

Director

• Fixed issue with director tests failing due to the token being created using the director URL instead of the discovery URL #1936

Full Changelog

Compare v7.12.4...v7.13.0

Bugs Fixed

• [Caches] - Rebuilt containers to ingest xrdcl-pelican v1.0.5, which comes with additional segfault bugfixes.
• [Origins] - Rebuilt containers to ingest upstream bugfixes for S3/HTTP origins.

Bugs Fixed

• [Cache Containers] - Grabbed patched version of xrootd 5.7.2 to mitigate further segfault bugs related to caches that are handling many concurrent requests for non-existent objects.

Bugs Fixed

[Cache] - Fixed a broken dependency for our cache containers that prevented caches from fetching non-cached objects from origins. [Cache] - Updated an underlying library version to prevent caches from segfaulting when non-existent objects are requested. [Client] - Added extra handling for client PUTs to accept XRootD's new HTTP 201 response.

Known Bugs

[Director/Cache] - This release contains a known bug where the Director's test utility fails for caches. This shouldn't affect the cache's ability to function, but will result in the cache showing errors in the Director's Web interface.

NOTE

Containerized versions of this release were dead on arrival due to a broken dependency for xrdcl-pelican.

Bugs Fixed

• [Servers] Restored the functionality of various tickers that had been erroneously stopped, preventing XRootD servers from refreshing some files like scitokens/auth config and preventing some metrics from being scraped by Prometheus.
• [Caches] Updated caches' underlying version of xrdcl-pelican from 0.9.4 to 1.0.2, which prevents a segfault in caches whenever a non-existent object is requested.

Bugs Introduced

• [Caches] The directorTest functionality will fail in caches due to the new xrdcl-pelican version until updates can be made in Directors. These test failures are limited in scope and should not affect the rest of the cache's functionality, but may generate excess noise in logs.

Pelican v7.12.0

Breaking Changes

[Cache] Corrected previously unsafe creation of namespace, meta and data directories in caches in #1709. This change may result in cache data invalidation, and will likely require manual intervention during upgrades. Failure to adjust the data hierarchy of existing caches could result in unintended data leaks or zombie data that can't be cleaned by the cache. If this affects you, please make sure you understand update requirements before beginning your upgrades.

New Features and Enhancements

[All] The config parameter Server.TLSCertificate has been deprecated in favor of Server.TLSCertificateChain to more clearly convey the contents of the file pointed to. The deprecated parameter should continue to function for the time being, but may removed in a future release. [All] Added a CLI tool pelican config for displaying and interacting with Pelican configuration in #1693. [All] Standardized and improved RPM packaging in #1666. [Director] Gave Directors the ability to collect Prometheus metrics for GeoIP resolution failures in #1772. [Director] Servers now advertise their version to the Director in #1753. [Director] Started displaying namespace capabilities in the Director in #1737. [Director] Improved Prometheus metrics for tracking the number of currently-advertising servers in #1684. [Director] Gave the ability for the Director to persist origin/cache downtime labels between restarts in #1673. [Origin] Gave S3 origins the ability to specify sentinel files in #1791. [Plugin] Allowed the plugin to detect when it is named pelican_plugin in addition to pelican_xfer_plugin in #1733.

Bugs Fixed

[Client] Cleaned up display of progress bars for object gets in the client in #1814. [Client] Fixed bug where object ls against namespaces that do not support listings caused nil pointer panics in #1710. [Client] Added additional client checks to prevent creation of empty objects with directory uploads in #1707. [Director] Fixed over-collection of server redirects in the Director's Prometheus database in #1743. [Director] Fixed OSDF Director startup bug that caused erroneous warnings/errors mentioning the proj label in #1708.

New Contributors

@alexandertuna made their first contribution in #1730 @mwestphall made their first contribution in #1791 @georgezzeng made their first contribution in #1727

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.11.8...v7.12.0

Changelog

For Admins

• [Origin] -- Configured public origins to use XRootD directive sec.protbind to allow backwards compatibility with legacy OSDF caches.

Changelog

• 1ee7db3 Add missing xrootd key to osx_install test
• 10c963d Add virtual-style url test
• 860f979 Allow Pelican to use path-style URLs for S3 origins
• b22440e Empty commit to force sync
• d260a07 Pre Release: Update npm version
• ea65a38 Update S3 test to use path-style URL

What's new

For Admins

• [Cache]: Caches now fetch new namespace ads every time they advertise with the director. This is necessary for discovering new origins if they join the federation after the cache is served.

What's new

For Admins

• [Origin]: Origins configured to export S3 backends are no longer explicitly tied to a single bucket.

What's new

For Admins

• [Cache]: Fixed bug that prevented the cache's pss.origin directive from being set correctly with pelican://-style URLs.

What's new

For Admins

• [Cache]: Fixed bug that prevented debug output from XRootD processes.

What's new

For Admins

• [Registry]: Fixes bugs related to the registry's topology table that introduced race conditions and caused excess load on the topology endpoint.

What's new

For Users

• [Client]: Bug fixes for pelican object put with pelican:// urls

For Admins

• [Director]: The director can now display more federation information without requiring authentication

Changelog

• a47a13b Fixed issue with added new line in cache authfile
• 3cc2452 Pre Release: Update npm version

Updates

Congratulations @williamnswanson for your first contribution in #684!

Highlights

For Users

• [General] – Improved logging doesn't report as many false errors
• [Client] – Improved file unpacking to handle more use cases
• [Client] – The plugin now downloads files in parallel

For Admins

• [Origin] – Pelican origins now support public namespaces
• [General] – Federation admins can now fine tune whether origins and caches require admin approval before joining the federation
• [Origin/Cache] – XRootD log levels are now configurable
• [Director] – The director can now report whether a specific origin has a requested object via the new director stat feature
• [Origin] – Fixed rate graph data loss issues in the origin
• [Registry] – Added the ability to configure a URL for fetching the registry's institution list

Changelog

• Fix duplicated director file transfer tests
• Added three Prometheus metrics recording number of test file transfers the director issues:
  • pelican_director_total_ftx_test_suite: total number of test suite the director issued, which means how many times the director started a new test go routine to an origin
  • pelican_director_active_ftx_test_suite: the number of active test suite running per origin, which should be 1, meaning only one active test go routine is running per origin. This will be a good indicator of whether the director is doing wild things.
  • pelican_director_total_ftx_test_runs: the total number of test file transfer runs per origin. i.e. How many tests in total were ran per origin. We run A test per 15s (by default) per origin, so this is also a good indicator of whether the origin is up or whether the director is going crazy. It comes with label status and report_status with value Success and Failure indicating if the transfer itself was successful and the reporting to the origin was successful.

Changelog

• Fix the issue where Pelican won't exit main process if error group goroutine returns error, unless an error returns in the main goroutine
• Fix the panic at the director when a cache advertise itself to the director, as we didn't have the case for cache when doing version compatibility check
• Improve logging at program exit to be consistent.
• Improve cache error group, remove duplicated exit handler that root.go already had.

Highlights

For Users

• [Client] Introduced pelican object get and pelican object put to specify a get or put rather than just running pelican object copy. #259
• [Client] Fixed issue where the stash_plugin file-transfer plugin was not reporting proper upload bytes. #510
• [Client] Fixed issue with progress bars not outputting properly with debug logs for recursive uploads/downloads #402
• [Client] Fixed issue where stash_plugin was not working with environment variable-based configuration #508
• [Client] pelican object copy now doesn’t rely on topology for PUTs, it gets the endpoint by querying the director #504

For Admins

• [Director] Introduced GeoIP Overrides that allow a director to pin IP ranges to a particular lat/long. #496
• [Registry] Introduced web-ui for registry that allows users to register their namespace through web interface. #264
• [Director] Added federation discovery endpoint to director. #417
• [General] Added topology components to Prometheus health metrics. #494
• [Cache] Added Admin approval to caches before they can advertise to director #439
• [Registry] Added support to use CILogon to login to registry UI #355
• [Registry] Deprecated Federation.NamespaceUrl which will be removed in the next major release #388
• [General] Web-ui now outputs more detailed error messages if login fails #424
• [Origin] Basic S3 origin functionality has been restored #592
• [Cache] Fixed issue where caches were logging sensitive information #604
• [Origin] Boolean flag added to starting origins indicating if they are writable or not (default is true) #507

Changelog

• 4e93152 Address code review feedback * Update minio package version as the existing one was removed from archive * Remove redundant code * Preserve dev environment for dev container
• 8904cde Replace xrd-http plugin by xrdcl-pelican plugin in dev-container
• d1e3b82 Replace xrd-http plugin by xrdcl-pelican plugin in prod container

Changelog

• 85cf48c Remove chmod on user-provided xrootd config files

Changelog

• ef6d348 Truncate the plugin output file as soon as possible

Highlights

For Users

• Users can now recursively upload and download files. #260, #261
• Users can upload a local directory to a data federation as a tarball and can unpack a tarball into a directory as it downloads.
• For improved debugging, writing a debug file into a known location is now supported. #22
• Introduced Javascript client allowing browsers to perform file transfers. Installation of the Javascript client enables users via their browser to download a file from a data federation supported by Pelican. #344

For Admins

• [Origin] Added support for S3 file backend so that admins may expose or configure the S3 backend. #169
• [Cache] Added authentication to cache service. #311
• [Issuer] Introduced new OAuth2 issuer built into the origin packaging. #262
• [General] Introduced web pages for the director and registry components. #265, #354, #355
• New feature to allow the director to aggregate metrics from registered origins and to allow the director to periodically test origins #291 #175

What's Changed

• Get rid of superfluous symlink by @timtheisen in https://github.com/PelicanPlatform/pelican/pull/348
• Code cleanup in serveNamespaceRegistry by @turetske in https://github.com/PelicanPlatform/pelican/pull/341
• Made Pelican logging able to output to file by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/346
• Fix problems with config_test.go by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/363
• Revert "Code cleanup in serveNamespaceRegistry" by @turetske in https://github.com/PelicanPlatform/pelican/pull/364
• Namespace registry serve cleanup by @turetske in https://github.com/PelicanPlatform/pelican/pull/365
• Renamed getOriginJWK to getServerJWK by @turetske in https://github.com/PelicanPlatform/pelican/pull/339
• Use Server.Hostname than os.Hostname() in generating CA by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/356
• Standardize server init and cleanup a few other small items by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/362
• Fix issue with progress bar panic on download by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/369
• Always generate a CA if no host certificate present by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/375
• Add authorization to Prometheus /metrics endpoint by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/373
• Set correct log level for Prometheus logger by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/358
• Unit test for namespace-registry by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/337
• Introduce enum for health status and component names for SetComponentHealthStatus by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/376
• Fix Config Bug by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/381
• Refactor init_server_cred to be more clear on their usage by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/382
• Adjusted dev.Dockerfile to run make web-build properly by @turetske in https://github.com/PelicanPlatform/pelican/pull/386
• Git test action updates by @turetske in https://github.com/PelicanPlatform/pelican/pull/385
• Fix small bugs in DiscoverFederation() by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/400
• Fixed scitokens generation bug by @turetske in https://github.com/PelicanPlatform/pelican/pull/392
• Add swagger spec and include auth APIs and common APIs by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/401
• Replace ComputeExternalAddress by param Server.ExternalWebUrl and rename Server.Port and Server.Address by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/378
• End-to-end test for XRootD by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/328
• Add unit tests to XRootD monitoring handler by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/347
• Hook up monitoring paths functionality by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/407
• Refactor origin_ui to web_ui to reuse basic logic for server web UI/API by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/398
• If no auth token is present, do not emit verbose error message by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/404
• Director-based origin health tests by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/305
• Always unmarshall config when /config API is called by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/395
• Update integration test file by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/422
• Rename namespace-registry to conform to go standards and move registry db test by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/425
• Update integration test file - update stashcp source by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/426
• Ensure TTL caches evict expired item by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/416
• Federation Overview #221 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/411
• Reimplimented recursive downloads by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/377
• Fixed non running e2e test by @turetske in https://github.com/PelicanPlatform/pelican/pull/434
• Remove content length from upload to enable chunked by @djw8605 in https://github.com/PelicanPlatform/pelican/pull/208
• Add authorization policies for OA4MP by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/393
• Add supervisor script to make origins servable by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/428
• Fix bad error message from director by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/433
• Fix racing conditions in various director-related code and tests by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/441
• Periodically prepare a combined custom + system CA bundle by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/406
• Add cache supervisord process by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/443
• Add test dependencies to dev.Dockerfile for PR #419 by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/450
• Pelican cache with auth by @turetske in https://github.com/PelicanPlatform/pelican/pull/423
• Automatically unpack tarballs when downloaded by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/408
• Web UI Move by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/449
• Fix doc missing space for client-usage.mdx by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/432
• Update Web UI src code location by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/455
• Upgrade NextJS to ^14 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/454
• Refactor director SD tests and add test for duplicated origins by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/430
• Refactor generate package by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/442
• Provide ConfigureOriginAPI a cancel context that can be cancelled on shutdown by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/431
• Implement recursive uploads by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/415
• Add listServer API to director Web UI by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/458
• Refactor file transfer tests into a utility class by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/447
• Swagger docs by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/456
• Provide better message (with no error) when we register an already-registered namespace prefix by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/461
• Arm-compatible Pelican container and fix 404 minio RPM by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/469
• Adding dir listing capability to Pelican Origins by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/468
• Enable key chaining to protect super/sub namespaces by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/460
• Fix progress bar for recursive downloads by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/471
• Director UI by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/466
• Build containers but don't push on PR by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/467
• Data Exports Overview Table #220 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/410
• Lock NextJS to 14.0.3 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/475
• Fix progress bars w/ recursive uploads and downloads by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/473
• Add S3 backend to Pelican by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/419
• Refactor checkAPIToken to be more generic and less verbose by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/429
• Make the registry aware of topology by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/470
• Add draft of SECURITY.md by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/324
• Placeholder namespace UI 354 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/478

New Contributors

• @timtheisen made their first contribution in https://github.com/PelicanPlatform/pelican/pull/348
• @djw8605 made their first contribution in https://github.com/PelicanPlatform/pelican/pull/208

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.2.0...v7.3.0

Changelog

• 8515fb4 Add cache supervisord process
• a2938c0 Add default environment var to origin supervisord script
• 4173537 Add supervisor script to make origins servable
• 0c5b1a0 Fix process name

Changelog

• fc84142 Fix small bugs in DiscoverFederation()

What's Changed

• fixed typos within plugin_stage.go by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/186
• Testing common transport timeouts by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/195
• Add --version flag to pelican-client by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/199
• Ship OA4MP in the Pelican docker image by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/181
• Update goreleaser command in test CI and enable --single-target by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/218
• Add cache for Go modules and npm packages by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/215
• Strip "Bearer " from authz query and header if they're present by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/198
• Refactored xrootd specific code out of pelican/cmd by @turetske in https://github.com/PelicanPlatform/pelican/pull/216
• Start the documentation folder by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/244
• Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from 0.42.0 to 0.44.0 by @dependabot in https://github.com/PelicanPlatform/pelican/pull/248
• Moved transport config options to parameters.yaml by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/250
• Federation component health status by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/217
• Json generation by @turetske in https://github.com/PelicanPlatform/pelican/pull/266
• Cleanup of pelican config files by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/243
• Fixed mispelled filename in .gitignore file by @turetske in https://github.com/PelicanPlatform/pelican/pull/278
• Documentation for user installation of the Pelican Client by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/242
• Add first draft of client usage docs by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/249
• Expose director's public key via Open ID style URLs by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/254
• Replaced some remaining viper.Get* calls with the appropriate param.GetString() call by @turetske in https://github.com/PelicanPlatform/pelican/pull/269
• Origin Installation Documentation by @turetske in https://github.com/PelicanPlatform/pelican/pull/245
• Fix broken Director auth headers and add regression test by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/274
• Added the StagePlugin.* params to the param table by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/270
• Make default log level part of pelican config by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/236
• Improve test of stash_plugin to be more like HTCSS environment by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/252
• Embed OA4MP as an issuer by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/240
• Fix dev container build failure by specify base image platform by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/286
• Update Dockerfile by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/282
• Fix some issues with Pelican RPMs by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/279
• Move npm path placeholder creation from GHA to Go by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/273
• Add some documentation on developing the website by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/267
• Bump golang.org/x/net from 0.12.0 to 0.17.0 by @dependabot in https://github.com/PelicanPlatform/pelican/pull/292
• Auto-generate config struct from parameters.yaml and provide REST API by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/268
• Implement creation of a sharing URL by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/257
• Web UI improvements by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/253
• Fixed issues with Progress Bars by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/302
• Add Web URL in origin's advertisement message by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/288
• Upgrade status component by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/316
• Updated the docker images to include xrdcl-http by @turetske in https://github.com/PelicanPlatform/pelican/pull/319
• Auto-configure origin issuer access by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/299
• Bump postcss and next in /origin_ui/src by @dependabot in https://github.com/PelicanPlatform/pelican/pull/296
• Closes #275 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/317
• Fix bypassed authentication for /config API by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/321
• Add pelican-osdf-compat package by @matyasselmeci in https://github.com/PelicanPlatform/pelican/pull/327
• Fix null dereference bug in Director by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/326
• Graph Robustness #276 by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/318
• Director Prometheus scraper of origins' metrics by @haoming29 in https://github.com/PelicanPlatform/pelican/pull/289
• Api view by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/284
• Add go generate step to make file by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/334
• Pelican cache noauth by @turetske in https://github.com/PelicanPlatform/pelican/pull/306
• Automatically register origin with the namespace daemon by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/295
• Add Parameters page and order/name docs by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/336
• Make director service host-aware by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/313

New Contributors

• @dependabot made their first contribution in https://github.com/PelicanPlatform/pelican/pull/248
• @matyasselmeci made their first contribution in https://github.com/PelicanPlatform/pelican/pull/327

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.1.0...v7.2.0

This release modifies the creation of Pelican sub-RPMs to allow compatibility with osdf-client.

This patch fixes authentication headers in the director so that pelican object copy can determine issuer metadata for some namespaces dynamically.

What's Changed

• Allow Pelican to run in some settings where $HOME is not defined @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/238

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.1.1...v7.1.2

What's Changed

• Fix director authz by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/223

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.1.0...v7.1.1

What's Changed

• Makefile updates by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/128
• Added redirect_test.go by @turetske in https://github.com/PelicanPlatform/pelican/pull/126
• Add pre-commit hooks for the repo by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/93
• Bump linter timeout to 3m by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/138
• Switched over remaining usage of http.Client to use common transport by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/137
• Added code coverage to github actions workflow by @turetske in https://github.com/PelicanPlatform/pelican/pull/134
• Run pre-commit against entire repo by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/139
• Remove commenting code that causes errors when using a fork by @turetske in https://github.com/PelicanPlatform/pelican/pull/145
• Incorporate PR #78 from old osdf-client by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/144
• Fixed Generate Private Key Problems in Windows by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/152
• Revive codeql by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/151
• Updated test.yml to attach code coverage to the action summary by @turetske in https://github.com/PelicanPlatform/pelican/pull/156
• Updated mpb bar progress bars by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/132
• Add a variety of configurable parameters to docs by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/133
• Match ns prefix by longest match, not first match by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/142
• Remove errant stdout message by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/158
• First step toward self-testing: configure authorization by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/149
• Test Origin Login APIs by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/154
• Remove redundant viper call by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/162
• Fix federation auto-discovery bug by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/161
• Removed isDir from handle_http.go by @turetske in https://github.com/PelicanPlatform/pelican/pull/165
• Parameter generation by @turetske in https://github.com/PelicanPlatform/pelican/pull/164
• Expose CLI for creating tokens by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/157
• Added tests for the three functions in origin_api.go by @turetske in https://github.com/PelicanPlatform/pelican/pull/160
• Refactor the client to be in a sub-package by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/177
• Add development Dockerfile by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/163
• Pelican transport fixes and generated after configs by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/183
• Origin self-test by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/167
• Query director for available namespaces by @Techming in https://github.com/PelicanPlatform/pelican/pull/185
• Put prometheus metrics behind federation/origin token by @turetske in https://github.com/PelicanPlatform/pelican/pull/176
• Put the token acquisition workflow behind a plugin check by @turetske in https://github.com/PelicanPlatform/pelican/pull/180
• Give the Director the ability to detect version mismatches by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/184
• Overhaul config names by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/182
• Fix template rendering failure by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/191
• Added check of cookie login for prometheus by @turetske in https://github.com/PelicanPlatform/pelican/pull/193
• Create Login Page for Post Initialization Login by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/196
• Port OSDF-Client PR #87 to Pelican by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/197
• Create new graph components by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/189
• Add Status Box Component by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/192

New Contributors

• @Techming made their first contribution in https://github.com/PelicanPlatform/pelican/pull/185

Full Changelog: https://github.com/PelicanPlatform/pelican/compare/v7.0.0...v7.1.0

The first release of the new Pelican codebase!

Pelican currently includes the functionality of the prior osdf-client / stashcp; it emulates the CLI of those clients if the binary is renamed to stashcp or stash_plugin.

Two new components have been developed, the director and the namespace registry. The director is responsible for routing clients to an appropriate cache or origin to serve up their request (and has similar functionality to the redirector for the OSDF). The namespace registry adds an authorization layer to the data federation, associating a prefix with the public key of an origin.

This includes experimental support for launching an origin via the Pelican command line. Over the next few monthly releases, this support will mature to the point of replacing currently running origin containers in the OSDF.

Release Notes:

• Merge OSDF Client by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/2
• Cleanup linter failures in origin_ui by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/11
• Finish implementation of the init login flow by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/13
• Simple landing page for Origin UI by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/15
• Director cli hookup by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/12
• Add Director capability into the Pelican CLI by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/7
• Have UI always use TLS by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/16
• Various Director cleanups by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/19
• Do not create key with group read permissions by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/26
• Dockerize by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/29
• Allow passing key directly via CL by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/28
• Add Namespace Management CLI Commands by @CohenQU in https://github.com/PelicanPlatform/pelican/pull/14
• Have origin advertise to the director service by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/18
• Dockerize by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/31
• Fix director tls by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/33
• Turn off autostart for director daemon by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/35
• Add cl flag to make default endpoint serve caches or origins by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/36
• Add xrootd, xrootd-client, xrootd-server to container by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/37
• Move groupadd and useradd before xrootd installation by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/38
• Namespace client cleanup by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/30
• Start adding documentation for existing parameters. by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/20
• Fix small bug in director's cache sorting by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/40
• Fix bad error message (bad content length) by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/10
• Additional user interfaces and APIs by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/17
• Embed of Prometheus monitoring by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/27
• Add support for running as root and 'multiuser' mode by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/51
• Draft integration of namespace registry app in Pelican by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/39
• Bump go version in the publish-container action by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/57
• Fix YAML error in go version for GHA by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/58
• Improve the use of the X-Pelican-* responses by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/48
• Reformat code base with go fmt by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/59
• Enable go fmt as one of the linters by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/49
• Clean UI by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/43
• Create (and explain) the need of a placeholder for golintci by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/60
• Create a single --port flag object by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/65
• Upgrade remainder of the code to lestrrat-go v2 by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/66
• Remove CVMFS shortcut within Pelican client by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/78
• Fix small bug in ns-registry auto creation by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/75
• Add copyright notices to Pelican source by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/80
• Use lestrrat jwks instead of custom structs in NS Registry by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/76
• Add deletion authorization by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/89
• Fix director responses by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/92
• Remove default user by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/95
• Add registry to container by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/102
• Create a privkey for registry by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/104
• Add simple verification to namespace registration by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/94
• UI updates by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/105
• Include token in redirect by @bbockelm in https://github.com/PelicanPlatform/pelican/pull/79
• Copy all supervisord confs into image by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/107
• Fix naming of oidc config vars by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/108
• Add default port to supervisord daemons by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/109
• Remove redundant env prefix by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/110
• Using common transport for https based writes by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/103
• UI changes by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/106
• Fix bug with common transport TLSSkipVerify by @joereuss12 in https://github.com/PelicanPlatform/pelican/pull/115
• Update Releaser by @CannonLock in https://github.com/PelicanPlatform/pelican/pull/116
• Send token and namespace ad to director by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/121
• Remove hardcoded originUrl by @jhiemstrawisc in https://github.com/PelicanPlatform/pelican/pull/124

New Contributors

• @bbockelm made their first contribution in https://github.com/PelicanPlatform/pelican/pull/2
• @jhiemstrawisc made their first contribution in https://github.com/PelicanPlatform/pelican/pull/12
• @CohenQU made their first contribution in https://github.com/PelicanPlatform/pelican/pull/14
• @joereuss12 made their first contribution in https://github.com/PelicanPlatform/pelican/pull/10
• @CannonLock made their first contribution in https://github.com/PelicanPlatform/pelican/pull/43

Full Changelog: https://github.com/PelicanPlatform/pelican/commits/v7.0.0