github.com/pelicanplatform/pelican@v1.0.5/docs/pages/index.mdx (about)

     1  import ImageRow from "@/components/ImageRow";
     2  
     3  # What Is the Pelican Platform?
     4  
     5  Pelican provides an open-source software platform for federating dataset repositories together and delivering the
     6  objects to computing capacity such as the [OSPool](https://osg-htc.org/services/open_science_pool.html).
     7  
     8  **Pelican Enables**:
     9  - Researchers to access their datasets at scales from a notebook to a campus cluster to the national computing fabric
    10  - Repositories and storage providers to export datasets in a scalable manner and helps implement FAIR principles
    11  - Compute providers to cache datasets on-site
    12  - Cyberinfrastructures to build gateways and portals to large-scale datasets
    13  
    14  Objects in a federation are accessible through a common namespace; given an object name,
    15  the Pelican client can discover the object’s location and download it through the access layer.
    16  The access layer consists of distributed caches which reduce the load on the origin for repeated accesses.
    17  
    18  <ImageRow alt={"Pelican and OSDF"} src={"/pelican/pelican-and-osdf.png"}>
    19  	A Pelican data federation provides an access layer that helps the origin
    20  	distribute datasets in the repositories.  A client wanting an object contacts
    21  	the manager to find the closest cache which either serves the objects from
    22  	local storage or streams it through the origin.
    23  </ImageRow>
    24  
    25  
    26  The flagship Pelican federation is the Open Science Data Federation (OSDF).
    27  The OSDF has approximately two dozen caches located throughout the world, often at
    28  points of presence within the global Research and Education networks such as ESNet and
    29  Internet2.
    30  
    31  <ImageRow alt={"Pelican and OSDF"} src={"/pelican/pelican-bus.png"}>
    32  	The OSDF serves as a transport bus, connecting a variety of backend storage types
    33  </ImageRow>
    34  
    35  
    36  
    37  
    38  Central to Pelican is the concept of the origin service.  The origin is the intermediary between
    39  the existing storage and the federation.  The origin is responsible for serving data as well
    40  as issuing tokens (credentials) authorizing access to datasets based on the local policy.