gitlab.com/SkynetLabs/skyd@v1.6.9/skymodules/renter/filesystem/README.md (about)

     1  # Filesystem
     2  The Filesystem is responsible for ensuring that all of its supported file
     3  formats can be accessed in a threadsafe manner. It doesn't handle any
     4  persistence directly but instead relies on the underlying format's package to
     5  handle that itself. This also means that methods like `Delete` and `Rename`
     6  should not be called on the underlying types directly but be called through
     7  the Filesystem's corresponding wrappers instead.
     8  
     9  To refer to a file or folder within the Filesystem, so-called SiaPaths are
    10  used. They are unix-like paths relative to the specified root of the
    11  Filesystem except for the fact that they don't start with a leading slash.
    12  Ideally all parts of the siad codebase only have to interact with SiaPaths
    13  instead of system paths and the Filesystem would handle all of the
    14  translations between SiaPaths and regular system paths. The Filesystem also
    15  enforces that files and folders can't share the same name.
    16  
    17  The Filesystem is an in-memory tree-like data structure of nodes. The nodes
    18  can either be files or directories. Directory nodes potentially point to
    19  other directory nodes or file nodes while file nodes can't have any children.
    20  When opening a file or folder, the Filesystem is traversed starting from the
    21  root until the required file or folder is found, opening all the nodes on
    22  demand and adding them to the tree. The tree will be pruned again as nodes
    23  are no longer needed. To do so a special locking convention was introduced
    24  which will be explained in its own paragraph.
    25  
    26  ## Locking Conventions
    27  Due to the nature of the Filesystem custom locking conventions were required
    28  to optimize the performance of the data structure.
    29  
    30  - A locked child node can't grab a parent's lock
    31  - A locked parent can grab its children's locks
    32  - A parent needs to be locked before adding/removing children
    33  - A parent doesn't need to be locked before modifying any children
    34  
    35  Locking like this avoids a lot of lock contention and enables us to easily
    36  and efficiently delete and rename folders.
    37  
    38  ## Submodules
    39  The Filesystem has several submodules that each perform a specific function
    40  for the Renter. This README will provide brief overviews of the submodules,
    41  but for more detailed descriptions of the inner workings of the submodules
    42  the respective README files should be reviewed.
    43   - SiaDir
    44   - SiaFile
    45  
    46  ### SiaDir
    47  The SiaDir module is the code that defines what a directory is on the Sia
    48  network. It also manages accesses and updates to the file, ensuring safety and
    49  ACIDity when performing file operations.
    50  
    51  ### SiaFile
    52  The SiaFile module is the code that defines what a file is on the Sia network.
    53  It also manages accesses and updates to the file, ensuring safety and ACIDity
    54  when performing file operations.
    55  
    56  
    57  # Subsystems
    58  The Filesystem has the following subsystems.
    59  - [Filesystem](#filesystem)
    60  - [DirNode](#file-node)
    61  - [FileNode](#dir-node)
    62  
    63  ### Filesystem
    64  **Key Files**
    65  - [filesystem.go](./filesystem.go)
    66  
    67  The Filesystem subsystem contains Filesystem specific errors, the definition
    68  for the common `node` type as well as all the exported methods which are
    69  called by other skymodules.
    70  
    71  It is a special `DirNode` which acts as the root of the Filesystem, is
    72  created upon startup and is always kept in memory instead of being pruned
    73  like the other nodes. The job of the FileSystem is to translate SiaPaths into
    74  regular paths and pass calls to its exported methods on to the correct child
    75  if possible. It also implements some high level methods which might require
    76  interacting with multiple nodes like `RenameDir` for example.
    77  
    78  ### DirNode
    79  **Key Files**
    80  - [dirnode.go](./dirnode.go)
    81  
    82  The DirNode extends the common `node` by fields relevant to a directory.
    83  Namely children and an embedded `SiaDir`. The special thing about the
    84  embedded `SiaDir` is that it is loaded lazily. That means it will only be
    85  loaded from disk once a method is called that requires it. The `SiaDir` will
    86  also be removed from memory again once the DirNode is no longer being
    87  accessed. That way we avoid caching the `SiaDir` for too long.
    88  
    89  ### FileNode
    90  **Key Files**
    91  - [filenode.go](./filenode.go)
    92  
    93  The FileNode is similar to the DirNode but it only extends the `node` by a
    94  single embedded `Siafile` field. Apart from that it contains wrappers for the
    95  `SiaFile` methods which correctly modify the parent directory when the
    96  underlying file is moved or deleted.