github.com/aarzilli/tools@v0.0.0-20151123112009-0d27094f75e0/os/fsi/readme.md (about)

     1  Overview
     2  --------------------
     3  This package serves as universal abstraction
     4  layer for all go programs that require a file system.
     5  
     6  The interface, http-fs and the memory-fs were taken 
     7  from [Steve Francia](https://twitter.com/spf13)'s afero:
     8  Afero can be found [here](https:github.com/spf13/afero).
     9  
    10  I had these purposes in mind:
    11  
    12  - Deploying content management systems on app engine.<br>
    13  Combining any apps with static pages and static HTML UI wrappers.
    14  
    15  - Any HTTP services on app engine, managing static resources.
    16  
    17  
    18  A similar effort was made by [rainycape](https://github.com/rainycape/vfs).
    19  
    20  Deployments can be made to local machines.
    21  Files are editable with all editors.
    22  
    23  ### Package fsi - filesystem interface
    24  Contains the minimal
    25  requirements for exchangeable filesystems.
    26  
    27  The required interface did not remain ultra slim,
    28  since apps need accessing all functions *via interface*.
    29  They cannot access conditional methods easily.
    30  
    31  All Fileystems provide a pck.Unrwap(fsi)(fsImpl,bool) to get access to
    32  the underlying fileystem. But we dont want to unwrap, only to access fsImpl.Chmod(...).
    33  We need fsiX.Chmod() directly.
    34  
    35  
    36  #### Subpackage common
    37  Holds common extensions to all filesystems.
    38  It contains an implementation of filepath.walk,
    39  that can be used for all contained filesystems.
    40  
    41  dsfs.RemoveAll is built on this walk.
    42  
    43  Contains standardization logic for paths.
    44  
    45  
    46  #### Subpackage tests 
    47  Contains tests for all filesystems.
    48  
    49  File level tests are taken from [afero](https:github.com/spf13/afero).
    50  
    51  I added a directory-tree test suite, with lots of subdirectories and files, and file-retrieval and removal.
    52  
    53  I also added a httpfs-wrapper test (incomplete).
    54  
    55  Tests for path standardization logic.
    56  
    57  #### osfs 
    58  Osfs is the wrapped operating system. Replace 
    59  	
    60  	os.Open() and ioutil.WriteFile()
    61  by 
    62  	
    63  	[filesys-instance].Open() and [filesys-instance].WriteFile()
    64  
    65  Then you can switch between hard disk and memory filesystem
    66  by changing the instantiation.
    67  
    68  osfs expects unix style paths, even under windows.
    69  Give it c:/dir/file.
    70  
    71  
    72  #### memfs
    73  Keeps directories and files completely in RAM.
    74  Good for quick testing and fast cleanup.
    75  
    76  You can change the MountPoint of a memfs after its creation,
    77  thus storing multiple trees consecutively, but not concurrently.
    78  
    79  
    80  #### stacking filesystems
    81  You can add a "shadow" filesystem to memfs.
    82  Memfs will now act as cache for the shadow filesystem.
    83  
    84  Its underimplemented though. Only Open() is looking.
    85  
    86  
    87  #### dsfs
    88  With dsfs you can write on google's datastore like onto a local hard disk.
    89  See doc.go for details.
    90  
    91  
    92  #### s3fs
    93  Not yet adapted: A filesystem layer for amazon s3 or ceph
    94  
    95  
    96  #### httpfs
    97  httpfs can wrap any previous filesystem and make it serveable by a go http fileserver.
    98  
    99  ## Improvements
   100  
   101  - memfs was substantially recoded.
   102  
   103  - All filesystems are initialized with variadic option functions, as Dave Cheney [suggested](http://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis).
   104  
   105  - (Deterioration) appengine/aetest is now needed for all tests.
   106  
   107  ## Todos
   108  
   109  #### Locking
   110  
   111  - The locking approach of memfs is incomplete.<br>
   112  memfs/0_init.go has details and solutions.<br>
   113  It is also impossible to improve, since you constantly run into deadlocks, when calling Open, Close, Create ... since they often nest.
   114  
   115  - memfs registerWithParent locking seems neglected.
   116  
   117  - I probably sync all directory tree structures with a go-routine-for-select.
   118  
   119  - dsfs needs a locking consideration for RemoveAll and Rename. 
   120  Behold the asynchroneus nature of dsfs directories.
   121  
   122  #### Tests
   123  
   124  - A concurrent access test suite is missing.
   125  
   126  
   127  Common Remarks
   128  --------------------
   129  All filesystems need to maintain compatibility
   130  to relative paths of osfs; that is to current working directory prefixing.
   131  Therefore all filesystems must support "." for current working dir.
   132  Currently - in memfs and dsfs - working dir always refers to fs.RootDir().
   133  
   134  memfs and dsfs interpret "", "/" and "." as starting with root.
   135  
   136  To access files directly under root, memfs and dsfs one must use ./filename
   137  
   138  The filesystem types are no longer exported.
   139  To access implementation specific functionality, use
   140  
   141  	subpck.Unwrap(fsi.FileSystem) SpecificFileSys
   142  
   143  
   144  Terminology
   145  --------------------
   146  "name" or "filename" can mean either the basename or the full path of the file,
   147  depending on the actual argument:
   148  
   149  	'app1.log'              # simply
   150  	'/tmp/logs/app1.log'    # full
   151  
   152  In the first case, it refers to 
   153  
   154  	[current dir]/app1.log.
   155  
   156  Which is for memfs and dsfs        
   157  
   158  	[root dir]/app1.log.
   159  
   160  Exception: 
   161  
   162  	os.FileInfo.Name() # always contains *only* the base name.
   163  
   164  
   165  Compare [discussion on stackoverflow](http:stackoverflow.com/questions/2235173/file-name-path-name-base-name-naming-standard-for-pieces-of-a-path)
   166  
   167  
   168  Fat architectural distinction to Afero
   169  --------------------
   170  fileinfo.Name() should return the filename without path.
   171  
   172  I think this keeps the file independent from its current location in a directory tree.
   173  
   174  It also means, the file has no method to access its current folder.
   175  We have to know its location, in order to open it.
   176  
   177  This is in striking contrast to the Afero implementation, where fileinfo.Name() returns the *full* path.
   178  But the documentation for os.FileInfo.Name() is on my side.
   179  Even Microsoft C# and C++ implementation [agree](https://msdn.microsoft.com/de-de/library/system.io.fileinfo.name(v=vs.110).aspx) and provide a separate FullName method.