github.com/aarzilli/tools@v0.0.0-20151123112009-0d27094f75e0/os/fsi/readme.md (about) 1 Overview 2 -------------------- 3 This package serves as universal abstraction 4 layer for all go programs that require a file system. 5 6 The interface, http-fs and the memory-fs were taken 7 from [Steve Francia](https://twitter.com/spf13)'s afero: 8 Afero can be found [here](https:github.com/spf13/afero). 9 10 I had these purposes in mind: 11 12 - Deploying content management systems on app engine.<br> 13 Combining any apps with static pages and static HTML UI wrappers. 14 15 - Any HTTP services on app engine, managing static resources. 16 17 18 A similar effort was made by [rainycape](https://github.com/rainycape/vfs). 19 20 Deployments can be made to local machines. 21 Files are editable with all editors. 22 23 ### Package fsi - filesystem interface 24 Contains the minimal 25 requirements for exchangeable filesystems. 26 27 The required interface did not remain ultra slim, 28 since apps need accessing all functions *via interface*. 29 They cannot access conditional methods easily. 30 31 All Fileystems provide a pck.Unrwap(fsi)(fsImpl,bool) to get access to 32 the underlying fileystem. But we dont want to unwrap, only to access fsImpl.Chmod(...). 33 We need fsiX.Chmod() directly. 34 35 36 #### Subpackage common 37 Holds common extensions to all filesystems. 38 It contains an implementation of filepath.walk, 39 that can be used for all contained filesystems. 40 41 dsfs.RemoveAll is built on this walk. 42 43 Contains standardization logic for paths. 44 45 46 #### Subpackage tests 47 Contains tests for all filesystems. 48 49 File level tests are taken from [afero](https:github.com/spf13/afero). 50 51 I added a directory-tree test suite, with lots of subdirectories and files, and file-retrieval and removal. 52 53 I also added a httpfs-wrapper test (incomplete). 54 55 Tests for path standardization logic. 56 57 #### osfs 58 Osfs is the wrapped operating system. Replace 59 60 os.Open() and ioutil.WriteFile() 61 by 62 63 [filesys-instance].Open() and [filesys-instance].WriteFile() 64 65 Then you can switch between hard disk and memory filesystem 66 by changing the instantiation. 67 68 osfs expects unix style paths, even under windows. 69 Give it c:/dir/file. 70 71 72 #### memfs 73 Keeps directories and files completely in RAM. 74 Good for quick testing and fast cleanup. 75 76 You can change the MountPoint of a memfs after its creation, 77 thus storing multiple trees consecutively, but not concurrently. 78 79 80 #### stacking filesystems 81 You can add a "shadow" filesystem to memfs. 82 Memfs will now act as cache for the shadow filesystem. 83 84 Its underimplemented though. Only Open() is looking. 85 86 87 #### dsfs 88 With dsfs you can write on google's datastore like onto a local hard disk. 89 See doc.go for details. 90 91 92 #### s3fs 93 Not yet adapted: A filesystem layer for amazon s3 or ceph 94 95 96 #### httpfs 97 httpfs can wrap any previous filesystem and make it serveable by a go http fileserver. 98 99 ## Improvements 100 101 - memfs was substantially recoded. 102 103 - All filesystems are initialized with variadic option functions, as Dave Cheney [suggested](http://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis). 104 105 - (Deterioration) appengine/aetest is now needed for all tests. 106 107 ## Todos 108 109 #### Locking 110 111 - The locking approach of memfs is incomplete.<br> 112 memfs/0_init.go has details and solutions.<br> 113 It is also impossible to improve, since you constantly run into deadlocks, when calling Open, Close, Create ... since they often nest. 114 115 - memfs registerWithParent locking seems neglected. 116 117 - I probably sync all directory tree structures with a go-routine-for-select. 118 119 - dsfs needs a locking consideration for RemoveAll and Rename. 120 Behold the asynchroneus nature of dsfs directories. 121 122 #### Tests 123 124 - A concurrent access test suite is missing. 125 126 127 Common Remarks 128 -------------------- 129 All filesystems need to maintain compatibility 130 to relative paths of osfs; that is to current working directory prefixing. 131 Therefore all filesystems must support "." for current working dir. 132 Currently - in memfs and dsfs - working dir always refers to fs.RootDir(). 133 134 memfs and dsfs interpret "", "/" and "." as starting with root. 135 136 To access files directly under root, memfs and dsfs one must use ./filename 137 138 The filesystem types are no longer exported. 139 To access implementation specific functionality, use 140 141 subpck.Unwrap(fsi.FileSystem) SpecificFileSys 142 143 144 Terminology 145 -------------------- 146 "name" or "filename" can mean either the basename or the full path of the file, 147 depending on the actual argument: 148 149 'app1.log' # simply 150 '/tmp/logs/app1.log' # full 151 152 In the first case, it refers to 153 154 [current dir]/app1.log. 155 156 Which is for memfs and dsfs 157 158 [root dir]/app1.log. 159 160 Exception: 161 162 os.FileInfo.Name() # always contains *only* the base name. 163 164 165 Compare [discussion on stackoverflow](http:stackoverflow.com/questions/2235173/file-name-path-name-base-name-naming-standard-for-pieces-of-a-path) 166 167 168 Fat architectural distinction to Afero 169 -------------------- 170 fileinfo.Name() should return the filename without path. 171 172 I think this keeps the file independent from its current location in a directory tree. 173 174 It also means, the file has no method to access its current folder. 175 We have to know its location, in order to open it. 176 177 This is in striking contrast to the Afero implementation, where fileinfo.Name() returns the *full* path. 178 But the documentation for os.FileInfo.Name() is on my side. 179 Even Microsoft C# and C++ implementation [agree](https://msdn.microsoft.com/de-de/library/system.io.fileinfo.name(v=vs.110).aspx) and provide a separate FullName method.