github.com/avfs/avfs@v0.33.1-0.20240303173310-c6ba67c33eb7/readme.md (about) 1 # AVFS 2 3 Another Virtual File System for Go 4 5  6 [](https://codecov.io/gh/avfs/avfs) 7 [](https://pkg.go.dev/github.com/avfs/avfs) 8 [](https://github.com/avfs/avfs/releases/latest) 9 [](/LICENSE) 10 [](https://magefile.org) 11 12 ## Overview 13 14 **AVFS** is a virtual file system abstraction, inspired mostly 15 by [Afero](http://github.com/spf13/afero) and Go standard library. It provides 16 an abstraction layer to emulate the behavior of a file system that provides several features : 17 - a set of **constants**, **interfaces** and **types** for all file systems 18 - a **test suite** for all file systems (emulated or real) 19 - each file system has its **own package** 20 - a very basic **identity manager** allows testing of user related functions 21 (Chown, Lchown) and file system permissions 22 23 Additionally, some file systems support : 24 - user file creation mode mask (**Umask**) (MemFS, OrefaFS) 25 - **chroot** (OSFS on Linux) 26 - **hard links** (MemFS, OrefaFS) 27 - **symbolic links** (MemFS) 28 - **multiple users concurrently** (MemFS) 29 - **Linux** and **Windows** emulation regardless of host operating system (MemFS, OrefaFS) 30 31 ## Installation 32 33 This package can be installed with the go install command : 34 35 ``` 36 go install github.com/avfs/avfs@latest 37 ``` 38 39 It is only tested with Go version >= 1.18 40 41 ## Getting started 42 43 To make an existing code work with **AVFS** : 44 45 - replace all references of `os`, `path/filepath` 46 with the variable used to initialize the file system (`vfs` in the following 47 examples) 48 - import the packages of the file systems and, if necessary, the `avfs` package 49 and initialize the file system variable. 50 - some file systems provide specific options available at initialization. For 51 instance `MemFS` needs `WithSystemDirs` option to create `/home`, `/root` 52 and `/tmp` directories. 53 54 ## Examples 55 56 ### Symbolic links 57 58 The example below demonstrates the creation of a file, a symbolic link to this 59 file, for a different file systems (depending on an environment variable). Error 60 management has been omitted for the sake of simplicity : 61 62 ```go 63 package main 64 65 import ( 66 "bytes" 67 "log" 68 "os" 69 70 "github.com/avfs/avfs" 71 "github.com/avfs/avfs/vfs/memfs" 72 "github.com/avfs/avfs/vfs/osfs" 73 ) 74 75 func main() { 76 var vfs avfs.VFS 77 78 switch os.Getenv("ENV") { 79 case "PROD": // The real file system for production. 80 vfs = osfs.NewWithNoIdm() 81 default: // in memory for tests. 82 vfs = memfs.New() 83 } 84 85 // From this point all references of 'os', 'path/filepath' 86 // should be replaced by 'vfs' 87 rootDir, _ := vfs.MkdirTemp("", "avfs") 88 defer vfs.RemoveAll(rootDir) 89 90 aFilePath := vfs.Join(rootDir, "aFile.txt") 91 92 content := []byte("randomContent") 93 _ = vfs.WriteFile(aFilePath, content, 0o644) 94 95 aFilePathSl := vfs.Join(rootDir, "aFileSymlink.txt") 96 _ = vfs.Symlink(aFilePath, aFilePathSl) 97 98 gotContentSl, _ := vfs.ReadFile(aFilePathSl) 99 if !bytes.Equal(content, gotContentSl) { 100 log.Fatalf("Symlink %s : want content to be %v, got %v", 101 aFilePathSl, content, gotContentSl) 102 } 103 104 log.Printf("content from symbolic link %s : %s", aFilePathSl, gotContentSl) 105 } 106 ``` 107 108 ### Multiple users creating simultaneously directories 109 110 The example below demonstrates the concurrent creation of subdirectories under a 111 root directory by several users in different goroutines (works only with 112 MemFS) : 113 114 ```go 115 package main 116 117 import ( 118 "fmt" 119 "log" 120 "sync" 121 122 "github.com/avfs/avfs" 123 "github.com/avfs/avfs/idm/memidm" 124 "github.com/avfs/avfs/vfs/memfs" 125 ) 126 127 func main() { 128 const ( 129 maxUsers = 100 130 groupName = "test_users" 131 ) 132 133 idm := memidm.New() 134 vfs := memfs.New() 135 136 rootDir, _ := vfs.MkdirTemp("", "avfs") 137 vfs.Chmod(rootDir, 0o777) 138 139 g, _ := idm.GroupAdd(groupName) 140 141 var wg sync.WaitGroup 142 wg.Add(maxUsers) 143 144 for i := 0; i < maxUsers; i++ { 145 go func(i int) { 146 defer wg.Done() 147 148 userName := fmt.Sprintf("user_%08d", i) 149 idm.UserAdd(userName, g.Name()) 150 151 vfsU, _ := vfs.Sub("/") 152 vfsU.SetUser(userName) 153 154 path := vfsU.Join(rootDir, userName) 155 vfsU.Mkdir(path, avfs.DefaultDirPerm) 156 }(i) 157 } 158 159 wg.Wait() 160 161 entries, _ := vfs.ReadDir(rootDir) 162 163 log.Println("number of dirs :", len(entries)) 164 for _, entry := range entries { 165 info, _ := entry.Info() 166 sst := vfs.ToSysStat(info) 167 168 u, _ := idm.LookupUserId(sst.Uid()) 169 170 log.Println("dir :", info.Name(), 171 ", mode :", info.Mode(), 172 ", owner :", u.Name()) 173 } 174 } 175 ``` 176 177 ## Status 178 179 Almost ready for Windows. 180 181 ## File systems 182 183 All file systems implement at least `avfs.FS` and `avfs.File` interfaces. 184 By default, each file system supported methods are the most commonly used from 185 packages `os` and `path/filepath`. All methods have identical names as their 186 functions counterparts. 187 The following file systems are currently available : 188 189 File system |Comments 190 ------------|-------- 191 [BasePathFS](vfs/basepathfs)|file system that restricts all operations to a given path within a file system 192 [MemFS](vfs/memfs)|In memory file system supporting major features of a linux file system (hard links, symbolic links, chroot, umask) 193 [OrefaFS](vfs/orefafs)|Afero like in memory file system 194 [OsFS](vfs/osfs)|Operating system native file system 195 [RoFS](vfs/rofs)|Read only file system 196 197 ## Supported methods 198 199 File system methods <br> `avfs.VFS`|Comments 200 -----------------------------------|-------- 201 `Abs`|equivalent to `filepath.Abs` 202 `Base`|equivalent to `filepath.Base` 203 `Chdir`|equivalent to `os.Chdir` 204 `Chmod`|equivalent to `os.Chmod` 205 `Chown`|equivalent to `os.Chown` 206 `Chtimes`|equivalent to `os.Chtimes` 207 `Clean`|equivalent to `filepath.Clean` 208 `Create`|equivalent to `os.Create` 209 `CreateTemp`|equivalent to `os.CreateTemp` 210 `Dir`|equivalent to `filepath.Dir` 211 `EvalSymlinks`|equivalent to `filepath.EvalSymlinks` 212 `FromSlash`|equivalent to `filepath.FromSlash` 213 `Features`|returns the set of features provided by the file system or identity manager 214 `Getwd`|equivalent to `os.Getwd` 215 `Glob`|equivalent to `filepath.Glob` 216 `HasFeature`|returns true if the file system or identity manager provides a given feature 217 `Idm`|returns the identity manager of the file system 218 `IsAbs`|equivalent to `filepath.IsAbs` 219 `IsPathSeparator`|equivalent to `filepath.IsPathSeparator` 220 `Join`|equivalent to `filepath.Join` 221 `Lchown`|equivalent to `os.Lchown` 222 `Link`|equivalent to `os.Link` 223 `Lstat`|equivalent to `os.Lstat` 224 `Match`|equivalent to `filepath.Match` 225 `Mkdir`|equivalent to `os.Mkdir` 226 `MkdirAll`|equivalent to `os.MkdirAll` 227 `MkdirTemp`|equivalent to `os.MkdirTemp` 228 `Open`|equivalent to `os.Open` 229 `OpenFile`|equivalent to `os.OpenFile` 230 `OSType`|returns the operating system type of the file system 231 `PathSeparator`|equivalent to `os.PathSeparator` 232 `ReadDir`|equivalent to `os.ReadDir` 233 `ReadFile`|equivalent to `os.ReadFile` 234 `Readlink`|equivalent to `os.Readlink` 235 `Rel`|equivalent to `filepath.Rel` 236 `Remove`|equivalent to `os.Remove` 237 `RemoveAll`|equivalent to `os.RemoveAll` 238 `Rename`|equivalent to `os.Rename` 239 `SameFile`|equivalent to `os.SameFile` 240 `SetUMask`|sets the file mode creation mask 241 `SetUser`|sets and returns the current user 242 `Split`|equivalent to `filepath.Split` 243 `Stat`|equivalent to `os.Stat` 244 `Sub`|equivalent to `fs.Sub` 245 `Symlink`|equivalent to `os.Symlink` 246 `TempDir`|equivalent to `os.TempDir` 247 `ToSlash`|equivalent to `filepath.ToSlash` 248 `ToSysStat`|takes a value from fs.FileInfo.Sys() and returns a value that implements interface avfs.SysStater 249 `Truncate`|equivalent to `os.Truncate` 250 `UMask`|returns the file mode creation mask 251 `User`|returns the current user 252 `Utils`|returns the file utils of the current file system 253 `WalkDir`|equivalent to `filepath.WalkDir` 254 `WriteFile`|equivalent to `os.WriteFile` 255 256 File methods <br> `avfs.File`|Comments 257 -----------------------------|-------- 258 `Chdir`|equivalent to `os.File.Chdir` 259 `Chmod`|equivalent to `os.File.Chmod` 260 `Chown`|equivalent to `os.File.Chown` 261 `Close`|equivalent to `os.File.Close` 262 `Fd`|equivalent to `os.File.Fd` 263 `Name`|equivalent to `os.File.Name` 264 `Read`|equivalent to `os.File.Read` 265 `ReadAt`|equivalent to `os.File.ReadAt` 266 `ReadDir`|equivalent to `os.File.ReadDir` 267 `Readdirnames`|equivalent to `os.File.Readdirnames` 268 `Seek`|equivalent to `os.File.Seek` 269 `Stat`|equivalent to `os.File.Stat` 270 `Truncate`|equivalent to `os.File.Truncate` 271 `Write`|equivalent to `os.File.Write` 272 `WriteAt`|equivalent to `os.File.WriteAt` 273 `WriteString`|equivalent to `os.File.WriteString` 274 275 ## Identity Managers 276 277 Identity managers allow users and groups management. The ones implemented 278 in `avfs` are just here to allow testing of functions related to users (Chown, 279 Lchown) and access rights, so they just allow one default group per user. 280 281 All file systems supporting identity manager implement by default the identity 282 manager `DummyIdm` 283 where all functions returns `avfs.ErrPermDenied`. 284 285 Identity Manager |Comments 286 -----------------|-------- 287 [DummyIdm](dummyidm.go)|dummy identity manager where all functions are not implemented 288 [MemIdm](idm/memidm)|In memory identity manager 289 [OsIdm](idm/osidm)|Identity manager using os functions 290 [SQLiteIdm](https://github.com/avfs/sqliteidm)|Identity manager backed by a SQLite database 291 292 Identity Manager methods <br>`avfs.FS` <br> `avfs.IdentityMgr`|Comments 293 --------------------------------------------------------------|-------- 294 `AdminGroup`|returns the administrator group (root for Linux) 295 `AdminUser`|returns the administrator user (root for Linux) 296 `GroupAdd`| adds a new group 297 `GroupDel`| deletes an existing group 298 `LookupGroup`| looks up a group by name 299 `LookupGroupId`| looks up a group by groupid 300 `LookupUser`| looks up a user by username 301 `LookupUserId`| looks up a user by userid 302 `UserAdd`| adds a new user 303 `UserDel`| deletes an existing user