github.com/AndrewDeryabin/doublestar/v4@v4.0.0-20230123132908-d9476b7d41be/README.md (about) 1 # doublestar 2 3 Path pattern matching and globbing supporting `doublestar` (`**`) patterns. 4 5 [](https://pkg.go.dev/github.com/bmatcuk/doublestar/v4) 6 [](https://github.com/bmatcuk/doublestar/releases) 7 [](https://github.com/bmatcuk/doublestar/actions) 8 [](https://codecov.io/github/bmatcuk/doublestar?branch=master) 9 [](https://github.com/sponsors/bmatcuk) 10 11 ## About 12 13 #### [Upgrading?](UPGRADING.md) 14 15 **doublestar** is a [golang] implementation of path pattern matching and 16 globbing with support for "doublestar" (aka globstar: `**`) patterns. 17 18 doublestar patterns match files and directories recursively. For example, if 19 you had the following directory structure: 20 21 ```bash 22 grandparent 23 `-- parent 24 |-- child1 25 `-- child2 26 ``` 27 28 You could find the children with patterns such as: `**/child*`, 29 `grandparent/**/child?`, `**/parent/*`, or even just `**` by itself (which will 30 return all files and directories recursively). 31 32 Bash's globstar is doublestar's inspiration and, as such, works similarly. 33 Note that the doublestar must appear as a path component by itself. A pattern 34 such as `/path**` is invalid and will be treated the same as `/path*`, but 35 `/path*/**` should achieve the desired result. Additionally, `/path/**` will 36 match all directories and files under the path directory, but `/path/**/` will 37 only match directories. 38 39 v4 is a complete rewrite with a focus on performance. Additionally, 40 [doublestar] has been updated to use the new [io/fs] package for filesystem 41 access. As a result, it is only supported by [golang] v1.16+. 42 43 ## Installation 44 45 **doublestar** can be installed via `go get`: 46 47 ```bash 48 go get github.com/bmatcuk/doublestar/v4 49 ``` 50 51 To use it in your code, you must import it: 52 53 ```go 54 import "github.com/bmatcuk/doublestar/v4" 55 ``` 56 57 ## Usage 58 59 ### ErrBadPattern 60 61 ```go 62 doublestar.ErrBadPattern 63 ``` 64 65 Returned by various functions to report that the pattern is malformed. At the 66 moment, this value is equal to `path.ErrBadPattern`, but, for portability, this 67 equivalence should probably not be relied upon. 68 69 ### Match 70 71 ```go 72 func Match(pattern, name string) (bool, error) 73 ``` 74 75 Match returns true if `name` matches the file name `pattern` ([see 76 "patterns"]). `name` and `pattern` are split on forward slash (`/`) characters 77 and may be relative or absolute. 78 79 Match requires pattern to match all of name, not just a substring. The only 80 possible returned error is `ErrBadPattern`, when pattern is malformed. 81 82 Note: this is meant as a drop-in replacement for `path.Match()` which always 83 uses `'/'` as the path separator. If you want to support systems which use a 84 different path separator (such as Windows), what you want is `PathMatch()`. 85 Alternatively, you can run `filepath.ToSlash()` on both pattern and name and 86 then use this function. 87 88 Note: users should _not_ count on the returned error, 89 `doublestar.ErrBadPattern`, being equal to `path.ErrBadPattern`. 90 91 92 ### PathMatch 93 94 ```go 95 func PathMatch(pattern, name string) (bool, error) 96 ``` 97 98 PathMatch returns true if `name` matches the file name `pattern` ([see 99 "patterns"]). The difference between Match and PathMatch is that PathMatch will 100 automatically use your system's path separator to split `name` and `pattern`. 101 On systems where the path separator is `'\'`, escaping will be disabled. 102 103 Note: this is meant as a drop-in replacement for `filepath.Match()`. It assumes 104 that both `pattern` and `name` are using the system's path separator. If you 105 can't be sure of that, use `filepath.ToSlash()` on both `pattern` and `name`, 106 and then use the `Match()` function instead. 107 108 ### GlobOption 109 110 Options that may be passed to `Glob`, `GlobWalk`, or `FilepathGlob`. Any number 111 of options may be passed to these functions, and in any order, as the last 112 argument(s). 113 114 ```go 115 WithFailOnIOErrors() 116 ``` 117 118 If passed, doublestar will abort and return IO errors when encountered. Note 119 that if the glob pattern references a path that does not exist (such as 120 `nonexistent/path/*`), this is _not_ considered an IO error: it is considered a 121 pattern with no matches. 122 123 ```go 124 WithFailOnPatternNotExist() 125 ``` 126 127 If passed, doublestar will abort and return `doublestar.ErrPatternNotExist` if 128 the pattern references a path that does not exist before any meta characters 129 such as `nonexistent/path/*`. Note that alts (ie, `{...}`) are expanded before 130 this check. In other words, a pattern such as `{a,b}/*` may fail if either `a` 131 or `b` do not exist but `*/{a,b}` will never fail because the star may match 132 nothing. 133 134 ```go 135 WithFilesOnly() 136 ``` 137 138 If passed, doublestar will only return "files" from `Glob`, `GlobWalk`, or 139 `FilepathGlob`. In this context, "files" are anything that is not a directory 140 or a symlink to a directory. 141 142 Note: if combined with the WithNoFollow option, symlinks to directories _will_ 143 be included in the result since no attempt is made to follow the symlink. 144 145 ```go 146 WithNoFollow() 147 ``` 148 149 If passed, doublestar will not follow symlinks while traversing the filesystem. 150 However, due to io/fs's _very_ poor support for querying the filesystem about 151 symlinks, there's a caveat here: if part of the pattern before any meta 152 characters contains a reference to a symlink, it will be followed. For example, 153 a pattern such as `path/to/symlink/*` will be followed assuming it is a valid 154 symlink to a directory. However, from this same example, a pattern such as 155 `path/to/**` will not traverse the `symlink`, nor would `path/*/symlink/*` 156 157 Note: if combined with the WithFilesOnly option, symlinks to directories _will_ 158 be included in the result since no attempt is made to follow the symlink. 159 160 ### Glob 161 162 ```go 163 func Glob(fsys fs.FS, pattern string, opts ...GlobOption) ([]string, error) 164 ``` 165 166 Glob returns the names of all files matching pattern or nil if there is no 167 matching file. The syntax of patterns is the same as in `Match()`. The pattern 168 may describe hierarchical names such as `usr/*/bin/ed`. 169 170 Glob ignores file system errors such as I/O errors reading directories by 171 default. The only possible returned error is `ErrBadPattern`, reporting that 172 the pattern is malformed. 173 174 To enable aborting on I/O errors, the `WithFailOnIOErrors` option can be 175 passed. 176 177 Note: this is meant as a drop-in replacement for `io/fs.Glob()`. Like 178 `io/fs.Glob()`, this function assumes that your pattern uses `/` as the path 179 separator even if that's not correct for your OS (like Windows). If you aren't 180 sure if that's the case, you can use `filepath.ToSlash()` on your pattern 181 before calling `Glob()`. 182 183 Like `io/fs.Glob()`, patterns containing `/./`, `/../`, or starting with `/` 184 will return no results and no errors. This seems to be a [conscious 185 decision](https://github.com/golang/go/issues/44092#issuecomment-774132549), 186 even if counter-intuitive. You can use [SplitPattern] to divide a pattern into 187 a base path (to initialize an `FS` object) and pattern. 188 189 Note: users should _not_ count on the returned error, 190 `doublestar.ErrBadPattern`, being equal to `path.ErrBadPattern`. 191 192 ### GlobWalk 193 194 ```go 195 type GlobWalkFunc func(path string, d fs.DirEntry) error 196 197 func GlobWalk(fsys fs.FS, pattern string, fn GlobWalkFunc, opts ...GlobOption) error 198 ``` 199 200 GlobWalk calls the callback function `fn` for every file matching pattern. The 201 syntax of pattern is the same as in Match() and the behavior is the same as 202 Glob(), with regard to limitations (such as patterns containing `/./`, `/../`, 203 or starting with `/`). The pattern may describe hierarchical names such as 204 usr/*/bin/ed. 205 206 GlobWalk may have a small performance benefit over Glob if you do not need a 207 slice of matches because it can avoid allocating memory for the matches. 208 Additionally, GlobWalk gives you access to the `fs.DirEntry` objects for each 209 match, and lets you quit early by returning a non-nil error from your callback 210 function. Like `io/fs.WalkDir`, if your callback returns `SkipDir`, GlobWalk 211 will skip the current directory. This means that if the current path _is_ a 212 directory, GlobWalk will not recurse into it. If the current path is not a 213 directory, the rest of the parent directory will be skipped. 214 215 GlobWalk ignores file system errors such as I/O errors reading directories by 216 default. GlobWalk may return `ErrBadPattern`, reporting that the pattern is 217 malformed. 218 219 To enable aborting on I/O errors, the `WithFailOnIOErrors` option can be 220 passed. 221 222 Additionally, if the callback function `fn` returns an error, GlobWalk will 223 exit immediately and return that error. 224 225 Like Glob(), this function assumes that your pattern uses `/` as the path 226 separator even if that's not correct for your OS (like Windows). If you aren't 227 sure if that's the case, you can use filepath.ToSlash() on your pattern before 228 calling GlobWalk(). 229 230 Note: users should _not_ count on the returned error, 231 `doublestar.ErrBadPattern`, being equal to `path.ErrBadPattern`. 232 233 ### FilepathGlob 234 235 ```go 236 func FilepathGlob(pattern string, opts ...GlobOption) (matches []string, err error) 237 ``` 238 239 FilepathGlob returns the names of all files matching pattern or nil if there is 240 no matching file. The syntax of pattern is the same as in Match(). The pattern 241 may describe hierarchical names such as usr/*/bin/ed. 242 243 FilepathGlob ignores file system errors such as I/O errors reading directories 244 by default. The only possible returned error is `ErrBadPattern`, reporting that 245 the pattern is malformed. 246 247 To enable aborting on I/O errors, the `WithFailOnIOErrors` option can be 248 passed. 249 250 Note: FilepathGlob is a convenience function that is meant as a drop-in 251 replacement for `path/filepath.Glob()` for users who don't need the 252 complication of io/fs. Basically, it: 253 254 * Runs `filepath.Clean()` and `ToSlash()` on the pattern 255 * Runs `SplitPattern()` to get a base path and a pattern to Glob 256 * Creates an FS object from the base path and `Glob()s` on the pattern 257 * Joins the base path with all of the matches from `Glob()` 258 259 Returned paths will use the system's path separator, just like 260 `filepath.Glob()`. 261 262 Note: the returned error `doublestar.ErrBadPattern` is not equal to 263 `filepath.ErrBadPattern`. 264 265 ### SplitPattern 266 267 ```go 268 func SplitPattern(p string) (base, pattern string) 269 ``` 270 271 SplitPattern is a utility function. Given a pattern, SplitPattern will return 272 two strings: the first string is everything up to the last slash (`/`) that 273 appears _before_ any unescaped "meta" characters (ie, `*?[{`). The second 274 string is everything after that slash. For example, given the pattern: 275 276 ``` 277 ../../path/to/meta*/** 278 ^----------- split here 279 ``` 280 281 SplitPattern returns "../../path/to" and "meta*/**". This is useful for 282 initializing os.DirFS() to call Glob() because Glob() will silently fail if 283 your pattern includes `/./` or `/../`. For example: 284 285 ```go 286 base, pattern := SplitPattern("../../path/to/meta*/**") 287 fsys := os.DirFS(base) 288 matches, err := Glob(fsys, pattern) 289 ``` 290 291 If SplitPattern cannot find somewhere to split the pattern (for example, 292 `meta*/**`), it will return "." and the unaltered pattern (`meta*/**` in this 293 example). 294 295 Of course, it is your responsibility to decide if the returned base path is 296 "safe" in the context of your application. Perhaps you could use Match() to 297 validate against a list of approved base directories? 298 299 ### ValidatePattern 300 301 ```go 302 func ValidatePattern(s string) bool 303 ``` 304 305 Validate a pattern. Patterns are validated while they run in Match(), 306 PathMatch(), and Glob(), so, you normally wouldn't need to call this. However, 307 there are cases where this might be useful: for example, if your program allows 308 a user to enter a pattern that you'll run at a later time, you might want to 309 validate it. 310 311 ValidatePattern assumes your pattern uses '/' as the path separator. 312 313 ### ValidatePathPattern 314 315 ```go 316 func ValidatePathPattern(s string) bool 317 ``` 318 319 Like ValidatePattern, only uses your OS path separator. In other words, use 320 ValidatePattern if you would normally use Match() or Glob(). Use 321 ValidatePathPattern if you would normally use PathMatch(). Keep in mind, Glob() 322 requires '/' separators, even if your OS uses something else. 323 324 ### Patterns 325 326 **doublestar** supports the following special terms in the patterns: 327 328 Special Terms | Meaning 329 ------------- | ------- 330 `*` | matches any sequence of non-path-separators 331 `/**/` | matches zero or more directories 332 `?` | matches any single non-path-separator character 333 `[class]` | matches any single non-path-separator character against a class of characters ([see "character classes"]) 334 `{alt1,...}` | matches a sequence of characters if one of the comma-separated alternatives matches 335 336 Any character with a special meaning can be escaped with a backslash (`\`). 337 338 A doublestar (`**`) should appear surrounded by path separators such as `/**/`. 339 A mid-pattern doublestar (`**`) behaves like bash's globstar option: a pattern 340 such as `path/to/**.txt` would return the same results as `path/to/*.txt`. The 341 pattern you're looking for is `path/to/**/*.txt`. 342 343 #### Character Classes 344 345 Character classes support the following: 346 347 Class | Meaning 348 ---------- | ------- 349 `[abc]` | matches any single character within the set 350 `[a-z]` | matches any single character in the range 351 `[^class]` | matches any single character which does *not* match the class 352 `[!class]` | same as `^`: negates the class 353 354 ## Performance 355 356 ``` 357 goos: darwin 358 goarch: amd64 359 pkg: github.com/bmatcuk/doublestar/v4 360 cpu: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz 361 BenchmarkMatch-8 285639 3868 ns/op 0 B/op 0 allocs/op 362 BenchmarkGoMatch-8 286945 3726 ns/op 0 B/op 0 allocs/op 363 BenchmarkPathMatch-8 320511 3493 ns/op 0 B/op 0 allocs/op 364 BenchmarkGoPathMatch-8 304236 3434 ns/op 0 B/op 0 allocs/op 365 BenchmarkGlob-8 466 2501123 ns/op 190225 B/op 2849 allocs/op 366 BenchmarkGlobWalk-8 476 2536293 ns/op 184017 B/op 2750 allocs/op 367 BenchmarkGoGlob-8 463 2574836 ns/op 194249 B/op 2929 allocs/op 368 ``` 369 370 These benchmarks (in `doublestar_test.go`) compare Match() to path.Match(), 371 PathMath() to filepath.Match(), and Glob() + GlobWalk() to io/fs.Glob(). They 372 only run patterns that the standard go packages can understand as well (so, no 373 `{alts}` or `**`) for a fair comparison. Of course, alts and doublestars will 374 be less performant than the other pattern meta characters. 375 376 Alts are essentially like running multiple patterns, the number of which can 377 get large if your pattern has alts nested inside alts. This affects both 378 matching (ie, Match()) and globbing (Glob()). 379 380 `**` performance in matching is actually pretty similar to a regular `*`, but 381 can cause a large number of reads when globbing as it will need to recursively 382 traverse your filesystem. 383 384 ## Sponsors 385 I started this project in 2014 in my spare time and have been maintaining it 386 ever since. In that time, it has grown into one of the most popular globbing 387 libraries in the Go ecosystem. So, if **doublestar** is a useful library in 388 your project, consider [sponsoring] my work! I'd really appreciate it! 389 390 [](https://reviewpad.com/) 391 392 Thanks for sponsoring me! 393 394 ## License 395 396 [MIT License](LICENSE) 397 398 [SplitPattern]: #splitpattern 399 [doublestar]: https://github.com/bmatcuk/doublestar 400 [golang]: http://golang.org/ 401 [io/fs]: https://pkg.go.dev/io/fs 402 [see "character classes"]: #character-classes 403 [see "patterns"]: #patterns 404 [sponsoring]: https://github.com/sponsors/bmatcuk