github.com/jvandenbroek/directory_stat_exporter@v0.2.0/README.md (about) 1 # directory statistics for prometheus 2 get simple directory statistics (metrics) for prometheus. if you want to know if a batch job did not process all the files in a directory or a certain folder contains files with error information, this is the right place. the directory_stat_exporter provides minimal metrics to see how many documents are in a folder and the timestamp (unix time in seconds) of the oldest one. with this information, you can decide, depending on the specs of an interface, if everything is ok or not. the analysis of a directory can be generated from only the directory itself, without the subdirectories, or also including all the subdirectories. depending on your needs. 3 4 the purpose of this exporter is to provide metrics so prometheus can generate an alert if there are files waiting longer than a certain time. 5 6 ## features 7 - all done without prometheus libraries 8 - don't know if this is a good decesion or a bad one. 9 - that's not really a feature, is it. oh, well, it's a fun project anyway. 10 - super simple, the only things provided so far: 11 - number of files in directory, with or without subdirectories 12 - last modified timestamp of oldest file in directoy, with or without subdirs. 13 - for calculation reasons the current timestamp 14 15 ## exports 16 - `dirstat_files_in_dir`: number of files in directory 17 - `dirstat_oldest_file_time`: timestamp (unix time) of oldest file in dir 18 - `dirstat_current_timestamp`: the current timestamp. because it's not provieded by prometheus (or I was not able to find it.) 19 20 ## todos 21 - test handling of unc paths in windows (yes, it's targeted for windows.) 22 - better logging (really? do I need logging here?) 23 - better error handling 24 - it should be fault tolerant and rather give useful metrics if an error occurs 25 - make information gathering concurrent, so more directories can be handled in the same time 26 - add the disk usage of the current directory as a metric 27 - think of usefulness first. do we really need this? -> so far: no. 28 - add performance measurements: 29 - how fast is the gathering 30 - where are the limits on local drives 31 - what are the limits on remote directories 32 - work out how to handle dependencies propperly. 33 34 ## notes to self 35 - *important* stack items correctly (types and help text must only appear once in a metric export / per request) 36 - labels must not contain a single backslash... I replaced all backslashes now with forward slashes. -> there must be a better solution 37 - e.g. add labels to the configuration and give them meaningful names. 38 39 ## problems 40 - large directories might not be handled well 41 - might use lot of memory, because whole directory is read once (untested) 42 43 ## configuration 44 ### service port 45 `serviceport: "9999"` 46 47 this does not really need more explanation 48 49 ### cache time 50 `cachetime: 5` 51 52 time of interval in minutes. must be an integer value. 53 54 this is the time interval which is used to poll the updates from the directories. it does not make sense to me to poll the directories every 15 seconds. if it is set to `0` then caching is disabled. 55 56 ### directories 57 ``` 58 directories: 59 - name: "tmp" 60 path: /tmp 61 recursive: true 62 ``` 63 |key|value| 64 |---|-----| 65 |path|a path to the directory to be monitored. this can be a relative or an absolute path. for windows it can also be a UNC path.| 66 |name|a useful name. must be unique. not sure if special characters and spaces are a good idea. I have to check that.| 67 |recursive|should subdirectories also be analysed. can be `true` or `false`| 68 69 ## usage 70 in your GOPATH type 71 `go get gopkg.in/yaml.v2` 72 73 `go get github.com/codestoke/directory_stat_exporter` 74 75 you'll find the binary in the `bin` directory of your GOPATH. the `config.yml` must be in the same directory of the executable (there are no other options yet) 76 77 currently, this generates the version in the master branch. this might not be very useful and can be buggy. 78 79 you can always download the source from a release, unpack it in your GOPATH and then compile it with 80 81 `go build` 82 83 have fun! 84 85 cheers, Oli