github.com/jvandenbroek/directory_stat_exporter@v0.2.0/README.md (about)

     1  # directory statistics for prometheus
     2  get simple directory statistics (metrics) for prometheus. if you want to know if a batch job did not process all the files in a directory or a certain folder contains files with error information, this is the right place. the directory_stat_exporter provides minimal metrics to see how many documents are in a folder and the timestamp (unix time in seconds) of the oldest one. with this information, you can decide, depending on the specs of an interface, if everything is ok or not. the analysis of a directory can be generated from only the directory itself, without the subdirectories, or also including all the subdirectories. depending on your needs.
     3  
     4  the purpose of this exporter is to provide metrics so prometheus can generate an alert if there are files waiting longer than a certain time.
     5  
     6  ## features
     7  - all done without prometheus libraries
     8    - don't know if this is a good decesion or a bad one.
     9    - that's not really a feature, is it. oh, well, it's a fun project anyway.
    10  - super simple, the only things provided so far:
    11    - number of files in directory, with or without subdirectories
    12    - last modified timestamp of oldest file in directoy, with or without subdirs.
    13    - for calculation reasons the current timestamp
    14  
    15  ## exports
    16  - `dirstat_files_in_dir`: number of files in directory
    17  - `dirstat_oldest_file_time`: timestamp (unix time) of oldest file in dir
    18  - `dirstat_current_timestamp`: the current timestamp. because it's not provieded by prometheus (or I was not able to find it.)
    19  
    20  ## todos
    21  - test handling of unc paths in windows (yes, it's targeted for windows.)
    22  - better logging (really? do I need logging here?)
    23  - better error handling
    24    - it should be fault tolerant and rather give useful metrics if an error occurs
    25  - make information gathering concurrent, so more directories can be handled in the same time
    26  - add the disk usage of the current directory as a metric
    27    - think of usefulness first. do we really need this? -> so far: no.
    28  - add performance measurements:
    29    - how fast is the gathering
    30    - where are the limits on local drives
    31    - what are the limits on remote directories 
    32  - work out how to handle dependencies propperly.
    33  
    34  ## notes to self
    35  - *important* stack items correctly (types and help text must only appear once in a metric export / per request)
    36  - labels must not contain a single backslash... I replaced all backslashes now with forward slashes. -> there must be a better solution
    37    - e.g. add labels to the configuration and give them meaningful names.
    38  
    39  ## problems
    40  - large directories might not be handled well
    41    - might use lot of memory, because whole directory is read once (untested)
    42  
    43  ## configuration
    44  ### service port
    45  `serviceport: "9999"`
    46  
    47  this does not really need more explanation
    48  
    49  ### cache time
    50  `cachetime: 5`
    51  
    52  time of interval in minutes. must be an integer value.
    53  
    54  this is the time interval which is used to poll the updates from the directories. it does not make sense to me to poll the directories every 15 seconds. if it is set to `0` then caching is disabled.
    55  
    56  ### directories
    57  ```
    58  directories:
    59       - name: "tmp"
    60         path: /tmp
    61         recursive: true
    62  ```
    63  |key|value|
    64  |---|-----|
    65  |path|a path to the directory to be monitored. this can be a relative or an absolute path. for windows it can also be a UNC path.|
    66  |name|a useful name. must be unique. not sure if special characters and spaces are a good idea. I have to check that.|
    67  |recursive|should subdirectories also be analysed. can be `true` or `false`|
    68  
    69  ## usage
    70  in your GOPATH type
    71  `go get gopkg.in/yaml.v2`
    72  
    73  `go get github.com/codestoke/directory_stat_exporter`
    74  
    75  you'll find the binary in the `bin` directory of your GOPATH. the `config.yml` must be in the same directory of the executable (there are no other options yet)
    76  
    77  currently, this generates the version in the master branch. this might not be very useful and can be buggy.
    78  
    79  you can always download the source from a release, unpack it in your GOPATH and then compile it with
    80  
    81  `go build`
    82  
    83  have fun!
    84  
    85  cheers, Oli