bosun.org@v0.0.0-20210513094433-e25bc3e69a1f/docs/scollector/external-collectors.md (about)

     1  ---
     2  layout: default
     3  title: External Collectors
     4  ---
     5     
     6  
     7  ## Using External Collectors with Scollector
     8  
     9  External collectors are executables or scripts that scollector invokes, collects output from, and forwards data points on to the server like any other metric. These can be binaries, .sh/.py/.rb/.cmd/.bat/.ps1 scripts, or even .jar files ( See [InitPrograms](http://godoc.org/bosun.org/cmd/scollector/collectors#InitPrograms) function for implementation details ). External collectors are a great way to get started collecting data, but when possible it is recommended for applications to send data directly to Bosun or to update scollector so that it natively supports additional systems.
    10  
    11  The [ColDir](http://godoc.org/bosun.org/cmd/scollector#hdr-Configuration_File) configuration key specifies the external collector directory, which is usually set to something like **/opt/scollector/collectors** in Linux or **C:\Program Files\scollector\collectors** in Windows. It should contain numbered directories just like the ones used in [OpenTSDB tcollector](http://opentsdb.net/docs/build/html/user_guide/utilities/tcollector.html#collecting-lots-of-metrics-with-tcollector). Each directory represents how often scollector will try to invoke the collectors in that folder (example: **60** = every 60 seconds). Use a directory named **0** for any executables or scripts that will run continuously and create output on their own schedule. Any non-numeric named directories will be ignored, and a lib and etc directory are often used for library and config data shared by all collectors.
    12  
    13  There are examples of external collectors in a few different languages on the Stack Overflow Documentation topic for [Scollector External Collectors](http://stackoverflow.com/documentation/bosun/720/scollector-external-collectors).
    14  
    15     
    16  
    17  ### Simple data output format
    18  
    19  The simple output format can be use to send raw metrics to scollector using the standard output stream. You won't be able to specify units, counter vs gauge, or descriptions, but those can always be specified manually in graphs/expressions or added later if desired.
    20  
    21  {% highlight text %}
    22  #Manually run script to see output: metric timestamp value tagk=tagv tagk=tagv
    23  /opt/scollector/collectors/0/twitter
    24  twitter.tweet_count 1441406996 0 query=stackoverflow-down
    25  twitter.follower_count 1441406996 1337 account=stackoverflow
    26  {% endhighlight %}
    27  
    28  `Metric` is the name of the metric you want to create.
    29  
    30  `Timestamp` is in Unix format (seconds since epoch).
    31  
    32  `Value` is the floating point value that will be used for that data point.
    33  
    34  `Tags` are optional. A host tag is automatically added, but overridden if specified.
    35  
    36  `Errors` anything written to the standard error stream is included in the scollector logs (syslog on Linux, event log on Windows).
    37  
    38     
    39  
    40  ### JSON data output format
    41  
    42  If you want to include metadata with your external collector metrics you can send JSON serialized instances of the [opentsdb.DataPoint](http://godoc.org/bosun.org/opentsdb#DataPoint) struct for metric data and the [metadata.Metasend](http://godoc.org/bosun.org/metadata#Metasend) struct for metadata. The JSON data would be streamed over standard output and look similar to the output listed below. It starts with metadata (only needed once, as scollector will re-send as needed) and then continues sending metrics. The 4th line is a log message which was sent to the standard error stream.
    43  
    44  {% highlight text %}
    45  {"metric":"exceptional.exceptions.count","name":"rate","value":"counter"}
    46  {"metric":"exceptional.exceptions.count","name":"unit","value":"errors"}
    47  {"metric":"exceptional.exceptions.count","name":"desc","value":"Exceptions per second stored in each Opserver data source."}
    48  2015/08/05 15:32:00 lookup OR-SQL03: no such host
    49  {"metric":"exceptional.exceptions.count","timestamp":1438788720,"value":5,"tags":{"application":"Careers","machine":"ny-web03","source":"NY_Status"}}
    50  {"metric":"exceptional.exceptions.count","timestamp":1438788720,"value":0,"tags":{"application":"AdServer","machine":"ny-web03","source":"NY_Status"}}
    51  {"metric":"exceptional.exceptions.count","timestamp":1438788720,"value":2,"tags":{"application":"AdServer","machine":"ny-web04","source":"NY_Status"}}
    52  {"metric":"exceptional.exceptions.count","timestamp":1438788720,"value":1,"tags":{"application":"AdServer","machine":"ny-web06","source":"NY_Status"}}
    53  {% endhighlight %}