github.com/qubitproducts/logspray@v0.2.14/indexer/doc.go (about) 1 // Package indexer processing logs and stores them for 2 // querying. This is probably confusing several concerns 3 // and needs work. 4 // 5 // On start up, each indexer is assigned a new UUID. 6 // The indexer creates shards, collections of shard files that span a liited 7 // time duration (shardDuration). Each shard created by an indexer is assigned 8 // a UUID. 9 // Each incoming stream is assigned a UUID. A sharfile is created for each stream. 10 // Each message in a stream is assigned its ordinal position in the stream 11 // 12 // The dataDir should be laid out as follows: 13 // dataDir/shardID/streamID.pb.log 14 package indexer