github.com/m3db/m3@v1.5.1-0.20231129193456-75a402aa583b/CHANGELOG.md (about) 1 # Changelog 2 3 # 1.5.0 4 5 ## Features and Performance 6 7 - **M3DB**: Optimize snapshotting by 30x or more which can speedup bootstrapping for workloads with large snapshots. ([#4093](https://github.com/m3db/m3/pull/4093)) 8 9 ## Bug Fixes 10 11 - **M3DB**: Fix multi-segment field iterator support of double underscore prefixed fields alphanumerically before `__m3ninx_id`. ([#4095](https://github.com/m3db/m3/pull/4095)) 12 - **M3DB**: Proactively GC Index entries belonging to a closed shard after cluster topology change. ([#4094](https://github.com/m3db/m3/pull/4094)) 13 14 # 1.4.2 15 16 ## Bug Fixes 17 - **M3Coordinator**: Guard against duplicate __rollup__ tags in aggregation matching logic. ([#3950](https://github.com/m3db/m3/pull/3950)) 18 19 ## Performance 20 - **M3DB**: Improve index query performance by avoid cloning of roaring bitmap in conjunctionSearcher. ([#3948](https://github.com/m3db/m3/pull/3948)) 21 - **M3DB**: Improve bootstrapping memory profile by fixing entry leak in series ref resolver. ([#3980](https://github.com/m3db/m3/pull/3980)) 22 - **M3DB**: Provide better balanced shard placement by fixing unbalanced initial shard allocation. ([#4020](https://github.com/m3db/m3/pull/4020) 23 24 # 1.4.1 25 26 ## Bug Fixes 27 - **M3Coordinator**: Do not Close singleton MessageProcessors when closing connections. This fixes a panic introduced that affects M3Coordinator -> M3Aggregator communication. ([#3934](https://github.com/m3db/m3/pull/3934)) 28 29 # 1.4.0 30 31 ## Features 32 - **M3Query**: Add write endpoint support for M3-Map-Tags-JSON header in InfluxDB path ([#3816](https://github.com/m3db/m3/pull/3816)) 33 - **M3Query**: Add support for `last_over_time` in M3Query engine ([#3884](https://github.com/m3db/m3/pull/3884)) 34 - **M3Aggregator**: Add p75/p25 as aggregation options ([#3867](https://github.com/m3db/m3/pull/3867)) 35 36 ## Bug Fixes 37 - **M3DB**: Fix M3TSZ to be deterministic when encoding high precision values ([#3872](https://github.com/m3db/m3/pull/3872)) 38 - **M3DB**: Gracefully handle reads including documents with stale index state ([#3905](https://github.com/m3db/m3/pull/3905)) 39 40 ## Performance 41 - **M3Aggregator**: Rework close and remove `persitFlushTimesEvery` semantics in leader flushing in favour of always persisting shard flush times on a successful flush for optimized graceful failovers ([#3890](https://github.com/m3db/m3/pull/3890)) 42 - **M3DB**: Optimize `filesetFiles` function during bootstrapping for namespaces with long retentions to prevent CPU spikes ([#3900](https://github.com/m3db/m3/pull/3900)) 43 - **M3DB**: Avoid loading blocks in memory for namespaces with snapshots disabled during bootstrapping to reduce memory usage ([#3919](https://github.com/m3db/m3/pull/3919)) 44 45 # 1.3.0 46 47 ## Features 48 49 - **M3Coordinator**: Add support for Prometheus Remote Write storage backend for sending aggregated and unaggregated metrics ([#3742](https://github.com/m3db/m3/pull/3742), [#3768](https://github.com/m3db/m3/pull/3768), [#3783](https://github.com/m3db/m3/pull/3783), [#3791](https://github.com/m3db/m3/pull/3791), [#3814](https://github.com/m3db/m3/pull/3814), [#3777](https://github.com/m3db/m3/pull/3777)) 50 - **M3Coordinator**: Add support for InfluxDB write endpoint GZip compression, setting timestamp precision and allowing an empty request body ([#3373](https://github.com/m3db/m3/pull/3373)) 51 - **M3DB**: Add SYSCTL_VM_MAX_MAP_COUNT env var for sysctl-setter sidecar allowing for custom VM max map count ([#3689](https://github.com/m3db/m3/pull/3689)) 52 53 ## Bug Fixes 54 55 - **M3DB**: Fix writes briefly degrading when creating a new namespace due to coarse lock acquisition ([#3765](https://github.com/m3db/m3/pull/3765)) 56 - **M3DB**: Fix compiled regexp DFA cache eviction on full bug that can lead to slow memory leak with large number of unique regexps ([#3806](https://github.com/m3db/m3/pull/3806)) 57 58 ## Performance 59 60 - **M3Coordinator**: Update default M3Msg retry initial backoff from 1s to 5s to reduces timeout and retries in large clusters ([#3820](https://github.com/m3db/m3/pull/3820)) 61 - **M3DB**: Fix performance of reverse index queries that cover huge time ranges ([#3813](https://github.com/m3db/m3/pull/3813)) 62 63 # 1.2.0 64 65 ## Features 66 67 - **M3Query**: Support Prometheus matchers with match[] URL parameters in label endpoints ([#3180](https://github.com/m3db/m3/pull/3180)) 68 - **M3Query**: Support Prometheus start and end time URL parameters for label and series metadata endpoints ([#3214](https://github.com/m3db/m3/pull/3214)) 69 - **M3Query**: Add Graphite functions and update functions with new arguments that were missing ([#3048](https://github.com/m3db/m3/pull/3048), [#3367](https://github.com/m3db/m3/pull/3367), [#3370](https://github.com/m3db/m3/pull/3370), [#3145](https://github.com/m3db/m3/pull/3145), [#3149](https://github.com/m3db/m3/pull/3149), [#3142](https://github.com/m3db/m3/pull/3142), [#3469](https://github.com/m3db/m3/pull/3469), [#3484](https://github.com/m3db/m3/pull/3484), [#3545](https://github.com/m3db/m3/pull/3545), [#3576](https://github.com/m3db/m3/pull/3576), [#3582](https://github.com/m3db/m3/pull/3582), [#3583](https://github.com/m3db/m3/pull/3583), [#3521](https://github.com/m3db/m3/pull/3521), [#3602](https://github.com/m3db/m3/pull/3602), [#3641](https://github.com/m3db/m3/pull/3641), [#3644](https://github.com/m3db/m3/pull/3644), [#3648](https://github.com/m3db/m3/pull/3648)) 70 - **M3Query**: Fix Graphite treatment of `**` to allow to match an empty segment instead of one or more ([#3366](https://github.com/m3db/m3/pull/3366), [#3593](https://github.com/m3db/m3/pull/3593)) 71 - **M3Query**: Add M3-Limit-Max-Range header to optionally truncate time range of queries ([#3538](https://github.com/m3db/m3/pull/3538)) 72 - **M3Coordinator**: Add ability to use an exclude by rollup rule to rollup metrics without specific dimensions ([#3318](https://github.com/m3db/m3/pull/3318)) 73 - **M3DB**: Use better heuristics to cap the series and aggregate query limits that individual DB nodes apply for a query so in larger clusters the query can be clamped earlier ([#3516](https://github.com/m3db/m3/pull/3516), [#3518](https://github.com/m3db/m3/pull/3518), [#3519](https://github.com/m3db/m3/pull/3519), [#3520](https://github.com/m3db/m3/pull/3520), [#3527](https://github.com/m3db/m3/pull/3527)) 74 - **M3DB**: Add repair option full_sweep and ability to force a repair via API call ([#3573](https://github.com/m3db/m3/pull/3573), [#3550](https://github.com/m3db/m3/pull/3550)) 75 76 ## Bug Fixes 77 78 - **M3DB**: Fix aggregate series metadata query limits ([#3112](https://github.com/m3db/m3/pull/3112)) 79 - **M3Coordinator**: Make bad aggregated namespace headers return bad request status code instead of internal server error ([#3070](https://github.com/m3db/m3/pull/3070)) 80 - **M3Coordinator**: Propagate Require-Exhaustive parameter for aggregate series metadata queries ([#3115](https://github.com/m3db/m3/pull/3115)) 81 - **M3Query**: Add determinism to Graphite sort and reduce functions ([#3164](https://github.com/m3db/m3/pull/3164)) 82 83 ## Performance 84 85 - **M3DB**: Rearchitect index segments to compact and expire series on block rotation instead of build a new segment for new block ([#3464](https://github.com/m3db/m3/pull/3464)) 86 - **M3DB**: Add postings list cache for searches and repopulate during active block index segment compaction before segment made visible for queries ([#3671](https://github.com/m3db/m3/pull/3671)) 87 - **M3DB**: Avoid allocating index entry fields per series and read from backing mmap directly ([#3050](https://github.com/m3db/m3/pull/3050), [#3062](https://github.com/m3db/m3/pull/3062), [#3057](https://github.com/m3db/m3/pull/3057)) 88 - **M3DB**: Avoid allocating series IDs when read from disk ([#3093](https://github.com/m3db/m3/pull/3093)) 89 - **M3DB**: Improve speed of tag byte reuse from ID for tags by speeding up search ([#3075](https://github.com/m3db/m3/pull/3075)) 90 - **M3DB**: Improve speed of bootstrap by using StreamingReadMetadata API for reads from disk ([#2938](https://github.com/m3db/m3/pull/2938)) 91 - **M3DB**: Improve speed of bootstrap by using an asynchronously evaluated series resolver API that can be written to while bootstrapping reliably ([#3316](https://github.com/m3db/m3/pull/3316)) 92 - **M3DB**: Add limits for total series being read at any one time globally ([#3141](https://github.com/m3db/m3/pull/3141)) 93 - **M3DB**: Restrict the time a query can hold an index worker to help allow small queries to continue to execute while larger ones are paused and resumed ([#3269](https://github.com/m3db/m3/pull/3269)) 94 - **M3DB**: Use adaptive WriteBatch allocations to dynamically match workload throughput and batch sizes ([#3429](https://github.com/m3db/m3/pull/3429)) 95 - **M3Coordinator**: Improve rule matching speed by improving per element rule matching and disabling cache which puts locks in the hot path ([#3080](https://github.com/m3db/m3/pull/3080), [#3083](https://github.com/m3db/m3/pull/3083)) 96 - **M3Query**: Improve speed of M3TSZ decoding by using 64 bit operations ([#2827](https://github.com/m3db/m3/pull/2827)) 97 - **M3Query**: Improve speed of M3TSZ decoding by using int64 type xtime.UnixNano instead of time.Time ([#3515](https://github.com/m3db/m3/pull/3515)) 98 - **M3Query**: Improve speed of quorum reads by improving multi-replica iterator ([#3512](https://github.com/m3db/m3/pull/3512)) 99 100 # 1.1.0 101 102 ## Features 103 104 - **M3Coordinator**: Add /ready endpoint for readiness probe which checks current write/read consistency level achievability ([#2976](https://github.com/m3db/m3/pull/2976)) 105 - **M3Coordinator**: Add per endpoint status code response codes and latency metrics ([#2880](https://github.com/m3db/m3/pull/2880)) 106 - **M3Coordinator**: Add Graphite carbon ingest latency metrics ([#3045](https://github.com/m3db/m3/pull/3045)) 107 - **M3Coordinator**: Add Graphite carbon ingest rule matcher contains to compliment regexp for faster matching ([#3046](https://github.com/m3db/m3/pull/3046)) 108 - **M3Coordinator**: Return 504 errors on timeout to downstream M3DB nodes or other cross-region coordinators ([#2886](https://github.com/m3db/m3/pull/2886)) 109 - **M3Coordinator**: Validate placements when using the raw placement upsert endpoint unless force set is specified ([#2922](https://github.com/m3db/m3/pull/2922)) 110 - **M3Query**: Add Graphite powSeries function ([#3038](https://github.com/m3db/m3/pull/3038)) 111 - **M3Query**: Add Graphite support for ** with metric path selectors ([#3020](https://github.com/m3db/m3/pull/3020)) 112 - **M3DB**: Add ability to configure regexp DFA and FSA limits ([#2926](https://github.com/m3db/m3/pull/2926)) 113 - **M3DB**: Add Alibaba Cloud storage class Kubernetes manifest for disk provisioning in Aliyun ([#2908](https://github.com/m3db/m3/pull/2908)) 114 115 ## Bug Fixes 116 117 - **M3Coordinator**: Always set content type JSON for error responses ([#2917](https://github.com/m3db/m3/pull/2917)) 118 - **M3Query**: Fix invalid query resulting in 500 instead of 400 ([#2910](https://github.com/m3db/m3/pull/2910)) 119 - **M3Query**: Allow Graphite variadic functions to omit variadic args ([#2882](https://github.com/m3db/m3/pull/2882)) 120 121 ## Performance 122 123 - **M3DB**: Skip out of retention index segments during bootstrap ([#2992](https://github.com/m3db/m3/pull/2992)) 124 125 ## Documentation 126 127 - **All**: Add clustering getting started guides for both Kubernetes operator and binaries deployment ([#2795](https://github.com/m3db/m3/pull/2795)) 128 129 # 1.0.0 130 131 ## Overview 132 133 This release makes breaking changes to the APIs and configuration to provide a simpler experience both for setup and operating M3. 134 135 - New [M3 website](https://m3db.io/). 136 - New [M3 documentation](https://m3db.io/docs). 137 - Simple [M3DB configuration](https://github.com/m3db/m3/blob/master/src/dbnode/config/m3dbnode-local-etcd.yml) and [guides](https://m3db.io/docs/quickstart/docker/). 138 - M3DB [hard limits](https://m3db.io/docs/operational_guide/resource_limits/) limits for high resiliency under load. 139 - Bootstrap rearchitecture, now able to boostrap hundreds of millions of recently written datapoints in minutes for reads on restart. 140 - Continued focus on baseline performance release-over-release. 141 142 ## Features 143 - **M3DB**: Namespace resolution and retention now configured dynamically via API and stored in etcd instead of being defined statically in M3Coordinator configuration. 144 ``` 145 message DatabaseCreateRequest { 146 // ... 147 148 // Optional aggregated namespace to create in 149 // addition to unaggregated namespace 150 AggregatedNamespace aggregated_namespace = 8; 151 } 152 ``` 153 - **M3DB**: Minimal configuration file with default settings looks like: 154 ``` 155 coordinator: {} 156 db: {} 157 ``` 158 and includes common settings such as global query limits. 159 160 ## Backwards Incompatible Changes 161 162 ### Configuration 163 - **M3DB**: `db.bootstrap.bootstrappers` removed 164 - **M3DB**: `db.config` nested under `db.discovery.config` (`discovery` can optionally accept different `type`s of defaults instead of a custom `config`) 165 - **M3Coordinator**: `cluster.namespaces.storageMetricsType` removed 166 - **M3Coordinator**: `tagOptions.tagOptions` no longer supports `legacy` type 167 - **M3Query**: `limits.perQuery.maxComputedDatapoints` removed 168 - **M3Query**: `limits.perQuery.maxFetchedDatapoints` removed 169 - **M3Query**: `limits.global.maxFetchedDatapoints` removed 170 - **M3Query**: `cache` removed 171 - **M3Query**: `listenAddress` changed to always be resolved as a string from config. Format changed from 172 ``` 173 listenAddress: 174 config: "..." 175 value: "..." 176 ``` 177 to 178 ``` 179 listenAddress: "..." 180 ``` 181 182 ### API 183 - **M3DB**: `/services/m3db/database/config/bootstrappers` dynamic bootstappers endpoint removed 184 - **M3Coordinator**: Removed deprecated URL `/api/v1/namespace` in favor of stable preferred URL `/api/v1/services/m3db/namespace` 185 - **M3Coordinator**: Removed deprecated URL `/api/v1/namespace/init` in favor of stable preferred URL `/api/v1/services/m3db/namespace/init` 186 - **M3Coordinator**: Removed deprecated URL `/api/v1/namespace/unagg` in favor of stable preferred URL `/api/v1/services/m3db/namespace/unagg` 187 - **M3Coordinator**: Removed deprecated URL `/api/v1/placement` in favor of stable preferred URL `/api/v1/services/m3db/placement` 188 - **M3Coordinator**: Removed deprecated URL `/api/v1/placement/init` in favor of stable preferred URL `/api/v1/services/m3db/placement/init` 189 190 ### Package 191 - `github.com/m3db/m3/src/x/close` removed in favor of `github.com/m3db/m3/src/x/resource` 192 - `github.com/m3db/m3/src/dbnode/clock` removed in favor of `github.com/m3db/m3/src/x/clock` 193 - `github.com/m3db/m3/src/x/dice/dice.go` moved to `github.com/m3db/m3/src/dbnode/storage/dice.go` 194 - `github.com/m3db/m3/src/x/lockfile/lockfile.go` moved to `github.com/m3db/m3/src/dbnode/server/lockfile.go` 195 196 ### Misc 197 - **M3Query**: Concept of data point limit enforcers removed in favor of the other remaining query limits (e.g. max series). This also removed metrics `cost_reporter_datapoints`, `cost_reporter_datapoints_counter`, and `cost_reporter_over_datapoints_limit`. 198 - Linter enabled 199 200 # 0.15.17 201 202 ## Features 203 - **M3Query**: Add aggregate Graphite function ([#2584](https://github.com/m3db/m3/pull/2584)) 204 - **M3Query**: Add applyByNode Graphite function ([#2654](https://github.com/m3db/m3/pull/2654)) 205 - **M3Query**: Graphite ParseTime function support greatly expanded to be more in line with Graphite allowances ([#2621](https://github.com/m3db/m3/pull/2621)) 206 207 ## Bug Fixes 208 209 - **M3Aggregator**: Add default m3msg write timeouts to mitigate deadlocking writes with a stale TCP connection ([#2698](https://github.com/m3db/m3/pull/2698)) 210 - **M3DB**: Fix a bug in bootstrap index caching that would cause long bootstrap times ([#2703](https://github.com/m3db/m3/pull/2703)) 211 - **M3Query**: Fix Graphite constantLine() function to return 3 steps ([#2699](https://github.com/m3db/m3/pull/2699)) 212 - **M3Query**: Fix Graphite limit snapping bug in movingAverage and movingMedian functions ([#2694](https://github.com/m3db/m3/pull/2694)) 213 214 # 0.15.16 215 216 ## Features 217 218 - **M3Query**: Add divideSeriesLists Graphite function ([#2585](https://github.com/m3db/m3/pull/2585)) 219 - **M3Query**: Add integralByInterval Graphite function ([#2596](https://github.com/m3db/m3/pull/2596)) 220 - **M3Query**: Add highest, lowest Graphite functions ([#2623](https://github.com/m3db/m3/pull/2623)) 221 - **M3Query**: Add resolution exceeds query range warning ([#2429](https://github.com/m3db/m3/pull/2429)) 222 223 ## Documentation 224 225 - **M3Coordinator**: Added OpenAPI specification for namespace update endpoint ([#2629](https://github.com/m3db/m3/pull/2629)) 226 227 ## Misc 228 229 - **M3Coordinator**: Add config option for writes to leaving shards to count towards consistency and read level unstrict all ([#2687](https://github.com/m3db/m3/pull/2687)) 230 - **All**: Upgrade TChannel to v1.14 ([#2659](https://github.com/m3db/m3/pull/2659)) 231 232 # 0.15.15 233 234 ## Features 235 236 - **M3DB**: Add configuration to limit bytes read for historical metrics in a given time window ([#2627](https://github.com/m3db/m3/pull/2627)) 237 - **M3DB**: Add configuration to toggle block caching ([#2613](https://github.com/m3db/m3/pull/2613)) 238 - **M3Coordinator**: Add extended configuration of label and tag validation ([#2647](https://github.com/m3db/m3/pull/2647)) 239 240 ## Performance 241 242 - **M3DB**: Perform single pass when reading commit log entry and reuse result for second bootstrap phase ([#2645](https://github.com/m3db/m3/pull/2645)) 243 244 ## Documentation 245 246 - **M3DB**: Documentation for fileset migrations, the forward and backwards compatibility guarantees and configuring migrations ([#2630](https://github.com/m3db/m3/pull/2630)) 247 248 # 0.15.14 249 250 ## Features 251 252 - **M3Query**: Add movingSum, movingMax, movingMin Graphite functions ([#2570](https://github.com/m3db/m3/pull/2570)) 253 - **M3Query**: Add aliasByTags Graphite function ([#2626](https://github.com/m3db/m3/pull/2626)) 254 - **M3Query**: Add exponentialMovingAverage Graphite function ([#2622](https://github.com/m3db/m3/pull/2622)) 255 - **M3Query**: Add timeSlice Graphite function ([#2581](https://github.com/m3db/m3/pull/2581)) 256 - **M3Query**: Add delay Graphite function ([#2567](https://github.com/m3db/m3/pull/2567)) 257 - **M3Query**: Add aggregateWithWildcards Graphite function ([#2582](https://github.com/m3db/m3/pull/2582)) 258 - **M3Query**: Add groupByNodes Graphite function ([#2579](https://github.com/m3db/m3/pull/2579)) 259 - **M3Coordinator**: Allow using the placement set API for creating new placements as well as upserting existing placements ([#2625](https://github.com/m3db/m3/pull/2625)) 260 - **M3DB**: Add bootstrap time migration config which supports seamless backwards and forwards compatible fileset upgrades ([#2521](https://github.com/m3db/m3/pull/2521)) 261 262 # 0.15.13 263 264 ## Bug Fixes 265 266 - **M3DB**: Fix case insensitive regexp modifiers (?i) implementation for metrics selector ([#2564](https://github.com/m3db/m3/pull/2564)) 267 268 ## Misc 269 270 - **M3DB**: Expose stream batch client options to config ([#2576](https://github.com/m3db/m3/pull/2576)) 271 - **M3Query**: Metrics visibility and ability to limit number of encoders per block ([#2516](https://github.com/m3db/m3/pull/2516)) 272 273 # 0.15.12 274 275 ## Bug Fixes 276 277 - **M3Query**: Fix to Graphite movingMedian and movingAverage functions that could skip data in certain cases or cause an out of bounds error after recovery ([#2549](https://github.com/m3db/m3/pull/2549)) 278 - **M3Coordinator**: Fix a Graphite carbon ingest tag lifecycle bug that could cause duplicate tags ([#2549](https://github.com/m3db/m3/pull/2549)) 279 280 # 0.15.11 281 282 ## Features 283 284 - **M3Coordinator**: Support for remapping rules and provide tags that are auto-appended to metrics when aggregations are applied ([#2414](https://github.com/m3db/m3/pull/2414)) 285 286 ## Bug Fixes 287 288 - **M3DB**: Extend lifetime of compactable index segments for aggregate queries ([#2550](https://github.com/m3db/m3/pull/2550)) 289 290 # 0.15.10 291 292 ## Features 293 294 - **M3DB**: Add migration task for filesets from v1.0 to v1.1 ([#2520](https://github.com/m3db/m3/pull/2520)) 295 296 ## Bug Fixes 297 298 - **M3DB**: Fix enqueue readers info file reading ([#2546](https://github.com/m3db/m3/pull/2546)) 299 300 ## Documentation 301 302 - **All**: Fix buildkite mkdocs script ([#2538](https://github.com/m3db/m3/pull/2538)) 303 304 # 0.15.9 305 306 ## Performance 307 308 - **M3DB**: Background cold flush process to no longer block data snapshotting or commit log rotation ([#2508](https://github.com/m3db/m3/pull/2508)) 309 - **M3DB**: Avoid sorting index entries when reading data filesets during bootstrap when not required ([#2533](https://github.com/m3db/m3/pull/2533)) 310 311 ## Bug Fixes 312 313 - **M3Coordinator**: Respect M3Cluster headers in namespace GET ([#2518](https://github.com/m3db/m3/pull/#2518)) 314 315 ## Documentation 316 317 - **M3Aggregator**: Add M3Aggregator documentation ([#1741](https://github.com/m3db/m3/pull/1741), [#2529](https://github.com/m3db/m3/pull/2529)) 318 - **M3DB**: Bootstrapper documentation fixes ([#2510](https://github.com/m3db/m3/pull/2510)) 319 - **All**: Update mkdocs ([#2524](https://github.com/m3db/m3/pull/2524), [#2527](https://github.com/m3db/m3/pull/2527)) 320 - **All**: Add M3 meetup recordings ([#2495](https://github.com/m3db/m3/pull/2524), [#2527](https://github.com/m3db/m3/pull/2495)) 321 - **All**: Update Twitter link ([#2530](https://github.com/m3db/m3/pull/2530)) 322 - **All**: Fix spelling in FAQ ([#2448](https://github.com/m3db/m3/pull/2448)) 323 324 ## Misc 325 326 - **M3DB**: Add bootstrap migration config and options ([#2519](https://github.com/m3db/m3/pull/2519)) 327 328 # 0.15.8 329 330 ## Misc 331 332 - **M3DB**: Pause rollout of background cold flush process by revert until further testing ([6830a8cb4](https://github.com/m3db/m3/commit/6830a8cb4)) 333 334 # 0.15.7 335 336 ## Performance 337 338 - **M3DB**: Background cold flush process to no longer block data snapshotting or commit log rotation ([#2460](https://github.com/m3db/m3/pull/2460)) 339 - **M3DB**: Validate individual index entries on decode instead of entire file on open, to further improve bootstrap speed ([#2468](https://github.com/m3db/m3/pull/2468)) 340 341 ## Bug Fixes 342 343 - **M3DB**: Strict JSON unmarshal (disallow unknown fields) for raw HTTP/JSON DB node APIs ([#2490](https://github.com/m3db/m3/pull/2490)) 344 - **M3Query**: Fix to regex selectors with leading wildcard ([#2505](https://github.com/m3db/m3/pull/#2505)) 345 346 ## Documentation 347 348 - **All**: Links to M3 meetup recordings ([#2494](https://github.com/m3db/m3/pull/2494)) 349 350 # 0.15.6 351 352 ## Features 353 354 - **M3DB**: Add per-namespace indexing runtime options to define concurrency weighted to indexing ([#2446](https://github.com/m3db/m3/pull/2446)) 355 356 ## Performance 357 358 - **M3DB**: Faster bootstrapping with deferred index checksum and significantly lower memory bump at block rotation ([#2446](https://github.com/m3db/m3/pull/2446)) 359 - **M3DB**: Faster series aggregate metadata queries by intersecting postings term results with resolved query postings list ([#2441](https://github.com/m3db/m3/pull/2441)) 360 361 ## Bug Fixes 362 363 - **M3Query**: Fix for label matching behavior not the same as regular Prometheus for when label value is ".+" or ".*" ([#2479](https://github.com/m3db/m3/pull/2479)) 364 - **M3Query**: Special case when request from Go process such as Prometheus Go client library for searching series metadata using Go min/max UTC values ([#2487](https://github.com/m3db/m3/pull/2487)) 365 - **M3Query**: Auto-detect querying for entire retention time range ([#2483](https://github.com/m3db/m3/pull/2483)) 366 - **M3Query**: Fix for Graphite query for metric with single identifier and no dot separated elements ([#2450](https://github.com/m3db/m3/pull/2450)) 367 368 ## Documentation 369 370 - **M3Coordinator**: Add rollup rules example documentation ([#2461](https://github.com/m3db/m3/pull/2461), [#2462](https://github.com/m3db/m3/pull/2462)) 371 372 ## Misc 373 374 - **M3DB**: Expose cluster total shards and replicas as metrics ([#2452](https://github.com/m3db/m3/pull/2452)) 375 376 # 0.15.5 377 378 ## Documentation 379 380 - **All**: Minor documentation fixes ([#2438](https://github.com/m3db/m3/pull/2438)) 381 - **M3Query**: Add M3-Restrict-By-Tags-JSON example ([#2437](https://github.com/m3db/m3/pull/2437)) 382 383 ## Misc 384 385 - **M3DB**: Add continuous performance profiler that conditionally triggers with RPC endpoint ([#2416](https://github.com/m3db/m3/pull/2416)) 386 387 # 0.15.4 388 389 ## Features 390 391 - **M3DB**: Performance increases for block rotation by streamlining indexing lock contention ([#2423](https://github.com/m3db/m3/pull/2423)) 392 - **M3DB**: Zero-copy of ID and fields on series index metadata re-indexing ([#2423](https://github.com/m3db/m3/pull/2423)) 393 - **M3Coordinator**: Add ability to restrict and block incoming series based on tag matchers ([#2430](https://github.com/m3db/m3/pull/2430)) 394 395 ## Bug Fixes 396 397 - **M3DB**: Fix an error where background compaction caused transient errors in queries ([#2432](https://github.com/m3db/m3/pull/2432)) 398 399 ## Documentation 400 401 - **M3Query**: Update config settings and cleaned up documentation for per query limits ([#2427](https://github.com/m3db/m3/pull/2427)) 402 403 # 0.15.3 404 405 ## Features 406 407 - **M3DB**: Ability to set per-query block limit ([#2415](https://github.com/m3db/m3/pull/2415)) 408 - **M3DB**: Ability to set global per-second query limit ([#2405](https://github.com/m3db/m3/pull/2405)) 409 410 ## Bug Fixes 411 412 - **M3DB**: Fix duplicate ID insertions causing transient error when flushing index block ([#2411](https://github.com/m3db/m3/pull/2411)) 413 - **M3Coordinator**: Mapping rules with drop policies now correctly apply to unaggregated metrics ([#2262](https://github.com/m3db/m3/pull/2262)) 414 - **M3Query**: Fix incorrect starting boundaries on some temporal queries ([#2413](https://github.com/m3db/m3/pull/2413)) 415 - **M3Query**: Fix bug in one to one matching in binary functions ([#2417](https://github.com/m3db/m3/pull/2417)) 416 - **M3DB**: Fix to edge case index data consistency on flush ([#2399](https://github.com/m3db/m3/pull/2399)) 417 418 # 0.15.2 419 420 ## Bug Fixes 421 422 - **M3DB**: Fix require exhaustive propagation of require exhaustive option through RPC ([#2409](https://github.com/m3db/m3/pull/2409)) 423 424 # 0.15.1 425 426 ## Features 427 428 - **M3DB**: Add ability to return an error when max time series limit is hit instead of partial result and warning ([#2400](https://github.com/m3db/m3/pull/2400)) 429 - **M3Coordinator**: Add support for namespace retention updates by API ([#2383](https://github.com/m3db/m3/pull/2383)) 430 431 ## Bug Fixes 432 433 - **M3Coordinator**: Fix Content-Type for OpenAPI handler ([#2403](https://github.com/m3db/m3/pull/2403)) 434 - **Build**: Build release binaries with goreleaser using Go 1.13 to match Go 1.13 docker images ([#2397](https://github.com/m3db/m3/pull/2397)) 435 436 ## Misc 437 438 - **M3DB**: Report a histogram of series blocks fetched per query ([#2381](https://github.com/m3db/m3/pull/2381)) 439 440 # 0.15.0 441 442 ## Features 443 444 - **M3Ctl**: Add M3 command line tool for calling APIs and using YAML files to apply commands ([#2097](https://github.com/m3db/m3/pull/2097)) 445 - **M3Coordinator**: Add public API to write annotations (i.e. arbitrary bytes), next to datapoints for things like storing exemplars ([#2022](https://github.com/m3db/m3/pull/2022), [#2029](https://github.com/m3db/m3/pull/2029), [#2031](https://github.com/m3db/m3/pull/2031)) 446 - **M3Coordinator**: Add support for mapping rules, allowing metrics to be stored at different resolutions based on their labels/tags ([#2036](https://github.com/m3db/m3/pull/2036)) 447 - **M3Coordinator**: Add Graphite mapping rule support ([#2060](https://github.com/m3db/m3/pull/2060)) ([#2063](https://github.com/m3db/m3/pull/2063)) 448 - **M3Coordinator**: Add community contributed InfluxDB write endpoint (at /api/v1/influxdb/write) ([#2083](https://github.com/m3db/m3/pull/2083)) 449 - **M3Coordinator**: Add headers to pass along with request to remote write forward targets ([#2249](https://github.com/m3db/m3/pull/2249)) 450 - **M3Coordinator**: Add retry to remote write forward targets ([#2299](https://github.com/m3db/m3/pull/2299)) 451 - **M3Coordinator**: Add in-place M3Msg topic consumer updates with a PUT request ([#2186](https://github.com/m3db/m3/pull/2186)) 452 - **M3Coordinator**: Add ability to rewrite tags for Prometheus remote write requests using header ([#2255](https://github.com/m3db/m3/pull/2255)) 453 - **M3Coordinator**: Add config for multi-process launcher and SO_REUSEPORT listen servers for non-container based multi-process scaling ([#2292](https://github.com/m3db/m3/pull/2292)) 454 - **M3Query**: Add Prometheus engine to compliment Prometheus Remote Read, improves performance by skipping serialization/deserialization/network overhead between Prometheus and M3Query ([#2343](https://github.com/m3db/m3/pull/2343), [#2369](https://github.com/m3db/m3/pull/2369)) 455 - **M3Query**: Add header to support enforcing all queries in request to implicitly always include a given label/tag matcher ([#2053](https://github.com/m3db/m3/pull/2053)) 456 - **M3Query**: Return headers indicating incomplete results for cross-regional fanout queries when remote fails or hits a limit ([#2053](https://github.com/m3db/m3/pull/2053)) 457 - **M3Query**: Refactor query server to allow for custom handlers ([#2073](https://github.com/m3db/m3/pull/2073)) 458 - **M3Query**: Add remote read debug parameters to look at raw data for a PromQL query and/or get results as JSON ([#2276](https://github.com/m3db/m3/pull/2276)) 459 - **M3Query**: Add warnings for Prometheus queries to Prometheus query JSON response ([#2265](https://github.com/m3db/m3/pull/2265)) 460 - **M3Query**: Add ability to set default query timeout by config ([#2226](https://github.com/m3db/m3/pull/2226)) 461 - **M3Aggregator**: Add M3Msg aggregator client for high throughput point to point clustered buffered delivery of metrics to aggregator ([#2171](https://github.com/m3db/m3/pull/2171)) 462 - **M3Aggregator**: Add rollup rule support for metrics aggregated with pre-existing timestamps, such as Prometheus metrics ([#2251](https://github.com/m3db/m3/pull/2251)) 463 - **M3Aggregator**: Add aggregator passthrough functionality for aggregation in a local region forwarding to a remote region for storage ([#2235](https://github.com/m3db/m3/pull/2235)) 464 465 ## Performance 466 467 - **M3DB**: Improve RSS memory management with madvise resulting in flat RSS usage with a steady workload as time passes block-over-block ([#2037](https://github.com/m3db/m3/pull/2037)) 468 - **M3DB**: Improve bootstrapping performance by allowing bootstrapping to be performed in a single pass, now possible for a lot of bootstraps to take just minutes depending on retention ([#1989](https://github.com/m3db/m3/pull/1989)) 469 - **M3DB**: Use zero-copy references to index data instead of copy-on-read index data for each query, substantially improving query throughput and performance ([#1839](https://github.com/m3db/m3/pull/1839)) 470 - **M3DB**: Further improve peer bootstrapping performance by using a document builder rather than raw memory segments ([#2078](https://github.com/m3db/m3/pull/2078)) 471 - **M3DB**: Concurrent indexing when building segments for newly inserted metrics ([#2146](https://github.com/m3db/m3/pull/2146)) 472 - **M3DB**: Decode ReadBits decompression improvements ([#2197](https://github.com/m3db/m3/pull/2197)) 473 - **M3DB**: Remove implicit cloning of time ranges to reduce allocs ([#2178](https://github.com/m3db/m3/pull/2178)) 474 - **M3Query**: Substantially improve temporal function performance ([#2049](https://github.com/m3db/m3/pull/2049)) 475 - **M3Query**: Improve datapoint decompression speed ([#2176](https://github.com/m3db/m3/pull/2176), [#2185](https://github.com/m3db/m3/pull/2185), [#2190](https://github.com/m3db/m3/pull/2190)) 476 - **M3Query**: Read bits uses an optimized byte reader ([#2205](https://github.com/m3db/m3/pull/2205)) 477 - **M3Coordinator**: Ensure coordinator not grow M3Msg buffer if message over max size ([#2207](https://github.com/m3db/m3/pull/2207)) 478 479 ## Bug Fixes 480 481 - **M3Aggregator**: Take last value by wall clock timestamp not arrival time to avoid late arrivals overwriting actual later occuring values ([#2199](https://github.com/m3db/m3/pull/2199)) 482 - **M3DB**: Validate indexable metrics for valid utf-8 prior to insert, also includes a utility for earlier M3DB versions to remove non-utf8 index data ([#2046](https://github.com/m3db/m3/pull/2046)) 483 - **M3DB**: Remove incorrect error log message for missing schema with default non-protobuf namespaces ([#2013](https://github.com/m3db/m3/pull/2013)) 484 - **M3DB**: Fixed memory leak causing index blocks to remain in memory after flushing ([#2037](https://github.com/m3db/m3/pull/2037)) 485 - **M3DB**: Fix long standing possibility of double RLock acqusition ([#2128](https://github.com/m3db/m3/pull/2128)) 486 - **M3DB**: Remove loop in fileset writer when previous fileset encountered an error writing out index files ([#2058](https://github.com/m3db/m3/pull/2058)) 487 - **M3DB**: Instead of encountering an error skip entries for unowned shards in commit log bootstrapper ([#2145](https://github.com/m3db/m3/pull/2145)) 488 - **M3DB**: Fix to avoid returning error when missing writable bucket with a cold flush ([#2188](https://github.com/m3db/m3/pull/2188)) 489 - **M3DB**: Set defaults and expose configuration of TChannel timeouts, this avoids idle connection growth ([#2173](https://github.com/m3db/m3/pull/2173)) 490 - **M3DB**: Account for Neg/Pos Offsets when building per field roaring bitmap posting lists ([#2213](https://github.com/m3db/m3/pull/2213)) 491 - **M3DB**: Fix to build flush errors ([#2229](https://github.com/m3db/m3/pull/2229), [#2217](https://github.com/m3db/m3/pull/2217)) 492 - **M3Coordinator**: Respect env and zone headers for topic API endpoints ([#2159](https://github.com/m3db/m3/pull/2159)) 493 - **M3Coordinator**: Add support for Graphite Grafana plugin /find POST requests ([#2153](https://github.com/m3db/m3/pull/2153)) 494 - **M3Coordinator**: Use tag options specified in config with M3Msg ingester ([#2212](https://github.com/m3db/m3/pull/2212)) 495 - **M3Coordinator**: Only honor default aggregation policies if not matched by mapping rule ([#2203](https://github.com/m3db/m3/pull/2203)) 496 - **M3Query**: Fix namespace resolve debug log not being written with multiple namespaces ([#2211](https://github.com/m3db/m3/pull/2211)) 497 - **M3Query**: Fix to temporal function regression leading to inconsistent results ([#2231](https://github.com/m3db/m3/pull/2231)) 498 - **M3Query**: Fix edge cases with cross-zonal query fanout and add verify utility ([#1993](https://github.com/m3db/m3/pull/1993)) 499 - **M3Query**: Fix issue with histogram grouping ([#2247](https://github.com/m3db/m3/pull/2247)) 500 501 ## Documentation 502 503 - **M3Aggregator**: Add M3 aggregator Grafana dashboard ([#2064](https://github.com/m3db/m3/pull/2064)) 504 - **M3Coordinator**: Add documentation to write to multiple clusters from a single coordinator ([#2187](https://github.com/m3db/m3/pull/2187)) 505 - **M3DB**: Add documentation about estimating number of unique time series ([#2062](https://github.com/m3db/m3/pull/2062)) 506 - **M3DB**: Update namespace configuration documentation to use simpler duration specific keys ([#2045](https://github.com/m3db/m3/pull/2045)) 507 508 ## Misc 509 510 - **All**: Upgrade to Go 1.13 and switch dependency management to Go modules ([#2221](https://github.com/m3db/m3/pull/2221)) 511 - **All**: Add gauge metrics to measure the number of active routines for any worker pool ([#2061](https://github.com/m3db/m3/pull/2061)) 512 - **All**: Allow for ${ENV_VAR_NAME} expansion with YAML configuration files ([#2033](https://github.com/m3db/m3/pull/2033)) 513 - **All**: Add a utility for comparing performance and correctness across different versions of M3DB, enabling diffing the perf of different versions ([#2044](https://github.com/m3db/m3/pull/2044)) 514 - **All**: Upgrade etcd client library to 3.4.3 ([#2101](https://github.com/m3db/m3/pull/2101)) 515 - **All**: Include key name in watch errors ([#2138](https://github.com/m3db/m3/pull/2138)) 516 - **Development**: Add HA Prometheus lab setup for dev M3 docker compose deployment ([#2206](https://github.com/m3db/m3/pull/2206)) 517 - **Development**: Temporarily disable kubeval validation to allow builds on go 1.12 ([#2241](https://github.com/m3db/m3/pull/2241)) 518 - **Development**: Add comparator value ingester for replaying functions against given data sets ([#2224](https://github.com/m3db/m3/pull/2224)) 519 - **Development**: Logging improvements ([#2222](https://github.com/m3db/m3/pull/2222),[#2225](https://github.com/m3db/m3/pull/2225)) 520 - **M3Aggregator**: Add a datasource variable and reuse it in all the panels of the aggregator dashboard ([#2182](https://github.com/m3db/m3/pull/2182)) 521 - **M3DB**: Add client bad request/internal error distinction for metrics and sampled logs ([#2201](https://github.com/m3db/m3/pull/2201)) 522 - **M3DB**: Add latency metrics to remote reads ([#2027](https://github.com/m3db/m3/pull/2027)) 523 - **M3DB**: Add metrics for async replication worker pool utilization ([#2059](https://github.com/m3db/m3/pull/2059)) 524 - **M3DB**: Remove carbon debug flag and rely on log debug level for debugging Carbon/Graphite mapping rules ([#2024](https://github.com/m3db/m3/pull/2024)) 525 - **M3DB**: Add metric for BootstrappedAndDurable ([#2210](https://github.com/m3db/m3/pull/2210)) 526 - **M3DB**: Use madvdontneed=1 in DB nodes to get a more accurate view of memory usage ([#2242](https://github.com/m3db/m3/pull/2242)) 527 - **M3DB**: Add trace spans for database bootstrap process helping to identify all remaining slow code paths ([#2216](https://github.com/m3db/m3/pull/2216)) 528 - **M3Coordinator**: Add power user API to custom set placement goal state for cluster membership and shards ([#2108](https://github.com/m3db/m3/pull/2108)) 529 - **M3Coordinator**: Delete M3 aggregator related etcd keys when aggregator placement deleted ([#2133](https://github.com/m3db/m3/pull/2133)) 530 - **M3Coordinator**: Add metrics for remote aggregator client and downsampler ([#2165](https://github.com/m3db/m3/pull/2165)) 531 - **M3Coordinator**: Add aggregator client maxBatchSize config for configuring buffer for data sent to aggregator ([#2166](https://github.com/m3db/m3/pull/2166)) 532 - **M3Query**: Allow both GET and POST for query APIs ([#2055](https://github.com/m3db/m3/pull/2055)) 533 - **M3Query**: Only build amd64 architecture for m3query releases ([#2202](https://github.com/m3db/m3/pull/2202)) 534 535 # 0.14.2 536 537 ## Bug Fixes 538 539 - **M3DB**: Fix the persist cycle not cleaning up state for reuse when flush times cannot be calculated ([#2007](https://github.com/m3db/m3/pull/2007)) 540 - **M3Query**: Add specialized matchers for empty EQ/NEQ matchers ([#1986](https://github.com/m3db/m3/pull/1986)) 541 542 ## Misc 543 544 - **M3Aggregator**: Do not require aggregator ID to be joined with port and add instance initialization debug logs ([#2012](https://github.com/m3db/m3/pull/2012)) 545 - **All**: Support env var expansion using [go.uber.org/config](go.uber.org/config) ([#2016](https://github.com/m3db/m3/pull/2016)) 546 - **All**: Deprecate listen address expansion in favor of [go.uber.org/config](go.uber.org/config) env var expansion ([#2017](https://github.com/m3db/m3/pull/2017)) 547 548 # 0.14.1 549 550 ## Features 551 552 - **M3Query**: Add endpoint that parses query to an AST ([#2002](https://github.com/m3db/m3/pull/2002)) 553 554 ## Bug Fixes 555 556 - **M3Query**: Fix label replace lazy block function execution ([#1985](https://github.com/m3db/m3/pull/1985)) 557 558 ## Misc 559 560 - **Build**: Ensure CGO is disabled in release binaries ([#2005](https://github.com/m3db/m3/pull/2005)) 561 562 # 0.14.0 563 564 ## Features 565 566 - **M3Query**: Add a limit exceeded header if too many series are matched for a query when limit is specified ([#1838](https://github.com/m3db/m3/pull/1838)) 567 - **M3Query**: Graphite datapoint timestamps are now truncated to resolution for deterministic steps independent of start/end query time ([#1997](https://github.com/m3db/m3/pull/1997)) 568 - **M3Coordinator**: Endpoint for /debug/dump now returns placement/namespace of the current configured database ([#1981](https://github.com/m3db/m3/pull/1981)) 569 - **M3DB**: Add support for batching writes across namespaces ([#1974](https://github.com/m3db/m3/pull/1974)) 570 - **M3DB**: Add support for batching fetches across namespaces ([#1987](https://github.com/m3db/m3/pull/1987)) 571 572 ## Performance 573 574 - **M3DB**: Optimize fetches to avoid decoding more data than required to satisfy consistency ([#1966](https://github.com/m3db/m3/pull/1966)) 575 576 ## Bug Fixes 577 578 - **M3DB**: Fix merge checksumming logic ([#1988](https://github.com/m3db/m3/pull/1988)) 579 580 ## Misc 581 582 - **M3Aggregator**: Add aggregator to release binaries ([#1943](https://github.com/m3db/m3/pull/1943)) 583 584 # 0.13.0 585 586 ## Features 587 588 - **M3Query**: Federated queries (e.g. for cross region/zone queries) configurable to return a warning and partial results than a hard error to route around unhealthy regions ([#1938](https://github.com/m3db/m3/pull/1938)) 589 - **M3Query**: Add ability to turn on GRPC reflection for testing query endpoints using utilities like grpcurl ([#1856](https://github.com/m3db/m3/pull/1856)) 590 - **M3Query**: Add ability to use multiple config files for config overrides ([#1934](https://github.com/m3db/m3/pull/1934)) 591 - **M3DB**: Add ability to replicate writes best effort directly to multiple clusters by specifying multiple etcd clusters for M3DB client ([#1859](https://github.com/m3db/m3/pull/1859)) 592 - **M3Coordinator**: Add ability to replicate Prometheus remote write requests by forwarding the compressed body to downstream remote write endpoints ([#1922](https://github.com/m3db/m3/pull/1922), [#1940](https://github.com/m3db/m3/pull/1940)) 593 594 ## Bug Fixes 595 596 - **M3Coordinator**: Fix ability to delete coordinator placements ([#1918](https://github.com/m3db/m3/pull/1918)) 597 - **M3Aggregator**: Ensure flush manager worker pool is at least size 1 irregardless of num CPUs ([#1881](https://github.com/m3db/m3/pull/1881)) 598 599 ## Documentation 600 601 - **M3**: Add FAQ and seed with some frequently asked questions asking ([#1927](https://github.com/m3db/m3/pull/1927)) 602 - **M3Coordinator**: Fix the cluster placement endpoint documentation in the manual cluster setup instructions ([#1936](https://github.com/m3db/m3/pull/1936)) 603 604 # 0.12.0 605 606 ## Features 607 608 - **M3DB**: Add support for limiting the amount of outstanding repaired data that can be held in memory at once ([#1885](https://github.com/m3db/m3/pull/1885)) 609 - **M3Coordinator**: Add namespace and placement information to the dump returned by the debug endpoint ([#1896](https://github.com/m3db/m3/pull/1896)) 610 - **M3Coordinator**: Add DELETE api for the /topic endpoint ([#1926](https://github.com/m3db/m3/pull/1926)) 611 - **M3Aggregator:** Make YAML serialization roundtrip for config related types ([#1864](https://github.com/m3db/m3/pull/1864)) 612 613 ## Performance 614 615 - **M3Query**: Improve performance of temporal functions ([#1917](https://github.com/m3db/m3/pull/1917)) 616 - **M3DB**: Only trigger bootstrap during topology change if a node receives new shards. This should significantly reduce the amount of time certain topology changes (such as node removes) took when performed on larger clusters ([#1868](https://github.com/m3db/m3/pull/1868)) 617 618 # 0.11.0 619 620 ## Migration Disclaimer 621 622 Version 0.11.0 of M3 includes further work on supporting writing data at arbitrary times (within the retention period). While most of these changes are transparent to the user in terms of functionality and performance, we had to make a change to the naming format of the files that get persisted to disk ([#1720](https://github.com/m3db/m3/pull/1720)). This change was required to handle multiple fileset volumes per block, which is necessary after introducing "cold flushes" ([#1624](https://github.com/m3db/m3/pull/1624)). 623 624 The redesign is **backwards compatible** but not **forwards compatible**. This means that you should be able upgrade your < 0.11.0 clusters to 0.11.0 with no issues, but you will not be able to downgrade without taking some additional steps. 625 626 ### Troubleshooting and Rolling Back 627 628 If you run into any issues with the upgrade or need to downgrade to a previous version for any reason, follow these steps: 629 630 1. Stop the node that is having trouble with the upgrade or that you're trying to downgrade. 631 2. Remove the filesets that include a volume index in them, e.g. this is a filename with the new volumed format: `fileset-1257890400000000000-0-data.db`, and this is the corresponding filename in the original format: `fileset-1257890400000000000-data.db`. 632 3. Modify the `bootstrappers` config in the M3DB YAML file from `filesystem, commitlog, peers, uninitialized_topology` to `filesystem, peers, commitlog, uninitialized_topology`. This will force the node to bootstrap from its peers instead of the local snapshot and commitlog files it has on disk, which is important otherwise when the node restarts, it will think that it has already been bootstrapped. 633 4. Turn the node back on. 634 635 ## Features 636 637 - **M3DB**: Add cluster repair capability for non-indexed data ([#1831](https://github.com/m3db/m3/pull/1831), [#1849](https://github.com/m3db/m3/pull/1849), [#1862](https://github.com/m3db/m3/pull/1862), [#1874](https://github.com/m3db/m3/pull/1874)) 638 - **M3DB**: Enable arbitrary out of order writes for non-indexed data ([#1780](https://github.com/m3db/m3/pull/1780), [#1790](https://github.com/m3db/m3/pull/1790), [#1829](https://github.com/m3db/m3/pull/1829)) 639 - **M3DB**: Expose debug dump on /debug/dump for zip file of profiles and data about the environment for debugging ([#1811](https://github.com/m3db/m3/pull/1811)) 640 - **M3Query**: Allow lookback duration to be set in query parameters for configurable lookback windowing ([#1793](https://github.com/m3db/m3/pull/1793)) 641 642 ## Performance 643 644 - **M3DB**: Improve base line write performance using pre-encoded tags instead of encoding just before commit log write ([#1898](https://github.com/m3db/m3/pull/1898), [#1904](https://github.com/m3db/m3/pull/1904)) 645 - **M3DB**: Avoid allocating expensive digest readers in seek.Open() ([#1835](https://github.com/m3db/m3/pull/1835)) 646 647 ## Bug Fixes 648 649 - **M3Query**: Fix PromQL variadic functions with no params specified ([#1846](https://github.com/m3db/m3/pull/1846)) 650 - **M3Query**: Fix PromQL absent function ([#1871](https://github.com/m3db/m3/pull/1871)) 651 - **M3Query**: Fix PromQL binary comparisons that use the time function ([#1888](https://github.com/m3db/m3/pull/1888)) 652 - **M3Query**: Fix multi-deployment queries (e.g. cross region) using temporal functions ([#1901](https://github.com/m3db/m3/pull/1901)) 653 - **M3DB**: Index queries to local node now returns only IDs owned by the node at time of read ([#1822](https://github.com/m3db/m3/pull/1822)) 654 - **M3DB**: Automatically create folder for KV cache, avoiding common error logs ([#1757](https://github.com/m3db/m3/pull/1757)) 655 - **M3Query**: Batch remote server responses to avoid large individual RPC messages ([#1784](https://github.com/m3db/m3/pull/1784)) 656 - **M3DB**: Use rlimit and setcap together to raise file descriptor limits to desired setting if possible ([#1792](https://github.com/m3db/m3/pull/1792), [#1745](https://github.com/m3db/m3/pull/1745), [#1850](https://github.com/m3db/m3/pull/1850)) 657 658 ## Documentation 659 660 - **M3DB**: Add shard and recommended block sizes ([#1890](https://github.com/m3db/m3/pull/1890)) 661 - **M3Coordinator**: Improve namespace specific read/write header documentation ([#1884](https://github.com/m3db/m3/pull/1884)) 662 663 # 0.10.2 664 665 ## Performance 666 667 - **M3DB**: First of a set of incremental bootstrap improvements, improve commit log reader speed ([#1724](https://github.com/m3db/m3/pull/1724)) 668 669 ## Bug Fixes 670 671 - **M3Query**: Add verbose step size parsing errors to identify problematic queries ([#1734](https://github.com/m3db/m3/pull/1734)) 672 - **M3Coordinator**: Add verbose errors to identify metrics arriving too late for aggregation and make lateness policies able to be set by config ([#1731](https://github.com/m3db/m3/pull/1731)) 673 674 # 0.10.1 675 676 ## Bug Fixes 677 678 - **M3Coordinator**: Use table based approach for aggregation tile buffer past calculation with good defaults for Prometheus remote write latency ([#1717](https://github.com/m3db/m3/pull/1717)) 679 680 # 0.10.0 681 682 ## Features 683 684 - **M3Query**: Add multi-zone and multi-region configuration for coordinator ([#1687](https://github.com/m3db/m3/pull/1687)) 685 - **M3Query**: Add debug param to `GET` `/api/v1/namespace` endpoint for better readability ([#1698](https://github.com/m3db/m3/pull/1698)) 686 - **M3Coordinator**: Add "ingest_latency" histogram metric and return datapoint too old/new errors with offending timestamps ([#1716](https://github.com/m3db/m3/pull/1716)) 687 688 ## Performance 689 690 - **M3DB**: Add forward index write capability which eases index pressure at block boundaries ([#1613](https://github.com/m3db/m3/pull/1613)) 691 - **M3Query**: Drop empty series from appearing in output when `keepNaNs` option is set to disabled ([#1682](https://github.com/m3db/m3/pull/1682), [#1684](https://github.com/m3db/m3/pull/1684)) 692 - **M3DB**: Build and release M3DB with Go 1.12 ([#1674](https://github.com/m3db/m3/pull/1674)) 693 694 ## Bug Fixes 695 696 - **M3DB**: Fix a bug where peer bootstrapping would sometimes get stuck for a short period of time due to other M3DB nodes (temporarily) returning more than one block of data for a given time period ([#1707](https://github.com/m3db/m3/pull/1707)) 697 - **M3DB**: Fix a bug that would cause multiple orders of magnitude slow-down in peer bootstrapping / node replacements when one of the nodes in the cluster was hard down ([#1677](https://github.com/m3db/m3/pull/1677)) 698 - **M3Query**: Fix a bug with parsing Graphite find queries by adding `MatchField` and `MatchNotField` match types, and explicitly making use of them in graphite queries ([#1676](https://github.com/m3db/m3/pull/1676)) 699 - **M3Query**: Propagate limit settings when using Prometheus Remote Read ([#1685](https://github.com/m3db/m3/pull/1685)) 700 - **M3Coordinator**: Return a 404 rather than a 500 if we attempt to delete a nonexistent placement ([#1701](https://github.com/m3db/m3/pull/1701)) 701 - **M3Coordinator**: Return HTTP 400 if all sent samples encounter too old/new or other bad request error ([#1692](https://github.com/m3db/m3/pull/1692)) 702 703 # 0.9.6 704 705 ## Bug Fixes 706 707 - **M3DB**: Revert Docker setcap change to allow image to run with no special flags ([#1672](https://github.com/m3db/m3/pull/1672)) 708 709 # 0.9.5 710 711 ## Performance 712 713 - **M3DB**: Use ReadAt() pread syscall to reduce syscalls per time series data read and half the number of open FDs with default configuration ([#1664](https://github.com/m3db/m3/pull/1664)) 714 715 ## Bug Fixes 716 717 - **M3Query**: Add missing support for PromQL unary expressions ([#1647](https://github.com/m3db/m3/pull/1647)) 718 - **M3DB**: Call setrlimit to set open FDs hard limit for docker image and in Dockerfile use setcap to enable capability ([#1666](https://github.com/m3db/m3/pull/1666)) 719 - **M3DB**: Updated Tally to 3.3.10 to issue correct error message when listen address conflict occurs with Prometheus reporter ([#1660](https://github.com/m3db/m3/pull/1660)) 720 721 # 0.9.4 722 723 ## Bug Fixes 724 725 - **M3Query**: Adding complete tags query fanout support with integration test ([#1656](https://github.com/m3db/m3/pull/1656)) 726 727 # 0.9.3 728 729 ## Bug Fixes 730 731 - **M3Query**: Add docker integration test for query fanout and fix query aggregating results ([#1652](https://github.com/m3db/m3/pull/1652)) 732 - **M3DB**: Restore process stats disabled by the open FD counting performance fix ([#1648](https://github.com/m3db/m3/pull/1648)) 733 734 # 0.9.2 735 736 ## Performance 737 738 - **M3Query**: Add default and config to limit the number of unique time series fetch from a single DB node to fulfill a query (default 10,000) ([#1644](https://github.com/m3db/m3/pull/1644)) 739 - **M3Query**: Fix and throttle file descriptor stat emission excessive allocation, also disable Prometheus process collector which had the same issue ([#1633](https://github.com/m3db/m3/pull/1633)) 740 741 ## Bug Fixes 742 743 - **M3DB**: Fix ReturnUnfulfilledForCorruptCommitLogFiles not propagating to bootstrapper options, causing peer bootstrapper to begin when it should not ([#1639](https://github.com/m3db/m3/pull/1639)) 744 - **M3Query**: Fix PromReadInstantHandler not correctly responding with error message when query encounters error ([#1611](https://github.com/m3db/m3/pull/1611)) 745 746 # 0.9.1 747 748 ## Performance 749 750 - **M3DB**: Use index aggregate query to speed up the [/label/<label_name>/values endpoint](https://prometheus.io/docs/prometheus/latest/querying/api/#querying-label-values) ([#1628](https://github.com/m3db/m3/pull/1628)) 751 752 ## Bug Fixes 753 754 - **M3Query**: Fixed an issue causing untyped step sizes to fail parsing, now defaults to using seconds as the time unit ([#1617](https://github.com/m3db/m3/pull/1617)) 755 - **M3Query**: Accept both GET and POST for series match endpoint ([#1606](https://github.com/m3db/m3/pull/1606)) 756 757 # 0.9.0 758 759 ## Features 760 761 - **M3Coordinator**: Emit caller I.P address in error logs ([#1583](https://github.com/m3db/m3/pull/1583)) 762 - **M3Query**: Add a list tags endpoint that displays every unique tag name stored in M3DB to match the Prometheus [/labels endpoint](https://prometheus.io/docs/prometheus/latest/querying/api/#getting-label-names) ([#1565](https://github.com/m3db/m3/pull/1565)) 763 - **M3DB**: Add native support for running M3DB in "index-only" mode ([#1596](https://github.com/m3db/m3/pull/1596)) 764 - **M3DB**: Reduce M3DB log spam for normal operations like cleanup ([#1592](https://github.com/m3db/m3/pull/1592)) 765 766 ## Bug Fixes 767 768 - **M3DB**: Filter out commit log files that were not present on disk before the process started when determining if files are corrupt in the commitlog bootstrapper. This will prevent a race in which the commitlog bootstrapper would interpret the current commitlog file as corrupt ([#1581](https://github.com/m3db/m3/pull/1581)) 769 - **M3Query**: Fixed binary arithmetic to properly ignore names when checking for matching series ([#1604](https://github.com/m3db/m3/pull/1604)) 770 - **M3Query**: Add start and end time to tag values endpoint to fix an issue preventing Grafana tag completion ([#1601](https://github.com/m3db/m3/pull/1601)) 771 772 ## Performance 773 774 - **M3DB**: Optimize some types aggregation queries in M3DB to not require materializing intermediary documents. This should substantially improve query performance for certain types of aggregation queries such as those used to perform auto-complete in Prometheus and Grafana ([#1545](https://github.com/m3db/m3/pull/1545)) 775 - **M3DB**: Open new commitlog files asynchronously. This should prevent large spikes in the commitlog queue during snapshotting / commitlog rotation for write-heavy workloads ([#1576](https://github.com/m3db/m3/pull/1576)) 776 777 # 0.8.4 (2019-04-20) 778 779 ## Performance 780 781 - **M3DB**: Fix index query and aggregate query pooling using finalizers ([#1567](https://github.com/m3db/m3/pull/1567)) 782 783 ## Bug Fixes 784 785 - **M3Query**: Fix PromQL `offset` modifier implementation ([#1561](https://github.com/m3db/m3/pull/1561)) 786 - **M3DB**: Fix index flush recovery when previous index flush attempts have failed ([#1574](https://github.com/m3db/m3/pull/1574)) 787 788 # 0.8.3 (2019-04-12) 789 790 ## Performance 791 792 - **M3DB**: Bump TagLiteralLength limit to 64K ([#1552](https://github.com/m3db/m3/pull/1552)) 793 794 # 0.8.2 (2019-04-11) 795 796 ## New Features 797 798 - **M3DB**: Add Jaeger tracing to M3DB QueryIDs path ([#1506](https://github.com/m3db/m3/pull/1506)) 799 800 ## Performance 801 802 - **M3DB**: Add postingsList caching for FieldQuery ([#1530](https://github.com/m3db/m3/pull/1530)) 803 - **M3DB**: Add write & read latency histogram metrics to M3DB client and emit from M3Coordinator ([#1533](https://github.com/m3db/m3/pull/1533)) 804 - **M3DB**: Persist block at retention edge when building index after peer streaming ([#1531](https://github.com/m3db/m3/pull/1531)) 805 806 ## Bug Fixes 807 808 - **M3Query**: Fixed a bug that could cause a panic when converting queries ([#1546](https://github.com/m3db/m3/pull/1546)) 809 810 # 0.8.1 (2019-04-02) 811 812 ## Bug Fixes 813 814 - **M3DB**: Fixed a bug that would sometimes prevent successfully retrieved time series data from being loaded into the cache. 815 - **M3DB**: Fixed a bug where an error was not being properly logged. 816 817 # 0.8.0 (2019-03-29) 818 819 ## Migration Disclaimer 820 821 Version 0.8.0 of M3 switches M3DB to use file descriptors and the Read() system call for reading time series data files instead of mmaps. 822 This should improve read latency and memory utilization for some workloads, particularly those in which the amount of data on disk vastly exceeds the amount of memory on the system. 823 This changes also enables the ability to increase the fetch concurrency past the default value. 824 825 As a result of this change, M3DB will allocate significantly less mmaps, but will create a corresponding amount of file descriptors. 826 827 Operators may need to tune their kernel configuration to allow a higher number of open file descriptors. Please follow our [Kernel Configuration Guide](https://docs.m3db.io/operational_guide/kernel_configuration/) for more details. 828 829 ## New Features 830 831 - **M3DB**: Add dedicated aggregation endpoint ([#1463](https://github.com/m3db/m3/pull/1463), [#1502](https://github.com/m3db/m3/pull/1502), [#1483](https://github.com/m3db/m3/pull/1483)) 832 833 ## Performance 834 835 - **M3DB**: Use Seek() and Read() APIs for index/data files instead of mmap + share expensive seeker resources among seekers ([#1421](https://github.com/m3db/m3/pull/1421)) 836 - **M3Query**: Tag completion endpoints (Prometheus (`match[]`, `/{foo}/series`), and Graphite (`find`) endpoints) now use the new dedicated aggregation endpoint ([#1481](https://github.com/m3db/m3/pull/1481)) 837 838 # 0.7.3 (2019-03-22) 839 840 ## New Features 841 842 - **M3DB**: Add an AllQuery index query type ([#1478](https://github.com/m3db/m3/pull/1478)) 843 844 ## Bug Fixes 845 846 - **M3DB**: Fix to annotation pooling ([#1476](https://github.com/m3db/m3/pull/1476)) 847 - **M3Coordinator**: Only panics log a stacktrace rather than expected errors ([#1480](https://github.com/m3db/m3/pull/1480)) 848 849 ## Performance 850 851 - **M3DB**: Use a single results object for merging postings lists across blocks, rather than creating a result per block and discarding them immediately thereafter. This should reduce memory utilization significantly for queries that look back over a large number of blocks ([#1474](https://github.com/m3db/m3/pull/1474)) 852 - **M3DB**: Improvement to applying back pressure on writes with a full commitlog, which should improve M3DB's ability to apply back pressure under load and recover quickly after being overwhelmed with traffic. ([#1482](https://github.com/m3db/m3/pull/1482)) 853 854 # 0.7.2 (2019-03-15) 855 856 ## Bug Fixes 857 858 - **All Binaries**: Fix LD flags in release so that version, branch, build date, etc are packaged into the binary properly. 859 860 # 0.7.1 (2019-03-15) 861 862 ## New Features 863 864 - **M3Query**: Add per-query cost accounting to allow fine-grained controls on datapoint limits to Prometheus and Graphite style queries ([#1207](https://github.com/m3db/m3/pull/1207)) ([#1449](https://github.com/m3db/m3/pull/1449)) 865 - **M3Query**: Add optional pickled return type for Graphite render endpoint ([#1446](https://github.com/m3db/m3/pull/1446)) 866 - **M3Query**: Drop NaNs from query results ([#1458](https://github.com/m3db/m3/pull/1458)) 867 868 ## Bug Fixes 869 870 - **M3DB**: Fix bug in postings list caching which could cause incorrect results on queries ([#1461](https://github.com/m3db/m3/pull/1461)) 871 872 # 0.7.0 (2019-03-12) 873 874 ## Migration Disclaimer 875 876 Version 0.7.0 of M3 includes a redesign of the snapshotting and commitlog components ([#1384](https://github.com/m3db/m3/pull/1384) among others). This redesign was critical to improve M3DB's consistency guarantees, reduce the amount of disk space that is wasted by commitlogs, and enable future feature development to support writing data at arbitrary times. 877 878 The redesign is **backwards compatible** but not **forwards compatible**. This means that you should be able upgrade your < 0.7.0 clusters to 0.7.0 with no issues, but you will not be able to downgrade without taking some additional steps. Note that the first bootstrap after the upgrade may take longer than usual, but subsequent bootstraps should be just as fast as they used to be, or even faster. 879 880 ### Troubleshooting and Rolling Back 881 882 If you run into any issues with the upgrade or need to downgrade to a previous version for any reason, follow these steps: 883 884 1. Stop the node that is having trouble with the upgrade or that you're trying to downgrade. 885 2. Modify the `bootstrappers` config in the M3DB YAML file from `filesystem, commitlog, peers, uninitialized_topology` to `filesystem, peers, commitlog, uninitialized_topology`. This will force the node to bootstrap from its peers instead of the local snapshot and commitlog files it has on disk, bypassing any issues related to file incompatibility between versions. 886 3. Turn the node back on and wait for it to finish bootstrapping and snapshotting. Once everything looks stable, change the config back to `filesystem, commitlog, peers, uninitialized_topology` so that the next time the node is restarted it will default to using the snapshot and commitlog files. 887 888 ## New Features 889 890 - **M3DB**: Restructuring of commitlog and snapshotting feature as described above ([#1384](https://github.com/m3db/m3/pull/1384)) 891 - **M3DB**: Obtain a lock on data directory on startup ([#1376](https://github.com/m3db/m3/pull/1376)) 892 - **M3Coordinator**: Add support for zone / environment override headers in namespace APIs so multiple M3DB clusters can be administered from the same M3Coordinator instance / etcd cluster ([#1427](https://github.com/m3db/m3/pull/1427)) 893 - **M3Query**: Add Jaeger tracing to M3Query, and mechanism to plugin custom opentracing implementations ([#1321](https://github.com/m3db/m3/pull/1321)) 894 - **M3nsch**: Add basic Grafana dashboard showing success rates and latencies ([#1401](https://github.com/m3db/m3/pull/1401)) 895 - **M3nsch**: Support generating new unique metrics ([#1397](https://github.com/m3db/m3/pull/1397)) 896 897 ## Performance 898 899 - **M3DB**: Optimize OStream implementation which reduces CPU synchronization for each write. Should result in modest improvement in load average for metrics workloads and a massive improvement in load average for any workload using large annotations ([#1399](https://github.com/m3db/m3/pull/1399), [#1437](https://github.com/m3db/m3/pull/1437)) 900 - **M3DB**: Prevent duplicate writes from being written to the commitlog ([#1375](https://github.com/m3db/m3/pull/1375)) 901 - **M3DB**: Construct RPC service once and share it with TChannel and HTTP servers to prevent pools from being initialized multiple times reducing memory footprint ([#1420](https://github.com/m3db/m3/pull/1420)) 902 - **M3Query**: Add LRU cache for query conversion. Should help dashboards with expensive regex query in particular ([#1398](https://github.com/m3db/m3/pull/1398)) 903 904 ## Bug Fixes 905 906 - **M3Coordinator**: Better error responses from namespace APIs when namespace is unknown ([#1412](https://github.com/m3db/m3/pull/1412)) 907 - **M3Query**: Fix panic in temporal functions ([#1429](https://github.com/m3db/m3/pull/1429)) 908 909 # 0.6.1 (2019-02-20) 910 911 ## Bug fixes 912 913 - **M3Coordinator**: Fix to panic caused by generating ID for series with no tags ([#1392](https://github.com/m3db/m3/pull/1392)) 914 - **M3Coordinator**: Fix to panic caused by reading placement when none available instead of return 404 ([#1391](https://github.com/m3db/m3/pull/1391)) 915 916 # 0.6.0 (2019-02-19) 917 918 ## Breaking changes 919 920 - **M3Coordinator**: ID generation scheme must be explicitly defined in configs ([Set "legacy" if unsure, further information on migrating to 0.6.0](https://docs.m3db.io/how_to/query/#migration)) ([#1381](https://github.com/m3db/m3/pull/1381)) 921 922 ## New Features 923 924 - **M3DB** (Config): Simplify M3 config options ([#1371](https://github.com/m3db/m3/pull/1371)) 925 - **M3Coordinator**: Improve database creation API ([#1350](https://github.com/m3db/m3/pull/1350)) 926 - **M3Query**: Add quantile_over_time and histogram_quantile Prometheus functions ([#1367](https://github.com/m3db/m3/pull/1367), [#1373](https://github.com/m3db/m3/pull/1373)) 927 - **Documentation**: Additional documentation for namespace setup and configuration, etcd, and M3Coordinator ID generations schemes ([#1350](https://github.com/m3db/m3/pull/1350), [#1354](https://github.com/m3db/m3/pull/1354), [#1381](https://github.com/m3db/m3/pull/1381), [#1385](https://github.com/m3db/m3/pull/1385)) 928 929 ## Performance 930 931 - **M3DB** (Index): Add posting list cache that should result in a massive improvement for M3DB query performance for most workloads ([#1370](https://github.com/m3db/m3/pull/1370)) 932 933 ## Bug fixes 934 935 - **M3DB** (Index): Fix race condition in index query ([#1356](https://github.com/m3db/m3/pull/1356)) 936 - **M3Coordinator**: Fix panic responder which was previously not reporting query panics to users ([#1353](https://github.com/m3db/m3/pull/1353)) 937 - **M3Query**: Fix bug in calculating temporal (over_time) functions ([#1271](https://github.com/m3db/m3/pull/1271)) 938 939 # 0.5.0 (2019-02-03) 940 941 ## New Features 942 943 - **M3Coordinator**: Add [Graphite support](https://docs.m3db.io/integrations/grafana/) in the form of Carbon ingestion (with configurable aggregation and storage policies), as well as direct and Grafana based Graphite querying support ([#1309](https://github.com/m3db/m3/pull/1309), [#1310](https://github.com/m3db/m3/pull/1310), [#1308](https://github.com/m3db/m3/pull/1308), [#1319](https://github.com/m3db/m3/pull/1319), [#1318](https://github.com/m3db/m3/pull/1318), [#1327](https://github.com/m3db/m3/pull/1327), [#1328](https://github.com/m3db/m3/pull/1328)) 944 - **M3Coordinator**: Add tag completion API ([#1175](https://github.com/m3db/m3/pull/1175)) 945 - **M3Coordinator**: Add new opt-in ID generation function that will never collide ([#1286](https://github.com/m3db/m3/pull/1286)) 946 - **M3DB**: Add [endpoint](https://m3db.io/openapi/#operation/databaseConfigSetBootstrappers) for setting database bootstrapers dynamically([#1239](https://github.com/m3db/m3/pull/1239)) 947 948 ## Performance 949 950 - **M3DB** (Index) Replace usage of slow "memory segment" for index segment with immutable F.S.Ts that are constantly being generated in the foreground as series are being inserted. Significantly reduces query latency (10x+) for some types of workloads that make heavy use of regex ([#1197](https://github.com/m3db/m3/pull/1197)) 951 - **M3DB**: (Index) Add support for concurrent index block queries (improves performance of queries that look back many blocks ([#1195](https://github.com/m3db/m3/pull/1195)) 952 - **M3DB**: (Index) Improve pooling configuration of one of the index array pools to prevent (almost) unbounded growth over time for certain workloads ([#1254](https://github.com/m3db/m3/pull/1254)) 953 - **M3DB**: (Index) Use only one roaring bitmap library (Pilosa), and upgrade to version of Pilosa with our upstreammed `UnionInPlace` improvements that reduces memory consumption of expensive queries by over a factor of 10x ([#1238](https://github.com/m3db/m3/pull/1238)), fixes: [#1192](https://github.com/m3db/m3/pull/1192) 954 - **M3DB**: (Index) Don't use object pool for allocating long-lived arrays of tag slices which reduces steady-state memory consumption because the default size is 16 which is much bigger than the number of tags most metrics have ([#1300](https://github.com/m3db/m3/pull/1300)) 955 - **M3DB**: Auto-calculate size of WriteBatchPool based on commitlog queue size and improve chance of batch being returned to pool ([#1236](https://github.com/m3db/m3/pull/1236)) 956 - **M3DB**: Don't allow msgpack library to allocate \*bufio.Reader (reduces allocations) and mmap bloomfilters and index summaries as files instead of anonymously so they can be paged out by the O.S if necessary ([#1289](https://github.com/m3db/m3/pull/1289)) 957 958 ## Bug fixes 959 960 - **M3DB**: Fix bug where namespace-level configuration "writes to commitlog" was not respected ([#1232](https://github.com/m3db/m3/pull/1232)) 961 - **M3DB**: Improve how M3DB handles durability during topology changes. Previously, a new node that was added to an M3DB cluster would not be completely durable (able to recover all data from disk) for a few minutes after the node join completed even though it marked all its shards as available. Now, newly added M3DB nodes will never mark their shards as available until they are completely durable ([#1183](https://github.com/m3db/m3/pull/1183)). Also, Make M3DB health check not return success until node is bootstrapped AND durable, not just bootstrapped. This makes automated operations (like those performed by the Kubernetes operator or various scripts) much safer ([#1287](https://github.com/m3db/m3/pull/1287)) 962 - **M3DB**: Make it possible to safely replace/add/remove M3DB seed nodes by exposing etcd configuration ([#1339](https://github.com/m3db/m3/pull/1339)) 963 - **M3Coordinator**: Fix bug in database create API so it respects number of shards ([#1188](https://github.com/m3db/m3/pull/1188)) 964 - **M3Coordinator**: Fix tag propagation for temporal functions ([#1307](https://github.com/m3db/m3/pull/1307)) 965 - **M3Coordinator**: Properly propagate M3DB fetch timeout from config instead of using default value ([#1342](https://github.com/m3db/m3/pull/1342)) 966 967 # 0.4.8 (2018-10-20) 968 969 - **Coordinator**: Reduce log spam for high latency requests ([#1164](https://github.com/m3db/m3/pull/1164)) 970 - **Coordinator**: Add support for replace API for db nodes ([#1162](https://github.com/m3db/m3/pull/1162)) 971 - **Coordinator**: Add sort and sort_desc and linear regression promql functions ([#1104](https://github.com/m3db/m3/pull/1104), [#1063](https://github.com/m3db/m3/pull/1063)) 972 - **DB**: Emit logs when process limits are misconfigured ([#1118](https://github.com/m3db/m3/pull/1118)) 973 - **DB**: Change the implementation of the snapshotting process to snapshot all unflushed blocks ([#1017](https://github.com/m3db/m3/pull/1017)) 974 - **DB**: Remove CacheAllMetadata policy ([#1110](https://github.com/m3db/m3/pull/1110)) 975 - FIX **Coordinator**: Fixed bug in database creation API: error not displayed in logs ([#1089](https://github.com/m3db/m3/pull/1089)) 976 - FIX **DB**: Various changes that improve M3DBs resiliency to corrupt commitlog files during bootstrap and cleanup as well as its resiliency to corrupt fileset files ([#1065](https://github.com/m3db/m3/pull/1065), [#1066](https://github.com/m3db/m3/pull/1066), [#1086](https://github.com/m3db/m3/pull/1086)) 977 - FIX **Coordinator**: Fix OpenAPI yml, was previously broken and would not render ([#1062](https://github.com/m3db/m3/pull/1062)) 978 - PERF **DB**: Sample M3DB write method timers to improve performance for high write throughput workloads ([#1057](https://github.com/m3db/m3/pull/1057)) 979 - PERF **DB**: Massive improvement in write throughput (2-3 in some cases) by improving the speed of the commitlog msgpack encoder and removing contention on the commitlog queue via batching ([#1160](https://github.com/m3db/m3/pull/1160), [#1157](https://github.com/m3db/m3/pull/1157)) 980 981 # 0.4.7 (2018-10-10) 982 983 - **Aggregator, Collector**: Add m3aggregator and m3collector for clustered downsampling (440b41657, df3999d58, [#1030](https://github.com/m3db/m3/pull/1030), [#1038](https://github.com/m3db/m3/pull/1038), [#1050](https://github.com/m3db/m3/pull/1050), [#1061](https://github.com/m3db/m3/pull/1061)) 984 - **Coordinator**: Add m3msg server and placement and topic APIs in m3coordinator to enable use as backend with m3aggregator ([#1028](https://github.com/m3db/m3/pull/1028), [#1055](https://github.com/m3db/m3/pull/1055), [#1060](https://github.com/m3db/m3/pull/1060)) 985 - DOCS **DB**: Add doc links to placement and namespace config operational guides ([#1029](https://github.com/m3db/m3/pull/1029)) 986 987 # 0.4.6 (2018-10-05) 988 989 - **Coordinator**: Add cluster namespace fanout heuristics supporting queries greater than retention ([#908](https://github.com/m3db/m3/pull/908)) 990 - **Coordinator**: Add ability for query storage to provide unconsolidated blocks ([#929](https://github.com/m3db/m3/pull/929)) 991 - FIX **Coordinator**: Multi-fetch fixes ([#989](https://github.com/m3db/m3/pull/989)) 992 - FIX **Coordinator**: Disable CGO on linux builds ([#969](https://github.com/m3db/m3/pull/969)) 993 - FIX **Coordinator**: Write fanouts with aggregated namespaces ([#991](https://github.com/m3db/m3/pull/991)) 994 - PERF **Coordinator**: Set namespace and ID as NoFinalize ([#956](https://github.com/m3db/m3/pull/956)) 995 - PERF **Coordinator**: Treat Prometheus TagName/Value as []byte instead of String ([#1004](https://github.com/m3db/m3/pull/1004)) 996 - PERF **Coordinator**: Improve performance of generatings IDs from tags in M3Coordinator ([#1000](https://github.com/m3db/m3/pull/1000)) 997 - PERF **Coordinator**: Significantly improve performance of FetchResultToPromResult and helper functions ([#1003](https://github.com/m3db/m3/pull/1003)) 998 - PERF **Coordinator**: Improve M3DB session performance part 2: Dont clone IDs if they are IsNoFinalize() ([#986](https://github.com/m3db/m3/pull/986)) 999 - PERF **Coordinator**: Improve M3DB session performance part 1: Pool goroutines in host queues ([#985](https://github.com/m3db/m3/pull/985)) 1000 - PERF **Coordinator**: Use pooling for writes to coordinator ([#942](https://github.com/m3db/m3/pull/942)) 1001 - PERF **Coordinator**: Use a better pool size for coordinator writes ([#976](https://github.com/m3db/m3/pull/976)) 1002 - PERF **Coordinator**: Avoid duplicate ident.ID conversions ([#935](https://github.com/m3db/m3/pull/935)) 1003 - DOCS **DB**: Add placement change operational guide ([#998](https://github.com/m3db/m3/pull/998)) 1004 - DOCS **DB**: Add namespace modification operational guide ([#995](https://github.com/m3db/m3/pull/995)) 1005 - DOCS **DB**: [integrations] systemd: add config flag to service ([#974](https://github.com/m3db/m3/pull/974)) 1006 - DOCS **DB**: Add operational guides for M3DB (topology and bootstrapping) ([#924](https://github.com/m3db/m3/pull/924)) 1007 - DOCS **DB**: [integrations] systemd: add systemd unit example ([#970](https://github.com/m3db/m3/pull/970)) 1008 - DOCS **DB**: Update bootstrapping operational guide ([#967](https://github.com/m3db/m3/pull/967)) 1009 - DOCS **DB**: Timeout topology watch and return better error messages for missing topology / namespaces ([#926](https://github.com/m3db/m3/pull/926)) 1010 - DOCS **DB**: Add Prom/Grafana dashboard for M3DB as well as docker-compose for local development of m3-stack ([#939](https://github.com/m3db/m3/pull/939)) 1011 1012 # 0.4.5 (2018-09-24) 1013 1014 - FIX **DB** Index data race in FST Segment reads ([#938](https://github.com/m3db/m3/pull/938)) 1015 1016 # 0.4.4 (2018-09-21) 1017 1018 - **DB:** Use commit log bootstrapper before peers bootstrapper ([#894](https://github.com/m3db/m3/pull/894)) 1019 - **Query:** Add more PromQL functions (irate, temporal functions, etc) ([#872](https://github.com/m3db/m3/pull/872), [#897](https://github.com/m3db/m3/pull/897)) 1020 - **Coordinator:** Add downsampling to all configured namespaces by default ([#890](https://github.com/m3db/m3/pull/890)) 1021 - **DB:** By default use upsert behavior for datapoints written to mutable series ([#876](https://github.com/m3db/m3/pull/876)) 1022 - PERF **DB:** Various performance updates ([#889](https://github.com/m3db/m3/pull/889), [#903](https://github.com/m3db/m3/pull/903)) 1023 1024 # 0.4.3 (2018-09-05) 1025 1026 - **Query:** Make compatible with Grafana Prometheus data source ([#871](https://github.com/m3db/m3/pull/871), [#877](https://github.com/m3db/m3/pull/877)) 1027 - FIX **DB:** Fix index correctness for multi-segment reads ([#883](https://github.com/m3db/m3/pull/883)) 1028 - PERF **DB:** Index performance improvements ([#880](https://github.com/m3db/m3/pull/880)) 1029 1030 # 0.4.2 (2018-08-30) 1031 1032 - FIX **DB:** Remove native pooling and remove possibility of it being used by any components ([#870](https://github.com/m3db/m3/pull/870)) 1033 - FIX **DB:** Fix LRU series cache locking during very high throughput ([#862](https://github.com/m3db/m3/pull/862)) 1034 1035 # 0.4.1 (2018-08-27) 1036 1037 - FIX **Coordinator:** Add support for negated match and regexp Prometheus remote read queries ([#858](https://github.com/m3db/m3/pull/858)) 1038 1039 # 0.4.0 (2018-08-09) 1040 1041 - **Coordinator:** Add downsampling capabilities ([#796](https://github.com/m3db/m3/pull/796)) 1042 - **Query:** Add dedicated m3query service for serving PromQL and other supported query languages ([#817](https://github.com/m3db/m3/pull/817)) 1043 - **DB:** Add commit log snapshotting support to significantly reduce disk space usage by removing the majority of uncompressed commit logs on a database node and speed up bootstrap time ([#757](https://github.com/m3db/m3/pull/757)) ([#802](https://github.com/m3db/m3/pull/802)) 1044 1045 - **Coordinator:** Add downsampling capabilities ([#796](https://github.com/m3db/m3/pull/796)) 1046 - **Query:** Add dedicated m3query service for serving PromQL and other supported query languages ([#817](https://github.com/m3db/m3/pull/817)) 1047 - **DB:** Add commit log snapshotting support to significantly reduce disk space usage by removing the majority of uncompressed commit logs on a database node and speed up bootstrap time ([#757](https://github.com/m3db/m3/pull/757)) ([#802](https://github.com/m3db/m3/pull/802)) 1048 1049 # 0.3.0 (2018-07-19) 1050 1051 - **Coordinator:** Add m3coordinator Dockerfile and update Prometheus test to use standalone coordinator ([#792](https://github.com/m3db/m3/pull/792)) 1052 - **Coordinator:** Add rudimentary multi-cluster support for m3coordinator ([#785](https://github.com/m3db/m3/pull/785)) 1053 - **DB:** Fix documentation for single node walkthrough ([#795](https://github.com/m3db/m3/pull/795)) 1054 1055 # 0.2.0 (2018-05-26) 1056 1057 - **Coordinator:** Build dedicated m3coordinator binary for release binaries 1058 1059 # 0.1.0 (2018-05-24) 1060 1061 - **DB:** Reverse indexing 1062 - **Coordinator:** Prometheus remote read/write support