github.com/nsqio/nsq@v1.3.0/ChangeLog.md (about) 1 # NSQ Changelog 2 3 ## Releases 4 5 ### 1.3.0 - 2023-12-26 6 7 **Upgrading** 8 9 * #1427 / #1373 / #1371 - fix staticcheck warnings, remove support for gobindata / go 1.16 10 11 Features: 12 13 * #1473 - `nsqd`: use --tls-root-ca-file in nsqauth request (thanks @intellitrend-team) 14 * #1470 / #1469 - `nsqadmin`: upgrade supported ECMA from ES5 to ES2020 (thanks @dudleycarr) 15 * #1468 - `nsqadmin`: add paused label to topic within the node view (thanks @dudleycarr) 16 * #1462 - `nsqadmin`: add admin check for topic/node thombstone endpoint (thanks @dudleycarr) 17 * #1434 - `nsqd`: add support of unix sockets for tcp, http, https listeners (thanks @telepenin) 18 * #1424 - `nsqd`: add /debug/freememory API (thanks @guozhao-coder) 19 * #1421 - `nsqd`: nicer tls-min-version help text default 20 * #1376 - `nsqd`: allow unbuffered memory chan if ephemeral or deferred 21 * #1380 - `nsqd`: use metadata struct for both marshal and unmarshal (thanks @karalabe) 22 * #1403 - `nsqd`: /info api returns more info (thanks @arshabbir) 23 * #1384 - `nsqd`: allow disabling both HTTP and HTTPS interfaces (thanks @karalabe) 24 * #1385 - `nsqd`: enable support for TLS1.3 (thanks @karalabe) 25 * #1372 - `nsqadmin`: new flag --dev-static-dir instead of debug build tag 26 27 Bugs: 28 29 * #1478 - `Dockerfile`: remove nsswitch.conf check (thanks @dudleycarr) 30 * #1467 - `nsqadmin`: fix counter by bounding animation steps (thanks @dudleycarr) 31 * #1466 - `nsqadmin`: fix broken graph template in nsqadmin node view (thanks @dudleycarr) 32 * #1455 / #1387 - update dependencies 33 * #1445 - `nsqd`: fix unsafe concurrency read in RemoveClient (thanks @gueFDF) 34 * #1441 - `nsqd`: fix panic when statsd enabled and memstats disabled with no topics (thanks @carl-reverb) 35 * #1428 - delete `validTopicChannelNameRegex` useless escape characters (thanks @sjatsh) 36 * #1419 - contrib: update nsqadmin.cfg.example (thanks @StellarisW) 37 38 ### 1.2.1 - 2021-08-15 39 40 **Upgrading** 41 42 * #1227 - bump dependencies, officially drop `dep` support, drop Go `1.9` support 43 44 Features: 45 46 * #1347 - `nsqadmin`: switch to go:embed for static assets 47 * #1355 / #1364 - arm64 builds (thanks @danbf) 48 * #1346 - `nsqd`: ability to skip ephemeral topics/channels in statsd output 49 * #1336 / #1341 / #1343 - `nsqd`: ability to configure auth endpoint path (thanks @tufanbarisyildirim) 50 * #1307 - remove `Context` to use stdlib `context` 51 * #1295 / #1296 - switch to GitHub Actions CI 52 * #1292 - `nsqd`: minimize allocations on message send (thanks @imxyb) 53 * #1289 - optimize `uniq` (thanks @donutloop) 54 * #1230 / #1232 - `nsqd`: ability to omit memory stats from `/stats` (thanks @creker) 55 * #1226 - `nsqd`: only update internal `RDY` count for client when it changes (thanks @andyxning) 56 * #1221 / #1363 - test against more recent versions of Go 57 * #1209 - `nsqd`: bump `go-diskqueue` (interface change) (thanks @bitpeng) 58 * #1206 - prefer idiomatic `sort.Ints` over `sort.Sort` (thanks @lelenanam) 59 * #1197 / #1362 - Dockerfile: update Alpine base image, use /data by default 60 * #1178 - `nsqd`: configurable queue scan worker pool (thanks @andyxning) 61 * #1159 - `nsqd`: don't buffer messages when `--mem-queue-size=0` (thanks @bitpeng) 62 * #1073 / #1297 - `nsqd`: support separate broadcast ports for TCP and HTTP (thanks @shyam-king) 63 64 Bugs: 65 66 * #1347 - `nsqadmin`: fix graphite key for ephemeral topics/channels 67 * #765 / #1195 / #1203 / #1205 - fix build on illumos (thanks @i-sevostyanov) 68 * #1333 - fix race detector tests on non-bash shells 69 * #1330 - fix `log_level` support in configuration file (thanks @edoger) 70 * #1319 / #1331 / #1361 - `nsqd`: handle SIGTERM 71 * #1287 - `nsqadmin`: fix `--proxy-graphite` support (thanks @fanlix) 72 * #1270 / #1271 - `nsqlookupd`: fix incorrect error message for HTTP listener (thanks @TangDH03) 73 * #1264 - fix benchmark script 74 * #1251 / #1314 / #1327 - `nsqd`: fix live lock for high churn ephemeral topic/channel reconnections (thanks @slayercat) 75 * #1237 - Dockerfile: add `nsswitch.conf` to ensure go resolver uses `/etc/hosts` first 76 * #1217 / #1220 - `nsqd`: improve error message when `--data-path` does not exist (thanks @mdh67899) 77 * #1198 / #1190 / #1262 - synchronize close of all connections on Exit (thanks @benjsto) 78 * #1188 / #1189 - `nsqadmin`: fix channel delete, fix source-maps in Firefox (thanks @avtrifanov) 79 * #1186 - `nsqadmin`: fix nodes list with ipv6 addresses (thanks @andyxning) 80 81 ### 1.2.0 - 2019-08-26 82 83 **Upgrading** 84 85 * #1055 - `nsqd`: removed support for old metadata scheme used in v0.3.8 and earlier 86 * you cannot upgrade directly from v0.3.8 to v1.2.0, you must go through v1.0.0-compat or v1.1.0 87 * #1115 - manage dependencies with go modules 88 * `dep` support still present for now, but deprecated 89 90 Features: 91 92 * #1136 - `nsqd`: add `--max-channel-consumers` (default unlimited) (thanks @andyxning) 93 * #1133 - `nsqd`: add `--min-output-buffer-timeout` (default 25ms) to limit how low a timeout a consumer can request 94 * and raise default `--max-output-buffer-timeout` to 30 seconds (lower timeout, more cpu usage) 95 * #1127 - `nsqd`: add topic total message bytes to stats (thanks @andyxning) 96 * #1125 - `nsqd`: add flag to adjust default `--output-buffer-timeout` (thanks @andyxning) 97 * #1163 - `nsqd`: add random load balancing for authd requests (thanks @shenhui0509) 98 * #1119 - `nsqd`: include client TLS cert CommonName in authd requests 99 * #1147 - `nsq_to_file`: include topic/channel in most log messages 100 * #1117 - `nsq_to_file`: add `--log-level` and `--log-prefix` flags 101 * #1117/#1120/#1123 - `nsq_to_file`: big refactor, more robust file switching and syncing and error handling 102 * #1118 - `nsqd`: add param to `/stats` endpoint to allow skipping per-client stats (much faster if many clients) 103 * #1118 - `nsqadmin`, `nsq_stat`: use `include_clients` param for `/stats` for a big speedup for big clusters 104 * #1110 - `nsq_to_file`: support for separate working directory with `--work-dir` (thanks @mccutchen) 105 * #856 - `nsqadmin`: add `--base-path` flag (thanks @blinklv) 106 * #1072 - `nsq_to_http`: add `--header` flag (thanks @alwindoss) 107 * #881 - `nsqd`: add producer client tcp connections to stats (thanks @sparklxb) 108 * #1071/#1074 - `nsq_to_file`: new flag `--sync-interval` (default same as previous behavior, 30 seconds) (thanks @alpaker) 109 110 Bugs: 111 112 * #1153 - `nsqd`: close connections that don't send "magic" header (thanks @JoseFeng) 113 * #1140 - `nsqd`: exit on all fatal Accept() errors - restart enables better recovery for some conditions (thanks @mdh67899) 114 * #1140 - `nsqd`, `nsqlookupd`, `nsqadmin`: refactor LogLevel, general refactor to better exit on all fatal errors 115 * #1140 - `nsqadmin`: switch to using `judwhite/go-svc` like `nsqd` and `nsqadmin` do 116 * #1134 - `nsqadmin`: fix clients count and channel total message rate (new bugs introduced in this cycle) 117 * #1132 - `nsqd`, `nsqlookupd`, `nsqadmin`: fix http error response unreliable json serialization 118 * #1116 - `nsqlookupd`: fix orphaned ephemeral topics in registration DB 119 * #1109 - `nsqd`: fix topic message mis-counting if there are backend write errors (thanks @SwanSpouse) 120 * #1099 - `nsqlookupd`: optimize `/nodes` endpoint, much better for hundreds of nsqd (thanks @andyxning) 121 * #1085 - switch `GOFLAGS` to `BLDFLAGS` in `Makefile` now that `GOFLAGS` is automatically used by go 122 * #1080 - `nsqadmin`: eslint reported fixes/cleanups 123 124 ### 1.1.0 - 2018-08-19 125 126 **Upgrading from 1.0.0-compat**: Just a few backwards incompatible changes: 127 128 * #1056 - Removed the `nsq_pubsub` utility 129 * #873 - `nsqd` flags `--msg-timeout` and `--statsd-interval` only take duration strings 130 * plain integer no longer supported 131 * #921 - `nsqd`: http `/mpub` endpoint `binary` param interprets "0" or "false" to mean text mode 132 * previously any value meant to use binary mode instead of text mode - (thanks @andyxning) 133 134 The previous release, version "1.0.0-compat", was curiously-named to indicate an almost 135 (but not quite) complete transition to a 1.0 api-stable release line. Confusingly, this 136 follow-up release which completes the transition comes more than a year later. Because there 137 have been a fair number of changes and improvements in the past year, an additional minor 138 version bump seems appropriate. 139 140 Features: 141 142 * #874 - `nsqd`: add memory stats to http `/stats` response (thanks @sparklxb) 143 * #892 - `nsqd`, `nsqlookupd`, `nsqadmin`: add `--log-level` option (deprecating `--verbose`) (thanks @antihax) 144 * #898 - `nsqd`, `nsqlookupd`, `nsqadmin`: logging refactor to use log levels everywhere 145 * #914 - `nsqadmin`: `X-Forwarded-User` based "admin" permission (thanks @chen-anders) 146 * #929 - `nsqd`: add topic/channel filter to `/stats`, use in `nsqadmin` and `nsq_stat` for efficiency (thanks @andyxning) 147 * #936 - `nsq_to_file`: refactor/cleanup 148 * #945 - `nsq_to_nsq`: support multiple `--topic` flags (thanks @jlr52) 149 * #957 - `nsq_tail`: support multiple `--topic` flags (thanks @soar) 150 * #946 - `nsqd`, `nsqadmin`: update internal http client with new go `http.Transport` features (keepalives, timeouts, dualstack) 151 * affects metadata/stats requests between `nsqadmin`, `nsqd`, `nsqlookupd` 152 * #954 - manage dependencies with `dep` (replacing `gpm`) (thanks @judwhite) 153 * #957 - multi-stage docker image build (thanks @soar) 154 * #996 - `nsqd`: better memory usage when messages have different sizes (thanks @andyxning) 155 * #1019 - `nsqd`: optimize random channel selection in queueScanLoop (thanks @vearne) 156 * #1025 - `nsqd`: buffer and spread statsd udp sends (avoid big burst of udp, less chance of loss) 157 * #1038 - `nsqlookupd`: optimize for many producers (thousands) (thanks @andyxning) 158 * #1050/#1053 - `nsqd`: new topic can be unlocked faster after creation 159 * #1062 - `nsqadmin`: update JS deps 160 161 Bugs: 162 163 * #753 - `nsqadmin`: fix missing channels in topic list 164 * #867 - `to_nsq`: fix divide-by-zero issue when `--rate` not specified (thanks @adamweiner) 165 * #868 - `nsqd`: clamp requeue timeout to range instead of dropping connection (thanks @tsholmes) 166 * #891 - `nsqd`: fix race when client subscribes to ephemeral topic or channel while it is being cleaned up (reported by @slayercat) 167 * #927 - `nsqd`: fix deflate level handling 168 * #934 - `nsqd`: fix channel shutdown flush race 169 * #935 - `nsq_to_file`: fix connection leaks when using `--topic-pattern` (thanks @jxskiss) 170 * #951 - mention docker images and binaries for additional platforms in README (thanks @DAXaholic) 171 * #950 - `nsqlookupd`: close connection when magic read fails (thanks @yc90s) 172 * #971 - `nsqd`: fix some races getting ChannelStats (thanks @daroot) 173 * #988 - `nsqd`: fix e2e timings config example, add range validation (thanks @protoss-player) 174 * #991 - `nsq_tail`: logging to stderr (only nsq messages to stdout) 175 * #1000 - `nsq_to_http`: fix http connect/request timeout flags (thanks @kamyanskiy) 176 * #993/#1008 - `nsqd`: fix possible lookupd-identify-error busy-loop (reported by @andyxning) 177 * #1005 - `nsqadmin`: fix typo "Delfate" in connection attrs list (thanks @arussellsaw) 178 * #1032 - `nsqd`: fix loading metadata with messages queued on un-paused topic with multiple channels (thanks @michaelyou) 179 * #1004 - `nsqlookupd`: exit with error when failed to listen on ports (thanks @stephens2424) 180 * #1068 - `nsqadmin`: fix html escaping for large_graph url inside javascript 181 * misc test suite improvements and updates (go versions, tls certs, ...) 182 183 ### 1.0.0-compat - 2017-03-21 184 185 **Upgrading from 0.3.8**: Numerous backwards incompatible changes: 186 187 * Deprecated `nsqd` features removed: 188 * Pre-V1 HTTP endpoints / response format: 189 * `/{m,}put` (use `/{m,}pub`) 190 * `/{create,delete,empty,pause,unpause}_{topic,channel}` (use `/{topic,channel}/<operation>`) 191 * `--max-message-size` flag (use `--max-msg-size`) 192 * `V2` protocol `IDENTIFY` command `short_id` and `long_id` properties (use `client_id`, `hostname`, and `user_agent`) 193 * `/stats` HTTP response `name` property (use `client_id`) 194 * Deprecated `nsqlookupd` features removed: 195 * Pre-V1 HTTP endpoints / response format: 196 * `/{create,delete}_{topic,channel}` (use `/{topic,channel}/<operation>`) 197 * `/tombstone_topic_producer` (use `/topic/tombstone`) 198 * Deprecated `nsqadmin` features removed: 199 * `--template-dir` flag (not required, templates are compiled into binary) 200 * `--use-statsd-prefixes` flag (use `--statsd-counter-format` and `--statsd-gauge-format`) 201 * `nsq_stat` `--status-every` flag (use `--interval`) 202 * `--reader-opt` on all binaries that had this flag (use `--consumer-opt`) 203 * `nsq_to_file` `--gzip-compression` flag (use `--gzip-level`) 204 * `nsq_to_http` `--http-timeout` and `--http-timeout-ms` flags (use `--http-connect-timeout` and `--http-request-timeout`) 205 * `nsq_to_http` `--round-robin` flag (use `--mode=round-robin`) 206 * `nsq_to_http` `--max-backoff-duration` flag (use `--consumer-opt=max_backoff_duration,X`) 207 * `nsq_to_http` `--throttle-fraction` flag (use `--sample=X`) 208 * `nsq_to_nsq` `--max-backoff-duration` flag (use `--consumer-opt=max_backoff_duration,X`) 209 * `nsqd` `--worker-id` deprecated in favor of `--node-id` (to be fully removed in subsequent release) 210 211 This is a compatibility release that drops a wide range of previously deprecated features (#367) 212 while introducing some new deprecations (#844) that we intend to fully remove in a subsequent 1.0 213 release. 214 215 Of note, all of the pre-1.0 HTTP endpoints (and response formats) are gone. Any clients or tools 216 that use these endpoints/response formats won't work with this release. These changes have been 217 available since 0.2.29 (released in July of 2014). Clients wishing to forwards-compatibly upgrade 218 can either use the new endpoints or send the following header: 219 220 Accept: application/vnd.nsq version=1.0 221 222 Also, many command line flags have been removed — in almost all cases an alternative is available 223 with a (hopefully) more obvious name. These changes have the same affect on config file option 224 names. 225 226 On Linux, this release will automatically migrate `nsq.<worker-id>.dat` named metadata files to 227 `nsq.dat` in a way that allows users to seamlessly _downgrade_ from this release back to 0.3.8, if 228 necessary. A subsequent release will clean up these convenience symlinks and observe only 229 `nsq.dat`. See the discussion in #741 and the changes #844 for more details. 230 231 Performance wise, #741 landed which significantly reduces global contention on internal message ID 232 generation, providing a ~1.75x speed improvement on multi-topic benchmarks. 233 234 Finally, a number of minor issues were resolved spanning contributions from 9 community members! 235 Thanks! 236 237 Features: 238 239 * #766 - use `alpine` base image for official Docker container (thanks @kenjones-cisco) 240 * #775 - `nsqadmin:` `/config` API (thanks @kenjones-cisco) 241 * #776 - `nsqadmin`, `nsq_stat`, `nsq_to_file`, `nsq_to_http`: HTTP client connect/request timeouts (thanks @kenjones-cisco) 242 * #777/#778/#783/#785 - improve test coverage (thanks @kenjones-cisco) 243 * #788 - `to_nsq`: add `--rate` message throttling option 244 * #367 - purge deprecated features (see above) 245 * #741 - `nsqd`: per-topic message IDs (multi-topic pub benchmarks up to ~1.75x faster) 246 * #850 - `nsqd`, `nsqlookupd`, `nsqadmin`: add `--log-prefix` option (thanks @ploxiln) 247 * #844 - `nsqd`: deprecate `--worker-id` for `--node-id` and drop ID from `nsqd.dat` file (thanks @ploxiln) 248 249 Bugs: 250 251 * #787 - `nsqlookupd`: properly close TCP connection in `IOLoop` (thanks @JackDrogon) 252 * #792 - `nsqdmin`: fix root CA verification (thanks @joshuarubin) 253 * #794 - `nsq_to_file`: require `--topic` or `--topic-pattern` (thanks @judwhite) 254 * #816/#823 - `nsqadmin`: fix handling of IPv6 broadcast addresses (thanks @magnetised) 255 * #805/#832 - `nsqd`: fix requeue and deferred message accounting (thanks @sdbaiguanghe) 256 * #532/#830 - `nsqd`: switch to golang/snappy to fix snappy deadlock 257 * #826/#831/#837/#839 - `nsqd`: fix default `--broadcast-address` and error when `nsqlookupd` reqs fail (thanks @ploxiln @stephensearles) 258 * #822/#835 - `nsqd`: prevent panic in binary `/mpub` (thanks @yangyifeng01) 259 * #841 - `nsqadmin`: allow ctrl/meta+click to open a new tab 260 * #843 - `nsqd`: check for exit before requeing 261 262 ### 0.3.8 - 2016-05-26 263 264 **Upgrading from 0.3.7**: Binaries contain no backwards incompatible changes. 265 266 This release fixes a critical regression in `0.3.7` that could result in message loss when 267 attempting to cleanly shutdown `nsqd` by sending it a `SIGTERM`. The expected behavior was for it 268 to flush messages in internal buffers to disk before exiting. See #757 and #759 for more details. 269 270 A few performance improvements landed including #743, which improves channel throughput by ~17%, 271 and #740, which reduces garbage when reading messages from disk. 272 273 We're now stripping debug info, reducing binary size, in the official binary downloads and Windows 274 binaries are now bundled with the appropriate `.exe` extension (#726 and #751). 275 276 Features: 277 278 * #743 - `nsqd`: remove channel `messagePump` 279 * #751 - strip debug info from binaries (thanks @ploxiln) 280 * #740 - `nsqd`: reduce garbage when reading from diskqueue (thanks @dieterbe) 281 282 Bugs: 283 284 * #757/#759 - `nsqd`: properly handle `SIGTERM` (thanks @judwhite) 285 * #738 - updates for latest `go-options` 286 * #730 - `nsqd`: diskqueue sync count on both read/write 287 * #734 - `nsqadmin`: make `rate` column work without `--proxy-graphite` (thanks @ploxiln) 288 * #726 - add `.exe` extension to Windows binaries (thanks @ploxiln) 289 * #722 - `nsqadmin`: fix connected duration > `1hr` 290 291 ### 0.3.7 - 2016-02-23 292 293 **Upgrading from 0.3.6**: Binaries contain no backwards incompatible changes. 294 295 This release has been built with Go 1.6. 296 297 Highlights include the various work done to reduce `nsqd` lock contention, significantly improving 298 the impact of high load on the `/stats` endpoint, addressing issues with timeouts and failures 299 in `nsqadmin` (#700, #701, #703, #709). 300 301 Thanks to @judwhite, `nsqd` and `nsqlookupd` now natively support being run as a Windows service 302 (#718). We're also now publishing official Windows releases. 303 304 `nsqd` will now `flock` its data directory on linux, preventing two `nsqd` from running 305 simultaneously pointed at the same path (#583). 306 307 On the bugfix side, the most noteworthy change is that `nsqd` will now correctly reset health state 308 on a successful backend write (#671). 309 310 Features: 311 312 * #700/#701/#703/#709 - `nsqd`: reduce lock contention (thanks @zachbadgett @absolute8511) 313 * #718 - `nsqd`/`nsqlookupd`: support running as a windows service (thanks @judwhite) 314 * #706 - `nsqd`: support enabling/disabling block profile via HTTP (thanks @absolute8511) 315 * #710 - `nsqd`: support `POST` `/debug/pprof/symbol` (thanks @absolute8511) 316 * #662 - `nsqadmin`: add flags for formatting statsd keys (thanks @kesutton) 317 * #583 - `nsqd`: `flock` `--data-path` on linux 318 * #663 - `nsqd`: optimize GUID generation (thanks @ploxiln) 319 320 Bugs: 321 322 * #672 - `nsqd`: fix max size accounting in `diskqueue` (thanks @judwhite) 323 * #671 - `nsqd`: reset health on successful backend write (thanks @judwhite) 324 * #615 - `nsqd`: prevent OOM when reading from `nsqlookupd` peer 325 * #664/#666 - dist.sh/Makefile cleanup (thanks @ploxiln) 326 327 ### 0.3.6 - 2015-09-24 328 329 **Upgrading from 0.3.5**: Binaries contain no backwards incompatible changes. 330 331 We've adopted the [Contributor Covenant 1.2 Code of Conduct](CODE_OF_CONDUCT.md) (#593). Help us 332 keep NSQ open and inclusive by reading and following this document. 333 334 We closed a few longstanding issues related to `nsqadmin`, namely (#323, et al.) converting it to 335 an API and single-page app (so that it is _much_ easier to develop), displaying fine-grained errors 336 (#421, #657), and enabling support for `--tls-required` configurations (#396). 337 338 For `nsqd`, we added support for deferred publishing aka `DPUB` (#293), which allows a producer to 339 specify a duration of time to delay initial delivery of the message. We also addressed performance 340 issues relating to large numbers of topics/channels (#577) by removing some per-channel goroutines 341 in favor of a centralized, periodic, garbage collection approach. 342 343 In order to provide more flexibility when deploying NSQ in dynamically orchestrated topologies, 344 `nsqd` now supports the ability to configure `nsqlookupd` peers at runtime via HTTP (#601), 345 eliminating the need to restart the daemon. 346 347 As part of the large `nsqadmin` refactoring, we took the opportunity to cleanup the internals for 348 _all_ of the daemon's HTTP code paths (#601, #610, #612, #641) as well as improving the test suite 349 so that it doesn't leave around temporary files (#553). 350 351 Features: 352 353 * #593 - add code of conduct 354 * #323/#631/#632/#642/#421/#649/#650/#651/#652/#654 - `nsqadmin`: convert to API / single-page app 355 * #653 - `nsqadmin`: expand notification context 356 * #293 - `nsqd`: add deferred pub (`DPUB`) 357 * #577 - `nsqd`: drop per-channel queue workers in favor of centralized queue GC 358 * #584 - `nsqlookupd`: improve registration DB performance (thanks @xiaost) 359 * #601 - `nsqd`: HTTP endpoints to dynamically configure `nsqlookupd` peers 360 * #608 - `nsqd`: support for filtering `/stats` to topic/channel (thanks @chrusty) 361 * #601/#610/#612/#641 - improved HTTP internal routing / log HTTP requests 362 * #628 - `nsqd`: clarify help text for `--e2e-processing-latency-percentile` 363 * #640 - switch `--{consumer,producer}-opt` to `nsq.ConfigFlag` 364 365 Bugs: 366 367 * #656 - `nsqadmin`: update `statsd` prefix to `stats.counters` 368 * #421/#657 - `nsqadmin`: display upstream/partial errors 369 * #396 - `nsqdamin`/`nsqd`: support for `--tls-required` 370 * #558 - don't overwrite docker root FS 371 * #582 - `nsqd`: ignore benign EOF errors 372 * #587 - `nsqd`: GUID error handling / catch errors if GUID goes backwards (thanks @mpe) 373 * #586 - `nsqd`: fix valid range for `--worker-id` 374 * #550/#602/#617/#618/#619/#620/#622 - `nsqd`: fix benchmarks (thanks @Dieterbe) 375 * #553 - cleanup test dirs 376 * #600 - `nsqd`: enforce diskqueue min/max message size (thanks @twmb) 377 378 ### 0.3.5 - 2015-04-26 379 380 **Upgrading from 0.3.3**: Binaries contain no backwards incompatible changes. 381 382 This is another quick bug fix release to address the broken `nsqadmin` binary in the distribution 383 (see #578). 384 385 ### 0.3.4 - 2015-04-26 386 387 **WARNING**: please upgrade to `v0.3.5` to address the broken `nsqadmin` binary. 388 389 **Upgrading from 0.3.3**: Binaries contain no backwards incompatible changes. 390 391 This is a quick bug fix release to fix the outdated `go-nsq` dependency in `v0.3.3` 392 for the bundled utilities (see 6e8504e). 393 394 ### 0.3.3 - 2015-04-26 395 396 **WARNING**: please upgrade to `v0.3.5` to address the outdated `go-nsq` dependency for the 397 bundled utilities and the broken `nsqadmin` binary. 398 399 **Upgrading from 0.3.2**: Binaries contain no backwards incompatible changes. 400 401 This release is primarily a bug fix release after cleaning up and reorganizing the codebase. 402 `nsqadmin` is now importable, which paves the way for completing #323. The bundled utilities 403 received a few feature additions and bug fixes (mostly from bug fixes on the `go-nsq` side). 404 405 Features: 406 407 * #569 - `nsqadmin`: re-org into importable package 408 * #562 - `nsq_to_{nsq,http}`: add `epsilon-greedy` mode (thanks @twmb) 409 * #547 - `nsqd`: adds `start_time` to `/stats` (thanks @ShawnSpooner) 410 * #544 - `nsq_to_http`: accept any `200` response as success (thanks @mikedewar) 411 * #548 - `nsq_to_http`: read entire request body (thanks @imgix) 412 * #552/#554/#555/#556/#561 - code cleanup and `/internal` package re-org (thanks @cespare) 413 414 Bugs: 415 416 * #573 - `nsqd`: don't persist metadata upon startup (thanks @xiaost) 417 * #560 - `nsqd`: do not print `EOF` error when client closes cleanly (thanks @twmb) 418 * #557 - `nsqd`: fix `--tls-required=tcp-https` with `--tls-client-auth-policy` (thanks @twmb) 419 * #545 - enable shell expansion in official Docker image (thanks @paddyforan) 420 421 NOTE: the bundled utilities are built against [`go-nsq` `v1.0.4`][go-nsq_104] and include all of 422 those features/fixes. 423 424 [go-nsq_104]: https://github.com/nsqio/go-nsq/releases/tag/v1.0.4 425 426 ### 0.3.2 - 2015-02-08 427 428 **Upgrading from 0.3.1**: Binaries contain no backwards incompatible changes however as of this 429 release we've updated our official Docker images. 430 431 We now provide a single Docker image [`nsqio/nsq`](https://registry.hub.docker.com/r/nsqio/nsq/) 432 that includes *all* of the NSQ binaries. We did this for several reasons, primarily because the 433 tagged versions in the previous incarnation were broken (and did not actually pin to a version!). 434 The new image is an order of magnitude smaller, weighing in around 70mb. 435 436 In addition, the impetus for this quick release is to address a slew of reconnect related bug fixes 437 in the utility apps (`nsq_to_nsq`, `nsq_to_file`, etc.), for details see the [`go-nsq` `v1.0.3` 438 release notes](https://github.com/nsqio/go-nsq/releases/tag/v1.0.3). 439 440 Features: 441 442 * #534/#539/#540 - single Dockerfile approach (thanks @paddyforan) 443 444 Bugs: 445 446 * #529 - nsqadmin: fix more `#ephemeral` topic deletion issues 447 * #530 - nsqd: fix the provided sample config file (thanks @jnewmano) 448 * #538 - nsqd: fix orphaned ephemeral channels (thanks @adamsathailo) 449 450 ### 0.3.1 - 2015-01-21 451 452 **Upgrading from 0.3.0**: No backwards incompatible changes. 453 454 This release contains minor bug fixes and feature additions. 455 456 There are a number of functionality improvements to the `nsq_stat` and `nsq_to_file` helper 457 applications (and general support for `#ephemeral` topics, broken in `0.2.30`). 458 459 Additionally, the TLS options continue to improve with support for setting `--tls-min-version` and 460 a work-around for a bug relating to `TLS_FALLBACK_SCSV` ([to be fixed in Go 461 1.5](https://go-review.googlesource.com/#/c/1776/)). 462 463 Features: 464 465 * #527 - nsq_stat: deprecate `--status-every` in favor of `--interval` 466 * #524 - nsq_stat: add `--count` option (thanks @nordicdyno) 467 * #518 - nsqd: set defaults for `--tls-min-version` and set TLS max version to 1.2 468 * #475/#513/#518 - nsqd: `--tls-required` can be disabled for HTTP / add `--tls-min-version` 469 (thanks @twmb) 470 * #496 - nsq_to_file: add `<PID>` to filename and rotation by size/interval (thanks @xiaost) 471 * #507 - nsq_stat: add rates (thanks @xiaost) 472 * #505 - nsqd: speed up failure path of `BytesToBase10` (thanks @iand) 473 474 Bugs: 475 476 * #522 - nsqadmin: fix `#ephemeral` topic deletion issues 477 * #509 - nsqd: fix `diskqueue` atomic rename on Windows (thanks @allgeek) 478 * #479 - nsqd: return `output_buffer_*` resolved settings in `IDENTIFY` response (thanks @tj) 479 480 ### 0.3.0 - 2014-11-18 481 482 **Upgrading from 0.2.31**: No backwards incompatible changes. 483 484 This release includes a slew of bug fixes and few key feature additions. 485 486 The biggest functional change is that `nsqd` no longer decrements its `RDY` count for clients. This 487 means that client libraries no longer have to periodically re-send `RDY`. For some context, `nsqd` 488 already provided back-pressure due to the fact that a client must respond to messages before 489 receiving new ones. The decremented `RDY` count only made the implementation of the server and 490 client more complex without additional benefit. Now the `RDY` command can be treated as an "on/off" 491 switch. For details see #404 and the associated changes in nsqio/go-nsq#83 and nsqio/pynsq#98. 492 493 The second biggest change (and oft-requested feature!) is `#ephemeral` topics. Their behavior 494 mirrors that of channels. This feature is incredibly useful for situations where you're using 495 topics to "route" messages to consumers (like RPC) or when a backlog of messages is undesirable. 496 497 There are now scripts in the `bench` directory that automate the process of running a distributed 498 benchmark. This is a work-in-progress, but it already provides a closer-to-production setup and 499 therefore more accurate results. There's much work to do here! 500 501 A whole bunch of bugs were fixed - notably all were 3rd-party contributions! Thanks! 502 503 * #305 - `#ephemeral` topics 504 * #404/#459 - don't decr `RDY` / send `RDY` before `FIN`/`REQ` 505 * #472 - improve `nsqd` `diskqueue` sync strategies 506 * #488 - ability to filter topics by regex in `nsq_to_file` (thanks @lxfontes) 507 * #438 - distributed pub-sub benchmark scripts 508 * #448 - better `nsqd` `IOLoop` logging (thanks @rexposadas) 509 * #458 - switch to [gpm](https://github.com/pote/gpm) for builds 510 511 Bugs: 512 513 * #493 - ensure all `nsqd` `Notify()` goroutines have exited prior to shutdown (thanks @allgeek) 514 * #492 - ensure `diskqueue` syncs at end of benchmarks (thanks @Dieterbe) 515 * #490 - de-flake `TestPauseMetadata` (thanks @allgeek) 516 * #486 - require ports to be specified for daemons (thanks @jnewmano) 517 * #482 - better bash in `dist.sh` (thanks @losinggeneration) 518 * #480 - fix panic when `nsqadmin` checks stats for missing topic (thanks @jnewmano) 519 * #469 - fix panic when misbehaving client sends corrupt command (thanks @prio) 520 * #461 - fix panic when `nsqd` decodes corrupt message data (thanks @twmb) 521 * #454/#455 - fix 32-bit atomic ops in `nsq_to_nsq`/`nsq_to_http` (thanks @leshik) 522 * #451 - fix `go get` compatibility (thanks @adams-sarah) 523 524 ### 0.2.31 - 2014-08-26 525 526 **Upgrading from 0.2.30**: No backwards incompatible changes. 527 528 This release includes a few key changes. First, we improved feedback and back-pressure when `nsqd` 529 writes to disk. Previously this was asynchronous and would result in clients not knowing that their 530 `PUB` had failed. Interestingly, this refactoring improved performance of `PUB` by 41%, by removing 531 the topic's goroutine responsible for message routing in favor of `N:N` Go channel communication. 532 For details see #437. 533 534 @paddyforan contributed official Dockerfiles that are now built automatically via Docker Hub. 535 Please begin to use (and improve these) as the various older images we had been maintaining will be 536 deprecated. 537 538 The utility apps deprecated the `--reader-opt` flag in favor of `--consumer-opt` and `nsq_to_nsq` 539 and `to_nsq` received a `--producer-opt` flag, for configuring details of the connection publishing 540 to `nsqd`. Additionally, it is now possible to configure client side TLS certificates via 541 `tls_cert` and `tls_key` opts. 542 543 As usual, we fixed a few minor bugs, see below for details. 544 545 New Features / Enhancements: 546 547 * #422/#437 - `nsqd`: diskqueue error feedback/backpressure (thanks @boyand) 548 * #412 - official Dockerfiles for `nsqd`, `nsqlookupd`, `nsqadmin` (thanks @paddyforan) 549 * #442 - utilities: add `--consumer-opt` alias for `--reader-opt` and 550 add `--producer-opt` to `nsq_to_nsq` (also support configuration 551 of `tls_cert` and `tls_key`) 552 * #448 - `nsqd`: improve IOLoop error messages (thanks @rexposadas) 553 554 Bugs: 555 556 * #440 - `nsqd`: fixed statsd GC stats reporting (thanks @jphines) 557 * #434/#435 - refactored/stabilized tests and logging 558 * #429 - `nsqd`: improve handling/documentation of `--worker-id` (thanks @bschwartz) 559 * #428 - `nsqd`: `IDENTIFY` should respond with materialized `msg_timeout` (thanks @visionmedia) 560 561 ### 0.2.30 - 2014-07-28 562 563 **Upgrading from 0.2.29**: No backwards incompatible changes. 564 565 **IMPORTANT**: this is a quick bug-fix release to address a panic in `nsq_to_nsq` and 566 `nsq_to_http`, see #425. 567 568 New Features / Enhancements: 569 570 * #417 - `nsqadmin`/`nsqd`: expose TLS connection state 571 * #425 - `nsq_to_nsq`/`nsq_to_file`: display per-destination-address timings 572 573 Bugs: 574 575 * #425 - `nsq_to_nsq`/`nsq_to_file`: fix shared mutable state panic 576 577 ### 0.2.29 - 2014-07-25 578 579 **Upgrading from 0.2.28**: No backwards incompatible changes. 580 581 This release includes a slew of new features and bug fixes, with contributions from 8 582 members of the community, thanks! 583 584 The most important new feature is authentication (the `AUTH` command for `nsqd`), added in #356. 585 When `nsqd` is configured with an `--auth-http-address` it will require clients to send the `AUTH` 586 command. The `AUTH` command body is opaque to `nsqd`, it simply passes it along to the configured 587 auth daemon which responds with well formed JSON, indicating which topics/channels and properties 588 on those entities are accessible to that client (rejecting the client if it accesses anything 589 prohibited). For more details, see [the spec](https://nsq.io/clients/tcp_protocol_spec.html) or [the 590 `nsqd` guide](https://nsq.io/components/nsqd.html#auth). 591 592 Additionally, we've improved performance in a few areas. First, we refactored in-flight handling in 593 `nsqd` to reduce garbage creation and improve baseline performance 6%. End-to-end processing 594 latency calculations are also significantly faster, thanks to improvements in the 595 [`perks`](https://github.com/bmizerany/perks/pulls/7) package. 596 597 HTTP response formats have been improved (removing the redundant response wrapper) and cleaning up 598 some of the endpoint namespaces. This change is backwards compatible. Clients wishing to move 599 towards the new response format can either use the new endpoint names or send the following header: 600 601 Accept: application/vnd.nsq version=1.0 602 603 Other changes including officially bumping the character limit for topic and channel names to 64 604 (thanks @svmehta), making the `REQ` timeout limit configurable in `nsqd` (thanks @AlphaB), and 605 compiling static asset dependencies into `nsqadmin` to simplify deployment (thanks @crossjam). 606 607 Finally, `to_nsq` was added to the suite of bundled apps. It takes a stdin stream and publishes to 608 `nsqd`, an extremely flexible solution (thanks @matryer)! 609 610 As for bugs, they're mostly minor, see the pull requests referenced in the section below for 611 details. 612 613 New Features / Enhancements: 614 615 * #304 - apps: added `to_nsq` for piping stdin to NSQ (thanks @matryer) 616 * #406 - `nsqadmin`: embed external static asset dependencies (thanks @crossjam) 617 * #389 - apps: report app name and version via `user_agent` 618 * #378/#390 - `nsqd`: improve in-flight message handling (6% faster, GC reduction) 619 * #356/#370/#386 - `nsqd`: introduce `AUTH` 620 * #358 - increase topic/channel name max length to 64 (thanks @svmehta) 621 * #357 - remove internal `go-nsq` dependencies (GC reduction) 622 * #330/#366 - version HTTP endpoints, simplify response format 623 * #352 - `nsqd`: make `REQ` timeout limit configurable (thanks @AlphaB) 624 * #340 - `nsqd`: bump perks dependency (E2E performance improvement, see 25086e4) 625 626 Bugs: 627 628 * #384 - `nsqd`: fix statsd GC time reporting 629 * #407 - `nsqd`: fix double `TOUCH` and use of client's configured msg timeout 630 * #392 - `nsqadmin`: fix HTTPS warning (thanks @juliangruber) 631 * #383 - `nsqlookupd`: fix race on last update timestamp 632 * #385 - `nsqd`: properly handle empty `FIN` 633 * #365 - `nsqd`: fix `IDENTIFY` `msg_timeout` response (thanks @visionmedia) 634 * #345 - `nsq_to_file`: set proper permissions on new directories (thanks @bschwartz) 635 * #338 - `nsqd`: fix windows diskqueue filenames (thanks @politician) 636 637 ### 0.2.28 - 2014-04-28 638 639 **Upgrading from 0.2.27**: No backwards incompatible changes. We've deprecated the `short_id` 640 and `long_id` options in the `IDENTIFY` command in favor of `client_id` and `hostname`, which 641 more accurately reflect the data typically used. 642 643 This release includes a few important new features, in particular enhanced `nsqd` 644 TLS support thanks to a big contribution by @chrisroberts. 645 646 You can now *require* that clients negotiate TLS with `--tls-required` and you can configure a 647 client certificate policy via `--tls-client-auth-policy` (`require` or `require-verify`): 648 649 * `require` - the client must offer a certificate, otherwise rejected 650 * `require-verify` - the client must offer a valid certificate according to the default CA or 651 the chain specified by `--tls-root-ca-file`, otherwise rejected 652 653 This can be used as a form of client authentication. 654 655 Additionally, `nsqd` is now structured such that it is importable in other Go applications 656 via `github.com/nsqio/nsq/nsqd`, thanks to @kzvezdarov. 657 658 Finally, thanks to @paddyforan, `nsq_to_file` can now archive *multiple* topics or 659 optionally archive *all* discovered topics (by specifying no `--topic` params 660 and using `--lookupd-http-address`). 661 662 New Features / Enhancements: 663 664 * #334 - `nsq_to_file` can archive many topics (thanks @paddyforan) 665 * #327 - add `nsqd` TLS client certificate verification policy, ability 666 to require TLS, and HTTPS support (thanks @chrisroberts) 667 * #325 - make `nsqd` importable (`github.com/nsqio/nsq/nsqd`) (thanks @kzvezdarov) 668 * #321 - improve `IDENTIFY` options (replace `short_id` and `long_id` with 669 `client_id` and `hostname`) 670 * #319 - allow path separator in `nsq_to_file` filenames (thanks @jsocol) 671 * #324 - display memory depth and total depth in `nsq_stat` 672 673 Bug Fixes: 674 675 * nsqio/go-nsq#19 and nsqio/go-nsq#29 - fix deadlocks on `nsq.Reader` connection close/exit, this 676 impacts the utilities packaged with the NSQ binary 677 distribution such as `nsq_to_file`, `nsq_to_http`, 678 `nsq_to_nsq` and `nsq_tail`. 679 * #329 - use heartbeat interval for write deadline 680 * #321/#326 - improve benchmarking tests 681 * #315/#318 - fix test data races / flakiness 682 683 ### 0.2.27 - 2014-02-17 684 685 **Upgrading from 0.2.26**: No backwards incompatible changes. We deprecated `--max-message-size` 686 in favor of `--max-msg-size` for consistency with the rest of the flag names. 687 688 IMPORTANT: this is another quick bug-fix release to address an issue in `nsqadmin` where templates 689 were incompatible with older versions of Go (pre-1.2). 690 691 * #306 - fix `nsqadmin` template compatibility (and formatting) 692 * #310 - fix `nsqadmin` behavior when E2E stats are disabled 693 * #309 - fix `nsqadmin` `INVALID_ERROR` on node page tombstone link 694 * #311/#312 - fix `nsqd` client metadata race condition and test flakiness 695 * #314 - fix `nsqd` test races (run w/ `-race` and `GOMAXPROCS=4`) deprecate `--max-message-size` 696 697 ### 0.2.26 - 2014-02-06 698 699 **Upgrading from 0.2.25**: No backwards incompatible changes. 700 701 IMPORTANT: this is a quick bug-fix release to address a regression identified in `0.2.25` where 702 `statsd` prefixes were broken when using the default (or any) prefix that contained a `%s` for 703 automatic host replacement. 704 705 * #303 - fix `nsqd` `--statsd-prefix` when using `%s` host replacement 706 707 ### 0.2.25 - 2014-02-05 708 709 **Upgrading from 0.2.24**: No backwards incompatible changes. 710 711 This release adds several commonly requested features. 712 713 First, thanks to [@elubow](https://twitter.com/elubow) you can now configure your clients to sample 714 the stream they're subscribed to. To read more about the details of the implementation see #286 and 715 the original discussion in #223. Eric also contributed an improvement to `nsq_tail` to add 716 the ability to tail the last `N` messages and exit. 717 718 We added config file support ([TOML](https://github.com/mojombo/toml/blob/master/README.md)) for 719 `nsqd`, `nsqlookupd`, and `nsqadmin` - providing even more deployment flexibility. Example configs 720 are in the `contrib` directory. Command line arguments override the equivalent option in the config 721 file. 722 723 We added the ability to pause a *topic* (it is already possible to pause individual *channels*). 724 This functionality stops all message flow from topic to channel for *all channels* of a topic, 725 queueing at the topic level. This enables all kinds of interesting possibilities like atomic 726 channel renames and trivial infrastructure wide operations. 727 728 Finally, we now compile the static assets used by `nsqadmin` into the binary, simplifying 729 deployment. This means that `--template-dir` is now deprecated and will be removed in a future 730 release and you can remove the templates you previously deployed and maintained. 731 732 New Features / Enhancements: 733 734 * #286 - add client `IDENTIFY` option to sample a % of messages 735 * #279 - add TOML config file support to `nsqd`, `nsqlookupd`, and `nsqadmin` 736 * #263 - add ability to pause a topic 737 * #291 - compile templates into `nsqadmin` binary 738 * #285/#288 - `nsq_tail` support for `-n #` to get recent # messages 739 * #287/#294 - display client `IDENTIFY` attributes in `nsqadmin` (sample rate, TLS, compression) 740 * #189/#296 - add client user agent to `nsqadmin`` 741 * #297 - add `nsq_to_nsq` JSON message filtering options 742 743 ### 0.2.24 - 2013-12-07 744 745 **Upgrading from 0.2.23**: No backwards incompatible changes. However, as you'll see below, quite a 746 few command line flags to the utility apps (`nsq_to_http`, `nsq_to_file`, `nsq_to_http`) were 747 deprecated and will be removed in the next release. Please use this release to transition over to 748 the new ones. 749 750 NOTE: we are now publishing additional binaries built against go1.2 751 752 The most prominent addition is the tracking of end-to-end message processing percentiles. This 753 measures the amount of time it's taking from `PUB` to `FIN` per topic/channel. The percentiles are 754 configurable and, because there is *some* overhead in collecting this data, it can be turned off 755 entirely. Please see [the section in the docs](https://nsq.io/components/nsqd.html) for 756 implementation details. 757 758 Additionally, the utility apps received comprehensive support for all configurable reader options 759 (including compression, which was previously missing). This necessitated a bit of command line flag 760 cleanup, as follows: 761 762 #### nsq_to_file 763 764 * deprecated `--gzip-compression` in favor of `--gzip-level` 765 * deprecated `--verbose` in favor of `--reader-opt=verbose` 766 767 #### nsq_to_http 768 769 * deprecated `--throttle-fraction` in favor of `--sample` 770 * deprecated `--http-timeout-ms` in favor of `--http-timeout` (which is a 771 *duration* flag) 772 * deprecated `--verbose` in favor of `--reader-opt=verbose` 773 * deprecated `--max-backoff-duration` in favor of 774 `--reader-opt=max_backoff_duration=X` 775 776 #### nsq_to_nsq 777 778 * deprecated `--verbose` in favor of `--reader-opt=verbose` 779 * deprecated `--max-backoff-duration` in favor of 780 `--reader-opt=max_backoff_duration=X` 781 782 New Features / Enhancements: 783 784 * #280 - add end-to-end message processing latency metrics 785 * #267 - comprehensive reader command line flags for utilities 786 787 ### 0.2.23 - 2013-10-21 788 789 **Upgrading from 0.2.22**: No backwards incompatible changes. 790 791 We now use [godep](https://github.com/kr/godep) in order to achieve reproducible builds with pinned 792 dependencies. If you're on go1.1+ you can now just use `godep get github.com/nsqio/nsq/...`. 793 794 This release includes `nsqd` protocol compression feature negotiation. 795 [Snappy](https://code.google.com/p/snappy/) and [Deflate](http://en.wikipedia.org/wiki/DEFLATE) are 796 supported, clients can choose their preferred format. 797 798 `--statsd-prefix` can now be used to modify the prefix for the `statsd` keys generated by `nsqd`. 799 This is useful if you want to add datacenter prefixes or remove the default host prefix. 800 801 Finally, this release includes a "bug" fix that reduces CPU usage for `nsqd` with many clients by 802 choosing a more reasonable default for a timer used in client output buffering. For more details 803 see #236. 804 805 New Features / Enhancements: 806 807 * #266 - use godep for reproducible builds 808 * #229 - compression (Snappy/Deflate) feature negotiation 809 * #241 - binary support for HTTP /mput 810 * #269 - add --statsd-prefix flag 811 812 Bug Fixes: 813 814 * #278 - fix nsqd race for client subscription cleanup (thanks @simplereach) 815 * #277 - fix nsqadmin counter page 816 * #275 - stop accessing simplejson internals 817 * #274 - nsqd channel pause state lost during unclean restart (thanks @hailocab) 818 * #236 - reduce "idle" CPU usage by 90% with large # of clients 819 820 ### 0.2.22 - 2013-08-26 821 822 **Upgrading from 0.2.21**: message timestamps are now officially nanoseconds. The protocol docs 823 always stated this however `nsqd` was actually sending seconds. This may cause some compatibility 824 issues for client libraries/clients that were taking advantage of this field. 825 826 This release also introduces support for TLS feature negotiation in `nsqd`. Clients can optionally 827 enable TLS by using the appropriate handshake via the `IDENTIFY` command. See #227. 828 829 Significant improvements were made to the HTTP publish endpoints and in flight message handling to 830 reduce GC pressure and eliminate memory abuse vectors. See #242, #239, and #245. 831 832 This release also includes a new utility `nsq_to_nsq` for performant, low-latency, copying of an NSQ 833 topic over the TCP protocol. 834 835 Finally, a whole suite of debug HTTP endpoints were added (and consolidated) under the 836 `/debug/pprof` namespace. See #238, #248, and #252. As a result `nsqd` now supports *direct* 837 profiling via Go's `pprof` tool, ie: 838 839 $ go tool pprof --web http://ip.address:4151/debug/pprof/heap 840 841 New Features / Enhancements: 842 843 * #227 - TLS feature negotiation 844 * #238/#248/#252 - support for more HTTP debug endpoints 845 * #256 - `nsqadmin` single node view (with GC/mem graphs) 846 * #255 - `nsq_to_nsq` utility for copying a topic over TCP 847 * #230 - `nsq_to_http` takes `--content-type` flag (thanks @michaelhood) 848 * #228 - `nsqadmin` displays tombstoned topics in the `/nodes` list 849 * #242/#239/#245 - reduced GC pressure for inflight and `/mput` 850 851 Bug Fixes: 852 853 * #260 - `tombstone_topic_producer` action in `nsqadmin` missing node info 854 * #244 - fix 64bit atomic alignment issues on 32bit platforms 855 * #251 - respect configured limits for HTTP publishing 856 * #247 - publish methods should not allow 0 length messages 857 * #231/#259 - persist `nsqd` metadata on topic/channel changes 858 * #237 - fix potential memory leaks with retained channel references 859 * #232 - message timestamps are now nano 860 * #228 - `nsqlookupd`/`nsqadmin` would display inactive nodes in `/nodes` list 861 * #216 - fix edge cases in `nsq_to_file` that caused empty files 862 863 ### 0.2.21 - 2013-06-07 864 865 **Upgrading from 0.2.20**: there are no backward incompatible changes in this release. 866 867 This release introduces a significant new client feature as well as a slew of consistency and 868 recovery improvements to diskqueue. 869 870 First, we expanded the feature negotiation options for clients. There are many cases where you want 871 different output buffering semantics from `nsqd` to your client. You can now control both 872 output buffer size and the output buffer timeout via new fields in the `IDENTIFY` command. You can 873 even disable output buffering if low latency is a priority. 874 875 You can now specify a duration between fsyncs via `--sync-timeout`. This is a far better way to 876 manage when the process fsyncs messages to disk (vs the existing `--sync-every` which is based on # 877 of messages). `--sync-every` is now considered a deprecated option and will be removed in a future 878 release. 879 880 Finally, `0.2.20` introduced a significant regression in #176 where a topic would not write messages 881 to its channels. It is recommended that all users running `0.2.20` upgrade to this release. For 882 additional information see #217. 883 884 New Features / Enhancements: 885 886 * #214 - add --sync-timeout for time based fsync, improve when diskqueue syncs 887 * #196 - client configurable output buffering 888 * #190 - nsq_tail generates a random #ephemeral channel 889 890 Bug Fixes: 891 892 * #218/#220 - expose --statsd-interval for nsqadmin to handle non 60s statsd intervals 893 * #217 - fix new topic channel creation regression from #176 (thanks @elubow) 894 * #212 - dont use port in nsqadmin cookies 895 * #214 - dont open diskqueue writeFile with O_APPEND 896 * #203/#211 - diskqueue depth accounting consistency 897 * #207 - failure to write a heartbeat is fatal / reduce error log noise 898 * #206 - use broadcast address for statsd prefix 899 * #205 - cleanup example utils exit 900 901 ### 0.2.20 - 2013-05-13 902 903 **Upgrading from 0.2.19**: there are no backward incompatible changes in this release. 904 905 This release adds a couple of convenient features (such as adding the ability to empty a *topic*) 906 and continues our work to reduce garbage produced at runtime to relieve GC pressure in the Go 907 runtime. 908 909 `nsqd` now has two new flags to control the max value clients can use to set their heartbeat 910 interval as well as adjust a clients maximum RDY count. This is all set/communicated via `IDENTIFY`. 911 912 `nsqadmin` now displays `nsqd` -> `nsqlookupd` connections in the "nodes" view. This is useful for 913 visualizing how the topology is connected as well as situations where `--broadcast-address` is being 914 used incorrectly. 915 916 `nsq_to_http` now has a "host pool" mode where upstream state will be adjusted based on 917 successful/failed requests and for failures, upstreams will be exponentially backed off. This is an 918 incredibly useful routing mode. 919 920 As for bugs, we fixed an issue where "fatal" client errors were not actually being treated as fatal. 921 Under certain conditions deleting a topic would not clean up all of its files on disk. There was a 922 reported issue where the `--data-path` was not writable by the process and this was only discovered 923 after message flow began. We added a writability check at startup to improve feedback. Finally. 924 `deferred_count` was being sent as a counter value to statsd, it should be a gauge. 925 926 New Features / Enhancements: 927 928 * #197 - nsqadmin nodes list improvements (show nsqd -> lookupd conns) 929 * #192 - add golang runtime version to daemon version output 930 * #183 - ability to empty a topic 931 * #176 - optimizations to reduce garbage, copying, locking 932 * #184 - add table headers to nsqadmin channel view (thanks @elubow) 933 * #174/#186 - nsq_to_http hostpool mode and backoff control 934 * #173/#187 - nsq_stat utility for command line introspection 935 * #175 - add nsqd --max-rdy-count configuration option 936 * #178 - add nsqd --max-heartbeat-interval configuration option 937 938 Bug Fixes: 939 940 * #198 - fix fatal errors not actually being fatal 941 * #195 - fix delete topic does not delete all diskqueue files 942 * #193 - fix data race in channel requeue 943 * #185 - ensure that --data-path is writable on startup 944 * #182 - fix topic deletion ordering to prevent race conditions with lookupd/diskqueue 945 * #179 - deferred_count as gauge for statsd 946 * #173/#188/#191 - fix nsqadmin counter template error; fix nsqadmin displaying negative rates 947 948 ### 0.2.19 - 2013-04-11 949 950 **Upgrading from 0.2.18**: there are no backward incompatible changes in this release. 951 952 This release is a small release that introduces one major client side feature and resolves one 953 critical bug. 954 955 `nsqd` clients can now configure their own heartbeat interval. This is important because as of 956 `0.2.18` *all* clients (including producers) received heartbeats by default. In certain cases 957 receiving a heartbeat complicated "simple" clients that just wanted to produce messages and not 958 handle asynchronous responses. This gives flexibility for the client to decide how it would like 959 behave. 960 961 A critical bug was discovered where emptying a channel would leave client in-flight state 962 inconsistent (it would not zero) which limited deliverability of messages to those clients. 963 964 New Features / Enhancements: 965 966 * #167 - 'go get' compatibility 967 * #158 - allow nsqd clients to configure (or disable) heartbeats 968 969 Bug Fixes: 970 971 * #171 - fix race conditions identified testing against go 1.1 (scheduler improvements) 972 * #160 - empty channel left in-flight count inconsistent (thanks @dmarkham) 973 974 ### 0.2.18 - 2013-02-28 975 976 **Upgrading from 0.2.17**: all V2 clients of nsqd now receive heartbeats (previously only clients 977 that subscribed would receive heartbeats, excluding TCP *producers*). 978 979 **Upgrading from 0.2.16**: follow the notes in the 0.2.17 changelog for upgrading from 0.2.16. 980 981 Beyond the important note above regarding heartbeats this release includes `nsq_tail`, an extremely 982 useful utility application that can be used to introspect a topic on the command line. If statsd is 983 enabled (and graphite in `nsqadmin`) we added the ability to retrieve rates for display in 984 `nsqadmin`. 985 986 We resolved a few critical issues with data consistency in `nsqlookupd` when channels and topics are 987 deleted. First, deleting a topic would cause that producer to disappear from `nsqlookupd` for all 988 topics. Second, deleting a channel would cause that producer to disappear from the topic list in 989 `nsqlookupd`. 990 991 New Features / Enhancements: 992 993 * #131 - all V2 nsqd clients get heartbeats 994 * #154 - nsq_tail example reader 995 * #143 - display message rates in nsqadmin 996 997 Bug Fixes: 998 999 * #148 - store tombstone data per registration in nsqlookupd 1000 * #153 - fix large graph formulas in nsqadmin 1001 * #150/#151 - fix topics disappearing from nsqlookupd when channels are deleted 1002 1003 ### 0.2.17 - 2013-02-07 1004 1005 **Upgrading from 0.2.16**: IDENTIFY and SUB now return success responses (they previously only 1006 responded to errors). The official Go and Python libraries are forwards/backwards compatible with 1007 this change however 3rd party client libraries may not be. 1008 1009 **Upgrading from 0.2.15**: in #132 deprecations in SUB were removed as well as support for the old, 1010 line oriented, `nsqd` metadata file format. For these reasons you should upgrade to `0.2.16` first. 1011 1012 New Features / Enhancements: 1013 1014 * #119 - add TOUCH command to nsqd 1015 * #142 - add --broadcast-address flag to nsqd/nsqadmin (thanks @dustismo) 1016 * #135 - atomic MPUB 1017 * #133 - improved protocol fatal error handling and responses; IDENTIFY/SUB success responses 1018 * #118 - switch nsqadmin actions to POST and require confirmation 1019 * #117/#147 - nsqadmin action POST notifications 1020 * #122 - configurable msg size limits 1021 * #132 - deprecate identify in SUB and old nsqd metadata file format 1022 1023 Bug Fixes: 1024 1025 * #144 - empty channel should clear inflight/deferred messages 1026 * #140 - fix MPUB protocol documentation 1027 * #139 - fix nsqadmin handling of legacy statsd prefixes for graphs 1028 * #138/#145 - fix nsqadmin action redirect handling 1029 * #134 - nsqd to nsqlookupd registration fixes 1030 * #129 - nsq_to_file gzip file versioning 1031 * #106 - nsqlookupd topic producer tombstones 1032 * #100 - sane handling of diskqueue read errors 1033 * #123/#125 - fix notify related exit deadlock 1034 1035 ### 0.2.16 - 2013-01-07 1036 1037 **Upgrading from 0.2.15**: there are no backward incompatible changes in this release. 1038 1039 However, this release introduces the `IDENTIFY` command (which supersedes sending 1040 metadata along with `SUB`) for clients of `nsqd`. The old functionality will be 1041 removed in a future release. 1042 1043 * #114 persist paused channels through restart 1044 * #121 fix typo preventing compile of bench_reader (thanks @datastream) 1045 * #120 fix nsqd crash when empty command is sent (thanks @michaelhood) 1046 * #115 nsq_to_file --filename-format --datetime-format parameter and fix 1047 * #101 fix topic/channel delete operations ordering 1048 * #98 nsqadmin fixes when not using lookupd 1049 * #90/#108 performance optimizations / IDENTIFY protocol support in nsqd. For 1050 a single consumer of small messages (< 4k) increases throughput ~400% and 1051 reduces # of allocations ~30%. 1052 * #105 strftime compatible datetime format 1053 * #103 nsq_to_http handler logging 1054 * #102 compatibility with Go tip 1055 * #99 nsq_to_file --gzip flag 1056 * #95 proxy graphite requests through nsqadmin 1057 * #93 fix nqd API response for no topics 1058 * #92 graph rendering options 1059 * #86 nsq_to_http Content-Length headers 1060 * #89 gopkg doc updates 1061 * #88 move pynsq to it's own repo 1062 * #81/#87 reader improvements / introduced MPUB. Fix bug for mem-queue-size < 10 1063 * #76 statsd/graphite support 1064 * #75 administrative ability to create topics and channels 1065 1066 ### 0.2.15 - 2012-10-25 1067 1068 * #84 fix lookupd hanging on to ephemeral channels w/ no producers 1069 * #82 add /counter page to nsqadmin 1070 * #80 message size benchmark 1071 * #78 send Content-Length for nsq_to_http requests 1072 * #57/#83 documentation updates 1073 1074 ### 0.2.14 - 2012-10-19 1075 1076 * #77 ability to pause a channel (includes bugfix for message pump/diskqueue) 1077 * #74 propagate all topic changes to lookupd 1078 * #65 create binary releases 1079 1080 ### 0.2.13 - 2012-10-15 1081 1082 * #70 deadlined nsq_to_http outbound requests 1083 * #69/#72 improved nsq_to_file sync strategy 1084 * #58 diskqueue metadata bug and refactoring 1085 1086 ### 0.2.12 - 2012-10-10 1087 1088 * #63 consolidated protocol V1 into V2 and fixed PUB bug 1089 * #61 added a makefile for simpler building 1090 * #55 allow topic/channel names with `.` 1091 * combined versions for all binaries 1092 1093 ### 0.2.7 - 0.2.11 1094 1095 * Initial public release. 1096 1097 ## go-nsq Client Library 1098 1099 * #264 moved **go-nsq** to its own [repository](https://github.com/nsqio/go-nsq) 1100 1101 ## pynsq Python Client Library 1102 1103 * #88 moved **pynsq** to its own [repository](https://github.com/nsqio/pynsq)