github.com/jackc/pgx/v5@v5.5.5/CHANGELOG.md (about) 1 # 5.5.5 (March 9, 2024) 2 3 Use spaces instead of parentheses for SQL sanitization. 4 5 This still solves the problem of negative numbers creating a line comment, but this avoids breaking edge cases such as 6 `set foo to $1` where the substitution is taking place in a location where an arbitrary expression is not allowed. 7 8 # 5.5.4 (March 4, 2024) 9 10 Fix CVE-2024-27304 11 12 SQL injection can occur if an attacker can cause a single query or bind message to exceed 4 GB in size. An integer 13 overflow in the calculated message size can cause the one large message to be sent as multiple messages under the 14 attacker's control. 15 16 Thanks to Paul Gerste for reporting this issue. 17 18 * Fix behavior of CollectRows to return empty slice if Rows are empty (Felix) 19 * Fix simple protocol encoding of json.RawMessage 20 * Fix *Pipeline.getResults should close pipeline on error 21 * Fix panic in TryFindUnderlyingTypeScanPlan (David Kurman) 22 * Fix deallocation of invalidated cached statements in a transaction 23 * Handle invalid sslkey file 24 * Fix scan float4 into sql.Scanner 25 * Fix pgtype.Bits not making copy of data from read buffer. This would cause the data to be corrupted by future reads. 26 27 # 5.5.3 (February 3, 2024) 28 29 * Fix: prepared statement already exists 30 * Improve CopyFrom auto-conversion of text-ish values 31 * Add ltree type support (Florent Viel) 32 * Make some properties of Batch and QueuedQuery public (Pavlo Golub) 33 * Add AppendRows function (Edoardo Spadolini) 34 * Optimize convert UUID [16]byte to string (Kirill Malikov) 35 * Fix: LargeObject Read and Write of more than ~1GB at a time (Mitar) 36 37 # 5.5.2 (January 13, 2024) 38 39 * Allow NamedArgs to start with underscore 40 * pgproto3: Maximum message body length support (jeremy.spriet) 41 * Upgrade golang.org/x/crypto to v0.17.0 42 * Add snake_case support to RowToStructByName (Tikhon Fedulov) 43 * Fix: update description cache after exec prepare (James Hartig) 44 * Fix: pipeline checks if it is closed (James Hartig and Ryan Fowler) 45 * Fix: normalize timeout / context errors during TLS startup (Samuel Stauffer) 46 * Add OnPgError for easier centralized error handling (James Hartig) 47 48 # 5.5.1 (December 9, 2023) 49 50 * Add CopyFromFunc helper function. (robford) 51 * Add PgConn.Deallocate method that uses PostgreSQL protocol Close message. 52 * pgx uses new PgConn.Deallocate method. This allows deallocating statements to work in a failed transaction. This fixes a case where the prepared statement map could become invalid. 53 * Fix: Prefer driver.Valuer over json.Marshaler for json fields. (Jacopo) 54 * Fix: simple protocol SQL sanitizer previously panicked if an invalid $0 placeholder was used. This now returns an error instead. (maksymnevajdev) 55 * Add pgtype.Numeric.ScanScientific (Eshton Robateau) 56 57 # 5.5.0 (November 4, 2023) 58 59 * Add CollectExactlyOneRow. (Julien GOTTELAND) 60 * Add OpenDBFromPool to create *database/sql.DB from *pgxpool.Pool. (Lev Zakharov) 61 * Prepare can automatically choose statement name based on sql. This makes it easier to explicitly manage prepared statements. 62 * Statement cache now uses deterministic, stable statement names. 63 * database/sql prepared statement names are deterministically generated. 64 * Fix: SendBatch wasn't respecting context cancellation. 65 * Fix: Timeout error from pipeline is now normalized. 66 * Fix: database/sql encoding json.RawMessage to []byte. 67 * CancelRequest: Wait for the cancel request to be acknowledged by the server. This should improve PgBouncer compatibility. (Anton Levakin) 68 * stdlib: Use Ping instead of CheckConn in ResetSession 69 * Add json.Marshaler and json.Unmarshaler for Float4, Float8 (Kirill Mironov) 70 71 # 5.4.3 (August 5, 2023) 72 73 * Fix: QCharArrayOID was defined with the wrong OID (Christoph Engelbert) 74 * Fix: connect_timeout for sslmode=allow|prefer (smaher-edb) 75 * Fix: pgxpool: background health check cannot overflow pool 76 * Fix: Check for nil in defer when sending batch (recover properly from panic) 77 * Fix: json scan of non-string pointer to pointer 78 * Fix: zeronull.Timestamptz should use pgtype.Timestamptz 79 * Fix: NewConnsCount was not correctly counting connections created by Acquire directly. (James Hartig) 80 * RowTo(AddrOf)StructByPos ignores fields with "-" db tag 81 * Optimization: improve text format numeric parsing (horpto) 82 83 # 5.4.2 (July 11, 2023) 84 85 * Fix: RowScanner errors are fatal to Rows 86 * Fix: Enable failover efforts when pg_hba.conf disallows non-ssl connections (Brandon Kauffman) 87 * Hstore text codec internal improvements (Evan Jones) 88 * Fix: Stop timers for background reader when not in use. Fixes memory leak when closing connections (Adrian-Stefan Mares) 89 * Fix: Stop background reader as soon as possible. 90 * Add PgConn.SyncConn(). This combined with the above fix makes it safe to directly use the underlying net.Conn. 91 92 # 5.4.1 (June 18, 2023) 93 94 * Fix: concurrency bug with pgtypeDefaultMap and simple protocol (Lev Zakharov) 95 * Add TxOptions.BeginQuery to allow overriding the default BEGIN query 96 97 # 5.4.0 (June 14, 2023) 98 99 * Replace platform specific syscalls for non-blocking IO with more traditional goroutines and deadlines. This returns to the v4 approach with some additional improvements and fixes. This restores the ability to use a pgx.Conn over an ssh.Conn as well as other non-TCP or Unix socket connections. In addition, it is a significantly simpler implementation that is less likely to have cross platform issues. 100 * Optimization: The default type registrations are now shared among all connections. This saves about 100KB of memory per connection. `pgtype.Type` and `pgtype.Codec` values are now required to be immutable after registration. This was already necessary in most cases but wasn't documented until now. (Lev Zakharov) 101 * Fix: Ensure pgxpool.Pool.QueryRow.Scan releases connection on panic 102 * CancelRequest: don't try to read the reply (Nicola Murino) 103 * Fix: correctly handle bool type aliases (Wichert Akkerman) 104 * Fix: pgconn.CancelRequest: Fix unix sockets: don't use RemoteAddr() 105 * Fix: pgx.Conn memory leak with prepared statement caching (Evan Jones) 106 * Add BeforeClose to pgxpool.Pool (Evan Cordell) 107 * Fix: various hstore fixes and optimizations (Evan Jones) 108 * Fix: RowToStructByPos with embedded unexported struct 109 * Support different bool string representations (Lev Zakharov) 110 * Fix: error when using BatchResults.Exec on a select that returns an error after some rows. 111 * Fix: pipelineBatchResults.Exec() not returning error from ResultReader 112 * Fix: pipeline batch results not closing pipeline when error occurs while reading directly from results instead of using 113 a callback. 114 * Fix: scanning a table type into a struct 115 * Fix: scan array of record to pointer to slice of struct 116 * Fix: handle null for json (Cemre Mengu) 117 * Batch Query callback is called even when there is an error 118 * Add RowTo(AddrOf)StructByNameLax (Audi P. Risa P) 119 120 # 5.3.1 (February 27, 2023) 121 122 * Fix: Support v4 and v5 stdlib in same program (Tomáš Procházka) 123 * Fix: sql.Scanner not being used in certain cases 124 * Add text format jsonpath support 125 * Fix: fake non-blocking read adaptive wait time 126 127 # 5.3.0 (February 11, 2023) 128 129 * Fix: json values work with sql.Scanner 130 * Fixed / improved error messages (Mark Chambers and Yevgeny Pats) 131 * Fix: support scan into single dimensional arrays 132 * Fix: MaxConnLifetimeJitter setting actually jitter (Ben Weintraub) 133 * Fix: driver.Value representation of bytea should be []byte not string 134 * Fix: better handling of unregistered OIDs 135 * CopyFrom can use query cache to avoid extra round trip to get OIDs (Alejandro Do Nascimento Mora) 136 * Fix: encode to json ignoring driver.Valuer 137 * Support sql.Scanner on renamed base type 138 * Fix: pgtype.Numeric text encoding of negative numbers (Mark Chambers) 139 * Fix: connect with multiple hostnames when one can't be resolved 140 * Upgrade puddle to remove dependency on uber/atomic and fix alignment issue on 32-bit platform 141 * Fix: scanning json column into **string 142 * Multiple reductions in memory allocations 143 * Fake non-blocking read adapts its max wait time 144 * Improve CopyFrom performance and reduce memory usage 145 * Fix: encode []any to array 146 * Fix: LoadType for composite with dropped attributes (Felix Röhrich) 147 * Support v4 and v5 stdlib in same program 148 * Fix: text format array decoding with string of "NULL" 149 * Prefer binary format for arrays 150 151 # 5.2.0 (December 5, 2022) 152 153 * `tracelog.TraceLog` implements the pgx.PrepareTracer interface. (Vitalii Solodilov) 154 * Optimize creating begin transaction SQL string (Petr Evdokimov and ksco) 155 * `Conn.LoadType` supports range and multirange types (Vitalii Solodilov) 156 * Fix scan `uint` and `uint64` `ScanNumeric`. This resolves a PostgreSQL `numeric` being incorrectly scanned into `uint` and `uint64`. 157 158 # 5.1.1 (November 17, 2022) 159 160 * Fix simple query sanitizer where query text contains a Unicode replacement character. 161 * Remove erroneous `name` argument from `DeallocateAll()`. Technically, this is a breaking change, but given that method was only added 5 days ago this change was accepted. (Bodo Kaiser) 162 163 # 5.1.0 (November 12, 2022) 164 165 * Update puddle to v2.1.2. This resolves a race condition and a deadlock in pgxpool. 166 * `QueryRewriter.RewriteQuery` now returns an error. Technically, this is a breaking change for any external implementers, but given the minimal likelihood that there are actually any external implementers this change was accepted. 167 * Expose `GetSSLPassword` support to pgx. 168 * Fix encode `ErrorResponse` unknown field handling. This would only affect pgproto3 being used directly as a proxy with a non-PostgreSQL server that included additional error fields. 169 * Fix date text format encoding with 5 digit years. 170 * Fix date values passed to a `sql.Scanner` as `string` instead of `time.Time`. 171 * DateCodec.DecodeValue can return `pgtype.InfinityModifier` instead of `string` for infinite values. This now matches the behavior of the timestamp types. 172 * Add domain type support to `Conn.LoadType()`. 173 * Add `RowToStructByName` and `RowToAddrOfStructByName`. (Pavlo Golub) 174 * Add `Conn.DeallocateAll()` to clear all prepared statements including the statement cache. (Bodo Kaiser) 175 176 # 5.0.4 (October 24, 2022) 177 178 * Fix: CollectOneRow prefers PostgreSQL error over pgx.ErrorNoRows 179 * Fix: some reflect Kind checks to first check for nil 180 * Bump golang.org/x/text dependency to placate snyk 181 * Fix: RowToStructByPos on structs with multiple anonymous sub-structs (Baptiste Fontaine) 182 * Fix: Exec checks if tx is closed 183 184 # 5.0.3 (October 14, 2022) 185 186 * Fix `driver.Valuer` handling edge cases that could cause infinite loop or crash 187 188 # v5.0.2 (October 8, 2022) 189 190 * Fix date encoding in text format to always use 2 digits for month and day 191 * Prefer driver.Valuer over wrap plans when encoding 192 * Fix scan to pointer to pointer to renamed type 193 * Allow scanning NULL even if PG and Go types are incompatible 194 195 # v5.0.1 (September 24, 2022) 196 197 * Fix 32-bit atomic usage 198 * Add MarshalJSON for Float8 (yogipristiawan) 199 * Add `[` and `]` to text encoding of `Lseg` 200 * Fix sqlScannerWrapper NULL handling 201 202 # v5.0.0 (September 17, 2022) 203 204 ## Merged Packages 205 206 `github.com/jackc/pgtype`, `github.com/jackc/pgconn`, and `github.com/jackc/pgproto3` are now included in the main 207 `github.com/jackc/pgx` repository. Previously there was confusion as to where issues should be reported, additional 208 release work due to releasing multiple packages, and less clear changelogs. 209 210 ## pgconn 211 212 `CommandTag` is now an opaque type instead of directly exposing an underlying `[]byte`. 213 214 The return value `ResultReader.Values()` is no longer safe to retain a reference to after a subsequent call to `NextRow()` or `Close()`. 215 216 `Trace()` method adds low level message tracing similar to the `PQtrace` function in `libpq`. 217 218 pgconn now uses non-blocking IO. This is a significant internal restructuring, but it should not cause any visible changes on its own. However, it is important in implementing other new features. 219 220 `CheckConn()` checks a connection's liveness by doing a non-blocking read. This can be used to detect database restarts or network interruptions without executing a query or a ping. 221 222 pgconn now supports pipeline mode. 223 224 `*PgConn.ReceiveResults` removed. Use pipeline mode instead. 225 226 `Timeout()` no longer considers `context.Canceled` as a timeout error. `context.DeadlineExceeded` still is considered a timeout error. 227 228 ## pgxpool 229 230 `Connect` and `ConnectConfig` have been renamed to `New` and `NewWithConfig` respectively. The `LazyConnect` option has been removed. Pools always lazily connect. 231 232 ## pgtype 233 234 The `pgtype` package has been significantly changed. 235 236 ### NULL Representation 237 238 Previously, types had a `Status` field that could be `Undefined`, `Null`, or `Present`. This has been changed to a 239 `Valid` `bool` field to harmonize with how `database/sql` represents `NULL` and to make the zero value useable. 240 241 Previously, a type that implemented `driver.Valuer` would have the `Value` method called even on a nil pointer. All nils 242 whether typed or untyped now represent `NULL`. 243 244 ### Codec and Value Split 245 246 Previously, the type system combined decoding and encoding values with the value types. e.g. Type `Int8` both handled 247 encoding and decoding the PostgreSQL representation and acted as a value object. This caused some difficulties when 248 there was not an exact 1 to 1 relationship between the Go types and the PostgreSQL types For example, scanning a 249 PostgreSQL binary `numeric` into a Go `float64` was awkward (see https://github.com/jackc/pgtype/issues/147). This 250 concepts have been separated. A `Codec` only has responsibility for encoding and decoding values. Value types are 251 generally defined by implementing an interface that a particular `Codec` understands (e.g. `PointScanner` and 252 `PointValuer` for the PostgreSQL `point` type). 253 254 ### Array Types 255 256 All array types are now handled by `ArrayCodec` instead of using code generation for each new array type. This also 257 means that less common array types such as `point[]` are now supported. `Array[T]` supports PostgreSQL multi-dimensional 258 arrays. 259 260 ### Composite Types 261 262 Composite types must be registered before use. `CompositeFields` may still be used to construct and destruct composite 263 values, but any type may now implement `CompositeIndexGetter` and `CompositeIndexScanner` to be used as a composite. 264 265 ### Range Types 266 267 Range types are now handled with types `RangeCodec` and `Range[T]`. This allows additional user defined range types to 268 easily be handled. Multirange types are handled similarly with `MultirangeCodec` and `Multirange[T]`. 269 270 ### pgxtype 271 272 `LoadDataType` moved to `*Conn` as `LoadType`. 273 274 ### Bytea 275 276 The `Bytea` and `GenericBinary` types have been replaced. Use the following instead: 277 278 * `[]byte` - For normal usage directly use `[]byte`. 279 * `DriverBytes` - Uses driver memory only available until next database method call. Avoids a copy and an allocation. 280 * `PreallocBytes` - Uses preallocated byte slice to avoid an allocation. 281 * `UndecodedBytes` - Avoids any decoding. Allows working with raw bytes. 282 283 ### Dropped lib/pq Support 284 285 `pgtype` previously supported and was tested against [lib/pq](https://github.com/lib/pq). While it will continue to work 286 in most cases this is no longer supported. 287 288 ### database/sql Scan 289 290 Previously, most `Scan` implementations would convert `[]byte` to `string` automatically to decode a text value. Now 291 only `string` is handled. This is to allow the possibility of future binary support in `database/sql` mode by 292 considering `[]byte` to be binary format and `string` text format. This change should have no effect for any use with 293 `pgx`. The previous behavior was only necessary for `lib/pq` compatibility. 294 295 Added `*Map.SQLScanner` to create a `sql.Scanner` for types such as `[]int32` and `Range[T]` that do not implement 296 `sql.Scanner` directly. 297 298 ### Number Type Fields Include Bit size 299 300 `Int2`, `Int4`, `Int8`, `Float4`, `Float8`, and `Uint32` fields now include bit size. e.g. `Int` is renamed to `Int64`. 301 This matches the convention set by `database/sql`. In addition, for comparable types like `pgtype.Int8` and 302 `sql.NullInt64` the structures are identical. This means they can be directly converted one to another. 303 304 ### 3rd Party Type Integrations 305 306 * Extracted integrations with https://github.com/shopspring/decimal and https://github.com/gofrs/uuid to 307 https://github.com/jackc/pgx-shopspring-decimal and https://github.com/jackc/pgx-gofrs-uuid respectively. This trims 308 the pgx dependency tree. 309 310 ### Other Changes 311 312 * `Bit` and `Varbit` are both replaced by the `Bits` type. 313 * `CID`, `OID`, `OIDValue`, and `XID` are replaced by the `Uint32` type. 314 * `Hstore` is now defined as `map[string]*string`. 315 * `JSON` and `JSONB` types removed. Use `[]byte` or `string` directly. 316 * `QChar` type removed. Use `rune` or `byte` directly. 317 * `Inet` and `Cidr` types removed. Use `netip.Addr` and `netip.Prefix` directly. These types are more memory efficient than the previous `net.IPNet`. 318 * `Macaddr` type removed. Use `net.HardwareAddr` directly. 319 * Renamed `pgtype.ConnInfo` to `pgtype.Map`. 320 * Renamed `pgtype.DataType` to `pgtype.Type`. 321 * Renamed `pgtype.None` to `pgtype.Finite`. 322 * `RegisterType` now accepts a `*Type` instead of `Type`. 323 * Assorted array helper methods and types made private. 324 325 ## stdlib 326 327 * Removed `AcquireConn` and `ReleaseConn` as that functionality has been built in since Go 1.13. 328 329 ## Reduced Memory Usage by Reusing Read Buffers 330 331 Previously, the connection read buffer would allocate large chunks of memory and never reuse them. This allowed 332 transferring ownership to anything such as scanned values without incurring an additional allocation and memory copy. 333 However, this came at the cost of overall increased memory allocation size. But worse it was also possible to pin large 334 chunks of memory by retaining a reference to a small value that originally came directly from the read buffer. Now 335 ownership remains with the read buffer and anything needing to retain a value must make a copy. 336 337 ## Query Execution Modes 338 339 Control over automatic prepared statement caching and simple protocol use are now combined into query execution mode. 340 See documentation for `QueryExecMode`. 341 342 ## QueryRewriter Interface and NamedArgs 343 344 pgx now supports named arguments with the `NamedArgs` type. This is implemented via the new `QueryRewriter` interface which 345 allows arbitrary rewriting of query SQL and arguments. 346 347 ## RowScanner Interface 348 349 The `RowScanner` interface allows a single argument to Rows.Scan to scan the entire row. 350 351 ## Rows Result Helpers 352 353 * `CollectRows` and `RowTo*` functions simplify collecting results into a slice. 354 * `CollectOneRow` collects one row using `RowTo*` functions. 355 * `ForEachRow` simplifies scanning each row and executing code using the scanned values. `ForEachRow` replaces `QueryFunc`. 356 357 ## Tx Helpers 358 359 Rather than every type that implemented `Begin` or `BeginTx` methods also needing to implement `BeginFunc` and 360 `BeginTxFunc` these methods have been converted to functions that take a db that implements `Begin` or `BeginTx`. 361 362 ## Improved Batch Query Ergonomics 363 364 Previously, the code for building a batch went in one place before the call to `SendBatch`, and the code for reading the 365 results went in one place after the call to `SendBatch`. This could make it difficult to match up the query and the code 366 to handle the results. Now `Queue` returns a `QueuedQuery` which has methods `Query`, `QueryRow`, and `Exec` which can 367 be used to register a callback function that will handle the result. Callback functions are called automatically when 368 `BatchResults.Close` is called. 369 370 ## SendBatch Uses Pipeline Mode When Appropriate 371 372 Previously, a batch with 10 unique parameterized statements executed 100 times would entail 11 network round trips. 1 373 for each prepare / describe and 1 for executing them all. Now pipeline mode is used to prepare / describe all statements 374 in a single network round trip. So it would only take 2 round trips. 375 376 ## Tracing and Logging 377 378 Internal logging support has been replaced with tracing hooks. This allows custom tracing integration with tools like OpenTelemetry. Package tracelog provides an adapter for pgx v4 loggers to act as a tracer. 379 380 All integrations with 3rd party loggers have been extracted to separate repositories. This trims the pgx dependency 381 tree.