github.com/cockroachdb/pebble@v0.0.0-20231214172447-ab4952c5f87b/README.md (about) 1 # Pebble [](https://github.com/cockroachdb/pebble/actions/workflows/ci.yaml) [](https://godoc.org/github.com/cockroachdb/pebble) <sup><sub><sub>[Coverage](https://storage.googleapis.com/crl-codecover-public/pebble/index.html)</sub></sub></sup> 2 3 #### [Nightly benchmarks](https://cockroachdb.github.io/pebble/) 4 5 Pebble is a LevelDB/RocksDB inspired key-value store focused on 6 performance and internal usage by CockroachDB. Pebble inherits the 7 RocksDB file formats and a few extensions such as range deletion 8 tombstones, table-level bloom filters, and updates to the MANIFEST 9 format. 10 11 Pebble intentionally does not aspire to include every feature in RocksDB and 12 specifically targets the use case and feature set needed by CockroachDB: 13 14 * Block-based tables 15 * Checkpoints 16 * Indexed batches 17 * Iterator options (lower/upper bound, table filter) 18 * Level-based compaction 19 * Manual compaction 20 * Merge operator 21 * Prefix bloom filters 22 * Prefix iteration 23 * Range deletion tombstones 24 * Reverse iteration 25 * SSTable ingestion 26 * Single delete 27 * Snapshots 28 * Table-level bloom filters 29 30 RocksDB has a large number of features that are not implemented in 31 Pebble: 32 33 * Backups 34 * Column families 35 * Delete files in range 36 * FIFO compaction style 37 * Forward iterator / tailing iterator 38 * Hash table format 39 * Memtable bloom filter 40 * Persistent cache 41 * Pin iterator key / value 42 * Plain table format 43 * SSTable ingest-behind 44 * Sub-compactions 45 * Transactions 46 * Universal compaction style 47 48 ***WARNING***: Pebble may silently corrupt data or behave incorrectly if 49 used with a RocksDB database that uses a feature Pebble doesn't 50 support. Caveat emptor! 51 52 ## Production Ready 53 54 Pebble was introduced as an alternative storage engine to RocksDB in 55 CockroachDB v20.1 (released May 2020) and was used in production 56 successfully at that time. Pebble was made the default storage engine 57 in CockroachDB v20.2 (released Nov 2020). Pebble is being used in 58 production by users of CockroachDB at scale and is considered stable 59 and production ready. 60 61 ## Advantages 62 63 Pebble offers several improvements over RocksDB: 64 65 * Faster reverse iteration via backwards links in the memtable's 66 skiplist. 67 * Faster commit pipeline that achieves better concurrency. 68 * Seamless merged iteration of indexed batches. The mutations in the 69 batch conceptually occupy another memtable level. 70 * L0 sublevels and flush splitting for concurrent compactions out of L0 and 71 reduced read-amplification during heavy write load. 72 * Faster LSM edits in LSMs with large numbers of sstables through use of a 73 copy-on-write B-tree to hold file metadata. 74 * Delete-only compactions that drop whole sstables that fall within the bounds 75 of a range deletion. 76 * Block-property collectors and filters that enable iterators to skip tables, 77 index blocks and data blocks that are irrelevant, according to user-defined 78 properties over key-value pairs. 79 * Range keys API, allowing KV pairs defined over a range of keyspace with 80 user-defined semantics and interleaved during iteration. 81 * Smaller, more approachable code base. 82 83 See the [Pebble vs RocksDB: Implementation 84 Differences](docs/rocksdb.md) doc for more details on implementation 85 differences. 86 87 ## RocksDB Compatibility 88 89 Pebble strives for forward compatibility with RocksDB 6.2.1 (the latest 90 version of RocksDB used by CockroachDB). Forward compatibility means 91 that a DB generated by RocksDB can be used by Pebble. Currently, Pebble 92 provides bidirectional compatibility with RocksDB (a Pebble generated DB 93 can be used by RocksDB) when using its FormatMostCompatible format. New 94 functionality that is backwards incompatible is gated behind new format 95 major versions. In general, Pebble only provides compatibility with the 96 subset of functionality and configuration used by CockroachDB. The scope 97 of RocksDB functionality and configuration is too large to adequately 98 test and document all the incompatibilities. The list below contains 99 known incompatibilities. 100 101 * Pebble's use of WAL recycling is only compatible with RocksDB's 102 `kTolerateCorruptedTailRecords` WAL recovery mode. Older versions of 103 RocksDB would automatically map incompatible WAL recovery modes to 104 `kTolerateCorruptedTailRecords`. New versions of RocksDB will 105 disable WAL recycling. 106 * Column families. Pebble does not support column families, nor does 107 it attempt to detect their usage when opening a DB that may contain 108 them. 109 * Hash table format. Pebble does not support the hash table sstable 110 format. 111 * Plain table format. Pebble does not support the plain table sstable 112 format. 113 * SSTable format version 3 and 4. Pebble does not support version 3 114 and version 4 format sstables. The sstable format version is 115 controlled by the `BlockBasedTableOptions::format_version` option. 116 See [#97](https://github.com/cockroachdb/pebble/issues/97). 117 118 ## Format major versions 119 120 Over time Pebble has introduced new physical file formats. Backwards 121 incompatible changes are made through the introduction of 'format major 122 versions'. By default, when Pebble opens a database, it defaults to 123 `FormatMostCompatible`. This version is bi-directionally compatible with RocksDB 124 6.2.1 (with the caveats described above). 125 126 To opt into new formats, a user may set `FormatMajorVersion` on the 127 [`Options`](https://pkg.go.dev/github.com/cockroachdb/pebble#Options) 128 supplied to 129 [`Open`](https://pkg.go.dev/github.com/cockroachdb/pebble#Open), or 130 upgrade the format major version at runtime using 131 [`DB.RatchetFormatMajorVersion`](https://pkg.go.dev/github.com/cockroachdb/pebble#DB.RatchetFormatMajorVersion). 132 Format major version upgrades are permanent; There is no option to 133 return to an earlier format. 134 135 The table below outlines the history of format major versions: 136 137 | Name | Value | Migration | 138 |------------------------------------|-------|------------| 139 | FormatMostCompatible | 1 | No | 140 | FormatVersioned | 3 | No | 141 | FormatSetWithDelete | 4 | No | 142 | FormatBlockPropertyCollector | 5 | No | 143 | FormatSplitUserKeysMarked | 6 | Background | 144 | FormatSplitUserKeysMarkedCompacted | 7 | Blocking | 145 | FormatRangeKeys | 8 | No | 146 | FormatMinTableFormatPebblev1 | 9 | No | 147 | FormatPrePebblev1Marked | 10 | Background | 148 | FormatSSTableValueBlocks | 12 | No | 149 | FormatFlushableIngest | 13 | No | 150 | FormatPrePebblev1MarkedCompacted | 14 | Blocking | 151 | FormatDeleteSizedAndObsolete | 15 | No | 152 | FormatVirtualSSTables | 16 | No | 153 154 Upgrading to a format major version with 'Background' in the migration 155 column may trigger background activity to rewrite physical file 156 formats, typically through compactions. Upgrading to a format major 157 version with 'Blocking' in the migration column will block until a 158 migration is complete. The database may continue to serve reads and 159 writes if upgrading a live database through 160 `RatchetFormatMajorVersion`, but the method call will not return until 161 the migration is complete. 162 163 For reference, the table below lists the range of supported Pebble format major 164 versions for CockroachDB releases. 165 166 | CockroachDB release | Earliest supported | Latest supported | 167 |---------------------|------------------------------------|---------------------------| 168 | 20.1 through 21.1 | FormatMostCompatible | FormatMostCompatible | 169 | 21.2 | FormatMostCompatible | FormatSetWithDelete | 170 | 21.2 | FormatMostCompatible | FormatSetWithDelete | 171 | 22.1 | FormatMostCompatible | FormatSplitUserKeysMarked | 172 | 22.2 | FormatMostCompatible | FormatPrePebblev1Marked | 173 | 23.1 | FormatSplitUserKeysMarkedCompacted | FormatFlushableIngest | 174 | 23.2 | FormatSplitUserKeysMarkedCompacted | FormatVirtualSSTables | 175 | 24.1 plan | FormatSSTableValueBlocks | | 176 177 ## Pedigree 178 179 Pebble is based on the incomplete Go version of LevelDB: 180 181 https://github.com/golang/leveldb 182 183 The Go version of LevelDB is based on the C++ original: 184 185 https://github.com/google/leveldb 186 187 Optimizations and inspiration were drawn from RocksDB: 188 189 https://github.com/facebook/rocksdb 190 191 ## Getting Started 192 193 ### Example Code 194 195 ```go 196 package main 197 198 import ( 199 "fmt" 200 "log" 201 202 "github.com/cockroachdb/pebble" 203 ) 204 205 func main() { 206 db, err := pebble.Open("demo", &pebble.Options{}) 207 if err != nil { 208 log.Fatal(err) 209 } 210 key := []byte("hello") 211 if err := db.Set(key, []byte("world"), pebble.Sync); err != nil { 212 log.Fatal(err) 213 } 214 value, closer, err := db.Get(key) 215 if err != nil { 216 log.Fatal(err) 217 } 218 fmt.Printf("%s %s\n", key, value) 219 if err := closer.Close(); err != nil { 220 log.Fatal(err) 221 } 222 if err := db.Close(); err != nil { 223 log.Fatal(err) 224 } 225 } 226 ```