github.com/cockroachdb/pebble@v0.0.0-20231214172447-ab4952c5f87b/testdata/indexed_batch_mutation (about) 1 # Set a key within the indexed batch. 2 new-batch 3 set foo foo 4 ---- 5 6 # Construct an iterator over the indexed batch. 7 8 new-batch-iter i0 9 ---- 10 11 # The key we set should be visible. 12 13 iter iter=i0 14 first 15 next 16 ---- 17 foo: (foo, .) 18 . 19 20 # Same behavior with a batch-only iterator. 21 new-batch-only-iter i-bo0 22 ---- 23 24 iter iter=i-bo0 25 first 26 next 27 ---- 28 foo: (foo, .) 29 . 30 31 # Set a new key, while the above iterator is still open. 32 33 mutate 34 set bar bar 35 ---- 36 37 # The new key should be invisible. 38 39 iter iter=i0 40 prev 41 next 42 ---- 43 foo: (foo, .) 44 . 45 46 # Same behavior with a batch-only iterator. 47 iter iter=i-bo0 48 prev 49 next 50 ---- 51 foo: (foo, .) 52 . 53 54 # A set-options operation should refresh the Iterator's view of the batch. The 55 # bar key should now be visibile. 56 57 iter iter=i0 58 set-options 59 first 60 next 61 next 62 ---- 63 . 64 bar: (bar, .) 65 foo: (foo, .) 66 . 67 68 # Same behavior with a batch-only iterator. 69 iter iter=i-bo0 70 set-options 71 first 72 next 73 next 74 ---- 75 . 76 bar: (bar, .) 77 foo: (foo, .) 78 . 79 80 # Delete foo with a range deletion. 81 82 mutate 83 del-range f g 84 ---- 85 86 # Both keys should still be visible. 87 88 iter iter=i0 89 prev 90 prev 91 ---- 92 foo: (foo, .) 93 bar: (bar, .) 94 95 # Same behavior with a batch-only iterator. 96 iter iter=i-bo0 97 prev 98 prev 99 ---- 100 foo: (foo, .) 101 bar: (bar, .) 102 103 # After refreshing the iterator's view of the batch, foo should be deleted. 104 105 iter iter=i0 106 set-options 107 seek-ge foo 108 seek-lt foo 109 ---- 110 . 111 . 112 bar: (bar, .) 113 114 # Same behavior with a batch-only iterator. 115 iter iter=i-bo0 116 set-options 117 seek-ge foo 118 seek-lt foo 119 ---- 120 . 121 . 122 bar: (bar, .) 123 124 # Write a range key set and a point key. 125 126 mutate 127 range-key-set a c @1 boop 128 set b b 129 ---- 130 131 # The mutations should not be visible. 132 133 iter iter=i0 134 prev 135 next 136 ---- 137 . 138 bar: (bar, .) 139 140 # Same behavior with a batch-only iterator. 141 iter iter=i-bo0 142 prev 143 next 144 ---- 145 . 146 bar: (bar, .) 147 148 # But refreshing the batch through a call to SetOptions should surface them. 149 150 iter iter=i0 151 set-options 152 first 153 next 154 next 155 ---- 156 . 157 a: (., [a-c) @1=boop UPDATED) 158 b: (b, [a-c) @1=boop) 159 bar: (bar, [a-c) @1=boop) 160 161 # Same behavior with a batch-only iterator. 162 iter iter=i-bo0 163 set-options 164 first 165 next 166 next 167 ---- 168 . 169 a: (., [a-c) @1=boop UPDATED) 170 b: (b, [a-c) @1=boop) 171 bar: (bar, [a-c) @1=boop) 172 173 # Remove part of the range key to fragment it. 174 175 mutate 176 range-key-del ace arc 177 ---- 178 179 iter iter=i0 180 next 181 prev 182 prev 183 prev 184 prev 185 ---- 186 . 187 bar: (bar, [a-c) @1=boop UPDATED) 188 b: (b, [a-c) @1=boop) 189 a: (., [a-c) @1=boop) 190 . 191 192 # Same behavior with a batch-only iterator. 193 iter iter=i-bo0 194 next 195 prev 196 prev 197 prev 198 prev 199 ---- 200 . 201 bar: (bar, [a-c) @1=boop UPDATED) 202 b: (b, [a-c) @1=boop) 203 a: (., [a-c) @1=boop) 204 . 205 206 iter iter=i0 207 set-options 208 first 209 next 210 next 211 next 212 ---- 213 . 214 a: (., [a-ace) @1=boop UPDATED) 215 arc: (., [arc-c) @1=boop UPDATED) 216 b: (b, [arc-c) @1=boop) 217 bar: (bar, [arc-c) @1=boop) 218 219 # Same behavior with a batch-only iterator. 220 iter iter=i-bo0 221 set-options 222 first 223 next 224 next 225 next 226 ---- 227 . 228 a: (., [a-ace) @1=boop UPDATED) 229 arc: (., [arc-c) @1=boop UPDATED) 230 b: (b, [arc-c) @1=boop) 231 bar: (bar, [arc-c) @1=boop) 232 233 # Create a new indexed batch and a new iterator over it. 234 235 new-batch 236 set foo foo 237 ---- 238 239 new-batch-iter i1 240 ---- 241 242 iter iter=i1 243 first 244 next 245 ---- 246 foo: (foo, .) 247 . 248 249 # Test interactions with cloned iterators. 250 # First, apply mutations to the batch. They should remain invisible. 251 252 mutate 253 set bar bar 254 range-key-set a z @1 boop 255 del-range f g 256 ---- 257 258 iter iter=i1 259 first 260 next 261 ---- 262 foo: (foo, .) 263 . 264 265 # Clone i1 to create i2. 266 267 clone from=i1 to=i2 refresh-batch=false 268 ---- 269 270 # i1 unchanged. 271 272 iter iter=i1 273 first 274 next 275 ---- 276 foo: (foo, .) 277 . 278 279 # i2 sees exactly the same stale state as i1 until SetOptions is called to 280 # explicitly refresh the view of the underlying batch. 281 282 iter iter=i2 283 first 284 next 285 set-options 286 first 287 next 288 next 289 ---- 290 foo: (foo, .) 291 . 292 . 293 a: (., [a-z) @1=boop UPDATED) 294 bar: (bar, [a-z) @1=boop) 295 . 296 297 # Clone i1 to create i3, this time passing RefreshBatchView: true. This clone 298 # should view the updated view of the underlying batch. 299 clone from=i1 to=i3 refresh-batch=true 300 ---- 301 302 iter iter=i3 303 first 304 next 305 ---- 306 a: (., [a-z) @1=boop UPDATED) 307 bar: (bar, [a-z) @1=boop) 308 309 # i1 should still have the old, stale view of the batch. 310 311 iter iter=i1 312 first 313 next 314 ---- 315 foo: (foo, .) 316 . 317 318 # Mutate the underlying batch again. 319 320 mutate 321 set foo foo 322 range-key-set a z @2 bax 323 del-range b c 324 ---- 325 326 # The new mutations should be invisible until SetOptions is called. 327 328 iter iter=i1 329 first 330 next 331 set-options 332 first 333 next 334 next 335 ---- 336 foo: (foo, .) 337 . 338 . 339 a: (., [a-z) @2=bax, @1=boop UPDATED) 340 foo: (foo, [a-z) @2=bax, @1=boop) 341 . 342 343 iter iter=i2 344 first 345 next 346 next 347 set-options 348 first 349 next 350 next 351 ---- 352 a: (., [a-z) @1=boop UPDATED) 353 bar: (bar, [a-z) @1=boop) 354 . 355 . 356 a: (., [a-z) @2=bax, @1=boop UPDATED) 357 foo: (foo, [a-z) @2=bax, @1=boop) 358 . 359 360 # Commit a separate batch to the underlying engine. 361 batch 362 range-key-set a z @5 poi 363 set apple apple 364 ---- 365 366 # The writes to the underlying engine should be invisible. 367 368 iter iter=i1 369 first 370 next 371 next 372 ---- 373 a: (., [a-z) @2=bax, @1=boop UPDATED) 374 foo: (foo, [a-z) @2=bax, @1=boop) 375 . 376 377 # Clone i1 to create i4. 378 379 clone from=i1 to=i4 refresh-batch=false 380 ---- 381 382 iter iter=i4 383 first 384 next 385 next 386 ---- 387 a: (., [a-z) @2=bax, @1=boop UPDATED) 388 foo: (foo, [a-z) @2=bax, @1=boop) 389 . 390 391 # Refresh i4's view of its batch. It should still not see the newly committed 392 # writes. 393 394 iter iter=i4 395 set-options 396 first 397 next 398 next 399 ---- 400 . 401 a: (., [a-z) @2=bax, @1=boop UPDATED) 402 foo: (foo, [a-z) @2=bax, @1=boop) 403 . 404 405 # Create a new iterator i5 over the indexed batch [not a Clone]. It should see 406 # all committed writes and uncommitted writes. 407 408 new-batch-iter i5 409 ---- 410 411 iter iter=i5 412 first 413 next 414 next 415 next 416 ---- 417 a: (., [a-z) @5=poi, @2=bax, @1=boop UPDATED) 418 apple: (apple, [a-z) @5=poi, @2=bax, @1=boop) 419 foo: (foo, [a-z) @5=poi, @2=bax, @1=boop) 420 . 421 422 # The batch-only iter only sees the contents of the batch. 423 new-batch-only-iter i-bo1 424 ---- 425 426 iter iter=i-bo1 427 first 428 next 429 next 430 ---- 431 a: (., [a-z) @2=bax, @1=boop UPDATED) 432 foo: (foo, [a-z) @2=bax, @1=boop) 433 . 434 435 # Mutate all the open iterators' underlying batch. 436 437 mutate 438 range-key-set a z @6 yaya 439 set c c 440 ---- 441 442 # The iterators should still not see the committed writes, even after refreshing 443 # to observe more recent batch writes. 444 445 iter iter=i1 446 first 447 next 448 next 449 ---- 450 a: (., [a-z) @2=bax, @1=boop UPDATED) 451 foo: (foo, [a-z) @2=bax, @1=boop) 452 . 453 454 iter iter=i4 455 first 456 next 457 next 458 set-options 459 first 460 next 461 next 462 ---- 463 a: (., [a-z) @2=bax, @1=boop UPDATED) 464 foo: (foo, [a-z) @2=bax, @1=boop) 465 . 466 . 467 a: (., [a-z) @6=yaya, @2=bax, @1=boop UPDATED) 468 c: (c, [a-z) @6=yaya, @2=bax, @1=boop) 469 foo: (foo, [a-z) @6=yaya, @2=bax, @1=boop) 470 471 472 # The batch-only iter sees the more recent batch writes after refreshing. 473 iter iter=i-bo1 474 first 475 next 476 next 477 set-options 478 first 479 next 480 next 481 next 482 ---- 483 a: (., [a-z) @2=bax, @1=boop UPDATED) 484 foo: (foo, [a-z) @2=bax, @1=boop) 485 . 486 . 487 a: (., [a-z) @6=yaya, @2=bax, @1=boop UPDATED) 488 c: (c, [a-z) @6=yaya, @2=bax, @1=boop) 489 foo: (foo, [a-z) @6=yaya, @2=bax, @1=boop) 490 . 491 492 # Test a scenario where constructing an Iterator should NOT use the cached 493 # fragmented tombstones / range keys, because the new Iterator is a Clone which 494 # must read at an earlier batch sequence number. 495 496 # Reset and start a new batch. 497 498 reset 499 ---- 500 501 new-batch 502 set foo foo 503 ---- 504 505 new-batch-iter i1 506 ---- 507 508 iter iter=i1 509 first 510 next 511 ---- 512 foo: (foo, .) 513 . 514 515 # Apply a range deletion and a range key. 516 517 mutate 518 del-range a z 519 range-key-set a z @1 foo 520 ---- 521 522 # Create a new iterator which will see both the range deletion and the range 523 # key, and cache both on the batch so that future iterators constructed over the 524 # batch do not need to. 525 526 new-batch-iter i2 527 ---- 528 529 iter iter=i2 530 first 531 next 532 ---- 533 a: (., [a-z) @1=foo UPDATED) 534 . 535 536 # Clone the original iterator from before the delete range and the range key 537 # were created. It should not use the cached fragments of range deletions or 538 # range keys, and should not see the effects of either. 539 540 clone from=i1 to=i3 refresh-batch=false 541 ---- 542 543 iter iter=i3 544 first 545 next 546 ---- 547 foo: (foo, .) 548 . 549 550 reset 551 ---- 552 553 new-batch 554 range-key-set a c @1 poi 555 range-key-set b d @2 yaya 556 ---- 557 558 new-batch-iter i1 559 ---- 560 561 # The batch contains 2 range keys, but the skiplist of fragmented range keys 562 # contains 3 elements (a-b, b-c, c-d). 563 564 iter iter=i1 565 first 566 next 567 next 568 ---- 569 a: (., [a-b) @1=poi UPDATED) 570 b: (., [b-c) @2=yaya, @1=poi UPDATED) 571 c: (., [c-d) @2=yaya UPDATED) 572 573 # Add a new range key to the batch. The batch contains 3 internal range keys, 574 # and the skiplist of fragmented range keys contains 3 elements. 575 576 mutate 577 range-key-set e f @3 foo 578 ---- 579 580 # Refreshing the iterator's view of the batch through SetOptions should surface 581 # the new range key. An earlier bug incorrectly compared the number of 582 # fragmented range keys to the number of internal batch range keys in order to 583 # determine when to refresh the iterator. 584 585 iter iter=i1 586 first 587 next 588 next 589 set-options 590 first 591 next 592 next 593 next 594 seek-ge bat 595 ---- 596 a: (., [a-b) @1=poi UPDATED) 597 b: (., [b-c) @2=yaya, @1=poi UPDATED) 598 c: (., [c-d) @2=yaya UPDATED) 599 . 600 a: (., [a-b) @1=poi UPDATED) 601 b: (., [b-c) @2=yaya, @1=poi UPDATED) 602 c: (., [c-d) @2=yaya UPDATED) 603 e: (., [e-f) @3=foo UPDATED) 604 bat: (., [b-c) @2=yaya, @1=poi UPDATED) 605 606 # Mutate the range key under the interleaving iterator's current position in the 607 # indexed batch. 608 # 609 # The last `seek-ge` operation landed on the range key [b-c). The top-level 610 # *pebble.Iterator needs to step the iterator again to see if there's a 611 # coincident point key at (`bat`), which would've advanced the interleaving 612 # iterator to the range key with bounds [c,d), so the underlying interleaving 613 # iterator is positioned ahead at: 614 # 615 # c: (., [c-d) @2=yaya) 616 # 617 # If we call set-options to refresh the iterator's view of the indexed batch, 618 # the range-key-unset [c,d)@2 becomes visible, and the range key that the 619 # underlying interleaving iterator is positioned over should not be visible. 620 # 621 # A bug previously allowed this range key to be visible when seeking into this 622 # span's bounds (see the optimization in InterleavingIter.SeekGE). Now, the call 623 # to SetOptions clears the interleaving iterator's positional state to avoid the 624 # SeekGE optimization. 625 626 mutate 627 range-key-unset b d @2 628 ---- 629 630 iter iter=i1 631 set-options 632 seek-ge cat 633 ---- 634 . 635 e: (., [e-f) @3=foo UPDATED) 636 637 reset 638 ---- 639 640 batch 641 range-key-set a e @1 foo 642 ---- 643 644 flush 645 ---- 646 647 new-batch 648 ---- 649 650 new-batch-iter batchiter 651 ---- 652 653 new-db-iter dbiter 654 ---- 655 656 # Test RangeKeyChanged() semantics. 657 # Seeking to the same prefix returns RangeKeyChanged()=false. 658 # Seeking to a new prefix returns RangeKeyChanged()=true. 659 # Seeking to the same prefix with a SetOptions call in between returns 660 # RangeKeyChanged()=true. 661 662 iter iter=dbiter 663 seek-prefix-ge b@3 664 seek-prefix-ge b@4 665 seek-prefix-ge c@3 666 seek-prefix-ge d@3 667 set-options 668 seek-prefix-ge d@1 669 ---- 670 b@3: (., [b-"b\x00") @1=foo UPDATED) 671 b@4: (., [b-"b\x00") @1=foo) 672 c@3: (., [c-"c\x00") @1=foo UPDATED) 673 d@3: (., [d-"d\x00") @1=foo UPDATED) 674 . 675 d@1: (., [d-"d\x00") @1=foo UPDATED) 676 677 # Test the same semantics on a batch iterator. 678 679 iter iter=batchiter 680 seek-prefix-ge b@3 681 seek-prefix-ge b@4 682 seek-prefix-ge c@3 683 seek-prefix-ge d@3 684 set-options 685 seek-prefix-ge d@1 686 ---- 687 b@3: (., [b-"b\x00") @1=foo UPDATED) 688 b@4: (., [b-"b\x00") @1=foo) 689 c@3: (., [c-"c\x00") @1=foo UPDATED) 690 d@3: (., [d-"d\x00") @1=foo UPDATED) 691 . 692 d@1: (., [d-"d\x00") @1=foo UPDATED) 693 694 # Test mutating the indexed batch's range keys, overlapping the existing seek 695 # position. It should not see the new mutations, but after a call to SetOptions 696 # it should AND it should return RangeKeyChanged()=true. 697 698 mutate 699 range-key-set d e @2 foo 700 ---- 701 702 iter iter=batchiter 703 seek-prefix-ge d@2 704 set-options 705 seek-prefix-ge d@2 706 ---- 707 d@2: (., [d-"d\x00") @1=foo) 708 . 709 d@2: (., [d-"d\x00") @2=foo, @1=foo UPDATED) 710 711 # Test cloning an iterator with a range-key mask block property filter 712 # configured. If the cloned and the clonee iterators have different suffixes 713 # configured, their suffixes should be respected. Previously, the 714 # RangeKeyMasking.Filter option was a footgun, because it was a single mutable 715 # instance. Cloning the iterator without supplying new iterator options would 716 # result in two iterators using the same filter. 717 718 reset 719 ---- 720 721 batch 722 range-key-set a e @5 foo 723 set b@4 b@4 724 ---- 725 726 new-db-iter iter-a 727 ---- 728 729 iter iter=iter-a 730 set-options mask-suffix=@3 mask-filter=true 731 seek-ge a 732 next 733 next 734 ---- 735 . 736 a: (., [a-e) @5=foo UPDATED) 737 b@4: (b@4, [a-e) @5=foo) 738 . 739 740 clone from=iter-a to=iter-b refresh-batch=false 741 ---- 742 743 iter iter=iter-b 744 set-options mask-suffix=@6 745 seek-ge a 746 next 747 ---- 748 . 749 a: (., [a-e) @5=foo UPDATED) 750 . 751 752 iter iter=iter-a 753 seek-ge a 754 next 755 next 756 ---- 757 a: (., [a-e) @5=foo UPDATED) 758 b@4: (b@4, [a-e) @5=foo) 759 .