github.com/zuoyebang/bitalostable@v1.0.1-0.20240229032404-e3b99a834294/testdata/rangekeys (about) 1 reset 2 ---- 3 4 # Use the key string as the value so that it's easy to tell when we surface the 5 # wrong value. 6 7 batch 8 set a a 9 set b b 10 set c c 11 set d d 12 range-key-set b c @5 boop 13 range-key-set cat dog @3 beep 14 ---- 15 wrote 6 keys 16 17 # Scan forward 18 19 combined-iter 20 seek-ge a 21 next 22 next 23 next 24 next 25 next 26 ---- 27 a: (a, .) 28 b: (b, [b-c) @5=boop UPDATED) 29 c: (c, . UPDATED) 30 cat: (., [cat-dog) @3=beep UPDATED) 31 d: (d, [cat-dog) @3=beep) 32 . 33 34 # Do the above forward iteration but with a mask suffix. The results should be 35 # identical despite range keys serving as masks, because none of the point keys 36 # have suffixes. 37 38 combined-iter mask-suffix=@9 39 seek-ge a 40 next 41 next 42 next 43 next 44 next 45 ---- 46 a: (a, .) 47 b: (b, [b-c) @5=boop UPDATED) 48 c: (c, . UPDATED) 49 cat: (., [cat-dog) @3=beep UPDATED) 50 d: (d, [cat-dog) @3=beep) 51 . 52 53 # Scan backward 54 55 combined-iter 56 seek-lt z 57 prev 58 prev 59 prev 60 prev 61 prev 62 ---- 63 d: (d, [cat-dog) @3=beep UPDATED) 64 cat: (., [cat-dog) @3=beep) 65 c: (c, . UPDATED) 66 b: (b, [b-c) @5=boop UPDATED) 67 a: (a, . UPDATED) 68 . 69 70 combined-iter 71 seek-ge ace 72 seek-ge b 73 seek-ge c 74 seek-ge cab 75 seek-ge cat 76 seek-ge d 77 seek-ge day 78 seek-ge dog 79 ---- 80 b: (b, [b-c) @5=boop UPDATED) 81 b: (b, [b-c) @5=boop) 82 c: (c, . UPDATED) 83 cat: (., [cat-dog) @3=beep UPDATED) 84 cat: (., [cat-dog) @3=beep) 85 d: (d, [cat-dog) @3=beep) 86 day: (., [cat-dog) @3=beep) 87 . 88 89 combined-iter 90 seek-lt 1 91 seek-lt ace 92 seek-lt b 93 seek-lt c 94 seek-lt cab 95 seek-lt cat 96 seek-lt d 97 seek-lt day 98 seek-lt dog 99 seek-lt zebra 100 ---- 101 . 102 a: (a, .) 103 a: (a, .) 104 b: (b, [b-c) @5=boop UPDATED) 105 c: (c, . UPDATED) 106 c: (c, .) 107 cat: (., [cat-dog) @3=beep UPDATED) 108 d: (d, [cat-dog) @3=beep) 109 d: (d, [cat-dog) @3=beep) 110 d: (d, [cat-dog) @3=beep) 111 112 rangekey-iter 113 first 114 next 115 next 116 set-bounds lower=bat upper=catatonic 117 first 118 next 119 next 120 ---- 121 b [b-c) @5=boop UPDATED 122 cat [cat-dog) @3=beep UPDATED 123 . 124 . 125 bat [bat-c) @5=boop UPDATED 126 cat [cat-catatonic) @3=beep UPDATED 127 . 128 129 rangekey-iter 130 seek-ge bat 131 ---- 132 bat [b-c) @5=boop UPDATED 133 134 # Delete 'b': The Iterator should still stop at b because of the range key 135 # with a start boundary at b. 136 137 batch 138 del b 139 ---- 140 wrote 1 keys 141 142 combined-iter 143 seek-ge b 144 seek-ge ace 145 ---- 146 b: (., [b-c) @5=boop UPDATED) 147 b: (., [b-c) @5=boop) 148 149 rangekey-iter 150 seek-ge b 151 seek-ge ace 152 ---- 153 b [b-c) @5=boop UPDATED 154 b [b-c) @5=boop 155 156 # Delete the b-c range key and the beginning of the cat-dog range key, 157 # truncating it to now begin at 'd'. 158 159 indexed-batch 160 range-key-del b d 161 ---- 162 created indexed batch with 1 keys 163 164 # Reading through the indexed batch, we should see the beginning of the cat-dog 165 # range key now beginning at 'd'. 166 167 combined-iter 168 seek-ge b 169 next 170 ---- 171 c: (c, .) 172 d: (d, [d-dog) @3=beep UPDATED) 173 174 commit-batch 175 ---- 176 wrote 1 keys 177 178 # Reading through the database after applying the batch, we should still see the 179 # beginning of the cat-dog range key now beginning at 'd'. 180 181 combined-iter 182 seek-ge b 183 next 184 ---- 185 c: (c, .) 186 d: (d, [d-dog) @3=beep UPDATED) 187 188 # Reading through the database after flushing, we should still see the 189 # beginning of the cat-dog range key now beginning at 'd'. 190 191 flush 192 ---- 193 194 combined-iter 195 seek-ge b 196 next 197 ---- 198 c: (c, .) 199 d: (d, [d-dog) @3=beep UPDATED) 200 201 reset 202 ---- 203 204 batch 205 range-key-set c d @1 boop 206 range-key-set apple c @3 beep 207 range-key-set ace apple @3 beep 208 set a a1 209 set b b1 210 set c c1 211 del a 212 set b b2 213 set c c2 214 ---- 215 wrote 9 keys 216 217 # Test that reverse iteration surfaces range key start boundaries alongside 218 # point keys at the same key, and defragments logically equivalent ranges. 219 220 combined-iter 221 last 222 prev 223 prev 224 prev 225 ---- 226 c: (c2, [c-d) @1=boop UPDATED) 227 b: (b2, [ace-c) @3=beep UPDATED) 228 ace: (., [ace-c) @3=beep) 229 . 230 231 # Test that forward iteration surfaces range key start boundaries alongside 232 # point keys at the same key, and defragments logically equivalent ranges. 233 234 combined-iter 235 first 236 next 237 next 238 next 239 ---- 240 ace: (., [ace-c) @3=beep UPDATED) 241 b: (b2, [ace-c) @3=beep) 242 c: (c2, [c-d) @1=boop UPDATED) 243 . 244 245 # NB: seek-prefix-ge truncates bounds to the prefix. 246 247 combined-iter 248 seek-prefix-ge b 249 next 250 ---- 251 b: (b2, [b-"b\x00") @3=beep UPDATED) 252 . 253 254 reset 255 ---- 256 257 batch 258 range-key-set a d @8 boop 259 set a@2 a@2 260 set a@3 a@3 261 set a@9 a@9 262 set a@10 a@10 263 set b b 264 ---- 265 wrote 6 keys 266 267 combined-iter 268 seek-prefix-ge a 269 next 270 next 271 next 272 next 273 next 274 ---- 275 a: (., [a-"a\x00") @8=boop UPDATED) 276 a@10: (a@10, [a-"a\x00") @8=boop) 277 a@9: (a@9, [a-"a\x00") @8=boop) 278 a@3: (a@3, [a-"a\x00") @8=boop) 279 a@2: (a@2, [a-"a\x00") @8=boop) 280 . 281 282 283 284 # Perform the above iteration with range-key masking enabled at a suffix equal 285 # to the range key's. The [a,d)@8 range key should serve as a masking, obscuring 286 # the points a@3 and a@2. 287 288 combined-iter mask-suffix=@8 289 seek-prefix-ge a 290 next 291 next 292 next 293 ---- 294 a: (., [a-"a\x00") @8=boop UPDATED) 295 a@10: (a@10, [a-"a\x00") @8=boop) 296 a@9: (a@9, [a-"a\x00") @8=boop) 297 . 298 299 # Perform the same thing but with a mask suffix below the range key's. All the 300 # points should be visible again. 301 # 302 # Then use SetOptions to raise the mask. The masked points should disappear. 303 304 combined-iter mask-suffix=@7 305 seek-prefix-ge a 306 next 307 next 308 next 309 next 310 next 311 set-options key-types=both mask-suffix=@8 312 seek-prefix-ge a 313 next 314 next 315 next 316 ---- 317 a: (., [a-"a\x00") @8=boop UPDATED) 318 a@10: (a@10, [a-"a\x00") @8=boop) 319 a@9: (a@9, [a-"a\x00") @8=boop) 320 a@3: (a@3, [a-"a\x00") @8=boop) 321 a@2: (a@2, [a-"a\x00") @8=boop) 322 . 323 . 324 a: (., [a-"a\x00") @8=boop UPDATED) 325 a@10: (a@10, [a-"a\x00") @8=boop) 326 a@9: (a@9, [a-"a\x00") @8=boop) 327 . 328 329 # Test that switching out of prefix iteration correctly expands the bounds 330 # beyond the scope of the previous prefix. 331 332 combined-iter 333 seek-prefix-ge a 334 next 335 seek-ge a@3 336 ---- 337 a: (., [a-"a\x00") @8=boop UPDATED) 338 a@10: (a@10, [a-"a\x00") @8=boop) 339 a@3: (a@3, [a-d) @8=boop UPDATED) 340 341 reset 342 ---- 343 344 # For all prefixes a, aa, ab, ... zz, write 3 keys at timestamps @1, @10, @100. 345 # This populates a total of (26**2 + 26) * 3 = 2106 keys. 346 347 populate keylen=2 timestamps=(1, 10, 100) 348 ---- 349 wrote 2106 keys 350 351 batch 352 range-key-set b c @5 beep 353 range-key-unset c d @1 354 range-key-del d e 355 ---- 356 wrote 3 keys 357 358 flush 359 ---- 360 361 metrics 362 ---- 363 Metrics.Keys.RangeKeySetsCount = 1 364 365 combined-iter 366 seek-ge az 367 next 368 next 369 next 370 next 371 next 372 seek-ge bz@10 373 next 374 next 375 ---- 376 az@100: (az@100, .) 377 az@10: (az@10, .) 378 az@1: (az@1, .) 379 b: (., [b-c) @5=beep UPDATED) 380 b@100: (b@100, [b-c) @5=beep) 381 b@10: (b@10, [b-c) @5=beep) 382 bz@10: (bz@10, [b-c) @5=beep) 383 bz@1: (bz@1, [b-c) @5=beep) 384 c@100: (c@100, . UPDATED) 385 386 # Perform the same iteration with all range keys serving as masks. The bz@1 387 # point key should be elided. 388 389 combined-iter mask-suffix=@100 390 seek-ge az 391 next 392 next 393 next 394 next 395 next 396 seek-ge bz@10 397 next 398 next 399 ---- 400 az@100: (az@100, .) 401 az@10: (az@10, .) 402 az@1: (az@1, .) 403 b: (., [b-c) @5=beep UPDATED) 404 b@100: (b@100, [b-c) @5=beep) 405 b@10: (b@10, [b-c) @5=beep) 406 bz@10: (bz@10, [b-c) @5=beep) 407 c@100: (c@100, . UPDATED) 408 c@10: (c@10, .) 409 410 # Ensure that a cloned iterator includes range keys. 411 412 combined-iter 413 seek-ge bz@10 414 clone 415 seek-ge bz@10 416 ---- 417 bz@10: (bz@10, [b-c) @5=beep UPDATED) 418 . 419 bz@10: (bz@10, [b-c) @5=beep UPDATED) 420 421 # Within a batch, later writes overwrite earlier writes. Here, the range-key-del 422 # of [bat, bus) overwrites the earlier writes of [b,c) and [b,e). 423 424 batch 425 range-key-set b c @5 beep 426 range-key-set b e @1 bop 427 range-key-set c z @1000 boop 428 range-key-del bat bus 429 ---- 430 wrote 4 keys 431 432 flush 433 ---- 434 435 lsm 436 ---- 437 0.1: 438 000008:[b#2111,RANGEKEYSET-z#72057594037927935,RANGEKEYSET] 439 0.0: 440 000006:[a@100#3,SET-zz@1#2104,SET] 441 442 scan-rangekeys 443 ---- 444 [b, bat) 445 @5=beep, @1=bop 446 [bus, c) 447 @5=beep, @1=bop 448 [c, e) 449 @1000=boop, @1=bop 450 [e, z) 451 @1000=boop 452 453 # NB: There are now 8 range key sets in the database. See the 7 range keys in 454 # the above scan-rangekeys. Additionally, the sstable flushed earlier up above 455 # included a rangekeyset [b,c) @5=beep. 456 457 metrics 458 ---- 459 Metrics.Keys.RangeKeySetsCount = 8 460 461 462 combined-iter 463 seek-prefix-ge ca 464 next 465 seek-prefix-ge ca@100 466 ---- 467 ca: (., [ca-"ca\x00") @1000=boop, @1=bop UPDATED) 468 ca@100: (ca@100, [ca-"ca\x00") @1000=boop, @1=bop) 469 ca@100: (ca@100, [ca-"ca\x00") @1000=boop, @1=bop) 470 471 472 # Perform the same iteration as above, but with @1000 range-key masking. The 473 # previously encountered point keys should be elided. 474 475 combined-iter mask-suffix=@1000 476 seek-prefix-ge ca 477 next 478 seek-prefix-ge ca@100 479 ---- 480 ca: (., [ca-"ca\x00") @1000=boop, @1=bop UPDATED) 481 . 482 ca@100: (., [ca-"ca\x00") @1000=boop, @1=bop UPDATED) 483 484 # Test masked, non-prefixed iteration. We should see the range keys, but all the 485 # points should be masked except those beginning with z which were excluded by 486 # the range key's exclusive z end bound. 487 488 combined-iter mask-suffix=@1000 489 seek-ge ca 490 next 491 next 492 next 493 next 494 next 495 ---- 496 ca: (., [c-e) @1000=boop, @1=bop UPDATED) 497 e: (., [e-z) @1000=boop UPDATED) 498 z@100: (z@100, . UPDATED) 499 z@10: (z@10, .) 500 z@1: (z@1, .) 501 za@100: (za@100, .) 502 503 reset 504 ---- 505 506 batch 507 range-key-set x z @5 boop 508 ---- 509 wrote 1 keys 510 511 combined-iter 512 last 513 next 514 prev 515 ---- 516 x: (., [x-z) @5=boop UPDATED) 517 . 518 x: (., [x-z) @5=boop UPDATED) 519 520 # Test limited reverse iteration. The seek-lt-limit z y must see the [x-z) range 521 # key because it covers a key within the range [y, z). The range key start 522 # boundary isn't until x. 523 524 combined-iter 525 seek-lt-limit z y 526 next 527 prev-limit y 528 ---- 529 x: valid (., [x-z) @5=boop UPDATED) 530 . 531 x: valid (., [x-z) @5=boop UPDATED) 532 533 # Test limited forward iteration. Since range keys are interleaved at the start 534 # boundaries, the iterator is guaranteed to encounter covering range keys 535 # without any special casing in the implementation. 536 537 combined-iter 538 seek-ge-limit w y 539 prev 540 next-limit y 541 ---- 542 x: valid (., [x-z) @5=boop UPDATED) 543 . 544 x: valid (., [x-z) @5=boop UPDATED) 545 546 # Test another limited backward iteration case where there exists a deleted 547 # point key and the underlying internalIterator is Prev'd to a key beyond the 548 # limit. This should still surface the covering range key. 549 550 batch 551 del yy 552 ---- 553 wrote 1 keys 554 555 combined-iter 556 seek-lt-limit z y 557 next 558 prev-limit y 559 ---- 560 x: valid (., [x-z) @5=boop UPDATED) 561 . 562 x: valid (., [x-z) @5=boop UPDATED) 563 564 # Applying range keys to a DB running with a version that doesn't support them 565 # results in an error. Range keys were added in version 7. 566 reset format-major-version=6 567 ---- 568 569 batch 570 range-key-set a z @5 boop 571 ---- 572 bitable: range keys require at least format major version 8 (current: 6) 573 574 # Constructing iterator over range keys on a DB that doesn't support them 575 # results in an error. 576 577 reset format-major-version=6 578 ---- 579 580 combined-iter 581 ---- 582 bitable: range keys require at least format major version 8 (current: 6) 583 584 # Test iterator bounds provided via IterOptions. 585 586 reset 587 ---- 588 589 batch 590 set a a 591 set b b 592 set c c 593 set d d 594 set f f 595 range-key-set a ap @6 foo 596 range-key-set ap c @5 bar 597 range-key-set cat zoo @3 bax 598 ---- 599 wrote 8 keys 600 601 # Ensure bounds provided at initialization are respected, and propagated to 602 # cloned iterators. 603 604 combined-iter lower=b upper=e 605 first 606 next 607 next 608 next 609 next 610 clone 611 first 612 next 613 next 614 next 615 next 616 ---- 617 b: (b, [b-c) @5=bar UPDATED) 618 c: (c, . UPDATED) 619 cat: (., [cat-e) @3=bax UPDATED) 620 d: (d, [cat-e) @3=bax) 621 . 622 . 623 b: (b, [b-c) @5=bar UPDATED) 624 c: (c, . UPDATED) 625 cat: (., [cat-e) @3=bax UPDATED) 626 d: (d, [cat-e) @3=bax) 627 . 628 629 # Ensure bounds provided during clone are propagated to cloned iterators. 630 631 combined-iter lower=b upper=e 632 first 633 next 634 next 635 next 636 next 637 clone lower=a upper=cat key-types=both 638 first 639 next 640 next 641 next 642 clone lower=a upper=cat key-types=point 643 first 644 next 645 next 646 next 647 ---- 648 b: (b, [b-c) @5=bar UPDATED) 649 c: (c, . UPDATED) 650 cat: (., [cat-e) @3=bax UPDATED) 651 d: (d, [cat-e) @3=bax) 652 . 653 . 654 a: (a, [a-ap) @6=foo UPDATED) 655 ap: (., [ap-c) @5=bar UPDATED) 656 b: (b, [ap-c) @5=bar) 657 c: (c, . UPDATED) 658 . 659 a:a 660 b:b 661 c:c 662 . 663 664 # Ensure bounds and key-types provided through SetOptions are respected. 665 666 combined-iter lower=b upper=e 667 first 668 next 669 next 670 next 671 next 672 set-options lower=a upper=cat key-types=both 673 first 674 next 675 next 676 next 677 set-options lower=a upper=cat key-types=point 678 first 679 next 680 next 681 next 682 ---- 683 b: (b, [b-c) @5=bar UPDATED) 684 c: (c, . UPDATED) 685 cat: (., [cat-e) @3=bax UPDATED) 686 d: (d, [cat-e) @3=bax) 687 . 688 . 689 a: (a, [a-ap) @6=foo UPDATED) 690 ap: (., [ap-c) @5=bar UPDATED) 691 b: (b, [ap-c) @5=bar) 692 c: (c, . UPDATED) 693 . 694 a:a 695 b:b 696 c:c 697 . 698 699 flush 700 ---- 701 702 metrics 703 ---- 704 Metrics.Keys.RangeKeySetsCount = 3 705 706 # Test Prev-ing back over a synthetic range key marker. Synthetic range-key 707 # markers (the keys interleaved at 'c' during a SeekGE(c) when there's a 708 # straddling range key) are ephemeral, and Prev-ing back must move back the 709 # appropriate number of times. 710 711 reset 712 ---- 713 714 batch 715 set a a 716 range-key-set b e @1 foo 717 ---- 718 wrote 2 keys 719 720 flush 721 ---- 722 723 combined-iter 724 seek-ge b 725 prev 726 seek-ge c 727 prev 728 ---- 729 b: (., [b-e) @1=foo UPDATED) 730 a: (a, . UPDATED) 731 c: (., [b-e) @1=foo UPDATED) 732 b: (., [b-e) @1=foo) 733 734 # Test a case during limited reverse iteration where a range key covers a 735 # portion of the keyspace within the limit. The iterator should NOT pause and 736 # should surface the range key. 737 738 reset 739 ---- 740 741 batch 742 del b 743 range-key-set a d @1 foo 744 ---- 745 wrote 2 keys 746 747 flush 748 ---- 749 750 combined-iter 751 seek-ge z 752 prev-limit c 753 ---- 754 . 755 a: valid (., [a-d) @1=foo UPDATED) 756 757 # Test a case during limited reverse iteration where there exists a range key 758 # but it ends before the limit. The iterator should pause. 759 760 reset 761 ---- 762 763 batch 764 del b 765 range-key-set a c @1 foo 766 ---- 767 wrote 2 keys 768 769 combined-iter 770 seek-ge z 771 prev-limit c 772 ---- 773 . 774 . at-limit 775 776 # Test a lazy-combined iteration edge. Consider the LSM: 777 # 778 # L5: 000003:[bar.DEL.3, foo.RANGEKEYSET.4] 779 # L6: 000001:[bar.SET.1] 000002:[bax.RANGEKEYSET.2] 780 # 781 # A call to First() seeks the levels to files L5.000003 and L6.000001. 782 # The L5 levelIter observes that L5.000003 contains the range key with 783 # start key `foo`, and triggers a switch to combined iteration, setting 784 # `combinedIterState.key` = `foo`. While switching to combined iteration, the 785 # iterator must recognize that `foo` > `bar`, and there may yet exist range keys 786 # that begin before `foo` (in this case `bax`). 787 788 reset 789 ---- 790 791 batch 792 set bar bar 793 ---- 794 wrote 1 keys 795 796 flush 797 ---- 798 799 batch 800 range-key-set bax zoo @1 foo 801 ---- 802 wrote 1 keys 803 804 flush 805 ---- 806 807 batch 808 del bar 809 range-key-set foo zoo @2 bar 810 ---- 811 wrote 2 keys 812 813 flush 814 ---- 815 816 lsm 817 ---- 818 0.1: 819 000009:[bar#3,DEL-zoo#72057594037927935,RANGEKEYSET] 820 0.0: 821 000005:[bar#1,SET-bar#1,SET] 822 000007:[bax#2,RANGEKEYSET-zoo#72057594037927935,RANGEKEYSET] 823 824 # Assert that First correctly finds [bax,zoo), despite the discovery of 825 # [foo,zoo) triggering the switch to combined iteration. 826 827 combined-iter 828 first 829 next 830 ---- 831 bax: (., [bax-foo) @1=foo UPDATED) 832 foo: (., [foo-zoo) @2=bar, @1=foo UPDATED) 833 834 # Test seeking into the middle of a range key during lazy-combined iteration. 835 # The iterator should surface Key() = the seek key. 836 837 combined-iter 838 seek-ge bop 839 ---- 840 bop: (., [bax-foo) @1=foo UPDATED) 841 842 combined-iter 843 last 844 ---- 845 foo: (., [foo-zoo) @2=bar, @1=foo UPDATED) 846 847 848 # Test a lazy combined iterator that must next/prev through fileMetdata when 849 # skipping through a RANGEDEL. 850 # 851 # L5 852 # b-----------------------y RANGEDEL 853 # L6 854 # [a] [[d,e)@1] [[l,m)@1] [z] 855 # 856 # A SeekGE(k) must surface [l,m)@1 and a SeekLT(k) must surface [d,e)@1. 857 858 reset 859 ---- 860 861 batch 862 set a a 863 ---- 864 wrote 1 keys 865 866 flush 867 ---- 868 869 batch 870 set z z 871 ---- 872 wrote 1 keys 873 874 flush 875 ---- 876 877 batch 878 range-key-set d e @1 foo 879 ---- 880 wrote 1 keys 881 882 flush 883 ---- 884 885 batch 886 range-key-set l m @1 foo 887 ---- 888 wrote 1 keys 889 890 flush 891 ---- 892 893 batch 894 del-range b y 895 ---- 896 wrote 1 keys 897 898 flush 899 ---- 900 901 lsm 902 ---- 903 0.1: 904 000013:[b#5,RANGEDEL-y#72057594037927935,RANGEDEL] 905 0.0: 906 000005:[a#1,SET-a#1,SET] 907 000009:[d#3,RANGEKEYSET-e#72057594037927935,RANGEKEYSET] 908 000011:[l#4,RANGEKEYSET-m#72057594037927935,RANGEKEYSET] 909 000007:[z#2,SET-z#2,SET] 910 911 combined-iter 912 seek-ge k 913 next 914 ---- 915 l: (., [l-m) @1=foo UPDATED) 916 z: (z, . UPDATED) 917 918 combined-iter 919 seek-lt k 920 prev 921 ---- 922 d: (., [d-e) @1=foo UPDATED) 923 a: (a, . UPDATED) 924 925 # Test a range key masking case where the range key is not immediately 926 # masking point keys, but masks point keys once positioned beneath it. 927 928 reset 929 ---- 930 931 batch 932 range-key-set d e @5 boop 933 set a@1 a1 934 set b@3 b3 935 set d@3 d3 936 ---- 937 wrote 4 keys 938 939 combined-iter mask-suffix=@9 940 first 941 next 942 next 943 next 944 ---- 945 a@1: (a1, .) 946 b@3: (b3, .) 947 d: (., [d-e) @5=boop UPDATED) 948 . 949 950 # Try a broad range key that masks all the point keys. 951 952 reset block-size=20 953 ---- 954 955 batch 956 range-key-set a z @5 boop 957 set a@1 foo 958 set b@3 foo 959 set c@3 foo 960 set d@1 foo 961 set e@3 foo 962 set f@3 foo 963 set g@2 foo 964 set h@2 foo 965 set i@2 foo 966 set j@2 foo 967 set k@0 foo 968 set l@2 foo 969 set m@1 foo 970 set n@3 foo 971 set o@4 foo 972 set p@2 foo 973 set q@2 foo 974 set r@1 foo 975 set s@2 foo 976 set t@3 foo 977 set u@2 foo 978 set v@0 foo 979 set w@0 foo 980 set x@2 foo 981 set y@4 foo 982 ---- 983 wrote 26 keys 984 985 flush 986 ---- 987 988 combined-iter mask-suffix=@9 989 first 990 next 991 stats 992 ---- 993 a: (., [a-z) @5=boop UPDATED) 994 . 995 stats: (interface (dir, seek, step): (fwd, 1, 1), (rev, 0, 0)), (internal (dir, seek, step): (fwd, 1, 1), (rev, 0, 0)), 996 (internal-stats: (block-bytes: (total 625 B, cached 0 B)), (points: (count 25, key-bytes 75, value-bytes 75, tombstoned: 0)) 997 998 # Repeat the above test, but with an iterator that uses a block-property filter 999 # mask. The internal stats should reflect fewer bytes read and fewer points 1000 # visited by the internal iterators. 1001 1002 combined-iter mask-suffix=@9 mask-filter 1003 first 1004 next 1005 stats 1006 ---- 1007 a: (., [a-z) @5=boop UPDATED) 1008 . 1009 stats: (interface (dir, seek, step): (fwd, 1, 1), (rev, 0, 0)), (internal (dir, seek, step): (fwd, 1, 1), (rev, 0, 0)), 1010 (internal-stats: (block-bytes: (total 50 B, cached 50 B)), (points: (count 2, key-bytes 6, value-bytes 6, tombstoned: 0)) 1011 1012 # Perform a similar comparison in reverse. 1013 1014 combined-iter mask-suffix=@9 1015 last 1016 prev 1017 stats 1018 ---- 1019 a: (., [a-z) @5=boop UPDATED) 1020 . 1021 stats: (interface (dir, seek, step): (fwd, 0, 0), (rev, 1, 1)), (internal (dir, seek, step): (fwd, 0, 0), (rev, 1, 1)), 1022 (internal-stats: (block-bytes: (total 625 B, cached 625 B)), (points: (count 25, key-bytes 75, value-bytes 75, tombstoned: 0)) 1023 1024 combined-iter mask-suffix=@9 mask-filter 1025 last 1026 prev 1027 stats 1028 ---- 1029 a: (., [a-z) @5=boop UPDATED) 1030 . 1031 stats: (interface (dir, seek, step): (fwd, 0, 0), (rev, 1, 1)), (internal (dir, seek, step): (fwd, 0, 0), (rev, 1, 1)), 1032 (internal-stats: (block-bytes: (total 50 B, cached 50 B)), (points: (count 2, key-bytes 6, value-bytes 6, tombstoned: 0)) 1033 1034 # Perform similar comparisons with seeks. 1035 1036 combined-iter mask-suffix=@9 1037 seek-ge m 1038 next 1039 stats 1040 ---- 1041 m: (., [a-z) @5=boop UPDATED) 1042 . 1043 stats: (interface (dir, seek, step): (fwd, 1, 1), (rev, 0, 0)), (internal (dir, seek, step): (fwd, 1, 1), (rev, 0, 0)), 1044 (internal-stats: (block-bytes: (total 325 B, cached 325 B)), (points: (count 13, key-bytes 39, value-bytes 39, tombstoned: 0)) 1045 1046 combined-iter mask-suffix=@9 mask-filter 1047 seek-ge m 1048 next 1049 stats 1050 ---- 1051 m: (., [a-z) @5=boop UPDATED) 1052 . 1053 stats: (interface (dir, seek, step): (fwd, 1, 1), (rev, 0, 0)), (internal (dir, seek, step): (fwd, 1, 1), (rev, 0, 0)), 1054 (internal-stats: (block-bytes: (total 50 B, cached 50 B)), (points: (count 2, key-bytes 6, value-bytes 6, tombstoned: 0)) 1055 1056 combined-iter mask-suffix=@9 1057 seek-lt m 1058 prev 1059 stats 1060 ---- 1061 a: (., [a-z) @5=boop UPDATED) 1062 . 1063 stats: (interface (dir, seek, step): (fwd, 0, 0), (rev, 1, 1)), (internal (dir, seek, step): (fwd, 0, 0), (rev, 1, 1)), 1064 (internal-stats: (block-bytes: (total 325 B, cached 325 B)), (points: (count 12, key-bytes 36, value-bytes 36, tombstoned: 0)) 1065 1066 combined-iter mask-suffix=@9 mask-filter 1067 seek-lt m 1068 prev 1069 stats 1070 ---- 1071 a: (., [a-z) @5=boop UPDATED) 1072 . 1073 stats: (interface (dir, seek, step): (fwd, 0, 0), (rev, 1, 1)), (internal (dir, seek, step): (fwd, 0, 0), (rev, 1, 1)), 1074 (internal-stats: (block-bytes: (total 75 B, cached 75 B)), (points: (count 2, key-bytes 6, value-bytes 6, tombstoned: 0)) 1075 1076 # Test repeated seeks into the same range key, while TrySeekUsingNext=true. 1077 # Test for regression fixed in #1849. 1078 1079 reset 1080 ---- 1081 1082 batch 1083 range-key-set a c @5 boop 1084 range-key-set c e @5 beep 1085 ---- 1086 wrote 2 keys 1087 1088 combined-iter 1089 seek-ge a 1090 seek-ge b 1091 ---- 1092 a: (., [a-c) @5=boop UPDATED) 1093 b: (., [a-c) @5=boop) 1094 1095 # Test a few cases around when a combined iterator should be lazy or not. 1096 1097 reset 1098 ---- 1099 1100 batch 1101 set a a 1102 set b b 1103 set c c 1104 set e e 1105 range-key-del a f 1106 range-key-unset a f @5 1107 ---- 1108 wrote 6 keys 1109 1110 flush 1111 ---- 1112 1113 wait-table-stats 1114 ---- 1115 1116 # The lazy iterator shouldn't switch to combined iteration when it encounters a 1117 # file that is known to only contain RANGEKEYDELs and RANGEKEYUNSETs. 1118 1119 combined-iter 1120 is-using-combined 1121 seek-ge a 1122 seek-ge b 1123 is-using-combined 1124 ---- 1125 using lazy iterator 1126 a: (a, .) 1127 b: (b, .) 1128 using lazy iterator 1129 1130 # Write a range key to the memtable. The combined iterator should be forced to 1131 # use non-lazy iteration. 1132 1133 batch 1134 range-key-set m z @5 foo 1135 set s s 1136 ---- 1137 wrote 2 keys 1138 1139 combined-iter 1140 is-using-combined 1141 seek-ge a 1142 is-using-combined 1143 seek-ge n 1144 is-using-combined 1145 ---- 1146 using combined (non-lazy) iterator 1147 a: (a, .) 1148 using combined (non-lazy) iterator 1149 n: (., [m-z) @5=foo UPDATED) 1150 using combined (non-lazy) iterator 1151 1152 flush 1153 ---- 1154 1155 # Now that the range key is flushed, a switch to combined iteration should only 1156 # happen once the sstable containing the set is encountered. 1157 1158 combined-iter 1159 is-using-combined 1160 seek-ge a 1161 is-using-combined 1162 seek-ge n 1163 is-using-combined 1164 ---- 1165 using lazy iterator 1166 a: (a, .) 1167 using lazy iterator 1168 n: (., [m-z) @5=foo UPDATED) 1169 using combined (non-lazy) iterator 1170 1171 # Regression test for a bug discovered in #1878. 1172 # A lazy-combined iterator triggers combined iteration during an initial 1173 # seek-prefix-ge call. The initial seek-prefix-ge call avoids defragmenting 1174 # fragments beyond the initial fragment [c,f). A subsequent seek-ge that seeks 1175 # within the bounds of the initial fragment [c,f) must not fall into the 1176 # optimization that reuses the span without reseeking the keypsan iterator, 1177 # because the span is not defragmented. 1178 # 1179 # In the bug surfaced by #1878, the initial seek-prefix-ge that switched to 1180 # combined iteration failed to record that the iterator was now in prefix mode, 1181 # allowing the subsequent seek-ge to incorrectly reuse the existing span. 1182 1183 reset 1184 ---- 1185 1186 batch 1187 range-key-set a c @5 foo 1188 ---- 1189 wrote 1 keys 1190 1191 flush 1192 ---- 1193 1194 batch 1195 range-key-set c f @5 foo 1196 ---- 1197 wrote 1 keys 1198 1199 flush 1200 ---- 1201 1202 batch 1203 range-key-set f m @5 foo 1204 ---- 1205 wrote 1 keys 1206 1207 flush 1208 ---- 1209 1210 lsm 1211 ---- 1212 0.0: 1213 000005:[a#1,RANGEKEYSET-c#72057594037927935,RANGEKEYSET] 1214 000007:[c#2,RANGEKEYSET-f#72057594037927935,RANGEKEYSET] 1215 000009:[f#3,RANGEKEYSET-m#72057594037927935,RANGEKEYSET] 1216 1217 combined-iter 1218 seek-prefix-ge d@5 1219 seek-ge d 1220 ---- 1221 d@5: (., [d-"d\x00") @5=foo UPDATED) 1222 d: (., [a-m) @5=foo UPDATED) 1223 1224 # Test that repeated SeekPrefixGEs correctly return truncated spans with 1225 # RangeKeyChanged() -> UPDATED. 1226 1227 combined-iter 1228 seek-prefix-ge c@5 1229 seek-prefix-ge d@5 1230 seek-ge d@7 1231 seek-prefix-ge d@7 1232 ---- 1233 c@5: (., [c-"c\x00") @5=foo UPDATED) 1234 d@5: (., [d-"d\x00") @5=foo UPDATED) 1235 d@7: (., [a-m) @5=foo UPDATED) 1236 d@7: (., [d-"d\x00") @5=foo UPDATED) 1237 1238 # Test a LSM with range keys fragmented within a prefix. 1239 # This is a regression test for cockroachdb/cockroach#86102. 1240 1241 reset target-file-size=1 1242 ---- 1243 1244 batch 1245 range-key-set a c @1 bar 1246 range-key-set c e @1 foo 1247 set c@9 c@9 1248 set c@8 c@8 1249 set c@7 c@7 1250 set c@6 c@6 1251 set c@5 c@5 1252 set c@4 c@4 1253 set c@3 c@3 1254 set c@2 c@2 1255 set d@0 d@0 1256 range-key-set y z @1 foo 1257 set z z 1258 ---- 1259 wrote 13 keys 1260 1261 flush 1262 ---- 1263 1264 lsm 1265 ---- 1266 0.0: 1267 000005:[a#1,RANGEKEYSET-c@8#72057594037927935,RANGEKEYSET] 1268 000006:[c@8#4,SET-c@7#72057594037927935,RANGEKEYSET] 1269 000007:[c@7#5,SET-c@6#72057594037927935,RANGEKEYSET] 1270 000008:[c@6#6,SET-c@5#72057594037927935,RANGEKEYSET] 1271 000009:[c@5#7,SET-c@4#72057594037927935,RANGEKEYSET] 1272 000010:[c@4#8,SET-c@3#72057594037927935,RANGEKEYSET] 1273 000011:[c@3#9,SET-c@2#72057594037927935,RANGEKEYSET] 1274 000012:[c@2#10,SET-d@0#72057594037927935,RANGEKEYSET] 1275 000013:[d@0#11,SET-e#72057594037927935,RANGEKEYSET] 1276 000014:[y#12,RANGEKEYSET-z#13,SET] 1277 1278 # The first seek-prefix-ge y@1 converts the iterator from lazy combined iterator 1279 # to combined iteration. 1280 # 1281 # The second seek-prefix-ge d@1 does not fully defragment the range key. The 1282 # underlying range key is defragmented to [c@2,e). This incomplete 1283 # defragmentation is still hidden from the user at this point, since the range 1284 # key is truncated to [d,d\x00). 1285 # 1286 # The third seek-prefix-ge c@0 seeks to a key that falls within the 1287 # range key currently defragmented on interleaving iterator. A previous bug 1288 # would use this span without defragmenting the span to include the full 1289 # span of the prefix [c,c\x00). 1290 1291 combined-iter 1292 seek-prefix-ge y@1 1293 seek-prefix-ge d@1 1294 seek-prefix-ge c@0 1295 ---- 1296 y@1: (., [y-"y\x00") @1=foo UPDATED) 1297 d@1: (., [d-"d\x00") @1=foo UPDATED) 1298 c@0: (., [c-"c\x00") @1=foo UPDATED) 1299 1300 # Test a LSM with range keys fragmented within a prefix. 1301 # This is a regression test for cockroachdb/cockroach#86102. 1302 1303 reset 1304 ---- 1305 1306 ingest ext1 1307 range-key-set a c@8 @1 bar 1308 set c@9 c@9 1309 ---- 1310 1311 ingest ext2 1312 range-key-set c@8 e @1 bar 1313 set c@8 c@8 1314 set c@7 c@7 1315 set c@6 c@6 1316 set c@5 c@5 1317 set c@4 c@4 1318 set c@3 c@3 1319 set c@2 c@2 1320 ---- 1321 1322 ingest ext2 1323 range-key-set y z @1 foo 1324 set z z 1325 ---- 1326 1327 lsm 1328 ---- 1329 6: 1330 000004:[a#1,RANGEKEYSET-c@8#72057594037927935,RANGEKEYSET] 1331 000005:[c@8#2,RANGEKEYSET-e#72057594037927935,RANGEKEYSET] 1332 000006:[y#3,RANGEKEYSET-z#3,SET] 1333 1334 1335 # The first seek-prefix-ge y@1 converts the iterator from lazy combined iterator 1336 # to combined iteration. 1337 # 1338 # The second seek-prefix-ge d@1 does not fully defragment the range key. The 1339 # underlying range key is defragmented to [a,c@8). This incomplete 1340 # defragmentation is still hidden from the user at this point, since the range 1341 # key is truncated to [a,a\x00). 1342 # 1343 # The third seek-prefix-ge c@10 seeks to a key that falls within the 1344 # range key currently defragmented on interleaving iterator. A previous bug 1345 # would use this span without defragmenting the span to include the full 1346 # span of the prefix [c,c\x00). 1347 1348 combined-iter 1349 seek-prefix-ge y@1 1350 seek-prefix-ge a@1 1351 seek-prefix-ge c@10 1352 ---- 1353 y@1: (., [y-"y\x00") @1=foo UPDATED) 1354 a@1: (., [a-"a\x00") @1=bar UPDATED) 1355 c@10: (., [c-"c\x00") @1=bar UPDATED) 1356 1357 # Ensure that no-op optimizations do not reuse range key iterator state across 1358 # SetOptions calls. No-op optimizations have the potential to fail to update 1359 # RangeKeyChanged(). 1360 1361 reset 1362 ---- 1363 1364 batch 1365 range-key-set p s @1 foo 1366 ---- 1367 wrote 1 keys 1368 1369 combined-iter lower=n@9 upper=x@5 1370 seek-lt y@3 1371 set-options lower=n@9 upper=x@5 1372 seek-lt-limit t o 1373 ---- 1374 p: (., [p-s) @1=foo UPDATED) 1375 . 1376 p: valid (., [p-s) @1=foo UPDATED) 1377 1378 combined-iter lower=n@9 upper=x@5 1379 seek-ge o 1380 set-options lower=n@9 upper=x@5 1381 seek-ge oat 1382 ---- 1383 p: (., [p-s) @1=foo UPDATED) 1384 . 1385 p: (., [p-s) @1=foo UPDATED) 1386 1387 combined-iter lower=n@9 upper=x@5 1388 seek-prefix-ge p@5 1389 set-options lower=n@9 upper=x@5 1390 seek-prefix-ge p 1391 ---- 1392 p@5: (., [p-"p\x00") @1=foo UPDATED) 1393 . 1394 p: (., [p-"p\x00") @1=foo UPDATED) 1395 1396 # Regression test for an invariant violation in the range key defragmenting 1397 # iterator during prefix iteration. [Related to #1893]. There is a lot of 1398 # subtlety here. Do not modify this test case without verifying that it still 1399 # exercises the right conditions. 1400 # 1401 # Normally during forward iteration, if a switch to lazy-combined iteration is 1402 # triggered, the lazy-combined iterator establishes a seek key for the range key 1403 # iterator such that the seek key is: 1404 # 1. greater than or equal to the key at previous iterator position. 1405 # 2. less than or equal to the first range key with a start key greater than 1406 # or equal to the previous iterator position. 1407 # These invariants are important so that the range key iterator is positioned 1408 # appropriately after the switch to combined iteration and no range keys are 1409 # missed. 1410 # 1411 # Parts of the iterator stack depend on the above invariants. For example, 1412 # during forward iteration the BoundedIter only checks span start keys against 1413 # iterator bounds and the configured prefix, with the expectation that the seek 1414 # is always already greater than or equal to the lower bound. In turn, the 1415 # DefragmentingIter indirectly relies on the same invariant, because it requires 1416 # a consistent view of the fragments. If the BoundedIter returns a span in one 1417 # direction, but skips it when iterating back, the defragmenting iterator will 1418 # end up on a different fragment. 1419 # 1420 # This test exercises a case in which previously, during prefix iteration, it 1421 # was possible for the switch to lazy-combined iteration to trigger using a seek 1422 # key k, such that there exist range key fragments between the current iterator 1423 # position and k (violating the 2nd invariant up above). 1424 # 1425 # The sequence of events is: 1426 # 1. SeekPrefixGE("b@9") = 'b@4': 1427 # a. This seek positions the two levels, L0 and L6. The L0 iterator seeks 1428 # to file 000006. This file does not contain any keys with the prefix 1429 # "b", and the bloom filter must succeed in excluding the file. Since the 1430 # file contains a range deletion, SeekPrefixGE returns the level's 1431 # largest point key (`d#inf,RANGEDEL`) to ensure the file stays open until 1432 # the iterator advances past the range deletion. 1433 # b. In L6, the level iterator seeks to 000004 which contains a key with 1434 # the prefix, returning 'b@4'. 1435 # 2. Next(): 1436 # a. Next advances the the L6 iterator to file 000005. This file contains a 1437 # range key [e,f)@1=bar, which updates the lazy-combined iterator's 1438 # state, recording the earliest observed range key as 'e'. The L6 level 1439 # iterator then returns the file single point key 'c'. 1440 # b. The merging iterator checks whether point key 'c' is deleted by any 1441 # range key deletions. It is. It's deleted by L0's [c,d) range deletion. 1442 # The merging iterator then seeks the iterator to the tombstone's end 1443 # key 'd'. 1444 # c. After seeking, the range deletion sentinel d is at the top of the 1445 # heap. At this point, the merging iterator checks whether the keyspace 1446 # of the prefix has been exceeded, and it has. It returns nil. 1447 # 3. Switch to combined iteration: 1448 # a. The Next has completed and triggered combined iteration. The only file 1449 # containing range keys that was observed was 000005, containing the 1450 # range key [e,f). The switch to combined iteration seeks the keyspan 1451 # iterator to 'e'. Note that the iterator never observed L0's [d,e) 1452 # range key that precedes [e,f) in the keyspace. 1453 # b. Seeking the keyspan iterator calls DefragmentingIter.SeekLT('e'), 1454 # which lands on the [d,e) fragment. This fragment does NOT check to see 1455 # if the span starts at a prefix greater than the current prefix 'b', 1456 # because only bounds in the direction of iteration are check. 1457 # c. The DefragmentingIter observes disappearing range key fragments when 1458 # it switches directions, as a result of (b). 1459 # 1460 1461 # Use 100-bits per key to ensure the bloom filter provides total recall. 1462 reset bloom-bits-per-key=100 1463 ---- 1464 1465 # Ingest L6 files: 1466 # 1467 # 000004: b@4 1468 # 000005: c, [e,f)@1=bar 1469 1470 ingest ext1 1471 set b@4 b@4 1472 ---- 1473 1474 ingest ext1 1475 set c c 1476 range-key-set e f @1 bar 1477 ---- 1478 1479 # Ingest L0 files: 1480 # 1481 # 000006: a, del-range(c, d) 1482 # 000007: [d,e)@1=bar 1483 1484 ingest ext2 1485 set a a 1486 del-range c d 1487 ---- 1488 1489 ingest ext3 1490 range-key-set d e @1 bar 1491 ---- 1492 1493 lsm 1494 ---- 1495 0.0: 1496 000006:[a#3,SET-d#72057594037927935,RANGEDEL] 1497 000007:[d#4,RANGEKEYSET-e#72057594037927935,RANGEKEYSET] 1498 6: 1499 000004:[b@4#1,SET-b@4#1,SET] 1500 000005:[c#2,SET-f#72057594037927935,RANGEKEYSET] 1501 1502 combined-iter 1503 seek-prefix-ge b@9 1504 next 1505 ---- 1506 b@4: (b@4, .) 1507 .