github.com/bytedance/gopkg@v0.0.0-20240514070511-01b2cbcf35e1/collection/skipset/readme.md (about) 1 <p align="center"> 2 <img src="https://raw.githubusercontent.com/zhangyunhao116/public-data/master/skipset-logo2.png"/> 3 </p> 4 5 ## Introduction 6 7 skipset is a high-performance concurrent set based on skip list. In typical pattern(100000 operations, 90%CONTAINS 9%ADD 1%REMOVE), the skipset up to 3x ~ 15x faster than the built-in sync.Map. 8 9 The main idea behind the skipset is [A Simple Optimistic Skiplist Algorithm](<https://people.csail.mit.edu/shanir/publications/LazySkipList.pdf>). 10 11 Different from the sync.Map, the items in the skipset are always sorted, and the `Contains` and `Range` operations are wait-free (A goroutine is guaranteed to complete an operation as long as it keeps taking steps, regardless of the activity of other goroutines). 12 13 14 ## Features 15 16 - Concurrent safe API with high-performance. 17 - Wait-free Contains and Range operations. 18 - Sorted items. 19 20 21 22 ## When should you use skipset 23 24 In these situations, `skipset` is better 25 26 - **Sorted elements is needed**. 27 - **Concurrent calls multiple operations**. such as use both `Contains` and `Add` at the same time. 28 - **Memory intensive**. The skipset save at least 50% memory in the benchmark. 29 30 In these situations, `sync.Map` is better 31 32 - Only one goroutine access the set for most of the time, such as insert a batch of elements and then use only `Contains` (use built-in map is even better). 33 34 35 36 ## QuickStart 37 38 ```go 39 package main 40 41 import ( 42 "fmt" 43 44 "github.com/bytedance/gopkg/collection/skipset" 45 ) 46 47 func main() { 48 l := NewInt() 49 50 for _, v := range []int{10, 12, 15} { 51 if l.Add(v) { 52 fmt.Println("skipset add", v) 53 } 54 } 55 56 if l.Contains(10) { 57 fmt.Println("skipset contains 10") 58 } 59 60 l.Range(func(value int) bool { 61 fmt.Println("skipset range found ", value) 62 return true 63 }) 64 65 l.Remove(15) 66 fmt.Printf("skipset contains %d items\r\n", l.Len()) 67 } 68 69 ``` 70 71 72 73 ## Benchmark 74 75 Go version: go1.16.2 linux/amd64 76 77 CPU: AMD 3700x(8C16T), running at 3.6GHz 78 79 OS: ubuntu 18.04 80 81 MEMORY: 16G x 2 (3200MHz) 82 83  84 85 ```shell 86 $ go test -run=NOTEST -bench=. -benchtime=100000x -benchmem -count=20 -timeout=60m > x.txt 87 $ benchstat x.txt 88 ``` 89 90 ``` 91 name time/op 92 Int64/Add/skipset-16 86.6ns ±11% 93 Int64/Add/sync.Map-16 674ns ± 6% 94 Int64/Contains50Hits/skipset-16 9.85ns ±12% 95 Int64/Contains50Hits/sync.Map-16 14.7ns ±30% 96 Int64/30Add70Contains/skipset-16 38.8ns ±18% 97 Int64/30Add70Contains/sync.Map-16 586ns ± 5% 98 Int64/1Remove9Add90Contains/skipset-16 24.9ns ±17% 99 Int64/1Remove9Add90Contains/sync.Map-16 493ns ± 5% 100 Int64/1Range9Remove90Add900Contains/skipset-16 25.9ns ±16% 101 Int64/1Range9Remove90Add900Contains/sync.Map-16 1.00µs ±12% 102 String/Add/skipset-16 130ns ±14% 103 String/Add/sync.Map-16 878ns ± 4% 104 String/Contains50Hits/skipset-16 18.3ns ± 9% 105 String/Contains50Hits/sync.Map-16 19.2ns ±18% 106 String/30Add70Contains/skipset-16 61.0ns ±15% 107 String/30Add70Contains/sync.Map-16 756ns ± 7% 108 String/1Remove9Add90Contains/skipset-16 31.3ns ±13% 109 String/1Remove9Add90Contains/sync.Map-16 614ns ± 6% 110 String/1Range9Remove90Add900Contains/skipset-16 36.2ns ±18% 111 String/1Range9Remove90Add900Contains/sync.Map-16 1.20µs ±17% 112 113 name alloc/op 114 Int64/Add/skipset-16 65.0B ± 0% 115 Int64/Add/sync.Map-16 128B ± 1% 116 Int64/Contains50Hits/skipset-16 0.00B 117 Int64/Contains50Hits/sync.Map-16 0.00B 118 Int64/30Add70Contains/skipset-16 19.0B ± 0% 119 Int64/30Add70Contains/sync.Map-16 77.7B ±16% 120 Int64/1Remove9Add90Contains/skipset-16 5.00B ± 0% 121 Int64/1Remove9Add90Contains/sync.Map-16 57.5B ± 4% 122 Int64/1Range9Remove90Add900Contains/skipset-16 5.00B ± 0% 123 Int64/1Range9Remove90Add900Contains/sync.Map-16 255B ±22% 124 String/Add/skipset-16 97.0B ± 0% 125 String/Add/sync.Map-16 152B ± 0% 126 String/Contains50Hits/skipset-16 15.0B ± 0% 127 String/Contains50Hits/sync.Map-16 15.0B ± 0% 128 String/30Add70Contains/skipset-16 40.0B ± 0% 129 String/30Add70Contains/sync.Map-16 98.2B ±11% 130 String/1Remove9Add90Contains/skipset-16 23.0B ± 0% 131 String/1Remove9Add90Contains/sync.Map-16 73.9B ± 4% 132 String/1Range9Remove90Add900Contains/skipset-16 23.0B ± 0% 133 String/1Range9Remove90Add900Contains/sync.Map-16 261B ±18% 134 135 name allocs/op 136 Int64/Add/skipset-16 1.00 ± 0% 137 Int64/Add/sync.Map-16 4.00 ± 0% 138 Int64/Contains50Hits/skipset-16 0.00 139 Int64/Contains50Hits/sync.Map-16 0.00 140 Int64/30Add70Contains/skipset-16 0.00 141 Int64/30Add70Contains/sync.Map-16 1.00 ± 0% 142 Int64/1Remove9Add90Contains/skipset-16 0.00 143 Int64/1Remove9Add90Contains/sync.Map-16 0.00 144 Int64/1Range9Remove90Add900Contains/skipset-16 0.00 145 Int64/1Range9Remove90Add900Contains/sync.Map-16 0.00 146 String/Add/skipset-16 2.00 ± 0% 147 String/Add/sync.Map-16 5.00 ± 0% 148 String/Contains50Hits/skipset-16 1.00 ± 0% 149 String/Contains50Hits/sync.Map-16 1.00 ± 0% 150 String/30Add70Contains/skipset-16 1.00 ± 0% 151 String/30Add70Contains/sync.Map-16 2.00 ± 0% 152 String/1Remove9Add90Contains/skipset-16 1.00 ± 0% 153 String/1Remove9Add90Contains/sync.Map-16 1.00 ± 0% 154 String/1Range9Remove90Add900Contains/skipset-16 1.00 ± 0% 155 String/1Range9Remove90Add900Contains/sync.Map-16 1.00 ± 0% 156 ```