github.com/bingoohuang/gg@v0.0.0-20240325092523-45da7dee9335/pkg/uid/ksuid.md (about)

     1  # ksuid
     2  
     3  ksuid is an efficient, comprehensive, battle-tested Go library for generating and parsing a specific kind of globally
     4  unique identifier called a *KSUID*. This library serves as its reference implementation.
     5  
     6  ## Install
     7  
     8  ```sh
     9  go get -u github.com/segmentio/ksuid
    10  ```
    11  
    12  ## What is a KSUID?
    13  
    14  KSUID is for K-Sortable Unique Identifier. It is a kind of globally unique identifier similar to
    15  a [RFC 4122 UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier), built from the ground-up to be "
    16  naturally"
    17  sorted by generation timestamp without any special type-aware logic.
    18  
    19  In short, running a set of KSUIDs through the UNIX `sort` command will result in a list ordered by generation time.
    20  
    21  ## Why use KSUIDs?
    22  
    23  There are numerous methods for generating unique identifiers, so why KSUID?
    24  
    25  1. Naturally ordered by generation time
    26  2. Collision-free, coordination-free, dependency-free
    27  3. Highly portable representations
    28  
    29  Even if only one of these properties are important to you, KSUID is a great choice! :) Many projects chose to use
    30  KSUIDs *just* because the text representation is copy-and-paste friendly.
    31  
    32  ### 1. Naturally Ordered By Generation Time
    33  
    34  Unlike the more ubiquitous UUIDv4, a KSUID contains a timestamp component that allows them to be loosely sorted by
    35  generation time. This is not a strong guarantee (an invariant) as it depends on wall clocks, but is still incredibly
    36  useful in practice. Both the binary and text representations will sort by creation time without any special sorting
    37  logic.
    38  
    39  ### 2. Collision-free, Coordination-free, Dependency-free
    40  
    41  While RFC 4122 UUIDv1s *do* include a time component, there aren't enough bytes of randomness to provide strong
    42  protection against collisions
    43  (duplicates). With such a low amount of entropy, it is feasible for a malicious party to guess generated IDs, creating a
    44  problem for systems whose security is, implicitly or explicitly, sensitive to an adversary guessing identifiers.
    45  
    46  To fit into a 64-bit number space, [Snowflake IDs](https://blog.twitter.com/2010/announcing-snowflake)
    47  and its derivatives require coordination to avoid collisions, which significantly increases the deployment complexity
    48  and operational burden.
    49  
    50  A KSUID includes 128 bits of pseudorandom data ("entropy"). This number space is 64 times larger than the 122 bits used
    51  by the well-accepted RFC 4122 UUIDv4 standard. The additional timestamp component can be considered "bonus entropy"
    52  which further decreases the probability of collisions, to the point of physical infeasibility in any practical
    53  implementation.
    54  
    55  ### Highly Portable Representations
    56  
    57  The text *and* binary representations are lexicographically sortable, which allows them to be dropped into systems which
    58  do not natively support KSUIDs and retain their time-ordered property.
    59  
    60  The text representation is an alphanumeric base62 encoding, so it "fits"
    61  anywhere alphanumeric strings are accepted. No delimiters are used, so stringified KSUIDs won't be inadvertently
    62  truncated or tokenized when interpreted by software that is designed for human-readable text, a common problem for the
    63  text representation of RFC 4122 UUIDs.
    64  
    65  ## How do KSUIDs work?
    66  
    67  Binary KSUIDs are 20-bytes: a 32-bit unsigned integer UTC timestamp and a 128-bit randomly generated payload. The
    68  timestamp uses big-endian encoding, to support lexicographic sorting. The timestamp epoch is adjusted to May 13th, 2014,
    69  providing over 100 years of life. The payload is generated by a cryptographically-strong pseudorandom number generator.
    70  
    71  The text representation is always 27 characters, encoded in alphanumeric base62 that will lexicographically sort by
    72  timestamp.
    73  
    74  ## High Performance
    75  
    76  This library is designed to be used in code paths that are performance critical. Its code has been tuned to eliminate
    77  all non-essential overhead. The `KSUID` type is derived from a fixed-size array, which eliminates the additional
    78  reference chasing and allocation involved in a variable-width type.
    79  
    80  The API provides an interface for use in code paths which are sensitive to allocation. For example, the `Append` method
    81  can be used to parse the text representation and replace the contents of a `KSUID` value without additional heap
    82  allocation.
    83  
    84  All public package level "pure" functions are concurrency-safe, protected by a global mutex. For hot loops that generate
    85  a large amount of KSUIDs from a single Goroutine, the `Sequence` type is provided to elide the potential contention.
    86  
    87  By default, out of an abundance of caution, the cryptographically-secure PRNG is used to generate the random bits of a
    88  KSUID. This can be relaxed in extremely performance-critical code using the included `FastRander`
    89  type. `FastRander` uses the standard PRNG with a seed generated by the cryptographically-secure PRNG.
    90  
    91  *_NOTE:_ While there is no evidence that `FastRander` will increase the probability of a collision, it shouldn't be used
    92  in scenarios where uniqueness is important to security, as there is an increased chance the generated IDs can be
    93  predicted by an adversary.*
    94  
    95  ## Battle Tested
    96  
    97  This code has been used in production at Segment for several years, across a diverse array of projects. Trillions upon
    98  trillions of KSUIDs have been generated in some of Segment's most performance-critical, large-scale distributed systems.
    99  
   100  ## Plays Well With Others
   101  
   102  Designed to be integrated with other libraries, the `KSUID` type implements many standard library interfaces, including:
   103  
   104  * `Stringer`
   105  * `database/sql.Scanner` and `database/sql/driver.Valuer`
   106  * `encoding.BinaryMarshal` and `encoding.BinaryUnmarshal`
   107  * `encoding.TextMarshal` and `encoding.TextUnmarshal`
   108    (`encoding/json` friendly!)
   109  
   110  ## Command Line Tool
   111  
   112  This package comes with a command-line tool `ksuid`, useful for generating KSUIDs as well as inspecting the internal
   113  components of existing KSUIDs. Machine-friendly output is provided for scripting use cases.
   114  
   115  ## CLI Usage Examples
   116  
   117  ### Generate a KSUID
   118  
   119  ```sh
   120  $ ksuid
   121  0ujsswThIGTUYm2K8FjOOfXtY1K
   122  ```
   123  
   124  ### Generate 4 KSUIDs
   125  
   126  ```sh
   127  $ ksuid -n 4
   128  0ujsszwN8NRY24YaXiTIE2VWDTS
   129  0ujsswThIGTUYm2K8FjOOfXtY1K
   130  0ujssxh0cECutqzMgbtXSGnjorm
   131  0ujsszgFvbiEr7CDgE3z8MAUPFt
   132  ```
   133  
   134  ### Inspect the components of a KSUID
   135  
   136  ```sh
   137  $ ksuid -f inspect 0ujtsYcgvSTl8PAuAdqWYSMnLOv
   138  
   139  REPRESENTATION:
   140  
   141    String: 0ujtsYcgvSTl8PAuAdqWYSMnLOv
   142       Raw: 0669F7EFB5A1CD34B5F99D1154FB6853345C9735
   143  
   144  COMPONENTS:
   145  
   146         Time: 2017-10-09 21:00:47 -0700 PDT
   147    Timestamp: 107608047
   148      Payload: B5A1CD34B5F99D1154FB6853345C9735
   149  ```
   150  
   151  ### Generate a KSUID and inspect its components
   152  
   153  ```sh
   154  $ ksuid -f inspect
   155  
   156  REPRESENTATION:
   157  
   158    String: 0ujzPyRiIAffKhBux4PvQdDqMHY
   159       Raw: 066A029C73FC1AA3B2446246D6E89FCD909E8FE8
   160  
   161  COMPONENTS:
   162  
   163         Time: 2017-10-09 21:46:20 -0700 PDT
   164    Timestamp: 107610780
   165      Payload: 73FC1AA3B2446246D6E89FCD909E8FE8
   166  
   167  ```
   168  
   169  ### Inspect a KSUID with template formatted inspection output
   170  
   171  ```sh
   172  $ ksuid -f template -t '{{ .Time }}: {{ .Payload }}' 0ujtsYcgvSTl8PAuAdqWYSMnLOv
   173  2017-10-09 21:00:47 -0700 PDT: B5A1CD34B5F99D1154FB6853345C9735
   174  ```
   175  
   176  ### Inspect multiple KSUIDs with template formatted output
   177  
   178  ```sh
   179  $ ksuid -f template -t '{{ .Time }}: {{ .Payload }}' $(ksuid -n 4)
   180  2017-10-09 21:05:37 -0700 PDT: 304102BC687E087CC3A811F21D113CCF
   181  2017-10-09 21:05:37 -0700 PDT: EAF0B240A9BFA55E079D887120D962F0
   182  2017-10-09 21:05:37 -0700 PDT: DF0761769909ABB0C7BB9D66F79FC041
   183  2017-10-09 21:05:37 -0700 PDT: 1A8F0E3D0BDEB84A5FAD702876F46543
   184  ```
   185  
   186  ### Generate KSUIDs and output JSON using template formatting
   187  
   188  ```sh
   189  $ ksuid -f template -t '{ "timestamp": "{{ .Timestamp }}", "payload": "{{ .Payload }}", "ksuid": "{{.String}}"}' -n 4
   190  { "timestamp": "107611700", "payload": "9850EEEC191BF4FF26F99315CE43B0C8", "ksuid": "0uk1Hbc9dQ9pxyTqJ93IUrfhdGq"}
   191  { "timestamp": "107611700", "payload": "CC55072555316F45B8CA2D2979D3ED0A", "ksuid": "0uk1HdCJ6hUZKDgcxhpJwUl5ZEI"}
   192  { "timestamp": "107611700", "payload": "BA1C205D6177F0992D15EE606AE32238", "ksuid": "0uk1HcdvF0p8C20KtTfdRSB9XIm"}
   193  { "timestamp": "107611700", "payload": "67517BA309EA62AE7991B27BB6F2FCAC", "ksuid": "0uk1Ha7hGJ1Q9Xbnkt0yZgNwg3g"}
   194  ```
   195  
   196  ## Implementations for other languages
   197  
   198  - Python: [svix-ksuid](https://github.com/svixhq/python-ksuid/)
   199  - Ruby: [ksuid-ruby](https://github.com/michaelherold/ksuid-ruby)
   200  - Java: [ksuid](https://github.com/ksuid/ksuid)
   201  - Rust: [rksuid](https://github.com/nharring/rksuid)
   202  - dotNet: [Ksuid.Net](https://github.com/JoyMoe/Ksuid.Net)
   203  
   204  ## License
   205  
   206  ksuid source code is available under an MIT License.