github.com/avahowell/sia@v0.5.1-beta.0.20160524050156-83dcc3d37c94/doc/Consensus.md (about) 1 Consensus Rules 2 =============== 3 4 This document is meant to provide a good high level overview of the Sia 5 cryptosystem, but does not fully explain all of the small details. The most 6 accurate explanation of the consensus rules is the consensus package (and all 7 dependencies). 8 9 This document will be more understandable if you have a general understanding 10 of proof of work blockchains, and does not try to build up from first 11 principles. 12 13 Cryptographic Algorithms 14 ------------------------ 15 16 Sia uses cryptographic hashing and cryptographic signing, each of which has 17 many potentially secure algorithms that can be used. We acknowledge our 18 inexperience, and that we have chosen these algorithms not because of our own 19 confidence in their properties, but because other people seem confident in 20 their properties. 21 22 For hashing, our primary goal is to use an algorithm that cannot be merge mined 23 with Bitcoin, even partially. A secondary goal is hashing speed on consumer 24 hardware, including phones and other low power devices. 25 26 For signing, our primary goal is verification speed. A secondary goal is an 27 algorithm that supports HD keys. A tertiary goal is an algorithm that supports 28 threshold signatures. 29 30 #### Hashing: blake2b 31 32 [blake2b](http://en.wikipedia.org/wiki/BLAKE_%28hash_function%29#BLAKE2 "Wiki page") has been chosen as a hashing algorithm because it is fast, it has had 33 substantial review, and it has invulnerability to length extension attacks. 34 Another particularly important feature of BLAKE2b is that it is not SHA-2. We 35 wish to avoid merge mining with Bitcoin, because that may result in many 36 apathetic Bitcoin miners mining on our blockchain, which may make soft forks 37 harder to coordinate. 38 39 #### Signatures: variable type signatures 40 41 Each public key will have an specifier (a 16 byte array) and a byte slice 42 containing an encoding of the public key. The specifier will tell the 43 signature verification which signing algorithm to use when verifying a 44 signature. Each signature will be a byte slice, the encoding can be 45 determined by looking at the specifier of the corresponding public key. 46 47 This method allows new signature types to be easily added to the currency in 48 a way that does not invalidate existing outputs and keys. Adding a new 49 signature type requires a hardfork, but allows easy protection against 50 cryptographic breaks, and easy migration to new cryptography if there are any 51 breakthroughs in areas like verification speed, ring signatures, etc. 52 53 Allowed algorithms: 54 55 ed25519: The specifier must match the string "ed25519". The public key 56 must be encoded into 32 bytes. Signatures and public keys will need to 57 follow the ed25519 specification. More information can be found at 58 ed25519.cr.yp.to 59 60 entropy: The specifier must match the string "entropy". The signature will 61 always be invalid. This provides a way to add entropy buffers to 62 SpendCondition objects to protect low entropy information, while being able 63 to prove that the entropy buffers are invalid public keys. 64 65 There are plans to also add ECDSA secp256k1 and Schnorr secp256k1. New 66 signing algorithms can be added to Sia through a soft fork, because 67 unrecognized algorithm types are always considered to have valid signatures. 68 69 Currency 70 -------- 71 72 The Sia cryptosystem has two types of currency. The first is the Siacoin. 73 Siacoins are generated every block and distributed to the miners. These miners 74 can then use the siacoins to fund file contracts, or can send the siacoins to 75 other parties. The siacoin is represented by an infinite precision unsigned 76 integer. 77 78 The second currency in the Sia cryptosystem is the Siafund, which is a special 79 asset limited to 10,000 indivisible units. Each time a file contract payout is 80 made, 3.9% of the payout is put into the siafund pool. The number of siacoins 81 in the siafund pool must always be divisible by 10,000; the number of coins 82 taken from the payout is rounded down to the nearest 10,000. The siafund is 83 also represented by an infinite precision unsigned integer. 84 85 Siafund owners can collect the siacoins in the siafund pool. For every 10,000 86 siacoins added to the siafund pool, a siafund owner can withdraw 1 siacoin. 87 Approx. 8790 siafunds are owned by Nebulous Inc. The remaining siafunds are 88 owned by early backers of the Sia project. 89 90 There are future plans to enable sidechain compatibility with Sia. This would 91 allow other currencies such as Bitcoin to be spent in all the same places that 92 the Siacoin can be spent. 93 94 Marshalling 95 ----------- 96 97 Many of the Sia types need to be hashed at some point, which requires having a 98 consistent algorithm for marshalling types into a set of bytes that can be 99 hashed. The following rules are used for hashing: 100 101 - Integers are little-endian, and are always encoded as 8 bytes. 102 - Bools are encoded as one byte, where zero is false and one is true. 103 - Variable length types such as strings are prefaced by 8 bytes containing 104 their length. 105 - Arrays and structs are encoded as their individual elements concatenated 106 together. The ordering of the struct is determined by the struct definition. 107 There is only one way to encode each struct. 108 - The Currency type (an infinite precision integer) is encoded in big endian 109 using as many bytes as necessary to represent the underlying number. As it 110 is a variable length type, it is prefixed by 8 bytes containing the length. 111 112 Block Size 113 ---------- 114 115 The maximum block size is 2e6 bytes. There is no limit on transaction size, 116 though it must fit inside of the block. Most miners enforce a size limit of 117 16e3 bytes per transaction. 118 119 Block Timestamps 120 ---------------- 121 122 Each block has a minimum allowed timestamp. The minimum timestamp is found by 123 taking the median timestamp of the previous 11 blocks. If there are not 11 124 previous blocks, the genesis timestamp is used repeatedly. 125 126 Blocks will be rejected if they are timestamped more than three hours in the 127 future, but can be accepted again once enough time has passed. 128 129 Block ID 130 -------- 131 132 The ID of a block is derived using: 133 Hash(Parent Block ID + 64 bit Nonce + Block Merkle Root) 134 135 The block Merkle root is obtained by creating a Merkle tree whose leaves are 136 the hash of the timestamp, the hashes of the miner outputs (one leaf per miner 137 output), and the hashes of the transactions (one leaf per transaction). 138 139 Block Target 140 ------------ 141 142 For a block to be valid, the id of the block must be below a certain target. 143 The target is adjusted once every 500 blocks, and it is adjusted by looking at 144 the timestamps of the previous 1000 blocks. The expected amount of time passed 145 between the most recent block and the 1000th previous block is 10e3 minutes. If 146 more time has passed, the target is lowered. If less time has passed, the 147 target is increased. Each adjustment can adjust the target by up to 2.5x. 148 149 The target is changed in proportion to the difference in time (If the time was 150 half of what was expected, the new target is 1/2 the old target). There is a 151 clamp on the adjustment. In one block, the target cannot adjust upwards by more 152 more than 1001/1000, and cannot adjust downwards by more than 999/1000. 153 154 The new target is calculated using (expected time passed in seconds) / (actual 155 time passed in seconds) * (current target). The division and multiplication 156 should be done using infinite precision, and the result should be truncated. 157 158 If there are not 1000 blocks, the genesis timestamp is used for comparison. 159 The expected time is (10 minutes * block height). 160 161 Block Subsidy 162 ------------- 163 164 The coinbase for a block is (300,000 - height) * 10^24, with a minimum of 165 30,000 \* 10^24. Any miner fees get added to the coinbase to create the block 166 subsidy. The block subsidy is then given to multiple outputs, called the miner 167 payouts. The total value of the miner payouts must equal the block subsidy. 168 169 The ids of the outputs created by the miner payouts is determined by taking the 170 block id and concatenating the index of the payout that the output corresponds 171 to. 172 173 The outputs created by the block subsidy cannot be spent for 50 blocks, and are 174 not considered a part of the consensus set until 50 blocks have transpired. 175 This limitation is in place because a simple blockchain reorganization is 176 enough to invalidate the output; double spend attacks and false spend attacks 177 are much easier to execute. 178 179 Transactions 180 ------------ 181 182 A Transaction is composed of the following: 183 184 - Siacoin Inputs 185 - Siacoin Outputs 186 - File Contracts 187 - File Contract Revisions 188 - Storage Proofs 189 - Siafund Inputs 190 - Siafund Outputs 191 - Miner Fees 192 - Arbitrary Data 193 - Transaction Signatures 194 195 The sum of all the siacoin inputs must equal the sum of all the miner fees, 196 siacoin outputs, and file contract payouts. There can be no leftovers. The sum 197 of all siafund inputs must equal the sum of all siafund outputs. 198 199 Several objects have unlock hashes. An unlock hash is the Merkle root of the 200 'unlock conditions' object. The unlock conditions contain a timelock, a number 201 of required signatures, and a set of public keys that can be used during 202 signing. 203 204 The Merkle root of the unlock condition objects is formed by taking the Merkle 205 root of a tree whose leaves are the timelock, the public keys (one leaf per 206 key), and the number of signatures. This ordering is chosen specifically 207 because the timelock and the number of signatures are low entropy. By using 208 random data as the first and last public key, you can make it safe to reveal 209 any of the public keys without revealing the low entropy items. 210 211 The unlock conditions cannot be satisfied until enough signatures have 212 provided, and until the height of the blockchain is at least equal to the value 213 of the timelock. 214 215 The unlock conditions contains a set of public keys which can each be used only 216 once when providing signatures. The same public key can be listed twice, which 217 means that it can be used twice. The number of required signatures indicates 218 how many public keys must be used to validate the input. If required signatures 219 is '0', the input is effectively 'anyone can spend'. If the required signature 220 count is greater than the number of public keys, the input is unspendable. 221 There must be exactly enough signatures. For example, if there are 3 public 222 keys and only two required signatures, then only two signatures can be included 223 into the transaction. 224 225 Siacoin Inputs 226 -------------- 227 228 Each input spends an output. The output being spent must exist in the consensus 229 set. The 'value' field of the output indicates how many siacoins must be used 230 in the outputs of the transaction. Valid outputs are miner fees, siacoin 231 outputs, and contract payouts. 232 233 Siacoin Outputs 234 --------------- 235 236 Siacoin outputs contain a value and an unlock hash (also called a coin 237 address). The unlock hash is the Merkle root of the spend conditions that must 238 be met to spend the output. 239 240 File Contracts 241 -------------- 242 243 A file contract is an agreement by some party to prove they have a file at a 244 given point in time. The contract contains the Merkle root of the data being 245 stored, and the size in bytes of the data being stored. 246 247 The Merkle root is formed by breaking the file into 64 byte segments and 248 hashing each segment to form the leaves of the Merkle tree. The final segment 249 is not padded out. 250 251 The storage proof must be submitted between the 'WindowStart' and 'WindowEnd' 252 fields of the contract. There is a 'Payout', which indicates how many siacoins 253 are given out when the storage proof is provided. 3.9% of this payout (rounded 254 down to the nearest 10,000) is put aside for the owners of siafunds. If the 255 storage proof is provided and is valid, the remaining payout is put in an 256 output spendable by the 'valid proof spend hash', and if a valid storage proof 257 is not provided to the blockchain by 'end', the remaining payout is put in an 258 output spendable by the 'missed proof spend hash'. 259 260 All contracts must have a non-zero payout, 'start' must be before 'end', and 261 'start' must be greater than the current height of the blockchain. A storage 262 proof is acceptable if it is submitted in the block of height 'end'. 263 264 File contracts are created with a 'Revision Hash', which is the Merkle root of 265 an unlock conditions object. A 'file contract revision' can be submitted which 266 fulfills the unlock conditions object, resulting in the file contract being 267 replaced by a new file contract, as specified in the revision. 268 269 File Contract Revisions 270 ----------------------- 271 272 A file contract revision modifies a contract. File contracts have a revision 273 number, and any revision submitted to the blockchain must have a higher 274 revision number in order to be valid. Any field can be changed except for the 275 payout - siacoins cannot be added to or removed from the file contract during a 276 revision, though the destination upon a successful or unsuccessful storage 277 proof can be changed. 278 279 The greatest application for file contract revisions is file-diff channels - a 280 file contract can be edited many times off-blockchain as a user uploads new or 281 different content to the host. This improves the overall scalability of Sia. 282 283 Storage Proofs 284 -------------- 285 286 A storage proof transaction is any transaction containing a storage proof. 287 Storage proof transactions are not allowed to have siacoin or siafund outputs, 288 and are not allowed to have file contracts. 289 290 When creating a storage proof, you only prove that you have a single 64 byte 291 segment of the file. The piece that you must prove you have is chosen 292 randomly using the contract id and the id of the 'trigger block'. The 293 trigger block is the block at height 'Start' - 1, where 'Start' is the value 294 'Start' in the contract that the storage proof is fulfilling. 295 296 The file is composed of 64 byte segments whose hashes compose the leaves of a 297 Merkle tree. When proving you have the file, you must prove you have one of the 298 leaves. To determine which leaf, take the hash of the contract id concatenated 299 to the trigger block id, then take the numerical value of the result modulus 300 the number of segments: 301 302 Hash(file contract id + trigger block id) % num segments 303 304 The proof is formed by providing the 64 byte segment, and then the missing 305 hashes required to fill out the remaining tree. The total size of the proof 306 will be 64 bytes + 32 bytes * log(num segments), and can be verified by anybody 307 who knows the root hash and the file size. 308 309 Storage proof transactions are not allowed to have siacoin outputs, siafund 310 outputs, or contracts. All outputs created by the storage proofs cannot be 311 spent for 50 blocks. 312 313 These limits are in place because a simple blockchain reorganization can change 314 the trigger block, which will invalidate the storage proof and therefore the 315 entire transaction. This makes double spend attacks and false spend attacks 316 significantly easier to execute. 317 318 Siafund Inputs 319 -------------- 320 321 A siafund input works similar to a siacoin input. It contains the id of a 322 siafund output being spent, and the unlock conditions required to spend the 323 output. 324 325 A special output is created when a siafund output is used as input. All of the 326 siacoins that have accrued in the siafund since its last spend are sent to the 327 'claim spend hash' found in the siafund output, which is a normal siacoin 328 address. The value of the siacoin output is determined by taking the size of 329 the siacoin pool when the output was created and comparing it to the current 330 size of the siacoin pool. The equation is: 331 332 ((Current Pool Size - Previous Pool Size) / 10,000) * siafund quantity 333 334 Like the miner outputs and the storage proof outputs, the siafund output cannot 335 be spent for 50 blocks because the value of the output can change if the 336 blockchain reorganizes. Reorganizations will not however cause the transaction 337 to be invalidated, so the ban on contracts and outputs does not need to be in 338 place. 339 340 Siafund Outputs 341 --------------- 342 343 Like siacoin outputs, siafund outputs contain a value and an unlock hash. The 344 value indicates the number of siafunds that are put into the output, and the 345 unlock hash is the Merkle root of the unlock conditions object which allows the 346 output to be spent. 347 348 Siafund outputs also contain a claim unlock hash field, which indicates the 349 unlock hash of the siacoin output that is created when the siafund output is 350 spent. The value of the output that gets created will depend on the growth of 351 the siacoin pool between the creation and the spending of the output. This 352 growth is measured by storing a 'claim start', which indicates the size of the 353 siafund pool at the moment the siafund output was created. 354 355 Miner Fees 356 ---------- 357 358 A miner fee is a volume of siacoins that get added to the block subsidy. 359 360 Arbitrary Data 361 -------------- 362 363 Arbitrary data is a set of data that is ignored by consensus. In the future, it 364 may be used for soft forks, paired with 'anyone can spend' transactions. In the 365 meantime, it is an easy way for third party applications to make use of the 366 siacoin blockchain. 367 368 Transaction Signatures 369 ---------------------- 370 371 Each signature points to a single public key index in a single unlock 372 conditions object. No two signatures can point to the same public key index for 373 the same set of unlock conditions. 374 375 Each signature also contains a timelock, and is not valid until the blockchain 376 has reached a height equal to the timelock height. 377 378 Signatures also have a 'covered fields' object, which indicates which parts of 379 the transaction get included in the signature. There is a 'whole transaction' 380 flag, which indicates that every part of the transaction except for the 381 signatures gets included, which eliminates any malleability outside of the 382 signatures. The signatures can also be individually included, to enforce that 383 your signature is only valid if certain other signatures are present. 384 385 If the 'whole transaction' is not set, all fields need to be added manually, 386 and additional parties can add new fields, meaning the transaction will be 387 malleable. This does however allow other parties to add additional inputs, 388 fees, etc. after you have signed the transaction without invalidating your 389 signature. If the whole transaction flag is set, all other elements in the 390 covered fields object must be empty except for the signatures field. 391 392 The covered fields object contains a slice of indexes for each element of the 393 transaction (siacoin inputs, miner fees, etc.). The slice must be sorted, and 394 there can be no repeated elements. 395 396 Entirely nonmalleable transactions can be achieved by setting the 'whole 397 transaction' flag and then providing the last signature, including every other 398 signature in your signature. Because no frivolous signatures are allowed, the 399 transaction cannot be changed without your signature being invalidated.