decred.org/dcrdex@v1.0.5/spec/fundamentals.mediawiki

=Distributed Exchange Design Fundamentals=

<!-- TODO: Previously, contents was just __TOC__, but scrolling does not seem to work
by default on github for section names with spaces. Fix scrolling with the default
_TOC_ and revert. Here and in other files. -->
__NOTOC__
==Contents==
* [[#exchange-variables|Exchange Variables]]
** [[#global-variables|Global Variables]]
** [[#asset-variables|Asset Variables]]
** [[#market-variables|Market Variables]]
** [[#bond-asset-variables|Bond Asset Variables]]
** [[#configuration-data-request|Configuration Data Request]]
* [[#bonds|Bonds]]
* [[#transaction-fees|Transaction Fees]]
* [[#epoch-based-order-matching|Epoch-based Order Matching]]
** [[#epoch-time|Epoch Time]]
** [[#pseudorandom-order-matching|Pseudorandom Order Matching]]
* [[#identities-based-on-public-key-infrastructure-pki-key-pairs|Identification]]
* [[#blockchain-interaction|Blockchain Interaction]]
* [[#adding-new-assets|Adding New Assets]]
* [[#api-version|API Version]]
** [[#v0|v0]]

----

There are several notable aspects of the DEX design that were chosen to permit
peer-to-peer trades with the mechanics of order execution existing entirely on
the separate blockchains via atomic swaps.
These are:
* Asset-specific order quantity increments and transaction fee rates
* Epoch-based pseudorandom order matching, with epoch duration configured per-market
* Client identities based on public key infrastructure (PKI)
* An open and rigidly-defined interface for integration of arbitrary assets

This section describes each of these design aspects.

==Exchange Variables==

There are a number of exchange-wide, market-specific, asset-specific, and user specific
variables that must be known by the client. These settings should be requested
by the user immediately after connecting via the
[[#configuration-data-request|<code>config</code> request]].

===Global Variables===

The '''api version''' (<code>apiver</code>) is the server's communications API
version. Check before attempting to trade.

The '''dex public key''' (<code>pubkey</code>) is the public side of a
[[#identities-based-on-public-key-infrastructure-pki-key-pairs|key pair]]
and uniquely identifies the dex. It can be used to verify messages from the dex.

The cancel rate for a user is the ratio of their canceled order count to
completed order count. This only applies to cancels made within two epochs of
the order being booked. After orders have sat for one epoch, they can be canceled
for free.  A user's cancellation rate must remain below the configured
[[community.mediawiki/#rule-3-an-accounts-cancellation-rate-must-not-exceed-the-threshold|'''cancellation threshold''']]
(<code>cancelmax</code>) to avoid penalty.

The '''broadcast timeout''' (<code>btimeout</code>) is the amount of time a
client has to broadcast a transaction. For the maker, the broadcast time is
measured from the time of match notification. For the taker, timeout is measured
from the time when the maker's swap receives its
<code>swapconf</code><sup>th</sup> confirmation (see [[#asset-variables|Asset Variables]]).

The '''bin sizes''' (<code>binSizes</code>) are the bin sizes for candlestick data sets. i.e.
"24h", "1h", "5m". Use in conjuction with <!-- TODO: create and link report notes --> epoch report notes to monitor trade history.

The config has three subsections detailed more below. They are '''asset variables'''
(<code>assets</code>), '''market variables''' (<code>markets</code>), and
'''bond asset variables''' (<code>bondAssets</code>).

===Asset Variables===

The '''version''' (<code>version</code>) is the asset version. Check before trading. A higher
version indicates updated swap configurations.

Every asset is assigned a unique integer '''ID''' (<code>id</code>) that will be used to identify the
asset in serialized structures. Whenever possible, the ID is the same as the
BIP-0044 registered [https://github.com/satoshilabs/slips/blob/master/slip-0044.md coin type index]
[[references.mediawiki|<nowiki>[6]</nowiki>]]. The '''symbol''' (<code>symbol</code>) is also found there.

The '''swap confirmations''' (<code>swapconf</code>) is the number of confirmations required during
settlement on the first swap transaction, before the second swap transaction is
broadcast, as well as on the second swap transaction, before the first
redemption is broadcast.

The '''max fee rate''' (<code>maxfeerate</code>) is the greatest fee the server
may expect the client to pay on initiation transactions. This can be used by
the client to determine how much needs to be reserved to trade.

The '''unit info''' (<code>unitinfo</code>) contains information on unit names
and includes the following variables:

* The '''atomic unit''' (<code>atomicUnit</code>) is the unit the asset package will use in communications. i.e. Sats for btc and gwei for eth.

* The '''conventional denomination''' (<code>conventional</code>) is the most commonly used denomination '''unit''' (<code>unit</code>) and its '''conversion factor''' (<code>conversionFactor</code>). i.e. BTC at 1e8 Sats or ETH at 1e9 gwei.

* The '''alternatives''' (<code>denominations</code>) are other possible '''unit''' and '''conversion factor''' combinations. i.e. µBTC at 1e2 Sats. They may not exist for some assets.

===Market Variables===

The '''name''' (<code>name</code>) is the market's name, composed of the base
and quote asset symbols, lower case, with a hyphen inbetween. i.e. dcr_btc.

The '''base''' (<code>base</code>) and '''quote''' (<code>quote</code>) are the
BIP-0044 registered [https://github.com/satoshilabs/slips/blob/master/slip-0044.md coin type index]
[[references.mediawiki|<nowiki>[6]</nowiki>]] of the base and quote assets that
make up the market.

The '''lot size''' (<code>lotsize</code>) for a market serves as both
the minimum order quantity and the order quantity increment for limit orders
and market buy orders, which are quantified in the market's base asset.
In particular, for ''lot size''  '''''l''''', the requested ''order quantity'',
'''''Q''''', must satisfy

<!--Q = n l, n \in \{1, 2, 3, ...\}-->
[[File:images/lot-sizes.png]]

A trade's rate ''r'' is in units of the quote asset, and must be specified in
integer multiples of the '''rate step''' (<code>ratestep</code>) shown here as ''p''.

<!--r = n p, n \in \{1, 2, 3, ...\}-->
[[File:images/price-increment.png]]

The '''epoch duration''' (<code>epochlen</code>) is the length of one [[#epoch-time|epoch]].


If the client connects shortly after a
[[orders.mediawiki/#trade-suspension|trade suspension]], it's possible that
trading will not commence until a future epoch. This information can be found in the config response's '''market status'''
(<code>status</code>) which inclues a few variables:

* If the market has been or will be susupended, the '''start epoch''' (<code>start epoch</code>) will be populated for each market and contains which epoch index trading did or will begin.

* A '''final epoch''' (<code>finalepoch</code>) may also be present if the dex is scheduled to go into trade suspension. If this variable is present, there may also be a '''persists''' (<code>persistbook</code>) boolean that is true when booked orders will not be purged after market suspension.

The [[orders.mediawiki/#market-buy-orders|'''market buy buffer''']] (<code>buybuffer</code>)
defines the minimum order size for a market buy order.

===Bond Asset Variables===

The '''version''' (<code>version</code>) is the bond version. A higher version indicates updated bond construction.

The '''id''' (<code>id</code>) is the bond asset's BIP-0044 registered [https://github.com/satoshilabs/slips/blob/master/slip-0044.md coin type index]
[[references.mediawiki|<nowiki>[6]</nowiki>]]

The '''confirmations''' (<code>confs</code>) are the confirmations needed in order for the bond to be
considered valid.

The '''amount''' (<code>amount</code>) is how much is required for one bond, and
the amount posted must be divisible by this amount. The amount will be in the
asset's [[#asset-variables|asset's]] <code>atomic unit</code>.

===Configuration Data Request===

'''Request route:''' <code>config</code>, '''originator:''' client

The <code>config</code> request <code>payload</code> can be null. The DEX will
respond with its current configuration.

<code>result</code>
{|
! field          !! type  !! description
|-
| cancelmax      || float || the [[community.mediawiki/#rule-3-an-accounts-cancellation-rate-must-not-exceed-the-threshold|cancellation threshold]]
|-
| btimeout       || int   || the broadcast timeout
|-
| apiver         || int   || the server's [[#api-version|api version]]
|-
| pubkey         || bytes || the server's public ecdsa key
|-
| binSizes       || <nowiki>[string]</nowiki>  || bin sizes for candlestick data sets (i.e. <nowiki>["24h", "1h", "5m"]</nowiki>)
|-
| bondAssets     || <nowiki>map[string]object</nowiki> || map of coin ticker symbols to Bond Asset objects (definition below)
|-
| assets         || <nowiki>[object]</nowiki> || list of Asset objects (definition below)
|-
| markets        || <nowiki>[object]</nowiki> || list of Market objects (definition below)
|}

'''Asset object'''

{|
! field      !! type !! description
|-
| version    || int   || the asset's version
|-
| symbol     || string || ticker symbol
|-
| id         || int || a unique per-asset ID
|-
| maxfeerate || int || the maximum fee rate for transactions (atoms/byte)
|-
| swapconf   || int || minimum confirmations for swap transactions
|-
| unitinfo   || object || a Unit Info object (definition below)
|}

'''Unit Info object'''

{|
! field      !! type !! description
|-
| atomicUnit  || string || the title of the smallest communicatable unit of the asset. i.e. Sats
|-
| conventional  || object || the most common Conversion Factor object (definition below)
|-
| denominations  || <nowiki>[object]</nowiki> || list of other optional Conversion Factor objects. May be empty (definition below)
|}

'''Conversion Factor object'''

{|
! field      !! type !! description
|-
| unit  || string || the title of the denomination. i.e. BTC
|-
| conversionFactor  || int || the number of atomic units in the denomination. i.e. 1e8
|}

'''Market object'''

{|
! field       !! type   !! description
|-
| name        || string || market name
|-
| base        || int    || base asset ID
|-
| quote       || int    || quote asset ID
|-
| lotsize     || int    || lot size (atoms)
|-
| ratestep    || int    || the price rate increment (atoms)
|-
| epochlen    || int    || the [[#epoch-based-order-matching|epoch duration]] (milliseconds)
|-
| buybuffer   || float  || the [[orders.mediawiki/#market-buy-orders|market buy buffer]]
|-
| status      || object || a Market Status object (definition below)
|}

'''Market Status object'''

{|
! field      !! type !! description
|-
| startepoch  || int    || the epoch number at which trading did or will commence. May be in the future e.g. [[orders.mediawiki/#trade-suspension|after maintenance]]
|-
| finalepoch  || int    || the epoch number at which trading will be suspended. Only present when a suspension is scheduled
|-
| persistbook || bool   || whether or not booked orders will be persisted through a scheduled suspension. Only present when a suspension is scheduled
|}

==Bonds==

The DEX collects no trading fees.
Collecting fees from trades executed via atomic swaps (where the server
is never in control of funds and settlement occurs directly on-chain) would
add considerable complexity to the swap process and incentivize DEX operators to
facilitate wash trading.
Instead, time locked bonds are considered during registration. <!-- TODO: Create and link to bond specifics. -->
Locked funds discourage certain spam attacks and enable punitive actions when
[[community.mediawiki/#rules-of-community-conduct|conduct rules]] are violated.

==Transaction Fees==

The clients will cover on-chain transaction fees at a minimum fee rate set by
the DEX operator.
Failure to provide the specified fee rate in a transaction will result in a
conduct violation.

As part of any submitted order, a client is required to demonstrate control of
funds that will back the atomic swap, and ensure that the backing funds are
sufficient to create the swap contract transactions transferring the full order
quantity as well as covering the network's transaction fees at the specified
rate.

Total on-chain fees associated with an order will increase as the number of swap
transactions required to settle the order increases. Maximum fees paid
[[orders.mediawiki/#calculating-transaction-fees|vary linearly with order size]], but the actual
fees realized can be significantly less than the maximum.
See the [[atomic.mediawiki/#atomic-settlement|atomic settlement]] section for examples of simple
and complex matches and how that affects the swap transaction sizes.

Fee rates can vary greatly between assets.
For many assets, low fees are possible without increasing the time until mined.
Transaction fees tend to rise as an asset pushes the limits of on-chain scaling.
For high-fee assets, the DEX operator must find a balance between lower fees,
which are preferable from an accounting standpoint, and higher fees, which can
decrease settlement time by increasing the speed at which transactions are
mined.

==Epoch-based Order Matching==

In order to devalue predatory behavior exhibited by certain high-frequency
trading algorithms, received orders are not processed continuously, but rather
after a shuffling step with all other orders received in a fixed duration period
called an ''epoch''.
The goal of this algorithm is to ensure that an individual client cannot obtain
a deterministic latency advantage over other clients when executing trades.
Limiting this possibility mitigates advantages gained from front-running,
spoofing, and other manipulative trading practices.

===Epoch Time===

For a given epoch duration '''''d > 0''''' in milliseconds, and current UNIX
epoch timestamp '''''t''''' (in milliseconds since Jan 01 00:00:00 1970 UTC),
the current order matching epoch index, '''''i''''', and epoch range are
computed as

<!--i = t / d, i d \leq t_i < d (i + 1)-->
[[File:images/epoch-index.png]]

where '''''/''''' is integer division. For example, at the time of writing,
'''''t = 1562008475123''''' , which for '''''d = 8000''''' (8 seconds)
corresponds to epoch number '''''i = 195251059''''' spanning
'''''&#91;1562008472000, 1562008480000)'''''. This convention allows epoch start
and end times for any epoch duration to be known without querying the server.

A clock synchronization protocol such as NTP will be used to ensure server and
client clocks are synchronized within acceptable tolerances.

===Pseudorandom Order Matching===

When the epoch ends, a match cycle begins. The preimage for each order is
collected and the orders are sorted lexicographically by their
[[orders.mediawiki/#order-id|order IDs]].

A random seed is derived from the Blake-256 hash of the concatenated
[[orders.mediawiki/#order-commitment|order preimages]].

<!--H_{seed} = \text{\footnotesize BLAKE256}(p_1 || p_2 || ... || p_N)-->
[[File:images/seed-eq.png]]

where '''''||''''' indicates concatenation and
'''''p<sub>i</sub>''''' is the preimage of the '''''i<sup>th</sup>''''' order.

The integer series produced by evaluating the hash
as a series of 8-byte big-endian integers is used to seed a Mersenne Twister
(mt19937) random number generator.

Shuffling is then performed using the The Fisher-Yates algorithm on the
'''''N''''' orders, where at each step '''''i : i < N''''', the order at index
'''''i''''' is swapped with the order at index '''''j''''',

<!--j = i + [r_i \bmod (N - i)]-->
[[File:images/fisher_yates_j.png]]

'''''r<sub>i</sub>''''' is the '''''i<sup>th</sup>''''' number in the seeded
mt19937 series.

Orders are processed one at a time. Each order is matched according to its type.

1. If the order is a '''cancel order''', any corresponding standing limit order is removed from the list and the cancel order is considered filled. If a cancellation is processed before the order that it cancels, the cancellation will fail, and will need to be re-submitted. That is, cancel orders do not affect down-queue orders, only standing orders.

2. If the order is a limit order with time in force ''standing'' that cannot match immediately (a '''maker'''), it is added to the standing orders. It is immediately able to match orders further down the queue.

3. If the order is a '''taker''', it is matched against the best available standing order. Clients for both orders are notified and the settlement process begins. The orders are set aside for monitoring. If a limit order with time in force ''standing'' on either side of the match is only partially filled, it is added to the standing orders with the appropriate modifications and is immediately available for matching again.

Any unmatched quantity on a limit order with time in force ''immediate'' is
left unfilled.
Market orders and immediate limit orders cannot match orders further down the
queue.

When a limit order from the queue matches a standing limit order on the book,
the match is assigned the price rate of the maker's order (the standing order's
price rate).

The process continues with the next order in the list and iterates until all
orders have been processed.

The preimages and seed are published at the start of the matching process. In
addition, a '''''checksum''''' defined as the Blake-256 hash of the
concatenation of the lexicographically sorted commitments of all orders in the
epoch queue, is sent to the client. Because the client already has the
commitments from their order book subscription, the checksum allows the client
to check that the orders underlying the preimages are indeed the same set
received in their order notification feed, and that no received orders were
omitted from either preimage collection or shuffling.

For the special case of an epoch with no orders, the checksum will be null.

==Identities based on Public Key Infrastructure (PKI) Key Pairs==

The server and the clients are identified and authenticated using [https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm ECDSA]
[[references.mediawiki|<nowiki>[7]</nowiki>]] public keys,
with matching private keys used to sign and authorize orders and other messages.
Establishing client identity with public keys keeps the notion of client
identity to a minimum, while providing a number of other security benefits
throughout the order placement and execution processes.

All data submitted to the exchange server from a client must be signed with the
client's private key and authenticated by the server using the corresponding
public key, so using client public keys directly for identity purposes is a
natural simplification that obviates the need for client user names and
passwords.

Further, since Politeia, Decred's governance platform, also employs PKI, the
same identities may be used on both services to facilitate time-stamping
exchange data via Politeia. For example, given common identities between the DEX
and Politeia, anchoring data related to DEX client and server conduct on the
Decred blockchain may be useful for establishing a reputation system.

==Blockchain Interaction==

'''Clients''' need wallets that support atomic swaps and the ability to
broadcast transactions to each of the blockchain networks involved in
[[atomic.mediawiki/#atomic-settlement|the swap]].

'''DEX operators''' need access to trusted full nodes for each of the
assets supported.
While operation via a surrogate blockchain data service such as a block explorer
is potentially feasible, it would entail significant security risks.
Initial development will require that the server have a direct connection to
full nodes of each asset's blockchain.

==Adding New Assets==

Adding support for an asset is accomplished by writing a
[https://golang.org/ Go] package with types that implement a particular set of
interfaces, defined [https://github.com/decred/dcrdex/blob/master/server/asset/common.go here]
and [https://github.com/decred/dcrdex/blob/master/server/asset/driver.go here].
There are then two ways to import the asset backend into the server software.

# The package is compiled with <code>-buildmode=plugin</code> and imported at runtime by specifying the plugin in the server configuration.
# The backend is added to the dcrdex repository, and imported directly at compile time.

With the exception of a small handful of assets which will be implemented during
initial phases of DEX development, it is expected that development communities
will release their own appropriately vetted plugins.

==API Version==

Since DEX v0.2, the dex server has an Application Programming Interface version.
Clients should check this version before attempting to communicate further with
the server. An unknown or incompatible version means that communications will
fail, possibly resulting in penalties or a ban.

===v0===

The server's pre-versioning api version with an apiver value of 0.