github.com/ghodss/etcd@v0.3.1-0.20140417172404-cc329bfa55cb/Documentation/internal-protocol-versioning.md (about)

     1  # Versioning
     2  
     3  Goal: We want to be able to upgrade an individual peer in an etcd cluster to a newer version of etcd.
     4  The process will take the form of individual followers upgrading to the latest version until the entire cluster is on the new version.
     5  
     6  Immediate need: etcd is moving too fast to version the internal API right now.
     7  But, we need to keep mixed version clusters from being started by a rolling upgrade process (e.g. the CoreOS developer alpha).
     8  
     9  Longer term need: Having a mixed version cluster where all peers are not running the exact same version of etcd itself but are able to speak one version of the internal protocol.
    10  
    11  Solution: The internal protocol needs to be versioned just as the client protocol is.
    12  Initially during the 0.\*.\* series of etcd releases we won't allow mixed versions at all.
    13  
    14  ## Join Control
    15  
    16  We will add a version field to the join command.
    17  But, who decides whether a newly upgraded follower should be able to join a cluster?
    18  
    19  ### Leader Controlled
    20  
    21  If the leader controls the version of followers joining the cluster then it compares its version to the version number presented by the follower in the JoinCommand and rejects the join if the number is less than the leader's version number.
    22  
    23  Advantages
    24  
    25  - Leader controls all cluster decisions still
    26  
    27  Disadvantages
    28  
    29  - Follower knows better what versions of the internal protocol it can talk than the leader
    30  
    31  
    32  ### Follower Controlled
    33  
    34  A newly upgraded follower should be able to figure out the leaders internal version from a defined internal backwards compatible API endpoint and figure out if it can join the cluster.
    35  If it cannot join the cluster then it simply exits.
    36  
    37  Advantages
    38  
    39  - The follower is running newer code and knows better if it can talk older protocols
    40  
    41  Disadvantages
    42  
    43  - This cluster decision isn't made by the leader
    44  
    45  ## Recommendation
    46  
    47  To solve the immediate need and to plan for the future lets do the following:
    48  
    49  - Add Version field to JoinCommand
    50  - Have a joining follower read the Version field of the leader and if its own version doesn't match the leader then sleep for some random interval and retry later to see if the leader has upgraded.
    51  
    52  # Research
    53  
    54  ## Zookeeper versioning
    55  
    56  Zookeeper very recently added versioning into the protocol and it doesn't seem to have seen any use yet.
    57  https://issues.apache.org/jira/browse/ZOOKEEPER-1633
    58  
    59  ## doozerd
    60  
    61  doozerd stores the version number of the peers in the datastore for other clients to check, no decisions are made off of this number currently.