github.com/technosophos/deis@v1.7.1-0.20150915173815-f9005256004b/docs/managing_deis/running-deis-without-ceph.rst (about) 1 :title: Running Deis without Ceph 2 :description: Configuring the cluster to remove Ceph from the control plane. 3 4 .. _running-deis-without-ceph: 5 6 Running Deis without Ceph 7 ========================= 8 9 .. include:: ../_includes/_ceph-dependency-description.rst 10 11 This guide is intended to assist users who are interested in removing the Ceph 12 dependency of the Deis control plane. 13 14 .. note:: 15 16 This guide was adapted from content graciously provided by Deis community member 17 `Arne-Christian Blystad`_. 18 19 Requirements 20 ------------ 21 22 External services are required to replace the internal store components: 23 24 * S3-compatible blob store (like `Amazon S3`_) 25 * PostgreSQL database (like `Amazon RDS`_) 26 * Log drain service with syslog log format compatibility (like `Papertrail`_) 27 28 Understanding component changes 29 ------------------------------- 30 31 Either directly or indirectly, all components in the :ref:`control-plane` 32 require Ceph (:ref:`store`). Some components require changes to accommodate 33 the removal of Ceph. The necessary changes are described below. 34 35 Logger 36 ^^^^^^ 37 38 The :ref:`logger` component provides a syslog-compatible endpoint to consume 39 application logs, which it writes to a shared Ceph filesystem. These logs are 40 read by the :ref:`controller` component. The :ref:`logspout` talks to the Docker 41 daemon on each host, listens for log events from running applications, and ships 42 them to the logger. 43 44 The Logger component is not necessary in a Ceph-less Deis cluster. Instead of 45 using the Logger, we will route all the logs directly to another syslog 46 compatible endpoint. 47 48 Database 49 ^^^^^^^^ 50 51 The :ref:`database` runs PostgreSQL and uses the Ceph S3 API (provided by 52 ``deis-store-gateway``) to store PostgreSQL backups and WAL logs. 53 Should the host running database fail, the database component will fail over to 54 a new host, start up, and replay backups and WAL logs to recover to its 55 previous state. 56 57 We will not be using the database component in the Ceph-less cluster, and will 58 instead rely on an external database. 59 60 Controller 61 ^^^^^^^^^^ 62 63 The :ref:`controller` component hosts the API that the Deis CLI consumes. The controller 64 mounts the same Ceph filesystem that the logger writes to. When users run ``deis logs`` 65 to view an application's log files, the controller reads from this shared filesystem. 66 67 A Ceph-less cluster will not store logs (instead sending them to an external service), 68 so the ``deis logs`` command will not work for users. 69 70 Registry 71 ^^^^^^^^ 72 73 The :ref:`registry` component is an instance of the offical Docker registry, and 74 is used to store application releases. The registry supports any S3 store, so 75 a Ceph-less cluster will simply reconfigure registry to use another store (typically 76 Amazon S3 itself). 77 78 Builder 79 ^^^^^^^ 80 81 The :ref:`builder` component is responsible for building applications deployed 82 to Deis via the ``git push`` workflow. It pushes to registry to store releases, 83 so it will require no changes. 84 85 Store 86 ^^^^^ 87 88 The :ref:`store` components implement Ceph itself. In a Ceph-less cluster, we 89 will skip the installation and starting of these components. 90 91 Deploying the cluster 92 --------------------- 93 94 This guide assumes a typical deployment on AWS by following the :ref:`deis_on_aws` 95 guide. 96 97 Deploy an AWS cluster 98 ^^^^^^^^^^^^^^^^^^^^^ 99 100 Follow the :ref:`deis_on_aws` installation documentation through the "Configure 101 DNS" portion. 102 103 Configure log shipping 104 ^^^^^^^^^^^^^^^^^^^^^^ 105 106 The :ref:`logspout` component must be configured to ship logs to somewhere other 107 than the :ref:`logger` component. 108 109 .. code-block:: console 110 111 $ HOST=logs.somewhere.com 112 $ PORT=98765 113 $ PROTOCOL=udp # Supported protocols are udp and tcp 114 $ deisctl config logs set host=${HOST} port=${PORT} protocol=${PROTOCOL} 115 116 Configure registry 117 ^^^^^^^^^^^^^^^^^^ 118 119 The :ref:`registry` component won't start until it's configured with an S3 store. 120 121 .. code-block:: console 122 123 $ BUCKET=MYS3BUCKET 124 $ AWS_ACCESS_KEY=something 125 $ AWS_SECRET_KEY=something 126 $ deisctl config registry set s3bucket=${BUCKET} \ 127 bucketName=${BUCKET} \ 128 s3accessKey=${AWS_ACCESS_KEY} \ 129 s3secretKey=${AWS_SECRET_KEY} \ 130 s3region=eu-west-1 \ 131 s3path=/ \ 132 s3encrypt=false \ 133 s3secure=false 134 $ deisctl config store set gateway/accessKey=${AWS_ACCESS_KEY} \ 135 gateway/secretKey=${AWS_SECRET_KEY} \ 136 gateway/host=s3.amazonaws.com \ 137 gateway/port=80 138 139 Configure database settings 140 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 141 142 Since we won't be running the :ref:`database`, we need to configure these settings 143 so the controller knows where to connect. 144 145 .. code-block:: console 146 147 $ HOST=something.rds.amazonaws.com 148 $ DB_USER=deis 149 $ DB_PASS=somethingsomething 150 $ DATABASE=deis 151 $ deisctl config database set engine=postgresql_psycopg2 \ 152 host=${HOST} \ 153 port=5432 \ 154 name=${DATABASE} \ 155 user=${DB_USER} \ 156 password=${DB_PASS} 157 158 Deploy the platform 159 ^^^^^^^^^^^^^^^^^^^ 160 161 The typical :ref:`install_deis_platform` documentation can be followed, with 162 one caveat: since we won't be deploying many of the typical Deis components, we cannot 163 use ``deisctl install platform`` or ``deisctl start platform`` -- instead, we 164 use ``deisctl install stateless-platform`` and ``deisctl start stateless-platform``. 165 166 These commands tell ``deisctl`` to skip the components that we don't need to use. 167 168 Confirm installation 169 ^^^^^^^^^^^^^^^^^^^^ 170 171 That's it! Deis is now running without Ceph. Issue a ``deisctl list`` to confirm 172 that the services are started, and see :ref:`using_deis` to start using the cluster. 173 174 Upgrading Deis 175 -------------- 176 177 When following the :ref:`upgrading-deis` documentation, be sure to use 178 ``stateless-platform`` instead of ``platform``. 179 180 .. _`Amazon RDS`: http://aws.amazon.com/rds/ 181 .. _`Amazon S3`: http://aws.amazon.com/s3/ 182 .. _`Arne-Christian Blystad`: https://github.com/blystad 183 .. _`Papertrail`: https://papertrailapp.com/