github.com/readium/readium-lcp-server@v0.0.0-20240509124024-799e77a0bbd6/README.md (about) 1 Readium LCP Server 2 ================== 3 4 Documentation 5 ============ 6 As a retailer, public library or specialized e-distributor, you are distributing EPUB or PDF ebooks, audiobooks packaged using the LPF or RPF (.audiobook) formats, or comics packaged using the RPF (.divina) format. You want them protected by the Readium LCP DRM. Your CMS (Content Management system) already handles publications, users, purchases or loans, your technical team is able to integrate this CMS with a License server by creating a new http endpoint in the CMS and requesting the License Server via its REST interface, and this team is also able to compile and deploy a Go open-source codebase. If you are in this situation, this open-source codebase is made for you. 7 8 Using the tools provided in this project you can: 9 * Encrypt your entire catalog of publications and store these encrypted files in a file system or S3 bucket, ready for download from any LCP compliant reading application; 10 * Generate LCP licenses; 11 * Let users request a loan extension or an early return; 12 * Cancel a license in case a user has declared he wasn't able to use it; 13 * Revoke a license in case of oversharing. 14 15 **A detailed documentation is found in the [Wiki pages of the project](../../wiki). You really have to read it before you start testing this application.** 16 17 Prerequisites 18 ============= 19 20 Binaries are only pre-built on demand and for a service fee, therefore in the general case you'll need to get a working Golang installation. 21 Please refer to the official Go documentation for installation procedures at https://golang.org/. 22 23 This software is working with *go 1.19* or higher. It is currently maintained using *go 1.21* (February 2024). 24 25 You must put in place: 26 27 1/ the License Server, active in your intranet, not accessible from the Web, only accessible from your CMS via a REST API. 28 29 2/ the Status Server, accessible from the Web via a REST API, using https (you'll need to install a reverse proxy). 30 31 3/ a License Gateway, i.e. a piece of sofware you'll have to develop, which takes a request for an existing LCP license from a reading app, interrogates your database in order to get user information relative to this license, calls the License Server endpoint and returns this fresh LCP license back to the caller app (more information in the project Wiki). 32 33 4/ a large storage volume for encrypted publications. It can be either a file system accessible from the Web via HTTP URLs, or an S3 bucket. Note that publications are encrypted once: every license generated for such publication is pointing at the same encrypted file. Because these publications are stronlgy encrypted and the decryption key is secured in your SQL database, public access to these files is not problematic. 34 35 The servers require the setup of an SQL Database. 36 37 - SQLite is sufficient for most needs. If the "database" property of each server defines a sqlite3 driver, the db setup is dynamically achieved when the server runs for the first time. SQLite database creation scripts are also provided in the "dbmodel" folder in case they are useful. 38 - MySQL, MS SQL and PostgreSQL database creation scripts are provided in the "dbmodel" folder. These scripts must be applied before launching the servers for the first time. 39 40 Encryption Profiles 41 =================== 42 Out of the box, this open-source software is using what we call the "basic" (or "test") LCP profile, i.e. a testing mode provided by the [LCP open standard](https://readium.org/lcp-specs/). Licenses generated with this "basic" profile are perfectly handled by reading applications based on [Readium Mobile](https://www.edrlab.org/software/readium-mobile/), as well as by [Thorium Reader](https://www.edrlab.org/software/thorium-reader/). 43 44 But this profile, because it is open, does not offer any security. Security is provided by a "production" profile, i.e. confidential crypto information and a personal X.509 certificate delivered to trusted implementers by [EDRLab](mailto:contact@edrlab.org). EDRLab is the wordwide LCP Certification Authority. Licenses generated with the "production" profile are handled by any LCP compliant Reading System. 45 46 Executables 47 =========== 48 The server software is composed of several independant parts: 49 50 ## [lcpencrypt] 51 52 A command line utility for content encryption. This utility can be included in any processing pipeline. 53 54 lcpencrypt can: 55 * Take an unprotected publication as input and generates an encrypted file as output. 56 * Store the encrypted file into a file system or S3 bucket. 57 * Notify the License server of the generation of the encrypted file. 58 * Optionnaly, notify the CMS of the generation of the encrypted file. 59 60 ## [lcpserver] 61 62 A License server implements [Readium Licensed Content Protection 1.0](https://readium.org/lcp-specs/releases/lcp/latest). 63 64 Its functionalities can only be accessed after client authentication. 65 66 Its private functionalities are: 67 * Generate a license or returns a fresh license 68 * Update the rights associated with a license 69 * Get a list of licenses (optionally filtered by publication) 70 71 ## [lsdserver] 72 73 A Status Server implements [Readium License Status Document 1.0](https://readium.org/lcp-specs/releases/lsd/latest). 74 75 Its public functionalities are: 76 * Return a license status document 77 * Process a device registration request 78 * Process a lending return request 79 * Process a lending renewal request 80 81 Its private functionalities (authentication required) are: 82 * Be notified of the generation of a new license 83 * Filter licenses by count of registered devices 84 * List all registered devices for a given license 85 * Revoke or cancel a license 86 87 ## [frontend] 88 89 A Frontend Test Server is also provided in the project. This is a demo server we developed to provide a micro-CMS and a user interface for testing LCP licenses. It is active on https://front-prod.edrlab.org/frontend/. We do not consider it production ready, we don't update it (despite evolutions in node, npm, and the multiple node modules used a dependencies) and it will disappear in the next major version of the codebase. The Frontend Test Server MUST NOT be used in production. 90 91 This is why the installation of the Frontend Test Server is not described in the following instructions. 92 93 Install 94 ======= 95 96 Assuming a working Go installation (*go 1.16* or higher) ... 97 98 ### On Linux and MacOS: the easy route 99 100 Simply use the go install command. 101 102 ```sh 103 # fetch, build and install the different packages and their dependencies 104 go install github.com/readium/readium-lcp-server/lcpencrypt@latest 105 go install github.com/readium/readium-lcp-server/lcpserver@latest 106 go install github.com/readium/readium-lcp-server/lsdserver@latest 107 ``` 108 109 "@latest" can be replaced by a specific version, e.g. "@V1.8.0" (warning: use a capital V). 110 111 You should now find the generated binaries in $GOPATH/bin: 112 113 - `lcpencrypt`: the command line encryption tool, 114 - `lcpserver`: the license server, 115 - `lsdserver`: the status server. 116 117 ### On Linux and MacOS: the developer's route 118 119 The project supports Go modules. Developers can therefore clone the codebase in the directory of their choice. This will be required to move the LCP Server to its production mode later. 120 121 Our recommendation is to use a structure like the one below: 122 123 ``` 124 |- <some root dir> 125 |-readium 126 |- readium-lcp-server // where the codebase is cloned 127 |- <config structure> // see below, configuration 128 129 ``` 130 131 Move to the `readium` directory and use: 132 133 ``` 134 git clone https://github.com/readium/readium-lcp-server.git 135 ``` 136 137 Then compile the code with: 138 139 ``` 140 cd readium-lcp-server 141 go build -o $GOPATH/bin ./lcpencrypt 142 go build -o $GOPATH/bin ./lcpserver 143 go build -o $GOPATH/bin ./lsdserver 144 ``` 145 146 You should now find the generated binaries in $GOPATH/bin (or $GOBIN if this environment variable is set). 147 148 149 ### On Windows 10 150 151 You must first install a GCC compiler in order to compile the SQLite module and to later move to "production mode". [TDM-GCC](http://tdm-gcc.tdragon.net/download) gives great results. 152 153 Also, in the previous instructions, replace: 154 155 * $GOPATH with %GOPATH% 156 * forward slashes with backslashes in paths. 157 158 Configuration 159 ============== 160 161 ## Config structure 162 163 Our recommendation is to use a structure like the one below: 164 165 ``` 166 |- <some root dir> 167 |-readium 168 |- readium-lcp-server // where the codebase is cloned 169 |- config // where the configuration files and X509 certificates are maintained 170 |- db // where the sqlite database files are stored (if sqlite is used) 171 |- tmp // where temporary files are created 172 ``` 173 174 ## Environment variables 175 176 The server is controlled by a yaml configuration file (e.g. "config.yaml") stored in a convenient folder, eg. `/usr/local/var/lcp`. 177 178 The License Server and Status Server will search their configuration file in the go bin directory by default; but the path to the file should be changed using an environment variable: 179 180 * `READIUM_LCPSERVER_CONFIG` for the License server 181 * `READIUM_LSDSERVER_CONFIG` for the Status server 182 183 The value of the each global variable is an absolute path to the configuration file for the given server. The two servers may share the same configuration file (useful if they are executed on the same server) or each server may get its own configuration file; this is your choice. 184 185 ## Password file 186 187 The LCP and LSD servers also require authenticated API requests for some of their functionalities. A password file formatted as an Apache "htpasswd" file is used for such authentication data. The htpasswd file format is of the form: 188 189 ```sh 190 User:$apr1$OMWGq53X$Qf17b.ezwEM947Vrr/oTh0 191 ``` 192 193 An example of password file generator is found [here](http://www.htaccesstools.com/htpasswd-generator/). 194 195 The password file may be shared between the LCP and LSD servers if the same credentials are used for both. The exact location and name of the file have no importance, as it will be referenced from the configuration file; but we recommand to name it `htpasswd` and place this file in the same folder as the configuration file, eg. `/usr/local/var/lcp`. 196 197 ## Certificate 198 199 The License server requires an X509 certificate and its associated private key. The exact location and name of these files have no importance, as they will be referenced from the configuration file; but we recommand to keep the file name of the file provided by EDRLab and place these files in a subfolder of the previous one, eg. `/usr/local/var/lcp/cert`. 200 201 A test certificate (`cert-edrlab-test.pem`) and private key (`privkey-edrlab-test.pem`) are provided in the `test/cert` directory of the project. These files are be used as long as the LCP server is configured in test mode (`profile` = `basic`). They are replaced by a provider specific certificate and private key when the server is moved to its production mode. 202 203 ## Quick-start configuration 204 205 A quick-start configuration meant only for test purposes is available in `test/config.yaml`. This file includes a default configuration for the the LCP, LSD and frontend servers. 206 207 1. Create a `<LCP_HOME>` folder, eg. `/usr/local/var/lcp` 208 2. Copy the `test/config.yaml` file into LCP_HOME 209 3. Replace any occurrence of `<LCP_HOME>` in config.yaml with the absolute path to the LCP_HOME folder 210 4. Setup the `READIUM_*_CONFIG` env variables, which must reference the configuration file 211 5. Generate a password file and place it into LCP_HOME 212 6. Create `cert`, `db`, `files/storage` folders in LCP_HOME 213 7. Copy the test certificate, test private key into the `cert` subfolder of LCP_HOME 214 215 ## Individual server configurations 216 217 Here are the details about the configuration properties of each server. In the samples, replace `<LCP_HOME>` with the absolute path to the folder containing the configuration file, password file, encrypted files, database and certificates. 218 219 ### License Server 220 221 #### profile section 222 `profile`: value of the LCP profile; allowed values are: 223 - `basic`: default value, as described in the Readium LCP specification, used for tests only. 224 - `1.0`: the current production profile, maintained by EDRLab. 225 226 #### lcp section 227 `lcp`: parameters associated with the License Server. 228 - `host`: the public server hostname, `hostname` by default. 229 - `port`: the listening port, `8989` by default. 230 - `public_base_url`: the URL used by the Status Server and the Frontend Test Server to communicate with this License server; combination of the host and port values on http by default. 231 - `auth_file`: mandatory; the path to the password file introduced in a preceding section. 232 - `cert_date`: new in v1.8, a date formatted as "yyyy-mm-dd", which corresponds to the date on which a new X509 certificate has been installed on the server. This is a patch related to a temporary flaw found in several LCP compliant reading applications. 233 - `database`: the URI formatted connection string to the database, see models below. 234 235 Here are models for the database property (variables in curly brackets): 236 - sqlite: `sqlite3://file:{path-to-dot-sqlite-file}?cache=shared&mode=rwc` 237 - MySQL: `mysql://{username}:{password}@/{dbname}?parseTime=true` 238 - MS SQL Server: `mssql://server={server-address};user id={username};password={password};database={dbname}` 239 - PostgrSQL: `postgres://{username}:{password}@{host}:{port}/{dbname}` 240 241 Note 1 relative to MS SQL Server: when using SQL Server Express, the server-address is of type `{ip-address}\\SQLEXPRESS)`. 242 243 Note 2 relative to MS SQL Server: we've seen installs with the additional parameters `;connection timeout=30;encrypt=disable`. 244 245 Note 3 relative to PostgrSQL: add `?sslmode=disable` for a local test install. 246 247 #### storage section 248 This section should be empty if the storage location of encrypted publications is managed by the lcpencrypt utility. 249 If this section is present and lcpencrypt does not manage the storage, all encrypted publications will be stored in the configured folder or s3 bucket. 250 251 `storage`: parameters related to the storage of encrypted publications. 252 - `mode` : optional. Possible values are "fs" (default value) and "s3". 253 254 If `mode` value is `s3`, the following parameters are expected: 255 - `bucket` (required): name of the target S3 bucket. 256 - `region` (optional): name of the target AWS region. 257 258 The S3 region and client credentials default to a chain of credential providers, searched in environment variables and shared files. See [Setting up an S3 Storage](https://github.com/readium/readium-lcp-server/wiki/Setting-up-an-S3-storage) for details. 259 Alternatively (but this is not recommended!), credentials can be stored in clear in the configuration file: 260 - `access_id`: value of the AWS access key id. 261 - `secret`: value of the AWS secret access key. 262 263 If `mode` value is NOT `s3`, the following paremeters are expected: 264 - `filesystem` subsection: parameters related to a file system storage. 265 - `directory`: absolute path of the directory in which all encrypted publications are stored. 266 - `url`: absolute http or https url of the storage volume in which all encrypted publications are stored. 267 268 #### certificate section 269 `certificate`: parameters related to the signature of licenses: 270 - `cert`: the path to provider certificate file (.pem or .crt). It will be inserted in the licenses and used by clients for checking the signature. 271 - `private_key`: the path to the private key (.pem) asociated with the certificate. It will be used for signing licenses. 272 273 #### license section 274 `license`: parameters related to static information to be included in all licenses generated by the License Server: 275 - `links`: subsection: links that will be included in all licenses. `hint` and `publication` links are required in a Readium LCP license. 276 If no such link exists in the partial license passed from the frontend when a new license his requested, 277 these link values will be inserted in the partial license. 278 If no value is present in the configuration file and no value is inserted in the partial license, 279 the License server will reply with a 500 Server Error at license creation. 280 The sub-properties of the `links` section are: 281 - `status`: required, URL template; location of the Status Document associated with a License Document. 282 The license identifier is inserted via the `{license_id}` variable. 283 The License Status Server is expecting the following form: `https://<url>/licenses/{license_id}/status` 284 - `hint`: required; location where a Reading System can redirect a user looking for additional information about the User Passphrase. 285 - `publication`: *deprecated in favor of the storage / filesystem / url parameter*, URL template; 286 Absolute http or https url of the storage volume in which all encrypted publications are stored. 287 The publication identifier is inserted via the `{publication_id}` variable. 288 289 #### lsd and lsd_notify_auth section 290 The License Server must be able to notify the Status Server of the generation of a new license. 291 292 The configuration of the License Server must therefore include: 293 294 `lsd` containing `public_base_url`: the public URL of the Status Server. 295 296 `lsd_notify_auth`: authentication parameters used by the License Server for notifying the Status Server 297 of the generation of a new license. This section contains: 298 - `username`: required, authentication username 299 - `password`: required, authentication password 300 301 #### Sample config 302 Here is a License Server sample config: 303 304 ```yaml 305 profile: "basic" 306 lcp: 307 host: "192.168.0.1" 308 port: 8989 309 public_base_url: "http://192.168.0.1:8989/lcpserver" 310 database: "sqlite3://file:/usr/local/var/readium/db/lcp.sqlite?cache=shared&mode=rwc" 311 auth_file: "/usr/local/var/readium/config/htpasswd" 312 certificate: 313 cert: "/usr/local/var/readium/config/cert.pem" 314 private_key: "/usr/local/var/readium/config/privkey.pem" 315 license: 316 links: 317 status: "https://www.example.net/lsdserver/licenses/{license_id}/status" 318 hint: "https://www.example.net/static/lcp_hint.html" 319 lsd: 320 public_base_url: "http://192.168.0.1:8990" 321 lsd_notify_auth: 322 username: "adm_username" 323 password: "adm_password" 324 325 ``` 326 327 ### Status Server 328 329 #### lsd section 330 `lsd`: parameters associated with the Status Server. 331 - `host`: the public server hostname, `hostname` by default. 332 - `port`: the listening port, `8990` by default. 333 - `public_base_url`: the URL used by the License Server to communicate with this Status Server; combination of the host and port values on http by default. 334 - `auth_file`: mandatory; the path to the password file introduced in a preceding section.. 335 - `database`: the URI formatted connection string to the database, see above for the format. 336 337 - `license_link_url`: URL template, mandatory; this is the url from which a fresh license can be fetched from the provider's frontend server. This url template supports a `{license_id}` parameter. The final url will be inserted in the 'license' link of every status document. It must be the url of a server acting as a proxy between the user request and the License Server. Such proxy is mandatory, as the License Server does not possess user information needed to craft a license from its identifier. If the test frontend server is used as a proxy (for tests only), the url template must be of the form "http://<frontend-server-url>/api/v1/licenses/{license_id}" (note the /api/v1 section). 338 339 #### license_status section 340 `license_status`: parameters related to the interactions implemented by the Status server, if any: 341 - `renting_days`: maximum number of days allowed for a loan, used for laon extensions. The maximum end date is calculated from the date the loan starts plus this value. If set to 0 or absent, no loan renewal is possible. 342 - `renew`: boolean; if `true`, the renewal of a loan is possible. 343 - `renew_days`: default number of additional days allowed during a renewal. 344 - `return`: boolean; if `true`, an early return is possible. 345 - `register`: boolean; if `true`, registering a device is possible. 346 - `renew_page_url`: URL template; if set, the renew feature is implemented as an HTML page. This url template supports a `{license_id}`, `{/license_id}` or `{?license_id}` parameter. The final url will be inserted in the 'renew' link of every status document. 347 - `renew_custom_url`: URL template; if set, the renew feature is managed by the license provider. This url template supports a `{license_id}`, `{/license_id}` or `{?license_id}` parameter. The final url will be inserted in the 'renew' link of every status document. 348 349 Detailed explanations about the use of `renew_page_url` and `renew_custom_url` are found in a [specific section of the wiki](https://github.com/readium/readium-lcp-server/wiki/Integrating-the-LCP-server-into-a-distribution-platform#option-manage-renew-requests-using-your-own-rules). 350 351 #### lcp_update_auth section 352 The Status Server must be able to get information from the License Server. 353 354 The configuration of the Status Server must therefore contain: 355 356 `lcp` containing `public_base_url`: the public URL of the License Server. 357 358 `lcp_update_auth`: authentication parameters used by the Status Server for updating a license via the License Server. This section contains: 359 - `username`: mandatory, authentication username 360 - `password`: mandatory, authentication password 361 362 #### Sample config 363 Here is a Status Server sample config: 364 365 ```yaml 366 lsd: 367 host: "192.168.0.1" 368 port: 8990 369 public_base_url: "http://192.168.0.1:8990" 370 database: "sqlite3://file:/usr/local/var/lcp/db/lsd.sqlite?cache=shared&mode=rwc" 371 auth_file: "/usr/local/var/lcp/htpasswd" 372 license_link_url: "https://www.example.net/lcp/licenses/{license_id}" 373 license_status: 374 register: true 375 renew: true 376 return: true 377 renting_days: 60 378 renew_days: 7 379 380 lcp: 381 public_base_url: "http://192.168.0.1:8989" 382 lcp_update_auth: 383 username: "adm_username" 384 password: "adm_password" 385 ``` 386 387 Execution 388 ========== 389 Each server must be launched in a different context (i.e. a different terminal for local use). If the path to the generated Go binaries ($GOPATH/bin) is properly set, each server can launched from any location: 390 391 - `lcpserver` 392 - `lsdserver` 393 394 NOTE: even if you deploy the server locally, using 127.0.0.1 is not recommended as you won't be able to access the modules from e.g. a mobile app. It is much better to use the WiFi IPv4 address of your computer and access the server from a mobile device via WiFi. 395 396 Contributing 397 ============ 398 Contributions are welcome. 399 400 Please make a Pull Request with tests at github.com/readium/readium-lcp-server