github.com/splucs/witchcraft-go-server@v1.7.0/README.md (about) 1 witchcraft-go-server 2 ==================== 3 [![](https://godoc.org/github.com/palantir/witchcraft-go-server?status.svg)](http://godoc.org/github.com/palantir/witchcraft-go-server) 4 5 `witchcraft-go-server` is a Go implementation of a Witchcraft server. It provides a way to quickly and easily create 6 servers that work in the Witchcraft ecosystem. 7 8 Implementation 9 -------------- 10 11 ### Configuration 12 A witchcraft server is provided with install configuration and runtime configuration. Install configuration specifies 13 configuration values that are static -- they are read in once at server startup and are known to never change. Runtime 14 configuration values are considered *refreshable*. When file-based configuration is used, whenever the runtime 15 configuration file is updated its contents are loaded and the corresponding values are refreshed. See the section on 16 refreshable configuration for more information on this. 17 18 The default configuration uses file-based configuration with the install configuration at `var/conf/install.yml` and 19 runtime configuration at `var/conf/runtime.yml`. It is possible to use code to specify different sources of 20 configuration (for example, in-memory providers). 21 22 `witchcraft-server` also supports using `encrypted-config-value` to automatically decrypt encrypted configuration 23 values. The default configuration expects a key file to be at `var/conf/encrypted-config-value.key`. It is possible to 24 use code to specify a different source for the key (or to specify that no key should be used). If the configuration 25 does not contain encrypted values, any specified ECV key will not be read. If the install configuration contains 26 encrypted values but the encryption key is missing or malformed, the server will fail to start. If the runtime config 27 contains encrypted values but fails to decrypt them, a warning will be logged and the encrypted values passed to the 28 server. 29 30 `witchcraft-server` defines base configuration for its install and runtime configuration. Servers that want to provide 31 their own install and/or runtime configuration should embed the base configuration structs within the definition of 32 their configuration structs. 33 34 ### Route registration 35 A witchcraft server is backed by a `wrouter.Router` and allows authors to register route handlers on the server. The 36 router uses a specific format for path templates to specify path parameters and has rules around the kinds of paths that 37 can be matched. witchcraft is opinionated about the path formats and does not support registering paths that cannot be 38 expressed using its template rules. All witchcraft routes are configured to emit request logs and trace logs and update 39 metrics for the requests using built-in middleware. The `context.Context` for the `http.Request` provided to the 40 handlers is configured with all of the standard loggers (service logger, event logger, trace logger, etc.). 41 42 When registering routes on the router, it is also possible to specify path/header/query param keys that should be 43 considered "safe" or "forbidden" when used as parameters in logging. These are combined with the default set of safe and 44 forbidden header parameters defined by the `req2log` package in `witchcraft-go-logging`. 45 46 ### Liveness, readiness, and health 47 `witchcraft-server` registers the endpoints `/status/liveness`, `/status/readiness` and `/status/health` to report the 48 server's liveness, readiness and health. By default, these endpoints use a built-in provider that reports liveness, 49 readiness and health based on the state of the server. It is possible to configure the liveness and readiness providers 50 in code, and health status providers can also be added via code (health supports specifying multiple sources to report 51 health, and the server's built-in health status provider will always be one of them). 52 53 The default behavior serves both the user-registered endpoints and the status endpoints from the same server. However, 54 if a "management port" is specified in the server's install configuration and its value differs from the "port" value in 55 configuration, then `witchcraft-server` starts a second management server on the specified port and serves the status 56 endpoints on that port. This can be useful in scenarios where all of the traffic to the main endpoints require client 57 certificates for TLS but the status endpoints need to be served without requiring client TLS certificates. 58 59 ### Debug Routes 60 The following routes are registered on the management server (if enabled, otherwise the main server) to aid in debugging 61 and telemetry collection: 62 * `/debug/pprof`: Provides an HTML index of the other endpoints at this route. 63 * `/debug/pprof/profile`: Returns the pprof-formatted cpu profile. See [pprof.Profile](https://golang.org/pkg/net/http/pprof/#Profile). 64 * `/debug/pprof/heap`: Returns the pprof-formatted heap profile as of the last GC. See [pprof.Profile](https://golang.org/pkg/runtime/pprof/#Profile). 65 * `/debug/pprof/cmdline`: Returns the process's command line invocation as `text/plain`. See [pprof.Cmdline](https://golang.org/pkg/net/http/pprof/#Cmdline). 66 * `/debug/pprof/symbol`: Looks up the program counters listed in the request, responding with a table mapping program counters to function names See [pprof.Symbol](https://golang.org/pkg/net/http/pprof/#Symbol). 67 * `/debug/pprof/trace`: Returns the execution trace in binary form. See [pprof.Trace](https://golang.org/pkg/net/http/pprof/#Trace). 68 69 ### Context path 70 If `context-path` is specified in the install configuration, all of the routes registered on the server will be prefixed 71 with the specified `context-path`. 72 73 ### Security 74 `witchcraft-server` only supports HTTPS. The TLS client authentication type is configurable in code. The base install 75 configuration has fields to specify the location of server key and certificate material for TLS connections. 76 77 Although it is not possible to run `witchcraft-server` using HTTP, it is possible to configure the server in code to use 78 a generated self-signed certificate on start-up. Running the server in this mode and connecting to it using TLS without 79 server certificate verification (equivalent of `curl -k` or an `http.Transport` with 80 `TLSClientConfig: &tls.Config{InsecureSkipVerify: true}`) provides an analog to using HTTP, with the benefit that the 81 traffic itself is still encrypted. 82 83 ### Logging 84 `witchcraft-server` is configured with service, event, metric, request and trace loggers from the 85 `witchcraft-go-logging` project and emits structured JSON logs using [`zap`](https://github.com/uber-go/zap) as the 86 logger implementation. The default behavior emits logs to the `var/log` directory (`var/log/service.log`, 87 `var/log/request.log`, etc.) unless the server is run in a Docker container, in which case the logs are always emitted 88 to `stdout`. The `use-console-log` property in the install configuration can also be set to "true" to always output logs 89 to `stdout`. The runtime configuration supports configuring the log output level for service logs. 90 91 The `context.Context` provided to request handlers is configured with all of the standard loggers (service logger, event 92 logger, trace logger, etc.). All of the handlers are also configured to emit request logs and trace logs. 93 94 ### Service logger origin 95 By default, the `origin` field of the service logger is set to be the package path of the package in which the 96 `witchcraft-server` is started. For example, if the server is started in the file 97 `github.com/palantir/project/server/server.go`, the origin for all service log lines will be 98 `github.com/palantir/project/server`. 99 100 It is possible to configure the origin to be a different value using code. The origin can be specified to be a string 101 constant or a function can be used that returns a specific package path based on supplied parameters (for example, the 102 function can specify that the caller package's parent package should be used as the origin). The origin can also be set 103 to empty, in which case it is omitted from the log output. 104 105 ### Trace IDs and instrumentation 106 `witchcraft-server` supports zipkin-compatible tracing and ensures that every request is instrumented for tracing. 107 `witchcraft-server` also recognizes that some code will use trace IDs without necessarily using full zipkin-compatible 108 spans, so some allowances are made to support this scenario. 109 110 The built-in `witchcraft-server` middleware that registers loggers on the context also ensures that a zipkin span is 111 started. If the incoming request header has valid zipkin span information (that is, it specifies both a `X-B3-TraceId` 112 and `X-B3-SpanId` in the header), then the span created by the middleware is a child span of the incoming span. If the 113 incoming request does not have a trace ID header, a new root span is created. If the header specifies a trace ID but not 114 a span ID, the middleware creates a new root zipkin span, but ensures that the trace ID of the created span matches what 115 is specified in the header. If an incoming request is routed to a registered endpoint, the built-in router middleware 116 will create another span (which is a child span of the one created by the request middleware) whose span name is the 117 HTTP method and template for the endpoint. 118 119 The trace information generated by the middleware is set on the header and will be visible to subsequent handlers. If 120 the request specifies a `X-B3-Sampled` header, the value specified in that header is used to determine sampling. If this 121 header is not present, whether or not the trace is sampled is determined by the sampling source configured for the 122 `witchcraft-server` (by default, all traces are sampled). If a trace is not sampled, `witchcraft-server` will not 123 generate any trace log output for it. However, the infrastructure will still perform all of the trace-related operations 124 (such as creating child spans and setting span information on headers). The install configuration field 125 `trace-sample-rate` represents a float between 0 and 1 (inclusive) to control the proportion of traces sampled by 126 default. If the `WithTraceSampler` server option is provided, it overrides this configuration. 127 128 `witchcraft-server` also ensures that the context for every request has a trace ID. After the logging middleware 129 executes, the request is guaranteed to have a trace ID (either from the incoming request or from the newly generated 130 root span), and that trace ID is registered on the context. The `witchcraft.TraceIDFromContext(context.Context) string` 131 function can be used to retrieve the trace ID from the context. 132 133 ### Creating new spans/trace log entries 134 Use the `wtracing.StartSpanFromContext` function to start a new span. 135 This function will create a new span that is a child span of the span in the provided context. 136 Defer the `Finish()` function of the returned span to ensure that the span is properly marked as finished (the "finish" 137 operation will also generate a trace log entry if the span is sampled). 138 139 ### Middleware 140 `witchcraft-server` supports registering middleware to perform custom handling/augmenting of incoming requests. There 141 are 2 different kinds of middleware: *request* and *route* middleware. 142 143 Request middleware is executed on every request received by the server. The function signature for request middleware is 144 `func(rw http.ResponseWriter, r *http.Request, next http.Handler)`. Request middleware is the most common kind of 145 middleware. The server has built-in request middleware that adds a panic handler, sets the loggers and trade ID on the 146 request context and updates request-related metrics. Any user-supplied request middleware is run after the built-in 147 request middleware in the order in which they were added (which means that the context has all of the loggers 148 configured). Request middleware is run before the request is handled by the router, which means that it is possible to 149 rewrite the URL and other properties of the request and the router will route the _modified_ request. However, note that 150 the built-in logging middleware extracts the UID, SID, TokenID and TraceID from the request and sets them on the loggers 151 before user-provided middleware is invoked, so if the user-defined middleware modifies the header in a manner that would 152 change any of these values, the middleware should also update the request context to have loggers that use the updated 153 values. 154 155 Route middleware is only executed on the routes that are registered on the router -- they wrap the handler registered on 156 the route, so they are executed after the path has been matched and the handler for the router has been located and the 157 path parameters have been extracted and set on the context. The function signature for route middleware is 158 `func(rw http.ResponseWriter, r *http.Request, reqVals RequestVals, next RouteRequestHandler)`. The `RequestVals` struct 159 stores the path template for the route along with the path parameters and their values. The server has built-in route 160 middleware that records a request log entry after the request has completed and creates a trace log span and logs a 161 trace log entry after the request has completed. Route middleware is run after all of the request middleware has run, 162 and any user-supplied route middleware is run after the built-in route middleware in the order in which they were added. 163 Because router middleware is executed after the routing has been determined, changing the URL of the request will not 164 change the handler that is invoked or the path parameter values that have been extracted/stored (although it may still 165 impact behavior based on the content of the actual handler that is registered). In general, most users will likely use 166 request middleware rather than route middleware. However, if users want to only execute middleware on matched routes and 167 want route-specific information such as the unrendered path template and the path parameter values, then route 168 middleware should be used. 169 170 ### Long-running execution not associated with a route 171 In some instances, a server may want a long-running task not associated with an endpoint. For example, the server may 172 want a long-running goroutine that performs an operation at some interval for the lifetime of the server. 173 174 It is recommended that such goroutines be launched in the initialization function provided to `witchcraft.With` and use 175 the `ctx Context` as its context. This context has the same lifecycle as the server and has all of the configured 176 loggers (service loggers, metric loggers, etc.) already configured on it. 177 178 The provided context does not have a span or trace ID associated with it. If a trace ID is desired, 179 [create a new span](#creating-new-spanstrace-log-entries) with `wtracing.StartSpanFromContext` and the provided context to derive a new context that has a 180 new root span associated with it. This function also updates any loggers in the context to use the new trace ID (for 181 example, service loggers will include the trace ID). 182 183 ### Metrics 184 `witchcraft-server` initializes a metrics registry that uses the `github.com/palantir/pkg/metrics` package (which uses 185 `github.com/rcrowley/go-metrics` internally) to track metrics for the server. All of the tracked metrics are emitted as 186 metric log entries once every metric emit interval, which is 60 seconds by default (and can be configured to be a custom 187 interval in the install configuration). 188 189 By default, `witchcraft-server` captures various Go runtime metrics (such as allocations, number of running goroutines, 190 etc.) at the same frequency as the metric emit frequency. The collection of Go runtime statistics can be disabled with 191 the `WithDisableGoRuntimeMetrics` server method. 192 193 ### SIGQUIT handling 194 `witchcraft-server` sets up a SIGQUIT handler such that, if the program is terminated using a SIGQUIT signal 195 (`kill -3`), a goroutine dump is written as a `diagnostic.1` log. This behavior can be disabled using 196 `server.WithDisableSigQuitHandler`. If `server.WithSigQuitHandlerWriter` is used, the stacks will also be written in 197 their unparsed form to the provided writer. 198 199 ### Shutdown signal handling 200 `witchcraft-server` attempts to drain active connections and gracefully shut down by calling `server.Shutdown` upon receiving a SIGTERM or SIGINT signal. This behavior can be disabled using `server.WithDisableShutdownSignalHandler`. 201 202 Example server initialization 203 ----------------------------- 204 205 ### Basic production server 206 The following is an example program that launches a `witchcraft-server` that registers a `GET /myNum` endpoint that 207 returns a randomly generated number encoded as JSON: 208 209 ```go 210 package main 211 212 import ( 213 "context" 214 "math/rand" 215 "net/http" 216 217 "github.com/palantir/witchcraft-go-server/rest" 218 "github.com/palantir/witchcraft-go-server/witchcraft" 219 "github.com/palantir/witchcraft-go-server/witchcraft/refreshable" 220 "github.com/palantir/witchcraft-go-server/wrouter" 221 ) 222 223 func main() { 224 if err := witchcraft.NewServer(). 225 WithInitFunc(func(ctx context.Context, info witchcraft.InitInfo) (func(), error) { 226 if err := registerMyNumEndpoint(info.Router); err != nil { 227 return nil, err 228 } 229 return nil, nil 230 }). 231 Start(); err != nil { 232 panic(err) 233 } 234 } 235 236 func registerMyNumEndpoint(router wrouter.Router) error { 237 return router.Get("/myNum", http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) { 238 rest.WriteJSONResponse(rw, rand.Intn(100), http.StatusOK) 239 })) 240 } 241 ``` 242 243 Creating a `witchcraft-server` starts with the `witchcraft.NewServer` function, which returns a new witchcraft server 244 with default configuration. The `*witchcraft.Server` struct has various `With*` functions that can be used to configure 245 the server, and the `Start()` function starts the server using the specified configuration. 246 247 The `WithInitFunc(InitFunc)` function is used to register routes on the server. The initialization function provided to 248 `WithInitFunc` is of the type `witchcraft.InitFunc`, which has the following definition: 249 `type InitFunc func(ctx context.Context, info InitInfo) (cleanup func(), rErr error)`. 250 251 The `ctx` provided to the function is valid for the duration of the server and has loggers configured on it. The `info` 252 struct contains fields that can be used to initialize various state and configuration for the server -- refer to the 253 `InitInfo` documentation for more information. 254 255 In this example, a "GET" endpoint is registered on the router using the "/myNum" path, and `rest` package is used to 256 write a JSON response. 257 258 This example server uses all of the `witchcraft` defaults -- it looks for install configuration in 259 `var/conf/install.yml` and uses `config.Install` as its type, looks for runtime configuration in `var/conf/runtime.yml` 260 and uses `config.Runtime` as its type, and looks for an encrypted-config-value key in 261 `var/conf/encrypted-config-value.key`. The install configuration must also specify paths to key and certificate files to 262 use for TLS. 263 264 ### Basic local/test server 265 The defaults for the server make sense for a production environment, but can make running the server locally (or in 266 tests) cumbersome. We can modify the `main` function as follows to configure the witchcraft server to use in-memory 267 defaults: 268 269 ```go 270 func main() { 271 if err := witchcraft.NewServer(). 272 WithInitFunc(func(ctx context.Context, info witchcraft.InitInfo) (func(), error) { 273 if err := registerMyNumEndpoint(info.Router); err != nil { 274 return nil, err 275 } 276 return nil, nil 277 }). 278 WithSelfSignedCertificate(). 279 WithECVKeyProvider(witchcraft.ECVKeyNoOp()). 280 WithRuntimeConfig(config.Runtime{}). 281 WithInstallConfig(config.Install{ 282 ProductName: "example-app", 283 Server: config.Server{ 284 Port: 8100, 285 }, 286 UseConsoleLog: true, 287 }). 288 Start(); err != nil { 289 panic(err) 290 } 291 } 292 ``` 293 294 The `WithSelfSignedCertificate()` function configures the server to start using a generated self-signed certificate, 295 which removes the need to specify server TLS material. The `WithECVKeyProvider(witchcraft.ECVKeyNoOp())` function 296 configures the server to use an empty ECV key source. The `WithRuntimeConfig(config.Runtime{})` function configures the 297 server to use the provided runtime configuration (in this case, it is empty), and the `WithInstallConfig` function 298 specifies the install configuration that should be used (it specifies that port 8100 should be used and that log output 299 should go to STDOUT). 300 301 With this configuration, the program can be run using `go run`: 302 303 ``` 304 ➜ go run main.go 305 {"level":"INFO","time":"2018-11-27T05:47:02.013456Z","message":"Listening to https","type":"service.1","origin":"github.com/palantir/witchcraft-go-server/app_example","params":{"address":":8100","server":"example-app"}} 306 ``` 307 308 Issuing a request to this server using `curl` produces the expected response (note that the `-k` option is used to skip 309 certificate verification because the server is using a self-signed certificate): 310 311 ```go 312 ➜ curl -k https://localhost:8100/myNum 313 81 314 ``` 315 316 You can also observe that the server emits trace and request logs based on receiving this request: 317 318 ``` 319 {"time":"2018-11-27T05:47:28.313585Z","type":"trace.1","span":{"traceId":"7e43bde2647413fc","id":"01228e628b3b3d22","name":"GET /myNum","parentId":"7e43bde2647413fc","timestamp":1543297648313551,"duration":29000}} 320 {"time":"2018-11-27T05:47:28.313719Z","type":"request.2","method":"GET","protocol":"HTTP/2.0","path":"/myNum","status":200,"requestSize":0,"responseSize":3,"duration":146,"traceId":"7e43bde2647413fc","params":{"Accept":"*/*","User-Agent":"curl/7.54.0","X-B3-Parentspanid":"7e43bde2647413fc","X-B3-Sampled":"1","X-B3-Spanid":"01228e628b3b3d22","X-B3-Traceid":"7e43bde2647413fc"}} 321 {"time":"2018-11-27T05:47:28.313802Z","type":"trace.1","span":{"traceId":"7e43bde2647413fc","id":"7e43bde2647413fc","name":"witchcraft-go-server request middleware","timestamp":1543297648313496,"duration":304000}} 322 ``` 323 324 ### Server using install configuration 325 The previous examples used the built-in install configuration. Most real servers will use custom install configuration 326 that specifies configuration for the server. Any struct can be used as install configuration, but it must support being 327 unmarshaled as YAML and must embed the `config.Install` struct. The install configuration is loaded once when the server 328 starts (it is never reloaded), so only values that are static for the lifetime of the server should be specified in this 329 configuration. 330 331 The following example modifies the previous example so that the endpoint returns the number defined in the install 332 configuration instead of a random number: 333 334 ```go 335 package main 336 337 import ( 338 "context" 339 "net/http" 340 341 "github.com/palantir/witchcraft-go-server/config" 342 "github.com/palantir/witchcraft-go-server/rest" 343 "github.com/palantir/witchcraft-go-server/witchcraft" 344 "github.com/palantir/witchcraft-go-server/witchcraft/refreshable" 345 "github.com/palantir/witchcraft-go-server/wrouter" 346 ) 347 348 type AppInstallConfig struct { 349 config.Install `yaml:",inline"` 350 351 MyNum int `yaml:"my-num"` 352 } 353 354 func main() { 355 if err := witchcraft.NewServer(). 356 WithInitFunc(func(ctx context.Context, info witchcraft.InitInfo) (func(), error) { 357 if err := registerMyNumEndpoint(info.Router, info.InstallConfig.(AppInstallConfig).MyNum); err != nil { 358 return nil, err 359 } 360 return nil, nil 361 }, 362 ). 363 WithSelfSignedCertificate(). 364 WithECVKeyProvider(witchcraft.ECVKeyNoOp()). 365 WithRuntimeConfig(config.Runtime{}). 366 WithInstallConfigType(AppInstallConfig{}). 367 WithInstallConfig(AppInstallConfig{ 368 Install: config.Install{ 369 ProductName: "example-app", 370 Server: config.Server{ 371 Port: 8100, 372 }, 373 UseConsoleLog: true, 374 }, 375 MyNum: 13, 376 }). 377 Start(); err != nil { 378 panic(err) 379 } 380 } 381 382 func registerMyNumEndpoint(router wrouter.Router, num int) error { 383 return router.Get("/myNum", http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) { 384 rest.WriteJSONResponse(rw, num, http.StatusOK) 385 })) 386 } 387 ``` 388 389 This example defines the `AppInstallConfig` struct, which embeds `config.Install` and also defines a `MyNum` field. The 390 `WithInstallConfigType(AppInstallConfig{})` function call is added to specify `AppInstallConfig{}` as the install struct 391 and the initialization function logic is modified to convert the provided `installConfig interface{}` into an 392 `AppInstallConfig` and uses the `MyNum` value as the value that is returned by the endpoint. The `WithInstallConfig` 393 function is also updated to use configuration that specifies a value for `MyNum`. 394 395 Running the updated program using `go run main.go` and issuing `curl -k https://localhost:8100/myNum` returns `13`. 396 397 A real program will generally read runtime configuration from disk rather than specifying it directly in code. We can 398 modify the example above to do this by simply removing the `WithInstallConfig` call: 399 400 ```go 401 package main 402 403 import ( 404 "context" 405 "net/http" 406 407 "github.com/palantir/witchcraft-go-server/config" 408 "github.com/palantir/witchcraft-go-server/rest" 409 "github.com/palantir/witchcraft-go-server/witchcraft" 410 "github.com/palantir/witchcraft-go-server/witchcraft/refreshable" 411 "github.com/palantir/witchcraft-go-server/wrouter" 412 ) 413 414 type AppInstallConfig struct { 415 config.Install `yaml:",inline"` 416 417 MyNum int `yaml:"my-num"` 418 } 419 420 func main() { 421 if err := witchcraft.NewServer(). 422 WithInitFunc(func(ctx context.Context, info witchcraft.InitInfo) (func(), error) { 423 if err := registerMyNumEndpoint(info.Router, info.InstallConfig.(AppInstallConfig).MyNum); err != nil { 424 return nil, err 425 } 426 return nil, nil 427 }, 428 ). 429 WithSelfSignedCertificate(). 430 WithECVKeyProvider(witchcraft.ECVKeyNoOp()). 431 WithInstallConfigType(AppInstallConfig{}). 432 Start(); err != nil { 433 panic(err) 434 } 435 } 436 437 func registerMyNumEndpoint(router wrouter.Router, num int) error { 438 return router.Get("/myNum", http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) { 439 rest.WriteJSONResponse(rw, num, http.StatusOK) 440 })) 441 } 442 ``` 443 444 By default, the install configuration is read from `var/conf/install.yml`. Create a file at that path relative to the 445 Go file and provide it with the YAML content for the configuration: 446 447 ```yaml 448 product-name: "example-app" 449 use-console-log: true 450 server: 451 port: 8100 452 my-num: 77 453 ``` 454 455 Running the updated program using `go run main.go` and issuing `curl -k https://localhost:8100/myNum` returns `77`. 456 457 ### Server using runtime configuration 458 Runtime configuration is similar to install configuration. The main difference is that runtime configuration supports 459 reloading configuration. When file-based runtime configuration is used, whenever the configuration file is updated, the 460 associated values are updated as well. 461 462 The following example defines a custom runtime configuration struct and returns the refreshable int value in the runtime 463 from its endpoint (the example uses a basic in-memory install configuration for simplicity): 464 465 ```go 466 package main 467 468 import ( 469 "context" 470 "net/http" 471 472 "github.com/palantir/witchcraft-go-server/config" 473 "github.com/palantir/witchcraft-go-server/rest" 474 "github.com/palantir/witchcraft-go-server/witchcraft" 475 "github.com/palantir/witchcraft-go-server/witchcraft/refreshable" 476 "github.com/palantir/witchcraft-go-server/wrouter" 477 ) 478 479 type AppRuntimeConfig struct { 480 config.Runtime `yaml:",inline"` 481 482 MyNum int `yaml:"my-num"` 483 } 484 485 func main() { 486 if err := witchcraft.NewServer(). 487 WithInitFunc(func(ctx context.Context, info witchcraft.InitInfo) (func(), error) { 488 myNumRefreshable := refreshable.NewInt(info.RuntimeConfig.Map(func(in interface{}) interface{} { 489 return in.(AppRuntimeConfig).MyNum 490 })) 491 if err := registerMyNumEndpoint(info.Router, myNumRefreshable); err != nil { 492 return nil, err 493 } 494 return nil, nil 495 }, 496 ). 497 WithSelfSignedCertificate(). 498 WithECVKeyProvider(witchcraft.ECVKeyNoOp()). 499 WithInstallConfig(config.Install{ 500 ProductName: "example-app", 501 Server: config.Server{ 502 Port: 8100, 503 }, 504 UseConsoleLog: true, 505 }). 506 WithRuntimeConfigType(AppRuntimeConfig{}). 507 Start(); err != nil { 508 panic(err) 509 } 510 } 511 512 func registerMyNumEndpoint(router wrouter.Router, numProvider refreshable.Int) error { 513 return router.Get("/myNum", http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) { 514 rest.WriteJSONResponse(rw, numProvider.CurrentInt(), http.StatusOK) 515 })) 516 } 517 ``` 518 519 The refreshable configuration warrants some closer examination. Note that the `registerMyNumEndpoint` takes a 520 `numProvider refreshable.Int` as an argument rather than an `int` and returns the result of `CurrentInt()`. 521 Conceptually, the `numProvider` is guaranteed to always return the current value of the number specified in the runtime 522 configuration. Using this pattern removes the need for writing code that listens for updates -- the code can simply 523 assume that the provider always returns the most recent value. `refreshable.Int` and `refreshable.String` are helper 524 types that provide functions that return the current value of the correct type. For types without helper functions, the 525 general `refreshable.Refreshable` should be used, and the `interface{}` returned by `Current()` must be explicitly 526 converted to the proper target type (this is required because Go does not support generics/templatization). 527 528 The `numProvider` provided to `registerMyNumEndpoint` is derived by applying a mapping function to the 529 `runtimeConfig refreshable.Refreshable` parameter. `runtimeConfig.Map` is provided with a function that, given an 530 updated runtime configuration, returns the portion of the configuration that is required. The input to the mapping 531 function must be explicitly cast to the runtime configuration type (in this case, `in.(AppRuntimeConfig)`), and then the 532 relevant section can be accessed (or derived) and returned. The result of the `Map` function is a `Refreshable` that 533 returns the mapped portion. In this case, because we know the result will always be an `int`, we wrap the returned 534 `Refreshable` in a `refreshable.NewInt` call, which provides the convenience function `CurrentInt()` that performs the 535 type conversion of the result to an `int`. 536 537 By default, the runtime configuration is read from `var/conf/runtime.yml`. Create a file at that path relative to the 538 Go file and provide it with the YAML content for the configuration: 539 540 ```yaml 541 my-num: 99 542 ``` 543 544 Running the updated program using `go run main.go` and issuing `curl -k https://localhost:8100/myNum` returns `99`. 545 While the program is still running, update the content of the file to be `my-num: 88`, save it, then run the `curl` 546 command again. The output is `88`. 547 548 ### Full server example 549 The following is an example of a server that defines and uses both custom install and runtime configuration: 550 551 ```go 552 package main 553 554 import ( 555 "context" 556 "net/http" 557 558 "github.com/palantir/witchcraft-go-server/config" 559 "github.com/palantir/witchcraft-go-server/rest" 560 "github.com/palantir/witchcraft-go-server/witchcraft" 561 "github.com/palantir/witchcraft-go-server/witchcraft/refreshable" 562 "github.com/palantir/witchcraft-go-server/wrouter" 563 ) 564 565 type AppInstallConfig struct { 566 config.Install `yaml:",inline"` 567 568 MyNum int `yaml:"my-num"` 569 } 570 571 type AppRuntimeConfig struct { 572 config.Runtime `yaml:",inline"` 573 574 MyNum int `yaml:"my-num"` 575 } 576 577 func main() { 578 if err := witchcraft.NewServer(). 579 WithInitFunc(func(ctx context.Context, info witchcraft.InitInfo) (func(), error) { 580 if err := registerInstallNumEndpoint(info.Router, info.InstallConfig.(AppInstallConfig).MyNum); err != nil { 581 return nil, err 582 } 583 584 myNumRefreshable := refreshable.NewInt(info.RuntimeConfig.Map(func(in interface{}) interface{} { 585 return in.(AppRuntimeConfig).MyNum 586 })) 587 if err := registerRuntimeNumEndpoint(info.Router, myNumRefreshable); err != nil { 588 return nil, err 589 } 590 return nil, nil 591 }, 592 ). 593 WithInstallConfigType(AppInstallConfig{}). 594 WithRuntimeConfigType(AppRuntimeConfig{}). 595 WithSelfSignedCertificate(). 596 WithECVKeyProvider(witchcraft.ECVKeyNoOp()). 597 Start(); err != nil { 598 panic(err) 599 } 600 } 601 602 func registerInstallNumEndpoint(router wrouter.Router, num int) error { 603 return router.Get("/installNum", http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) { 604 rest.WriteJSONResponse(rw, num, http.StatusOK) 605 })) 606 } 607 608 func registerRuntimeNumEndpoint(router wrouter.Router, numProvider refreshable.Int) error { 609 return router.Get("/runtimeNum", http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) { 610 rest.WriteJSONResponse(rw, numProvider.CurrentInt(), http.StatusOK) 611 })) 612 } 613 ``` 614 615 With `var/conf/install.yml`: 616 617 ```yaml 618 product-name: "example-app" 619 use-console-log: true 620 server: 621 port: 8100 622 my-num: 7 623 ``` 624 625 And `var/conf/runtime.yml`: 626 627 ```yaml 628 my-num: 13 629 ``` 630 631 Querying `installNum` returns `7`, while querying `runtimeNum` returns `13`: 632 633 ``` 634 ➜ curl -k https://localhost:8100/installNum 635 7 636 ➜ curl -k https://localhost:8100/runtimeNum 637 13 638 ``` 639 640 In a production server, `WithSelfSignedCertificate()` and `WithECVKeyProvider(witchcraft.ECVKeyNoOp())` would not be 641 called and the proper security and key material would exist in their expected locations. 642 643 Refreshable configuration 644 ------------------------- 645 The runtime configuration for `witchcraft-server` uses the `refreshable.Refreshable` interface. Conceptually, a 646 `Refreshable` is a container that holds a value of a specific type that may be updated/refreshed. The following is the 647 interface definition for `Refreshable`: 648 649 ```go 650 type Refreshable interface { 651 // Current returns the most recent value of this Refreshable. 652 Current() interface{} 653 654 // Subscribe subscribes to changes of this Refreshable. The provided function is called with the value of Current() 655 // whenever the value changes. 656 Subscribe(consumer func(interface{})) (unsubscribe func()) 657 658 // Map returns a new Refreshable based on the current one that handles updates based on the current Refreshable. 659 Map(func(interface{}) interface{}) Refreshable 660 } 661 ``` 662 663 The `runtimeConfig refreshable.Refreshable` parameter provided to the initialization function specified using 664 `WithInitFunc` stores the latest unmarshaled runtime configuration as its current value, and the type of the value is 665 specified using the `WithRuntimeConfigType` function (if this function is not called, `config.Runtime` is used as the 666 default type). 667 668 For example, for the call: 669 670 ```go 671 witchcraft.NewServer(). 672 WithInitFunc(func(ctx context.Context, info witchcraft.InitInfo) (func(), error) { 673 return nil, nil 674 }). 675 WithRuntimeConfigType(AppRuntimeConfig{}) 676 ``` 677 678 The `WithRuntimeConfigType(AppRuntimeConfig{})` function specifies that the type of the runtime configuration is 679 `AppRuntimeConfig`, so the value returned by `runtimeConfig.Current()` in `WithInitFunc` will have the type 680 `AppRuntimeConfig`. Because Go does not have a notion of generics, the author must make this association manually and 681 perform the conversion of the current value into the desired type when using it (for example, 682 `runtimeConfig.Current().(AppRuntimeConfig)`). 683 684 The `Refreshable` interface supports using the `Map` function to derive a new refreshable based on the value of the 685 current refreshable. This allows downstream functions that are only interested in a subset of the refreshable to observe 686 just the relevant portion. 687 688 For example, consider the `AppRuntimeConfig` definition: 689 690 ```go 691 type AppRuntimeConfig struct { 692 config.Runtime `yaml:",inline"` 693 694 MyNum int `yaml:"my-num"` 695 } 696 ``` 697 698 A downstream function may only be interested in updates to the `MyNum` variable -- if updates to `config.Runtime` are 699 not relevant to the function, there is no need to subscribe to it. The following code derives a new `Refreshable` from 700 the `runtimeConfig` refreshable: 701 702 ```go 703 myNumRefreshable := runtimeConfig.Map(func(in interface{}) interface{} { 704 return in.(AppRuntimeConfig).MyNum 705 }) 706 ``` 707 708 The `Current()` function for `myNumRefreshable` returns the `MyNum` field of `in.(AppRuntimeConfig)`, and the derived 709 `Refreshable` is only updated when the derived value changes. Accessing a field is the most common usage of `Map`, but 710 any arbitrary logic can be performed in the mapping function. Just note that the mapping will be performed whenever the 711 parent refreshable is updated and the result will be compared using `reflect.DeepEqual`. 712 713 The general `Refreshable` interface returns an `interface{}` and its result must always be converted to the actual 714 underlying type. However, if a `Refreshable` is known to return an `int`, `string` or `bool`, convenience wrapper types 715 are provided to return typed values. For example, `refreshable.NewInt(in Refreshable)` returns a `refreshable.Int`, 716 which is defined as: 717 718 ```go 719 type Int interface { 720 Refreshable 721 CurrentInt() int 722 } 723 ``` 724 725 The `CurrentInt()` function returns the current value converted to an `int`, which makes it easier to use in code and 726 alleviates the need for clients to manually remember the type stored in the `Refreshable`. 727 728 If a `Refreshable` with a particular value/type is used widely throughout a code base, it may make sense to define a 729 similar interface so that clients do not have to manually track the type information. For example, a typed `Refreshable` 730 for `AppRuntimeConfig` can be defined as follows: 731 732 ```go 733 type RefreshableAppRuntimeConfig interface { 734 Refreshable 735 CurrentAppRuntimeConfig() AppRuntimeConfig 736 } 737 738 type refreshableAppRuntimeConfig struct { 739 Refreshable 740 } 741 742 func (r refreshableAppRuntimeConfig) CurrentAppRuntimeConfig() AppRuntimeConfig { 743 return rt.Current().(AppRuntimeConfig) 744 } 745 ``` 746 747 ### Updating refreshable configuration: provider-based vs. push-based 748 The "provider" model of configuration updates takes the philosophy that executing code simply needs the most up-to-date 749 value of a `Refreshable` when it executes. This model makes the most sense when the value is read whenever an endpoint 750 is executed or when a long-running or periodically executed background task executes. In these scenarios, the latest 751 value of the `Refreshable` is only needed when the logic executes. This update model is typically the most common, and 752 is achieved by passing down specific `Refreshable` providers for the required values to the handlers/routines. 753 754 However, in some cases, an application may want to be notified of every update to a field and react to that update 755 immediately -- for example, if updating a specific configuration field triggers an expensive computation that should 756 happen immediately, the logic wants to be notified as soon as the update is made. 757 758 In this scenario, the `Subscribe` function should be used for the `Refreshable` that has the value for which updates are 759 needed. For example, consider the following configuration: 760 761 ```go 762 type AppRuntimeConfig struct { 763 config.Runtime `yaml:",inline"` 764 765 AssetURLs []string `yaml:"asset-urls"` 766 } 767 ``` 768 769 The `AssetURLs` field specifies URLs that should be downloaded by the program whenever the value is updated. This 770 can be handled as follows: 771 772 ```go 773 unsubscribe := runtimeConfig.Map(func(in interface{}) interface{} { 774 return in.(AppRuntimeConfig).AssetURLs 775 }).Subscribe(func(in interface{}) { 776 assetURLs := in.([]string) 777 // perform work 778 }) 779 // unsubscribe should be deferred or stored and run at shutdown 780 ``` 781 782 The `Map` function returns a new `Refreshable` that updates only when the `AssetURLs` field is updated, and the 783 `Subscribe` function subscribes a listener that performs work as soon as the value is updated. This ensures that the 784 logic is run as soon as the value is refreshed every time the value is updated. 785 786 License 787 ------- 788 This project is made available under the [Apache 2.0 License](http://www.apache.org/licenses/LICENSE-2.0).