github.com/jonsyu1/godel@v0.0.0-20171017211503-64567a0cf169/docs/Philosophy.md (about) 1 Philosophy 2 ========== 3 The design of gödel is based on some core philosophical principles. Although they are not particularly novel, these 4 principles drive most of the design decisions of gödel. 5 6 Builds and deployment are part of a project 7 ------------------------------------------- 8 Building and deployment should be core concerns for all projects. Code is not useful in a vacuum, and it is all too 9 often the case that developers focus on writing code or implementing functionality without giving deep thought to how 10 the software will eventually be built and deployed. Building the binaries for a product and creating its distribution is 11 just as critical to a product as its core code, and the code and mechanisms for performing these tasks should be held to 12 the same standard as the core code for a project. 13 14 gödel provides tasks for building and publishing and includes them as part of the core project. 15 16 Separate configuration and logic 17 -------------------------------- 18 Configuration and logic should be distinct. The place where this is violated most often is build scripts -- typically, 19 scripts that are written to build and distribute a project conflate the configuration for the project with the logic for 20 performing actions such as building and publishing the product. This makes it hard to track changes, since changes in 21 these files may be either due to changing configuration or changing logic. It also makes logic harder to test and 22 re-use. If a script is useful, other projects will tend to copy them. However, if they mingle logic and configuration, 23 they tend to fork as they are copied, and it's hard (or impossible) to roll out updates later in a uniform manner. 24 25 gödel establishes clear separation between configuration and logic. All of the project-specific configuration is stored 26 in config files in `godel/config` and all of the generic logic for running actions is in gödel. This allows the logic to 27 be tested and generalized, and also makes it possible to roll out updates cleanly across multiple different projects. 28 29 If a convention isn't enforced with automation, it will decay 30 ------------------------------------------------------------- 31 Whether it be indentation level, coding style, license headers on files or tasks that create generated code, if a 32 convention is not enforced in an automated manner, it will inevitably decay over time (this is especially true for 33 larger code bases with multiple contributors). This is similar to the philosophy behind [gofmt](https://blog.golang.org/go-fmt-your-code). 34 Establishing a convention and asking people to follow it (whether it be by running a program themselves, creating a Git 35 hook for it, or trying to catch it in code reviews) is great. However, if the convention is not checked and enforced in 36 an automated manner, it is doomed to fail (a.k.a. [second law of thermodynamics](https://en.wikipedia.org/wiki/Second_law_of_thermodynamics)). 37 38 gödel tasks are designed to be able to run as part of CI to verify and enforce best practices for formatting, linting 39 checks, license headers and more. 40 41 Embed expertise and knowledge into tools so that it scales 42 ---------------------------------------------------------- 43 As people spend time working in a codebase or ecosystem, they start to develop expertise. If someone has been working in 44 Go for a while, knowing that a project must be in a `$GOPATH` is second nature. If `go build ./...` fails due to failing 45 to build something in the vendor directory, they can easily reason/Google their way to running 46 `go build $(go list ./... | grep -v /vendor/)` instead. One may also know tricks like the fact that running `go install std` 47 to build the standard library for different OS/architectures before cross-compiling saves a ton of time for repeated 48 cross-platform compilations. Pushing out tips and tricks like this to all of the developers that might work on a project 49 is hard to do, and even if an effective mechanism for it exists inertia is a powerful force and many people will not 50 alter their practices. However, adding this kind of information/logic to tooling provides it for free to anyone who uses 51 the tools. 52 53 gödel bakes in many of these kinds of optimizations and expertise into the tooling so that even people who are new to Go 54 can immediately create projects and start writing, building and publishing code in Go without having to learn these 55 kinds of optimizations themselves. 56 57 Tasks that apply should be able to verify 58 ----------------------------------------- 59 Some tasks like applying correct formatting or ensuring that source files have a specific license header can be applied 60 automatically without user feedback. Ideally, any task that can apply a change should also have a mode that verifies 61 whether the state of the world matches the expected state without making any modifications. If the state of the world 62 differs, the task should fail and provide feedback about what needs to be changed to make it pass. Having this mode of 63 operation makes it much easier to run the task as part of continuous integration. 64 65 gödel provides a verification mode for all of its tasks that make modifications. 66 67 Optimize the default use case for humans 68 ---------------------------------------- 69 When thinking about the default behavior for commands or flags, optimize on the choice that is easier for humans that 70 use the tool in an interactive manner. This came up when trying to determine the default behavior for `./godelw verify`. 71 The command has an `--apply` flag that, when true, applies the changes detected by verify, and there was a question as 72 to what the default value of the flag should be. Conceptually, there's a case to be made for the default value being 73 false -- `verify` should only verify by default and apply changes only when specifically instructed to do so. However, 74 most developers invoking the command locally will always want to apply the changes that are flagged. Requiring them to 75 run `./godelw verify --apply=true` every time is much more onerous than just making `./godelw verify` apply changes by 76 default. Based on this, we made the default value of the flag `true`. This means that CI environments typically have to 77 invoke `./godelw verify --apply=false`. However, in CI, this value is defined once and then forgotten, so this trade-off 78 made sense. 79 80 gödel is designed to have sensible defaults that do what a developer would expect when invoking the command locally. 81 82 Checks must be repeatable 83 ------------------------- 84 The result of a given set of checks on a specific input should stay constant over time. This property is not true for 85 builds that use `go get` to retrieve checks or build tools at the beginning of each build because the result of `go get` 86 can change over time (either because the version of the tool changes or because it becomes unavailable). 87 88 gödel contains the code for all of its checks and requires projects to declare the version of gödel that is uses as part 89 of its configuration, so a given version of a project is tied to a specific version of gödel and the result of the 90 checks stay constant over time. 91 92 Although the gödel executable itself is not included in projects, as long as the distribution for the version of gödel 93 used is available it will build with the same result. gödel distributions can also be downloaded/saved locally or 94 on-prem to avoid dependencies on external services. 95 96 Checks should be fast and idempotent 97 ------------------------------------ 98 Developers iterate quickly and have little patience. Tasks should complete on the order of seconds when possible. It 99 should also be possible to kill checks at any point without any adverse effects. 100 101 Most gödel tasks complete in under 1 second on small projects, and even on larger projects most tasks complete on the 102 order of seconds (checks and tests can take longer). All tasks are designed to be idempotent and fail gracefully and 103 clean up when terminated. 104 105 Failures should be obvious and provide contextual information 106 ------------------------------------------------------------- 107 If a task or operation fails, it should provide a user-readable explanation of what failed along with any relevant 108 context for diagnosing or reproducing the issue. 109 110 gödel strives to provide readable error messages with context rather than just stack traces on failures. Common failure 111 paths have been identified and the error messages strive to include common causes and work-arounds. Failures that occur 112 in sub-processes include information on the command that was invoked and the environment variables provided to it so 113 that users can attempt to diagnose the issue manually. 114 115 Provide building blocks that can be composed 116 -------------------------------------------- 117 Most tasks are composed of multiple different parts or actions, and by default tasks should provide an all-inclusive 118 experience that works out-of-the-box. However, tasks should also provide the flexibility to be used in other ways so 119 other tools can use them to compose their own tasks. When possible, internal tasks should also be structured in a way 120 that composes these distinct tasks so that it is possible to manually perform partial tasks or debug failures of 121 compound tasks. 122 123 gödel provides tasks such as `./godelw packages` and `./godelw products` that echo out state defined by gödel in a 124 manner that can easily be consumed by other tasks or tools. It also provides the `__` invocation mechanism that can be 125 used to run individual pieces of subprograms in isolation (for example, running `./godelw __check __errcheck` invokes 126 the exact piece of sub-functionality used by `./godelw check` when it runs `errcheck`). 127 128 Use a single source of truth with good abstractions 129 --------------------------------------------------- 130 If there is state about the world that is true, define it in a single place and re-use it across the project using the 131 correct abstractions. For example, most build tools want to ignore the "vendor" directory and any directories that 132 contain generated code. These directories should be excluded from tests, checks, formatting, etc. Rather than adding 133 logic to each of these different tasks to ignore these directories, recognize that the abstract issue is that there are 134 a set of directories that are not considered part of the core source of a project -- define them in a single place and 135 then share that information across all tasks and logic. 136 137 gödel does this for things like exclude directories and projects. It establishes abstractions for things like products, 138 projects and packages and uses them consistently in its own code and exposes them as tasks so that other tools can use 139 them as well. 140 141 Orchestrate, don't obfuscate 142 ---------------------------- 143 Go has standard tooling that people understand well. Whenever possible, build tools should delegate to the built-in 144 tooling to do things in a standard manner. 145 146 gödel is designed to do as little work as possible -- its main concerns are reading declarative configuration and then 147 orchestrating standard tools to perform the actual work. When errors or failures occur, gödel strives to expose the 148 failure in a way that can be repeated/verified using standard tooling to verify that the failure was not introduced by 149 gödel. 150 151 If it's not tested, it can't be trusted 152 --------------------------------------- 153 If a piece of functionality isn't tested, it can't be trusted to work. Even if it works now, there are no guarantees 154 that the behavior won't regress in the future. 155 156 gödel has extensive unit and integration tests that are run in CI against multiple different environments. Almost all 157 fixes for issues that are identified are accompanied by tests that ensure that the issue stays fixed. The code and build 158 is designed in a manner that almost every aspect is testable. 159 160 Anticipate user needs and enable them 161 ------------------------------------- 162 If a set of checks are run and one fails, a user will probably want the ability to re-run just the failing check. If 163 the "build" task builds all products for all platforms by default, a user will probably want to be able to build just a 164 specific product for a specific platform. If a task caches results by default, the user will probably want a way to run 165 the task in a manner that ignores the cache. 166 167 gödel tries to anticipate all of the needs or requests that a user will have for a task enable them. The goal is to 168 optimize for the common case, but to provide configuration and options for customizing behavior as necessary. 169 170 Trust is hard to earn and easy to lose 171 -------------------------------------- 172 Having someone opt to use a piece of software is one of the highest compliments that can be paid to it. People are 173 inherently skeptical of new products, and nothing is more frustrating than a build tool that becomes a source of build 174 issues. Build tools are often the bearer of bad news so people tend to have a negative reaction towards interacting with 175 them in the first place, so people have very little patience with build tools. 176 177 gödel strives to be a tool that enhances productivity and can be trusted as a core part of a development and continuous 178 integration setup. It is dog-fooded extensively and maintained with care.