github.com/kubecost/golang-migrate-duckdb/v4@v4.17.0-duckdb.1/README.md (about) 1 # NOTE 2 # NOTE: This is a Kubecost internal fork of https://github.com/golang-migrate/migrate for implementing DuckDB migration support. If this project goes well and our internal fork is adopted, we should strongly consider contributed back to upstream. 3 # NOTE 4 # Forked on 2023-07-12 5 6 # migrate 7 8 __Database migrations written in Go. Use as [CLI](#cli-usage) or import as [library](#use-in-your-go-project).__ 9 10 * Migrate reads migrations from [sources](#migration-sources) 11 and applies them in correct order to a [database](#databases). 12 * Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. 13 (Keeps the drivers lightweight, too.) 14 * Database drivers don't assume things or try to correct user input. When in doubt, fail. 15 16 Forked from [mattes/migrate](https://github.com/mattes/migrate) 17 18 ## Databases 19 20 Database drivers run migrations. [Add a new database?](database/driver.go) 21 22 * [PostgreSQL](database/postgres) 23 * [PGX v4](database/pgx) 24 * [PGX v5](database/pgx/v5) 25 * [Redshift](database/redshift) 26 * [Ql](database/ql) 27 * [Cassandra](database/cassandra) 28 * [SQLite](database/sqlite) 29 * [SQLite3](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165)) 30 * [SQLCipher](database/sqlcipher) 31 * [MySQL/ MariaDB](database/mysql) 32 * [Neo4j](database/neo4j) 33 * [MongoDB](database/mongodb) 34 * [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170)) 35 * [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171)) 36 * [Google Cloud Spanner](database/spanner) 37 * [CockroachDB](database/cockroachdb) 38 * [YugabyteDB](database/yugabytedb) 39 * [ClickHouse](database/clickhouse) 40 * [Firebird](database/firebird) 41 * [MS SQL Server](database/sqlserver) 42 43 ### Database URLs 44 45 Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: `dbdriver://username:password@host:port/dbname?param1=true¶m2=false` 46 47 Any [reserved URL characters](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters) need to be escaped. Note, the `%` character also [needs to be escaped](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_the_percent_character) 48 49 Explicitly, the following characters need to be escaped: 50 `!`, `#`, `$`, `%`, `&`, `'`, `(`, `)`, `*`, `+`, `,`, `/`, `:`, `;`, `=`, `?`, `@`, `[`, `]` 51 52 It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below: 53 54 ```bash 55 $ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))' 56 String to encode: FAKEpassword!#$%&'()*+,/:;=?@[] 57 FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D 58 $ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")' 59 String to encode: FAKEpassword!#$%&'()*+,/:;=?@[] 60 FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D 61 $ 62 ``` 63 64 ## Migration Sources 65 66 Source drivers read migrations from local or remote sources. [Add a new source?](source/driver.go) 67 68 * [Filesystem](source/file) - read from filesystem 69 * [io/fs](source/iofs) - read from a Go [io/fs](https://pkg.go.dev/io/fs#FS) 70 * [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata)) 71 * [pkger](source/pkger) - read from embedded binary data ([markbates/pkger](https://github.com/markbates/pkger)) 72 * [GitHub](source/github) - read from remote GitHub repositories 73 * [GitHub Enterprise](source/github_ee) - read from remote GitHub Enterprise repositories 74 * [Bitbucket](source/bitbucket) - read from remote Bitbucket repositories 75 * [Gitlab](source/gitlab) - read from remote Gitlab repositories 76 * [AWS S3](source/aws_s3) - read from Amazon Web Services S3 77 * [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage 78 79 ## CLI usage 80 81 * Simple wrapper around this library. 82 * Handles ctrl+c (SIGINT) gracefully. 83 * No config search paths, no config files, no magic ENV var injections. 84 85 __[CLI Documentation](cmd/migrate)__ 86 87 ### Basic usage 88 89 ```bash 90 $ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2 91 ``` 92 93 ### Docker usage 94 95 ```bash 96 $ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate 97 -path=/migrations/ -database postgres://localhost:5432/database up 2 98 ``` 99 100 ## Use in your Go project 101 102 * API is stable and frozen for this release (v3 & v4). 103 * Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies. 104 * To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`. 105 * Bring your own logger. 106 * Uses `io.Reader` streams internally for low memory overhead. 107 * Thread-safe and no goroutine leaks. 108 109 __[Go Documentation](https://pkg.go.dev/github.com/golang-migrate/migrate/v4)__ 110 111 ```go 112 import ( 113 "github.com/golang-migrate/migrate/v4" 114 _ "github.com/golang-migrate/migrate/v4/database/postgres" 115 _ "github.com/golang-migrate/migrate/v4/source/github" 116 ) 117 118 func main() { 119 m, err := migrate.New( 120 "github://mattes:personal-access-token@mattes/migrate_test", 121 "postgres://localhost:5432/database?sslmode=enable") 122 m.Steps(2) 123 } 124 ``` 125 126 Want to use an existing database client? 127 128 ```go 129 import ( 130 "database/sql" 131 _ "github.com/lib/pq" 132 "github.com/golang-migrate/migrate/v4" 133 "github.com/golang-migrate/migrate/v4/database/postgres" 134 _ "github.com/golang-migrate/migrate/v4/source/file" 135 ) 136 137 func main() { 138 db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable") 139 driver, err := postgres.WithInstance(db, &postgres.Config{}) 140 m, err := migrate.NewWithDatabaseInstance( 141 "file:///migrations", 142 "postgres", driver) 143 m.Up() // or m.Step(2) if you want to explicitly set the number of migrations to run 144 } 145 ``` 146 147 ## Getting started 148 149 Go to [getting started](GETTING_STARTED.md) 150 151 ## Tutorials 152 153 * [CockroachDB](database/cockroachdb/TUTORIAL.md) 154 * [PostgreSQL](database/postgres/TUTORIAL.md) 155 156 (more tutorials to come) 157 158 ## Migration files 159 160 Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-files-up-and-down-for-a-migration) 161 162 ```bash 163 1481574547_create_users_table.up.sql 164 1481574547_create_users_table.down.sql 165 ``` 166 167 [Best practices: How to write migrations.](MIGRATIONS.md) 168 169 ## Coming from another db migration tool? 170 171 Check out [migradaptor](https://github.com/musinit/migradaptor/). 172 *Note: migradaptor is not affliated or supported by this project* 173 174 ## Versions 175 176 Version | Supported? | Import | Notes 177 --------|------------|--------|------ 178 **master** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | New features and bug fixes arrive here first | 179 **v4** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | Used for stable releases | 180 **v3** | :x: | `import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended) | **DO NOT USE** - No longer supported | 181 182 ## Development and Contributing 183 184 Yes, please! [`Makefile`](Makefile) is your friend, 185 read the [development guide](CONTRIBUTING.md). 186 187 Also have a look at the [FAQ](FAQ.md). 188 189 --- 190 191 Looking for alternatives? [https://awesome-go.com/#database](https://awesome-go.com/#database).