github.com/mithrandie/csvq@v1.18.1/README.md (about) 1 # csvq 2 3 SQL-like query language for csv 4 5 [](https://github.com/mithrandie/csvq/actions/workflows/test.yml) 6 [](https://codecov.io/gh/mithrandie/csvq) 7 [](https://opensource.org/licenses/MIT) 8 9 Csvq is a command line tool to operate CSV files. 10 You can read, update, delete CSV records with SQL-like query. 11 12 You can also execute multiple operations sequentially in managed transactions by passing a procedure or using the interactive shell. 13 In the multiple operations, you can use variables, cursors, temporary tables, and other features. 14 15 ## Latest Release 16 [](https://github.com/mithrandie/csvq/releases/latest) 17 [](https://launchpad.net/~mithrandie/+archive/ubuntu/csvq) 18 19 ## Intended Use 20 Csvq is intended for one-time queries and routine processing described in source files on the amount of data that can be handled by spreadsheet applications. 21 22 It is not suitable for handling very large data since all data is kept on memory when queries are executed. 23 There is no indexing, calculation order optimization, etc., and the execution speed is not fast due to the inclusion of mechanisms for updating data and handling various other features. 24 25 However, it can be run with a single executable binary, and you don't have to worry about troublesome dependencies during installation. 26 You can not only write and run your own queries, but also share source files with co-workers on multiple platforms. 27 28 This tool may be useful for those who want to handle data easily and roughly, without having to think about troublesome matters. 29 30 ## Features 31 32 * CSV File Operation 33 * Select Query 34 * Insert Query 35 * Update Query 36 * Replace Query 37 * Delete Query 38 * Create Table Query 39 * Alter Table Query 40 * Cursor 41 * Temporary Table 42 * Transaction Management 43 * Support loading data from Standard Input 44 * Support following file formats 45 * [CSV](https://datatracker.ietf.org/doc/html/rfc4180) 46 * TSV 47 * [LTSV](http://ltsv.org) 48 * Fixed-Length Format 49 * [JSON](https://datatracker.ietf.org/doc/html/rfc8259) 50 * [JSON Lines](https://jsonlines.org) 51 * Support following file encodings 52 * UTF-8 53 * UTF-16 54 * Shift_JIS 55 56 > JSON and JSON Lines formats support only UTF-8. 57 58 ## Reference Manual 59 60 [Reference Manual - csvq](https://mithrandie.github.io/csvq/reference) 61 62 ## Installation 63 64 ### Install executable binary 65 66 1. Download an archive file from [release page](https://github.com/mithrandie/csvq/releases). 67 2. Extract the downloaded archive and add a binary file in it to your path. 68 69 ### Build from source 70 71 #### Requirements 72 73 Go 1.18 or later (cf. [Getting Started - The Go Programming Language](https://golang.org/doc/install)) 74 75 #### Build command 76 77 ```$ go install github.com/mithrandie/csvq``` 78 79 ### Install using package manager 80 81 Installing using a package manager does not ensure that you always get the latest version, but it may make installation and updating easier. 82 83 #### Ubuntu 84 85 1. ```$ sudo add-apt-repository ppa:mithrandie/csvq``` 86 2. ```$ sudo apt update``` 87 3. ```$ sudo apt install csvq``` 88 89 #### Arch Linux (unofficial) 90 91 Install the [csvq-git](https://aur.archlinux.org/packages/csvq-git) or [csvq-bin](https://aur.archlinux.org/packages/csvq-bin) from the [Arch User Repository](https://wiki.archlinux.org/title/Arch_User_Repository) (e.g. `yay -S csvq-git`) 92 93 #### macOS (unofficial) 94 95 1. Install homebrew (cf. [The missing package manager for macOS (or Linux) — Homebrew](https://brew.sh)) 96 2. ```$ brew install csvq``` 97 98 ## Usage 99 100 ```shell 101 # Simple query 102 csvq 'select id, name from `user.csv`' 103 csvq 'select id, name from user' 104 105 # Specify data delimiter as tab character 106 csvq -d '\t' 'select count(*) from `user.csv`' 107 108 # Load no-header-csv 109 csvq --no-header 'select c1, c2 from user' 110 111 # Load from redirection or pipe 112 csvq 'select * from stdin' < user.csv 113 cat user.csv | csvq 'select *' 114 115 # Load from Fixed-Length Format 116 cat /var/log/syslog | csvq -n -i fixed -m '[15, 24, 124]' 'select *' 117 118 # Split lines with spaces automatically 119 ps | csvq -i fixed -m spaces 'select * from stdin' 120 121 # Output in JSON format 122 csvq -f json 'select integer(id) as id, name from user' 123 124 # Output to a file 125 csvq -o new_user.csv 'select id, name from user' 126 127 # Load statements from file 128 $ cat statements.sql 129 VAR @id := 0; 130 SELECT @id := @id + 1 AS id, 131 name 132 FROM user; 133 134 $ csvq -s statements.sql 135 136 # Execute statements in the interactive shell 137 $ csvq 138 csvq > UPDATE users SET name = 'Mildred' WHERE id = 2; 139 1 record updated on "/home/mithrandie/docs/csv/users.csv". 140 csvq > COMMIT; 141 Commit: file "/home/mithrandie/docs/csv/users.csv" is updated. 142 csvq > EXIT; 143 144 # Show help 145 csvq -h 146 ``` 147 148 More details >> [https://mithrandie.github.io/csvq](https://mithrandie.github.io/csvq) 149 150 ## Execute csvq statements in Go 151 152 [csvq-driver](https://github.com/mithrandie/csvq-driver) 153 154 ## Example of cooperation with other applications 155 156 - [csvq emacs extension](https://github.com/mithrandie/csvq-emacs-extension)