github.com/minio/minio@v0.0.0-20240328213742-3f72439b8a27/docs/logging/README.md (about) 1 # MinIO Logging Quickstart Guide [](https://slack.min.io) 2 3 This document explains how to configure MinIO server to log to different logging targets. 4 5 ## Log Targets 6 7 MinIO supports currently two target types 8 9 - console 10 - http 11 12 ### Logging Console Target 13 14 Console target is on always and cannot be disabled. 15 16 ### Logging HTTP Target 17 18 HTTP target logs to a generic HTTP endpoint in JSON format and is not enabled by default. To enable HTTP target logging you would have to update your MinIO server configuration using `mc admin config set` command. 19 20 Assuming `mc` is already [configured](https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart) 21 22 ``` 23 mc admin config get myminio/ logger_webhook 24 logger_webhook:name1 auth_token="" endpoint="" 25 ``` 26 27 ``` 28 mc admin config set myminio logger_webhook:name1 auth_token="" endpoint="http://endpoint:port/path" 29 mc admin service restart myminio 30 ``` 31 32 NOTE: `http://endpoint:port/path` is a placeholder value to indicate the URL format, please change this accordingly as per your configuration. 33 34 MinIO also honors environment variable for HTTP target logging as shown below, this setting will override the endpoint settings in the MinIO server config. 35 36 ``` 37 export MINIO_LOGGER_WEBHOOK_ENABLE_target1="on" 38 export MINIO_LOGGER_WEBHOOK_AUTH_TOKEN_target1="token" 39 export MINIO_LOGGER_WEBHOOK_ENDPOINT_target1=http://localhost:8080/minio/logs 40 minio server /mnt/data 41 ``` 42 43 ## Audit Targets 44 45 Assuming `mc` is already [configured](https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart) 46 47 ### Audit HTTP Target 48 49 ``` 50 mc admin config get myminio/ audit_webhook 51 audit_webhook:name1 enable=off endpoint= auth_token= client_cert= client_key= 52 ``` 53 54 ``` 55 mc admin config set myminio audit_webhook:name1 auth_token="" endpoint="http://endpoint:port/path" 56 mc admin service restart myminio 57 ``` 58 59 NOTE: `http://endpoint:port/path` is a placeholder value to indicate the URL format, please change this accordingly as per your configuration. 60 61 MinIO also honors environment variable for HTTP target Audit logging as shown below, this setting will override the endpoint settings in the MinIO server config. 62 63 ``` 64 export MINIO_AUDIT_WEBHOOK_ENABLE_target1="on" 65 export MINIO_AUDIT_WEBHOOK_AUTH_TOKEN_target1="token" 66 export MINIO_AUDIT_WEBHOOK_ENDPOINT_target1=http://localhost:8080/minio/logs 67 export MINIO_AUDIT_WEBHOOK_CLIENT_CERT="/tmp/cert.pem" 68 export MINIO_AUDIT_WEBHOOK_CLIENT_KEY=="/tmp/key.pem" 69 minio server /mnt/data 70 ``` 71 72 Setting this environment variable automatically enables audit logging to the HTTP target. The audit logging is in JSON format as described below. 73 74 NOTE: 75 76 - `timeToFirstByte` and `timeToResponse` will be expressed in Nanoseconds. 77 - Additionally in the case of the erasure coded setup `tags.objectErasureMap` provides per object details about 78 - Pool number the object operation was performed on. 79 - Set number the object operation was performed on. 80 - The list of drives participating in this operation belong to the set. 81 82 ```json 83 { 84 "version": "1", 85 "deploymentid": "51bcc7b9-a447-4251-a940-d9d0aab9af69", 86 "time": "2021-10-08T00:46:36.801714978Z", 87 "trigger": "incoming", 88 "api": { 89 "name": "PutObject", 90 "bucket": "testbucket", 91 "object": "hosts", 92 "status": "OK", 93 "statusCode": 200, 94 "rx": 380, 95 "tx": 476, 96 "timeToResponse": "257694819ns" 97 }, 98 "remotehost": "127.0.0.1", 99 "requestID": "16ABE7A785E7AC2C", 100 "userAgent": "MinIO (linux; amd64) minio-go/v7.0.15 mc/DEVELOPMENT.2021-10-06T23-39-34Z", 101 "requestHeader": { 102 "Authorization": "AWS4-HMAC-SHA256 Credential=minio/20211008/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length,Signature=4c60a59e5eb3b0a68693c7fee9dbb5a8a509e0717668669194d37bf182fde031", 103 "Content-Length": "380", 104 "Content-Type": "application/octet-stream", 105 "User-Agent": "MinIO (linux; amd64) minio-go/v7.0.15 mc/DEVELOPMENT.2021-10-06T23-39-34Z", 106 "X-Amz-Content-Sha256": "STREAMING-AWS4-HMAC-SHA256-PAYLOAD", 107 "X-Amz-Date": "20211008T004636Z", 108 "X-Amz-Decoded-Content-Length": "207", 109 "X-Amz-Server-Side-Encryption": "aws:kms" 110 }, 111 "responseHeader": { 112 "Accept-Ranges": "bytes", 113 "Content-Length": "0", 114 "ETag": "4939450d1beec11e10a91ee7700bb593", 115 "Server": "MinIO", 116 "Strict-Transport-Security": "max-age=31536000; includeSubDomains", 117 "Vary": "Origin,Accept-Encoding", 118 "X-Amz-Request-Id": "16ABE7A785E7AC2C", 119 "X-Amz-Server-Side-Encryption": "aws:kms", 120 "X-Amz-Server-Side-Encryption-Aws-Kms-Key-Id": "arn:aws:kms:my-minio-key", 121 "X-Content-Type-Options": "nosniff", 122 "X-Xss-Protection": "1; mode=block", 123 "x-amz-version-id": "ac4639f6-c544-4f3f-af1e-b4c0736f67f9" 124 }, 125 "tags": { 126 "objectErasureMap": { 127 "hosts": { 128 "poolId": 1, 129 "setId": 1, 130 "drives": [ 131 "/mnt/data1", 132 "/mnt/data2", 133 "/mnt/data3", 134 "/mnt/data4" 135 ] 136 } 137 } 138 } 139 } 140 ``` 141 142 ### Kafka Target 143 144 Assuming that you already have Apache Kafka configured and running. 145 146 ``` 147 mc admin config set myminio/ audit_kafka 148 KEY: 149 audit_kafka[:name] send audit logs to kafka endpoints 150 151 ARGS: 152 brokers* (csv) comma separated list of Kafka broker addresses 153 topic (string) Kafka topic used for bucket notifications 154 sasl_username (string) username for SASL/PLAIN or SASL/SCRAM authentication 155 sasl_password (string) password for SASL/PLAIN or SASL/SCRAM authentication 156 sasl_mechanism (string) sasl authentication mechanism, default 'plain' 157 tls_client_auth (string) clientAuth determines the Kafka server's policy for TLS client auth 158 sasl (on|off) set to 'on' to enable SASL authentication 159 tls (on|off) set to 'on' to enable TLS 160 tls_skip_verify (on|off) trust server TLS without verification, defaults to "on" (verify) 161 client_tls_cert (path) path to client certificate for mTLS auth 162 client_tls_key (path) path to client key for mTLS auth 163 version (string) specify the version of the Kafka cluster 164 comment (sentence) optionally add a comment to this setting 165 ``` 166 167 Configure MinIO to send audit logs to locally running Kafka brokers 168 169 ``` 170 mc admin config set myminio/ audit_kafka:target1 brokers=localhost:29092 topic=auditlog 171 mc admin service restart myminio/ 172 ``` 173 174 On another terminal assuming you have `kafkacat` installed 175 176 ``` 177 kafkacat -b localhost:29092 -t auditlog -C 178 179 {"version":"1","deploymentid":"8a1d8091-b874-45df-b9ea-e044eede6ace","time":"2021-07-13T02:00:47.020547414Z","trigger":"incoming","api":{"name":"ListBuckets","status":"OK","statusCode":200,"timeToFirstByte":"261795ns","timeToResponse":"312490ns"},"remotehost":"127.0.0.1","requestID":"16913736591C237F","userAgent":"MinIO (linux; amd64) minio-go/v7.0.11 mc/DEVELOPMENT.2021-07-09T02-22-26Z","requestHeader":{"Authorization":"AWS4-HMAC-SHA256 Credential=minio/20210713/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=7fe65c5467e05ca21de64094688da43f96f34fec82e8955612827079f4600527","User-Agent":"MinIO (linux; amd64) minio-go/v7.0.11 mc/DEVELOPMENT.2021-07-09T02-22-26Z","X-Amz-Content-Sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","X-Amz-Date":"20210713T020047Z"},"responseHeader":{"Accept-Ranges":"bytes","Content-Length":"547","Content-Security-Policy":"block-all-mixed-content","Content-Type":"application/xml","Server":"MinIO","Vary":"Origin,Accept-Encoding","X-Amz-Request-Id":"16913736591C237F","X-Xss-Protection":"1; mode=block"}} 180 ``` 181 182 MinIO also honors environment variable for Kafka target Audit logging as shown below, this setting will override the endpoint settings in the MinIO server config. 183 184 ``` 185 mc admin config set myminio/ audit_kafka --env 186 KEY: 187 audit_kafka[:name] send audit logs to kafka endpoints 188 189 ARGS: 190 MINIO_AUDIT_KAFKA_ENABLE* (on|off) enable audit_kafka target, default is 'off' 191 MINIO_AUDIT_KAFKA_BROKERS* (csv) comma separated list of Kafka broker addresses 192 MINIO_AUDIT_KAFKA_TOPIC (string) Kafka topic used for bucket notifications 193 MINIO_AUDIT_KAFKA_SASL_USERNAME (string) username for SASL/PLAIN or SASL/SCRAM authentication 194 MINIO_AUDIT_KAFKA_SASL_PASSWORD (string) password for SASL/PLAIN or SASL/SCRAM authentication 195 MINIO_AUDIT_KAFKA_SASL_MECHANISM (string) sasl authentication mechanism, default 'plain' 196 MINIO_AUDIT_KAFKA_TLS_CLIENT_AUTH (string) clientAuth determines the Kafka server's policy for TLS client auth 197 MINIO_AUDIT_KAFKA_SASL (on|off) set to 'on' to enable SASL authentication 198 MINIO_AUDIT_KAFKA_TLS (on|off) set to 'on' to enable TLS 199 MINIO_AUDIT_KAFKA_TLS_SKIP_VERIFY (on|off) trust server TLS without verification, defaults to "on" (verify) 200 MINIO_AUDIT_KAFKA_CLIENT_TLS_CERT (path) path to client certificate for mTLS auth 201 MINIO_AUDIT_KAFKA_CLIENT_TLS_KEY (path) path to client key for mTLS auth 202 MINIO_AUDIT_KAFKA_VERSION (string) specify the version of the Kafka cluster 203 MINIO_AUDIT_KAFKA_COMMENT (sentence) optionally add a comment to this setting 204 ``` 205 206 ``` 207 export MINIO_AUDIT_KAFKA_ENABLE_target1="on" 208 export MINIO_AUDIT_KAFKA_BROKERS_target1="localhost:29092" 209 export MINIO_AUDIT_KAFKA_TOPIC_target1="auditlog" 210 minio server /mnt/data 211 ``` 212 213 Setting this environment variable automatically enables audit logging to the Kafka target. The audit logging is in JSON format as described below. 214 215 NOTE: 216 217 - `timeToFirstByte` and `timeToResponse` will be expressed in Nanoseconds. 218 - Additionally in the case of the erasure coded setup `tags.objectErasureMap` provides per object details about 219 - Pool number the object operation was performed on. 220 - Set number the object operation was performed on. 221 - The list of drives participating in this operation belong to the set. 222 223 ## Explore Further 224 225 - [MinIO Quickstart Guide](https://min.io/docs/minio/linux/index.html#quickstart-for-linux) 226 - [Configure MinIO Server with TLS](https://min.io/docs/minio/linux/operations/network-encryption.html)