github.com/GoogleContainerTools/skaffold/v2@v2.13.2/docs-v2/content/en/docs/workflows/debug.md (about) 1 --- 2 title: "Debugging With Skaffold" 3 linkTitle: "Debugging" 4 weight: 30 5 featureId: debug 6 aliases: [/docs/how-tos/debug] 7 --- 8 9 Skaffold lets you set breakpoints and step through your application, even when deployed to remote Kubernetes clusters, as if the code were running locally on your machine. Skaffold detects the language runtimes of your container images, reconfigures the pods for debugging, forwards debugging ports, and then monitors for when debuggable containers come online. IDE integrations like Cloud Code leverage Skaffold's events to automatically set up debugging sessions. 10 11 Debugging is currently supported for five language runtimes. 12 13 - Go 1.13+ (runtime ID: `go`) using [Delve](https://github.com/go-delve/delve) 14 - NodeJS (runtime ID: `nodejs`) using the NodeJS Inspector (Chrome DevTools) 15 - Java and JVM languages (runtime ID: `jvm`) using JDWP 16 - Python 3.5-3.10 runtimes (runtime ID: `python`) using `debugpy` (Debug Adapter Protocol) or `pydevd` 17 - NOTE: Python 3.11 support is planned and coming soon but is currently still in progress 18 - .NET Core (runtime ID: `netcore`) using `vsdbg` (only for VS Code) 19 20 Skaffold can usually detect the correct language runtime if present. However if you encounter difficulties then checkout the [Supported Language Runtimes]({{< relref "#supported-language-runtimes">}}) section for the exact heuristics that Skaffold uses and you can modify your application accordingly, or read about [how you can manually configure your container image]({{< relref "#can-images-be-debugged-without-the-runtime-support-images" >}}). 21 22 ## (Recommended) Debugging using Cloud Code 23 24 The easiest way to debug using Skaffold is by using the [Cloud Code IDE extension]({{< relref "../install/#managed-ide">}}) for Visual Studio Code, JetBrains IDEs, and Cloud Shell. Cloud Code will automatically configure container images for debugging so you can debug Kubernetes services just like how you would debug a local service in your IDE. 25 26 For more information, see the corresponding documentation for [VS Code](https://cloud.google.com/code/docs/vscode/debug), [IntelliJ](https://cloud.google.com/code/docs/intellij/kubernetes-debugging) and [Cloud Shell](https://cloud.google.com/code/docs/shell/debug). 27 28 ## Detailed debugger configuration and setup 29 30 This section describes how Skaffold configures the container image for debugging for the supported language runtimes, and how to setup with certain IDEs. 31 32 Note that many debuggers may require additional information for the location of source files. 33 We are looking for ways to identify this information and to pass it back if found. 34 35 {{% tabs %}} 36 {{% tab "GO" %}} 37 38 #### Go (runtime: `go`, protocols: `dlv`) 39 40 Go-based applications are configured to run under 41 [Delve](https://github.com/go-delve/delve) in its headless-server mode. 42 43 On recognizing a Go-based container image, `debug` rewrites the container image's 44 entrypoint to invoke your application using `dlv`: 45 46 ``` 47 dlv exec --headless --continue --accept-multiclient --listen=:56268 --api-version=2 <app> -- <args> ... 48 ``` 49 50 ##### Skaffold debug using the JetBrains GoLand and IntelliJ Ultimate 51 52 Debugging is only supported in JetBrains IDEs for Go applications built using Go Modules. 53 The IDE settings must also be explicitly configured to use Go Modules. Errors like 54 `cannot find debugger path` indicate misconfiguration. 55 56 57 ##### Skaffold debug using the VS Code Go extension 58 59 If you use the debug functionality of the 60 [VS Code Go extension](https://github.com/golang/vscode-go) to connect to 61 `dlv`, you may need to configure _local_ and _remote_ source paths, via 62 either the `cwd` and `remotePath` properties, or the `substitutePath` array 63 property 64 ([reference](https://github.com/golang/vscode-go/blob/master/docs/debugging-legacy.md#launch-configurations)). 65 66 The `cwd` or `from` properties should be the absolute path of the top-level 67 directory of your source files and should generally match the artifact's 68 `context` directory in `skaffold.yaml`. You can represent this using the 69 `${workspaceFolder}` variable. 70 71 The `remotePath` or `to` properties should be set to the absolute path of the 72 source location _during compilation_. For example, the `golang` images, which 73 are 74 [often used in multi-stage builds](https://github.com/GoogleContainerTools/skaffold/tree/main/examples/getting-started/Dockerfile), 75 have a default working directory of `/go`. The following 76 [configuration](https://github.com/golang/vscode-go/blob/master/docs/debugging-legacy.md#remote-debugging) 77 works in this case: 78 79 ```json 80 { 81 "name": "Skaffold Debug", 82 "type": "go", 83 "request": "attach", 84 "mode": "remote", 85 "host": "localhost", 86 "port": 56268, 87 "cwd": "${workspaceFolder}", 88 "remotePath": "/go" 89 } 90 ``` 91 92 You can use the `substitutePath` property instead of `cwd` and `remotePath`: 93 94 ```json 95 { 96 "name": "Skaffold Debug", 97 "type": "go", 98 "request": "attach", 99 "mode": "remote", 100 "host": "localhost", 101 "port": 56268, 102 "substitutePath": [ 103 { 104 "from": "${workspaceFolder}", 105 "to": "/go", 106 }, 107 ], 108 } 109 ``` 110 111 If you use the `ko` local builder, the source path during compilation matches 112 the source path in your IDE. In this case, the value for `remotePath` will 113 match `cwd`. If the values of both `cwd` and `remotePath` are 114 `${workspaceFolder}`, you can omit these properties. 115 116 {{% /tab %}} 117 {{% tab "NODEJS" %}} 118 119 #### NodeJS (runtime: `nodejs`, protocols: `devtools`) 120 121 NodeJS applications are configured to use the Chrome DevTools inspector via the `--inspect` argument. 122 123 On recognizing a NodeJS-based container image, `debug` rewrites the container image's 124 entrypoint to invoke your application with `--inspect`: 125 ``` 126 node --inspect=9229 <app.js> 127 ``` 128 129 {{< alert title="Note" >}} 130 Many applications use NodeJS-based tools as part of their launch, like <tt>npm</tt>, rather than 131 invoke <tt>node</tt> directly. These intermediate <tt>node</tt> instances may interpret the 132 <tt>--inspect</tt> arguments. Skaffold introduces a <tt>node</tt> wrapper that 133 only invokes the real <tt>node</tt> with <tt>--inspect</tt> if running an application script, 134 and skips scripts located in <tt>node_modules</tt>. For more details see the 135 <a href="https://github.com/GoogleContainerTools/container-debug-support/pull/34">associated PR</a>. 136 {{< /alert >}} 137 138 A debugging client must first obtain [the inspector UUID](https://github.com/nodejs/node/issues/9185#issuecomment-254872466). 139 140 {{% /tab %}} 141 {{% tab "JAVA" %}} 142 143 #### Java and Other JVM Languages (runtime: `jvm`, protocols: `jdwp`) 144 145 Java/JVM applications are configured to expose the JDWP agent using the `JAVA_TOOL_OPTIONS` 146 environment variable. 147 Note that the use of `JAVA_TOOL_OPTIONS` causes extra debugging output from the JVM on launch. 148 149 On recognizing a JVM-based container image, `debug` rewrites the container image's 150 environment to set: 151 ``` 152 JAVA_TOOL_OPTIONS=-agentlib:jdwp=transport=dt_socket,server=y,address=5005,suspend=n,quiet=y 153 ``` 154 {{% /tab %}} 155 {{% tab "PYTHON" %}} 156 157 #### Python (runtime: `python`, protocols: `dap` or `pydevd`) 158 159 Python applications are configured to use either [`debugpy`](https://github.com/microsoft/debugpy/) or 160 [`pydevd`](https://github.com/fabioz/PyDev.Debugger). `debugpy` uses the 161 [_debug adapter protocol_ (DAP)](https://microsoft.github.io/debug-adapter-protocol/) which 162 is supported by Visual Studio Code, [Eclipse LSP4e](https://projects.eclipse.org/projects/technology.lsp4e), 163 [and other editors and IDEs](https://microsoft.github.io/debug-adapter-protocol/implementors/tools/). 164 165 On recognizing a Python-based container image, `debug` rewrites the container image's 166 entrypoint to invoke Python using either the `pydevd` or `debugpy` modules: 167 ``` 168 python -m debugpy --listen 5678 <app> 169 ``` 170 171 or 172 ``` 173 python -m pydevd --server --port 5678 <app.py> 174 ``` 175 176 {{< alert title="Note" >}} 177 As many Python web frameworks use launcher scripts, like `gunicorn`, Skaffold now uses 178 a debug launcher that examines the app command-line. 179 {{< /alert >}} 180 181 {{% /tab %}} 182 {{% tab ".NETCORE" %}} 183 184 #### .NET Core (runtime: `dotnet`, protocols: `vsdbg`) 185 186 .NET Core applications are configured to be deployed along with `vsdbg` 187 for VS Code. 188 189 {{< alert title="JetBrains Rider" >}} 190 This set up does not yet work automatically with [Cloud Code for IntelliJ in JetBrains Rider](https://github.com/GoogleCloudPlatform/cloud-code-intellij/issues/2903). 191 There is [a manual workaround](https://github.com/GoogleCloudPlatform/cloud-code-intellij/wiki/Manual-set-up-for-remote-debugging-in-Rider). 192 {{< /alert >}} 193 194 {{< alert title="Omnisharp for VS Code" >}} 195 For users of [VS Code's debug adapter for C#](https://github.com/OmniSharp/omnisharp-vscode):** 196 the following configuration can be used to debug a container. It assumes that your code is deployed 197 in `/app` or `/src` folder in the container. If that is not the case, the `sourceFileMap` property 198 should be changed to match the correct folder. `processId` is usually 1 but might be different if you 199 have an unusual entrypoint. You can also use `"${command:pickRemoteProcess}"` instead if supported by 200 your base image. (`//` comments must be stripped.) 201 ```json 202 { 203 "name": "Skaffold Debug", 204 "type": "coreclr", 205 "request": "attach", 206 "processId" : 1, 207 "justMyCode": true, // set to `true` in debug configuration and `false` in release configuration 208 "pipeTransport": { 209 "pipeProgram": "kubectl", 210 "pipeArgs": [ 211 "exec", 212 "-i", 213 "<NAME OF YOUR POD>", // name of the pod you debug. 214 "--" 215 ], 216 "pipeCwd": "${workspaceFolder}", 217 "debuggerPath": "/dbg/netcore/vsdbg", // location where vsdbg binary installed. 218 "quoteArgs": false 219 }, 220 "sourceFileMap": { 221 // Change this mapping if your app in not deployed in /src or /app in your docker image 222 "/src": "${workspaceFolder}", 223 "/app": "${workspaceFolder}" 224 // May also be like this, depending of your repository layout 225 // "/src": "${workspaceFolder}/src", 226 // "/app": "${workspaceFolder}/src/<YOUR PROJECT TO DEBUG>" 227 } 228 } 229 ``` 230 {{< /alert >}} 231 232 {{% /tab %}} 233 {{% /tabs %}} 234 235 ## Supported Language Runtimes 236 237 The language runtime for the artifacts can be specified in the `skaffold.yaml` file. 238 {{% readfile file="samples/debug/runtime-type.yaml" %}} 239 240 Otherwise Skaffold `debug` recognizes the language runtime used in a 241 container image for specific language runtimes using standard heuristics described below. 242 243 {{% tabs %}} 244 {{% tab "GO" %}} 245 #### Go (runtime: `go`) 246 247 Go-based container images are recognized by: 248 249 - the presence of one of the 250 [standard Go runtime environment variables](https://godoc.org/runtime): 251 `GODEBUG`, `GOGC`, `GOMAXPROCS`, or `GOTRACEBACK`, or 252 - the presence of the 253 [`KO_DATA_PATH` environment variable](https://github.com/google/ko#static-assets) 254 in container images built by 255 [`ko`]({{< relref "/docs/builders/builder-types/ko" >}}), or 256 - is launching using `dlv`. 257 258 Unless you built your container image using `ko`, set one of the standard Go 259 environment variables to allow Skaffold to detect your container as Go. 260 `GOTRACEBACK=single` is the default setting for Go, so you can add this 261 environment variable without changing behavior. Alternatively, `GOTRACEBACK=all` 262 is a useful setting in many situtions. 263 264 Your application should be built with the `-gcflags='all=-N -l'` flag to 265 disable optimizations and inlining. If you do not add this flag, debugging can 266 becoming confusing due to seemingly-random execution jumps that happen due to 267 statement reordering and inlining. Skaffold configures Docker builds with a 268 `SKAFFOLD_GO_GCFLAGS` build argument that you should use in your `Dockerfile`: 269 270 ``` 271 FROM golang 272 ENV GOTRACEBACK=all 273 COPY . . 274 ARG SKAFFOLD_GO_GCFLAGS 275 RUN go build -gcflags="${SKAFFOLD_GO_GCFLAGS}" -o /app . 276 ``` 277 278 Note that the Alpine Linux-based `golang` container images do not include a 279 C compiler which is required to build with `-gcflags='all=-N -l'`. 280 281 The ko builder adds `-gcflags='all=-N -l'` automatically when you use 282 `skaffold debug`. 283 284 We recommend that you do _not_ add the `-trimpath` flag when debugging, as this 285 flag removes information from the resulting binary that aids debugging. 286 287 {{% /tab %}} 288 {{% tab "JAVA" %}} 289 290 #### Java and Other JVM Languages (runtime: `jvm`) 291 292 JVM application are recognized by: 293 - the presence of a `JAVA_VERSION` or `JAVA_TOOL_OPTIONS` environment variable, or 294 - the container command-line invokes `java`. 295 296 {{% /tab %}} 297 {{% tab "NODEJS" %}} 298 299 #### NodeJS (runtime: `nodejs`) 300 301 NodeJS images are recognized by: 302 - the presence of a `NODE_VERSION`, `NODEJS_VERSION`, or `NODE_ENV` environment variable, or 303 - the container command-line invokes `node` or `npm`. 304 305 {{% /tab %}} 306 {{% tab "PYTHON" %}} 307 308 #### Python (runtime: `python`) 309 310 Python application are recognized by: 311 - the presence of a standard Python environment variable: 312 `PYTHON_VERSION`, `PYTHONVERBOSE`, `PYTHONINSPECT`, `PYTHONOPTIMIZE`, 313 `PYTHONUSERSITE`, `PYTHONUNBUFFERED`, `PYTHONPATH`, `PYTHONUSERBASE`, 314 `PYTHONWARNINGS`, `PYTHONHOME`, `PYTHONCASEOK`, `PYTHONIOENCODING`, 315 `PYTHONHASHSEED`, `PYTHONDONTWRITEBYTECODE`, or 316 - the container command-line invokes `python`, `python2`, or `python3`. 317 318 {{% /tab %}} 319 {{% tab ".NETCORE" %}} 320 321 #### .NET Core (runtime: `dotnet`) 322 323 .NET Core application are recognized by: 324 - the presence of a standard .NET environment variable: 325 `ASPNETCORE_URLS`, `DOTNET_RUNNING_IN_CONTAINER`, 326 `DOTNET_SYSTEM_GLOBALIZATION_INVARIANT`, or 327 - the container command-line invokes the [dotnet](https://github.com/dotnet/sdk) cli 328 329 Furthermore, your app must be built with the `--configuration Debug` options to disable optimizations. 330 331 {{% /tab %}} 332 {{% /tabs %}} 333 334 ## How It works 335 336 `skaffold debug` acts like `skaffold dev`, but it configures containers in pods 337 for debugging as required for each container's runtime technology. 338 The associated debugging ports are exposed and labelled so that they can be port-forwarded to the 339 local machine. 340 One notable difference from `skaffold dev` is that `debug` disables image rebuilding and 341 syncing as it leads to users accidentally terminating debugging sessions by saving file changes. 342 These behaviours can be re-enabled with the `--auto-build`, `--auto-deploy`, and `--auto-sync` 343 flags, or triggering a [devloop iteration]({{< relref "/docs/design/api#control-api" >}}) using the Skaffold API. 344 345 Enabling debugging has two phases: 346 347 1. **Configuring:** Skaffold automatically examines each built container image and 348 attempts to recognize the underlying language runtime. Container images can be 349 explicitly configured too. 350 3. **Monitoring:** Skaffold watches the cluster to detect when debuggable containers 351 start execution. 352 353 ### Configuring container images for debugging 354 355 `skaffold debug` examines the *built artifacts* to determine the underlying language runtime technology. 356 Kubernetes manifests that reference these artifacts are transformed on-the-fly to enable the 357 language runtime's debugging functionality. These transforms add or alter environment variables 358 and entrypoints, and more. 359 360 Some language runtimes require additional support files to enable debugging. 361 For these languages, a special set of [runtime-specific images](https://github.com/GoogleContainerTools/container-debug-support) 362 are configured as _init-containers_ to populate a shared-volume that is mounted into 363 each of the appropriate containers. These images are hosted at 364 `gcr.io/k8s-skaffold/skaffold-debug-support`; alternative locations can be 365 specified in [Skaffold's global configuration]({{< relref "/docs/design/global-config.md" >}}). 366 367 For images that are successfully recognized, Skaffold adds a `debug.cloud.google.com/config` 368 annotation to the corresponding Kubernetes pod-spec that encode the debugging parameters. 369 370 ### Monitoring for debuggable containers 371 372 Once the application is deployed, `debug` monitors the cluster looking for debuggable pods with a 373 `debug.cloud.google.com/config` annotation. For each new debuggable pod, Skaffold emits 374 an event that can be used by tools like IDEs to establish a debug session. 375 376 ### Additional changes 377 378 `debug` makes some other adjustments to simplify the debug experience: 379 380 - *Replica Counts*: `debug` rewrites the replica counts to 1 for 381 deployments, replica sets, and stateful sets. This results in 382 requests being serialized so that one request is processed at a time. 383 384 - *Kubernetes Probes*: `debug` changes the timeouts on HTTP-based 385 [liveness, readiness, and startup probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) 386 to 600 seconds (10 minutes) from the default of 1 second. 387 This change allows probes to be debugged, and avoids negative 388 consequences from blocked probes when the app is already suspended 389 during a debugging session. 390 Failed liveness probes in particular result in the container 391 being terminated and restarted. 392 393 The probe timeout value can be set on a per-podspec basis by setting 394 a `debug.cloud.google.com/probe/timeouts` annotation on the podspec's metadata 395 with a valid duration (see [Go's time.ParseDuration()](https://pkg.go.dev/time#ParseDuration)). 396 This probe timeout-rewriting can be skipped entirely by using `skip`. For example: 397 ```yaml 398 metadata: 399 annotations: 400 debug.cloud.google.com/probe/timeouts: skip 401 spec: ... 402 ``` 403 404 ## Troubleshooting 405 406 ### My container is not being made debuggable? 407 408 **Was this image built by Skaffold?** 409 `debug` only works for images that were built by 410 Skaffold so as to avoid affecting system- or infrastructure-level containers such as proxy sidecars. 411 412 **Was Skaffold able to recognize the image?** 413 `debug` emits a warning when it is unable to configure an image for debugging: 414 ``` 415 WARN[0005] Image "image-name" not configured for debugging: unable to determine runtime for "image-name" 416 ``` 417 418 See the language runtime section details on how container images are recognized. 419 420 **Is your container command-line using shell constructs?** 421 422 Automatic debugging of Go and Python applications requires that Skaffold 423 adjust the command-line to use a specific launcher (`dlv` for Go, or our 424 custom Python launcher to configure the appropriate debugging module). 425 These launchers require that the container command-line invokes the 426 binary or launch script (i.e., a `pip`-generated launcher script such as `gunicorn`). 427 Skaffold cannot debug Go and Python applications that use user-created 428 shell scripts, or that use shell constructs like `exec` or `eval`. 429 Either rewrite your container image command-line or 430 [manually configure your container for debugging]({{< relref "#can-images-be-debugged-without-the-runtime-support-images" >}}). 431 For example: 432 ``` 433 ❌ CMD launch.sh 434 ❌ CMD exec gunicorn app:app 435 ✅ CMD gunicorn app:app 436 ✅ CMD [gunicorn, app:app] 437 ✅ CMD [python, -m, gunicorn, app:app] 438 ``` 439 440 441 ### Why aren't my breakpoints being hit? 442 443 **Missing required language extension.** 444 Go, Python, and JavaScript debugging in IDEA Ultimate requires the 445 corresponding language plugin to be installed. If it is not installed, 446 the debugger will not attach and breakpoints will not be hit. 447 448 **Breakpoints are not hit during program startup.** 449 Skaffold configures the containers to run-on-start (sometimes called 450 _continue_ mode), such that the containers do not wait for the 451 debugger to connect. The container will have likely finished its 452 startup by the time the debugger is able to connect and configure 453 the breakpoints, and so breakpoints in the normal startup path are 454 unlikely to be hit. 455 456 **File mapping misconfiguration between local and container filesystems.** 457 Some language runtimes require configuring the IDE with a _source map_ 458 to map files in the container to their corresponding local files. 459 If this mapping is incorrect, then the remote debugging runtime 460 will not be able to find the source file for the breakpoint. The Cloud Code plugins generally prompt the user when required. 461 462 - Some builds use multi-stage dockerfiles to compile the binary using 463 a build-time image and then copy the binary into a smaller image 464 intended for deployment. In these cases, the source mapping must 465 use the build-time paths. For example, many Go builds use a `golang` 466 image for building, where the source files are typically copied into `/go`. 467 - JetBrains GoLand and IDEA Ultimate only support remote breakpoints 468 for Go applications built using Go Modules. The IDE settings must 469 also be explicitly configured to use Go Modules. 470 - JVM languages do not require the mapping as the JVM uses class 471 names rather than file paths. 472 473 ### Can images be debugged without the runtime support images? 474 475 The special [runtime-support images](https://github.com/GoogleContainerTools/container-debug-support) 476 are provided as a convenience for automatic configuration. You can 477 manually configure your images for debugging by one of the following 478 means: 479 480 - `skaffold debug` usually recognizes already-configured container images 481 that use the appropriate debugger. For example, you could use a 482 multi-stage Dockerfile with a `debug` stage that invokes the 483 application using the debugger. 484 - Use a `debug.cloud.google.com/config` workload annotation on the 485 pod-spec to describe the debug configuration of each container 486 image in the pod, as described in [_Workload Annotations_](#workload-annotations). 487 488 ## Limitations 489 490 `skaffold debug` has some limitations. 491 492 ### Unsupported Container Entrypoints 493 494 `skaffold debug` requires being able to examine and alter the 495 command-line used in the container entrypoint. This transformation 496 will not work with images that use intermediate launch scripts or 497 binaries. 498 499 ### Supported Deployers 500 501 `skaffold debug` is only supported with the `kubectl`, `kustomize`, and `helm` deployers. 502 503 {{< alert title="Note" >}} 504 Helm support requires using Helm v3.1.0 or greater. 505 {{< /alert >}} 506 507 508 ### Deprecated Workload API Objects 509 510 `skaffold debug` does not support deprecated versions of Workload API objects: 511 512 - `extensions/v1beta1` and `apps/v1beta1` was [deprecated in Kubernetes 1.8](https://github.com/kubernetes/kubernetes/blob/HEAD/CHANGELOG/CHANGELOG-1.8.md#other-notable-changes-16) 513 and [removed in Kubernetes 1.16](https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/). 514 - `apps/v1beta2` was [deprecated in Kubernetes 1.9](https://github.com/kubernetes/kubernetes/blob/HEAD/CHANGELOG/CHANGELOG-1.9.md#apps) 515 and [removed in Kubernetes 1.16](https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/). 516 517 Applications should transition to the `apps/v1` APIs, 518 [introduced in Kubernetes 1.9](https://kubernetes.io/blog/2017/12/kubernetes-19-workloads-expanded-ecosystem/#workloads-api-ga). 519 520 ---- 521 522 ## Appendix: IDE Support via Events and Metadata {#metadata-events} 523 524 `debug` provides additional support for IDEs to detect the debuggable containers and to determine 525 appropriate configuration parameters. 526 527 ### Workload Annotations 528 529 Each transformed workload object carries a `debug.cloud.google.com/config` annotation with 530 a JSON object describing the debug configurations for the pod's containers (linebreaks for readability): 531 ``` 532 debug.cloud.google.com/config={ 533 "<containerName>":{"runtime":"<runtimeId>",...}, 534 "<containerName>":{"runtime":"<runtimeId>",...}, 535 } 536 ``` 537 538 For example the following annotation indicates that the container named `web` is a Go application 539 that is being debugged by a headless Delve session on port `56268` (linebreaks for readability): 540 ``` 541 debug.cloud.google.com/config={ 542 "web":{ 543 "artifact":"gcr.io/random/image", 544 "runtime":"go", 545 "ports":{"dlv":56268}, 546 "workingDir":"/some/path"}} 547 ``` 548 549 `artifact` is the corresponding artifact's image name in the `skaffold.yaml`. 550 `runtime` is the language runtime detected (one of: `go`, `jvm`, `nodejs`, `python`). 551 `ports` is a list of debug ports keyed by the language runtime debugging protocol. 552 `workingDir` is the working directory (if not an empty string). 553 554 555 ### API: Events 556 557 Each debuggable container being started or stopped raises a _debug-container-event_ through 558 Skaffold's event mechanism ([gRPC]({{< ref "/docs/references/api/grpc#debuggingcontainerevent" >}}), 559 [REST]({{< ref "/docs/references/api/swagger#/SkaffoldService/Events" >}})). 560 561 <details> 562 <summary>`/v1/events` stream of `skaffold debug` within `examples/jib`</summary> 563 564 In this example, we do a `skaffold debug`, and then kill the deployed pod. The deployment starts a new pod. 565 We get a terminated event for the container for the killed pod. 566 567 ```json 568 { 569 "result": { 570 "timestamp": "2020-02-05T03:27:30.114354Z", 571 "event": { 572 "debuggingContainerEvent": { 573 "status": "Started", 574 "podName": "web-f6d56bcc5-6csgs", 575 "containerName": "web", 576 "namespace": "default", 577 "artifact": "skaffold-jib", 578 "runtime": "jvm", 579 "debugPorts": { 580 "jdwp": 5005 581 } 582 } 583 }, 584 "entry": "Debuggable container started pod/web-f6d56bcc5-6csgs:web (default)" 585 } 586 } 587 ``` 588 589 </details> 590 591 592 593 ### API: State 594 595 The API's _state_ ([gRPC]({{< relref "/docs/references/api/grpc#skaffoldservice" >}}), 596 [REST]({{< relref "/docs/references/api/swagger#/SkaffoldService/GetState" >}})) also includes a list of debuggable containers. 597 598 <details> 599 <summary>The `/v1/state` listing debugging containers</summary> 600 601 ```json 602 { 603 "buildState": { 604 "artifacts": { 605 "skaffold-jib": "Complete" 606 } 607 }, 608 "deployState": { 609 "status": "Complete" 610 }, 611 "forwardedPorts": { 612 "5005": { 613 "localPort": 5005, 614 "remotePort": 5005, 615 "podName": "web-f6d56bcc5-6csgs", 616 "containerName": "web", 617 "namespace": "default", 618 "portName": "jdwp", 619 "resourceType": "pod", 620 "resourceName": "web-f6d56bcc5-6csgs", 621 "address": "127.0.0.1" 622 }, 623 "8080": { 624 "localPort": 8080, 625 "remotePort": 8080, 626 "namespace": "default", 627 "resourceType": "service", 628 "resourceName": "web", 629 "address": "127.0.0.1" 630 }, 631 "8081": { 632 "localPort": 8081, 633 "remotePort": 8080, 634 "podName": "web-f6d56bcc5-6csgs", 635 "containerName": "web", 636 "namespace": "default", 637 "resourceType": "pod", 638 "resourceName": "web-f6d56bcc5-6csgs", 639 "address": "127.0.0.1" 640 } 641 }, 642 "statusCheckState": { 643 "status": "Not Started" 644 }, 645 "fileSyncState": { 646 "status": "Not Started" 647 }, 648 "debuggingContainers": [ 649 { 650 "status": "Started", 651 "podName": "web-f6d56bcc5-6csgs", 652 "containerName": "web", 653 "namespace": "default", 654 "artifact": "skaffold-jib", 655 "runtime": "jvm", 656 "debugPorts": { 657 "jdwp": 5005 658 } 659 } 660 ] 661 } 662 663 ``` 664 665 </details> 666