github.com/inspektor-gadget/inspektor-gadget@v0.28.1/docs/builtin-gadgets/trace/exec.md (about)

     1  ---
     2  title: 'Using trace exec'
     3  weight: 20
     4  description: >
     5    Trace new processes.
     6  ---
     7  
     8  ![Screencast of the trace exec gadget](exec.gif)
     9  
    10  The trace exec gadget streams new processes creation events.
    11  
    12  ### On Kubernetes
    13  
    14  Let's deploy an example application that will spawn few new processes:
    15  
    16  ```bash
    17  $ kubectl apply -f docs/examples/ds-myapp.yaml
    18  daemonset.apps/myapp1-pod created
    19  daemonset.apps/myapp2-pod created
    20  
    21  $ kubectl get pod --show-labels -o wide
    22  NAME               READY   STATUS    RESTARTS   AGE   IP               NODE              NOMINATED NODE   READINESS GATES   LABELS
    23  myapp1-pod-sbtvw   1/1     Running   0          9s    10.244.192.133   minikube-docker   <none>           <none>            controller-revision-hash=865c886d8f,myapp=app-one,name=myapp1-pod,pod-template-generation=1,role=demo
    24  myapp2-pod-5pg4w   1/1     Running   0          9s    10.244.192.132   minikube-docker   <none>           <none>            controller-revision-hash=677d884fc,myapp=app-two,name=myapp2-pod,pod-template-generation=1,role=demo
    25  ```
    26  
    27  Using the trace exec gadget, we can see which new processes are spawned on node
    28  minikube-docker where myapp1-pod-sbtvw and myapp2-pod-5pg4w are running:
    29  
    30  ```bash
    31  $ kubectl gadget trace exec --selector role=demo --node minikube-docker
    32  K8S.NODE        K8S.NAMESPACE K8S.POD          K8S.CONTAINER PID     PPID    COMM  PCOMM RET ARGS
    33  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226276 2221571 true  sh    0   /bin/true
    34  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226277 2221571 date  sh    0   /bin/date
    35  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226278 2221571 cat   sh    0   /bin/cat /proc/version
    36  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226279 2221571 true  sh    0   /bin/true
    37  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226280 2221571 date  sh    0   /bin/date
    38  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226281 2221571 cat   sh    0   /bin/cat /proc/version
    39  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226282 2221571 true  sh    0   /bin/true
    40  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226283 2221571 date  sh    0   /bin/date
    41  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226284 2221571 cat   sh    0   /bin/cat /proc/version
    42  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226286 2221571 true  sh    0   /bin/true
    43  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226287 2221571 date  sh    0   /bin/date
    44  minikube-docker default       myapp1-pod-sbtvw myapp1-pod    2226288 2221571 cat   sh    0   /bin/cat /proc/version
    45  minikube-docker default       myapp2-pod-5pg4w myapp2-pod    2226289 2221280 true  sh    0   /bin/true
    46  minikube-docker default       myapp2-pod-5pg4w myapp2-pod    2226290 2221280 date  sh    0   /bin/date
    47  minikube-docker default       myapp2-pod-5pg4w myapp2-pod    2226291 2221280 echo  sh    0   /bin/echo sleep-10
    48  minikube-docker default       myapp2-pod-5pg4w myapp2-pod    2226292 2221280 sleep sh    0   /bin/sleep 10
    49  ^C
    50  ```
    51  Processes of both pods are spawned: myapp1 spawns `cat /proc/version` and `sleep 1`,
    52  myapp2 spawns `echo sleep-10` and `sleep 10`, both spawn `true` and `date`.
    53  We can stop to trace again by hitting Ctrl-C.
    54  
    55  Finally, we clean up our demo app.
    56  
    57  ```bash
    58  $ kubectl delete -f docs/examples/ds-myapp.yaml
    59  ```
    60  
    61  ### With `ig`
    62  
    63  Let's start the gadget in a terminal:
    64  
    65  ```bash
    66  $ sudo ig trace exec -c test-trace-exec
    67  RUNTIME.CONTAINERNAME PID     PPID    COMM   PCOMM           RET ARGS
    68  ```
    69  
    70  Run a container that executes some binaries:
    71  
    72  ```bash
    73  $ docker run --name test-trace-exec -it --rm busybox /bin/sh -c 'while /bin/true ; do whoami ; sleep 3 ; done'
    74  ```
    75  
    76  The tool will show the different processes executed by the container:
    77  
    78  ```bash
    79  $ sudo ig trace exec -c test-trace-exec
    80  RUNTIME.CONTAINERNAME PID     PPID    COMM   PCOMM           RET ARGS
    81  test-trace-exec       2233189 2233166 sh     containerd-shim 0   /bin/sh -c while /bin/true ; do whoami ; sleep 3 ; done
    82  test-trace-exec       2233214 2233189 true   sh              0   /bin/true
    83  test-trace-exec       2233215 2233189 whoami sh              0   /bin/whoami
    84  test-trace-exec       2233567 2233189 true   sh              0   /bin/true
    85  test-trace-exec       2233570 2233189 whoami sh              0   /bin/whoami
    86  test-trace-exec       2233642 2233189 true   sh              0   /bin/true
    87  test-trace-exec       2233643 2233189 whoami sh              0   /bin/whoami
    88  test-trace-exec       2233757 2233189 true   sh              0   /bin/true
    89  test-trace-exec       2233758 2233189 whoami sh              0   /bin/whoami
    90  test-trace-exec       2233931 2233189 true   sh              0   /bin/true
    91  test-trace-exec       2233932 2233189 whoami sh              0   /bin/whoami
    92  ```
    93  
    94  ### `--paths`
    95  
    96  Optionally, this gadget can provide the current working directory of the process calling `exec()` and the full path of the executable.
    97  This is disabled by default and can be enabled by passing the `--paths` flag:
    98  
    99  ```bash
   100  $ sudo ig trace exec
   101  RUNTIME.CONTAINERNAME PID    PPID   COMM  RET ARGS
   102  test                  644871 639225 mkdir 0   /usr/bin/mkdir -p /tmp/bar/foo/
   103  test                  644888 639225 cat   0   /usr/bin/cat /dev/null
   104  ```
   105  ```
   106  $ sudo ig trace exec --paths
   107  RUNTIME.CONTAINERN… PID    PPID   COMM  RET ARGS                     CWD EXEPATH
   108  test                644377 639225 mkdir 0   /usr/bin/mkdir -p /tmp/… /   /usr/bin/mkdir
   109  test                644497 639225 cat   0   /usr/bin/cat /dev/null   /   /usr/bin/cat
   110  ```
   111  
   112  
   113  ### Overlay filesystem upper layer
   114  
   115  It can be useful to know if the executable in a container was modified or part
   116  of the original container image. If it was modified, it will be located in the
   117  upper layer of the overlay filesystem. For this reason, this gadget provides
   118  the upper layer field which is true if the executable is located in the upper
   119  layer of the overlay filesystem.
   120  
   121  ```bash
   122  $ sudo ig trace exec -c test -o columns=comm,ret,upperlayer
   123  COMM             RET UPPERLAYER
   124  sh               0   false
   125  cp               0   false
   126  echo             0   false
   127  echo2            0   true
   128  ```
   129  
   130  ```bash
   131  $ docker run -ti --rm --name=test ubuntu \
   132      sh -c 'cp /bin/echo /bin/echo2 ; /bin/echo lower ; /bin/echo2 upper'
   133  lower
   134  upper
   135  ```
   136  
   137  Limitations:
   138  - The upper layer field is only available when the executable is executed
   139   correctly (ret=0). For example, if the executable does not have the "execute"
   140   permission, the execution will fail and the upper layer field will not be
   141   defined.
   142  - In case of a shell script, the upper layer field will refer to the location
   143   of the shell program (e.g. `/bin/sh`) and not the script file.