github.com/grafana/pyroscope@v1.18.0/docs/sources/configure-client/grafana-alloy/ebpf/troubleshooting.md (about)

     1  ---
     2  title: "Troubleshoot eBPF installation"
     3  menuTitle: "Troubleshoot"
     4  description: "Troubleshoot Grafana eBPF installation."
     5  weight: 40
     6  ---
     7  
     8  # Troubleshoot eBPF installation
     9  
    10  Learn how to troubleshoot and resolve eBPF installation issues.
    11  
    12  ## Profile interpreted languages
    13  
    14  Profiling interpreted languages like Ruby, JavaScript, etc., isn't ideal using this implementation.
    15  The JIT-compiled methods in these languages are typically not in ELF file format, demanding additional steps for
    16  profiling. For instance, using perf-map-agent and enabling frame pointers for Java.
    17  
    18  Interpreted methods display the interpreter function’s name rather than the actual function.
    19  
    20  ## Troubleshoot unknown symbols
    21  
    22  Symbols are extracted from various sources, including:
    23  
    24  * The `.symtab` and `.dynsym` sections in the ELF file.
    25  * The `.symtab` and `.dynsym` sections in the debug ELF file.
    26  * The `.gopclntab` section in Go language ELF files.
    27  
    28  The search for debug files follows [gdb algorithm](https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html).
    29  For example, if the profiler wants to find the debug file
    30  for `/lib/x86_64-linux-gnu/libc.so.6`
    31  with a `.gnu_debuglink` set to `libc.so.6.debug` and a build ID `0123456789abcdef`. The following paths are examined:
    32  
    33  * `/usr/lib/debug/.build-id/01/0123456789abcdef.debug`
    34  * `/lib/x86_64-linux-gnu/libc.so.6.debug`
    35  * `/lib/x86_64-linux-gnu/.debug/libc.so.6.debug`
    36  * `/usr/lib/debug/lib/x86_64-linux-gnu/libc.so.6.debug`
    37  
    38  ### Deal with unknown symbols
    39  
    40  Unknown symbols in the profiles you’ve collected indicate that the profiler couldn't access an ELF file associated with a given address in the trace.
    41  
    42  This can occur for several reasons:
    43  
    44  * The process has terminated, making the ELF file inaccessible.
    45  * The ELF file is either corrupted or not recognized as an ELF file.
    46  * There is no corresponding ELF file entry in `/proc/pid/maps` for the address in the stack trace.
    47  
    48  ### Address unresolved symbols
    49  
    50  If you only see module names (e.g., `/lib/x86_64-linux-gnu/libc.so.6`) without corresponding function names, this
    51  indicates that the symbols couldn't be mapped to their respective function names.
    52  
    53  This can occur for several reasons:
    54  
    55  * The binary has been stripped, leaving no `.symtab`, `.dynsym`, or `.gopclntab` sections in the ELF file.
    56  * The debug file is missing or could not be located.
    57  
    58  To fix this for your binaries, ensure that they are either not stripped or that you have separate
    59  debug files available. You can achieve this by running:
    60  
    61  ```bash
    62  objcopy --only-keep-debug elf elf.debug
    63  strip elf -o elf.stripped
    64  objcopy --add-gnu-debuglink=elf.debug elf.stripped elf.debuglink
    65  ```
    66  
    67  For system libraries, ensure that debug symbols are installed. On Ubuntu, for example, you can install debug symbols
    68  for `libc` by executing:
    69  
    70  ```bash
    71  apt install libc6-dbg
    72  ```
    73  
    74  ### Understand flat stack traces
    75  
    76  If your profiles show many shallow stack traces, typically 1-2 frames deep, your binary might have been compiled without frame pointers.
    77  
    78  To compile your code with frame pointers, include the `-fno-omit-frame-pointer` flag in your compiler options.
    79  
    80  
    81  ### Ensure Python process data is discoverable
    82  
    83  This error indicates that Pyroscope cannot locate required Python runtime symbols, potentially due to nonstandard library naming:
    84  
    85  `pyperf get python process data failed: missing symbols pyRuntimeAddr autoTLSkeyAddr`
    86   
    87  This can occur if the application build process uses custom naming for libraries, such as:
    88  
    89  - `libpython3-custom.10.so.1.0`
    90  
    91  Pyroscope expects standard naming patterns like:
    92  
    93  - `libpython3.10.so.1.0`
    94  
    95  To resolve this, ensure Python libraries follow standard naming conventions.
    96