github.com/grafana/pyroscope@v1.18.0/examples/language-sdk-instrumentation/python/README.md (about) 1 ## Continuous Profiling for Python applications 2 3 ### Profiling a Python Rideshare App with Pyroscope 4 5  6 7 #### _Read this in other languages._ 8 9 <kbd>[简体中文](README_zh.md)</kbd> 10 11 Note: For documentation on the Pyroscope pip package visit [our website](https://grafana.com/docs/pyroscope/latest/configure-client/language-sdks/python/) 12 13 ## Interactive Tutorial 14 15 Explore our [interactive Ride Share tutorial](https://killercoda.com/grafana-labs/course/pyroscope/ride-share-tutorial) on KillerCoda, where you can learn how to use Pyroscope by profiling a "Ride Share" application. 16 17 ## Live Demo 18 19 Feel free to check out the [live demo](https://play.grafana.org/a/grafana-pyroscope-app/profiles-explorer?searchText=&panelType=time-series&layout=grid&hideNoData=off&explorationType=flame-graph&var-serviceName=pyroscope-rideshare-python&var-profileMetricId=process_cpu:cpu:nanoseconds:cpu:nanoseconds&var-dataSource=grafanacloud-profiles) of this example on our demo page. 20 21 ## Background 22 23 In this example we show a simplified, basic use case of Pyroscope. We simulate a "ride share" company which has three endpoints found in `server.py`: 24 25 - `/bike` : calls the `order_bike(search_radius)` function to order a bike 26 - `/car` : calls the `order_car(search_radius)` function to order a car 27 - `/scooter` : calls the `order_scooter(search_radius)` function to order a scooter 28 29 We also simulate running 3 distinct servers in 3 different regions: 30 31 - us-east 32 - eu-north 33 - ap-south 34 35 One of the most useful capabilities of Pyroscope is the ability to tag your data in a way that is meaningful to you. In this case, we have two natural divisions, and so we "tag" our data to represent those: 36 37 - `region`: statically tags the region of the server running the code 38 - `vehicle`: dynamically tags the endpoint (similar to how one might tag a controller rails) 39 40 ## Tagging static region 41 42 Tagging something static, like the `region`, can be done in the initialization code in the `config.tags` variable: 43 44 ```python 45 pyroscope.configure( 46 application_name = "ride-sharing-app", 47 server_address = "http://pyroscope:4040", 48 tags = { 49 "region": f'{os.getenv("REGION")}', # Tags the region based off the environment variable 50 } 51 ) 52 ``` 53 54 ## Tagging dynamically within functions 55 56 Tagging something more dynamically, like we do for the `vehicle` tag can be done inside our utility `find_nearest_vehicle()` function using a `with pyroscope.tag_wrapper()` block 57 58 ```python 59 def find_nearest_vehicle(n, vehicle): 60 with pyroscope.tag_wrapper({ "vehicle": vehicle}): 61 i = 0 62 start_time = time.time() 63 while time.time() - start_time < n: 64 i += 1 65 ``` 66 67 What this block does, is: 68 69 1. Add the tag `{ "vehicle" => "car" }` 70 2. execute the `find_nearest_vehicle()` function 71 3. Before the block ends it will (behind the scenes) remove the `{ "vehicle" => "car" }` from the application since that block is complete 72 73 ## Resulting flame graph / performance results from the example 74 75 ### Running the example 76 77 Try out one of the Django, Flask, or FastAPI examples located in the `rideshare` directory by running the following commands: 78 79 ```shell 80 # Pull latest pyroscope and grafana images: 81 docker pull grafana/pyroscope:latest 82 docker pull grafana/grafana:latest 83 84 # Run the example project: 85 docker compose up --build 86 87 # Reset the database (if needed): 88 docker compose down 89 ``` 90 91 What this example will do is run all the code mentioned above and also send some mock-load to the 3 servers as well as their respective 3 endpoints. If you select our application: `ride-sharing-app` from the dropdown, you should see a flame graph that looks like this (below). After we give 20-30 seconds for the flame graph to update and then click the refresh button we see our 3 functions at the bottom of the flame graph taking CPU resources _proportional to the size_ of their respective `search_radius` parameters. 92 93 ## Where's the performance bottleneck? 94 95  96 97 The first step when analyzing a profile outputted from your application, is to take note of the _largest node_ which is where your application is spending the most resources. In this case, it happens to be the `order_car` function. 98 99 The benefit of using the Pyroscope package, is that now that we can investigate further as to _why_ the `order_car` function is problematic. Tagging both `region` and `vehicle` allows us to test two good hypotheses: 100 - Something is wrong with the `/car` endpoint code 101 - Something is wrong with one of our regions 102 103 To analyze this we can select one or more labels on the "Labels" page: 104 105  106 107 ## Narrowing in on the Issue Using Labels 108 109 Knowing there is an issue with the `order_car` function we automatically select that tag. Then, after inspecting multiple `region` tags, it becomes clear by looking at the timeline that there is an issue with the `eu-north` region, where it alternates between high-cpu times and low-cpu times. 110 111 We can also see that the `find_nearest_vehicle` function is consuming almost 70% of CPU resources during this time period. 112 113  114 115 ## Visualizing diff between two flame graphs 116 117 While the difference _in this case_ is stark enough to see in the comparison view, sometimes the diff between the two flame graphs is better visualized with them overlayed over each other. Without changing any parameters, we can simply select the diff view tab and see the difference represented in a color-coded diff flame graph. 118 119  120 121 ### More use cases 122 123 We have been beta testing this feature with several different companies and some of the ways that we've seen companies tag their performance data: 124 - Linking profiles with trace data 125 - Tagging controllers 126 - Tagging regions 127 - Tagging jobs from a redis / sidekiq / rabbitmq queue 128 - Tagging commits 129 - Tagging staging / production environments 130 - Tagging different parts of their testing suites 131 - Etc... 132 133 ### Future Roadmap 134 135 We would love for you to try out this example and see what ways you can adapt this to your python application. Continuous profiling has become an increasingly popular tool for the monitoring and debugging of performance issues (arguably the fourth pillar of observability). 136 137 We'd love to continue to improve this pip package by adding things like integrations with popular tools, memory profiling, etc. and we would love to hear what features _you would like to see_.