Hello,
I am testing the new sampling profiler Tachyon. For my workload, I am consistently seeing error-rates exceeeding 97%, producing poor quality results. Is this due to me using Tachyon incorrectly, or is something about my particular workload throwing it off?
I am using python 3.15.0 beta 2, built with pyenv:
PYTHON_CONFIGURE_OPTS=‘–enable-optimizations --with-lto’ PYTHON_CFLAGS=‘-march=native -mtune=native -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer’ ~/.pyenv/bin/pyenv install -v 3.15.0b2
The target is a pure-python webserver:
git clone https://github.com/9001/copyparty/; cd copyparty; ~/.pyenv/versions/3.15.0b2/bin/python -m profiling.sampling run --heatmap -o a.html -d 10 -r 1khz --blocking -a -m copyparty -q --no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1 -v srv::ra
Load is generated by sending http-requests from another terminal:
while true; do curl -s 127.1:3923/?ls{,,,,,,,,,,,,,,,,,,,,,,,,,,,,}{,,,,,,,,,,,,,,,,,,,,,,,,,,,,} >/dev/null; done
I have tried with/without `–blocking`, and various sampling-rates (1k through 20k), all with no difference. The arguments given to the profiling target (`–no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1`) are to reduce the number of threads, but there is still 8 threads at idle, listed at http://127.0.0.1:3923/?stack .
With --realtime-stats I get:
Stats: 1,000.2Hz (999.8µs) Min: 930.8Hz Max: 1,082.5Hz N=9003 Cache: 98.7% (2135+82/30)
Captured 10,001 samples in 10.00 seconds
Sample rate: 1,000.09 samples/sec
Error rate: 97.52
Unwinder Statistics:
Frame Cache:
Total samples: 10,001
Full hits: 2,309 (94.8%)
Partial hits: 95 (3.9%)
Misses: 32 (1.3%)
Frame Reads:
From cache: 9,924 (74.6%)
From memory: 3,375 (25.4%)
Code Object Cache:
Hits: 16,236 (59.9%)
Misses: 10,851 (40.1%)
Batched Reads:
Attempts: 12,187
Successes: 12,167 (99.8%)
Misses: 20 (0.2%)
Segments read: 34,171/34,191 (99.9%)
Memory:
Read operations: 51,643 (13.6 MB)
if I attach to a running process rather than run it directly under tachyon, the error rate drops to 80%, however the output is still confusing; I know that almost all of the cpu-time is spent in httpcli.run, yet that does not get a single hit.
That aside, I really like everything else about Tachyon so far! The UI looks awesome, and I’m certain it’ll be a fantastic way to see where the cycles are going, as soon as I can get it to sing 