Code coverage with clang

Core Development
1

I’d like to measure code coverage of the _csv.c module. My Mac has clang installed (/usr/bin/gcc is a hard link). When I run configure with --enable-profiling, it dutifully adds “-pg” to the gcc command line. This doesn’t work with clang.

This doesn’t seem to be something Python’s configure script can do currently. Is there a recipe floating around to enable this?

2

It seems that Clang offers a -pg command line option now, but I’m nowhere near familiar enough with either GCC or Clang (and I don’t know anything about gprof at all) to know if these are really equivalent.

Assuming that “mcount instrumentation” is good enough, my best guess is that configure was designed (at some point before 2020 or so) not to add an unrecognized option for Clang command lines, and nobody thought to update this when it became available.

(That said, if you have /usr/bin/gcc hard-linked to Clang, I can’t guess how the configure script knows that it isn’t GCC…)

3

For the one extension where I do collect coverage information I use clang with --coverage -fprofile-arcs -ftest-coverage. That said, there flags are cargo culted and I don’t know if this is the best way to collect coverage information with clang.

4

Thanks. The first thing I tried doing was enabling profiling in the usual way. Whatever version of clang I have on my Mac said something like “ignoring -pg”. Second try was too edit configure to replace -pg with --coverage, then rerun it. That flag was nowhere to be found in the make output. Maybe I edited the wrong (or not enough) place.

My final attempt was just to trim make -n with output to a shell script. A quick edit and Bob’s yet Uncle. Well, except for not having llcm-cov. I left for a bike ride at that point and will revisit this later.

5

Thanks. Will check into the two -f* flags later.

6

You’ll need to add -fprofile-instr-generate -fcoverage-mapping to CFLAGS and LDFLAGS. I add them explicitly[1]; I never use the --coverage option.

Here’s a dump of my “generate coverage” script:

#!/bin/sh
set -ve

MACOS_VER="13.2"
PY_MAJOR="3"
PY_MINOR="12"
ARCH=arm64
PREFIX="build/lib.macosx-${MACOS_VER}-${ARCH}-${PY_MAJOR}.${PY_MINOR}-pydebug/"
SUFFIX="cpython-${PY_MAJOR}${PY_MINOR}d-darwin.so"

# Convenience variables, used for BIN
SOCKET_LIB="${PREFIX}/_socket.${SUFFIX}"
SQLITE_LIB="${PREFIX}/_sqlite3.${SUFFIX}"
ELEMENTTREE_LIB="$PREFIX/_elementtree.${SUFFIX}"
READLINE_LIB="$PREFIX/readline.${SUFFIX}"
TESTCAPI="$PREFIX/_testcapi.${SUFFIX}"
SYSLOG_LIB="$PREFIX/syslog.${SUFFIX}"

BIN=$SOCKET_LIB # python.exe $SQLITE_LIB, etc.
PROFRAW=default.profraw
PROFDATA=default.profdata
SRC=$HOME/src/cpython.git
REPORT=llvm-report

# Tweak this as you like it
IGNORE="(clinic|Include)"

llvm-profdata merge \
  -sparse $PROFRAW  \
  -o $PROFDATA

llvm-cov report                    \
  -ignore-filename-regex="$IGNORE" \
  -instr-profile=$PROFDATA         \
  $BIN                             \
  $SRC

llvm-cov show                      \
  -format=html                     \
  -output-dir=$REPORT              \
  -ignore-filename-regex="$IGNORE" \
  -instr-profile=$PROFDATA         \
  $BIN                             \
  $SRC

rm -f $PROFDATA

  1. passing CFLAGS=... and LDFLAGS=... to configure ↩︎