insertdict() (dictobject.c:2051) first calls _Py_dict_lookup() (dictobject.c:2071). On a miss, it then calls insert_combined_dict() (dictobject.c:1902), which calls find_empty_slot() (dictobject.c:1924) and probes the table again.
Both searches start from the same position and use the same probing sequence, and both stop at the same DKIX_EMPTY slot. The slot found by _Py_dict_lookup() could therefore be reused directly.
This is more noticeable in free-threaded builds. On the miss path, insert_split_key() (dictobject.c:1946) does three probes: unicodekeys_lookup_unicode_threadsafe() (1954), unicodekeys_lookup_unicode() after taking the keys lock (1967), and find_empty_slot() (1977). The latter two could be combined into a single locked lookup that also returns the empty slot. This path is used when adding new instance attributes to split dictionaries.
However, A cached slot becomes invalid if the keys table is resized or its kind changes, so it must only be reused after those checks. For split dictionaries, the slot from the initial lock-free lookup cannot be reused because the shared keys table may have changed. The slot found by the locked lookup can be reused safely while holding the keys lock. A final dictkeys_get_index(keys, slot) == DKIX_EMPTY check can be used as a conservative fallback if needed.
find_empty_slot() searches for DUMMY entries as well as EMPTY slots as insertion candidates. However, since free-threaded builds do not use DUMMY entries, it may be possible to integrate this with _Py_dict_lookup().
The problem is that _Py_dict_lookup() does not return hashpos. Adding an &hashpos argument would slightly slow down dict lookups, one of Python’s most important operations. We need to carefully evaluate whether this would improve overall performance.
I was thinking we could make the hashpos return optional. In that case, the compiler may be able to optimize the NULL case so that the existing lookup path remains unchanged.
Another way to avoid affecting the regular lookup path would be to add specialized helper functions for the cases where we actually need hashpos like insertion and deletion.
This would touch one of Python’s most important operations, so I think it is a difficult trade-off and worth discussing carefully. I also benchmarked the redundant probe, and it does not seem to be very expensive, although it is not negligible either. It accounts for roughly 3–4% of the time for deletion and somewhat less for insertion.
I prototyped the idea recently. The hard part is to balance code duplication, performance gains, compiler inlining etc. Also the idea works best for unicode keys (for other types of keys arbitrary code may be executed, potentially mutating the dict).