The 3.11 linetable for generator expressions seems to be missing an entry

Core Development
1

I’m writing a pure python pyc file reader and disassembler, and I ran into an issue with generator expressions in 3.11. I’m stepping through code.co_code one opcode at a time, and reading one entry from the linetable for each opcode, which mostly works, except for a generator expression there is no linetable entry for the last RETURN_VALUE opcode.

If I just add an “assume there is an INFO_NONE if we have read past the end of the linetable” it all seems to work, and it does make sense as an optimisation because the RETURN_VALUE will never actually appear in the source code, but I’m wondering if there is actually no final entry or if I have some bug elsewhere.

The code looks like

        lt = linetable.linetable_reader(self.code)
        ret = []
        for o in wordcode_reader(self.code.co_code):
            if o.op == 0:  # CACHE
                continue
            pos = lt.get(o.start)

and lt.get reads a single entry from the linetable reader:

    def read(self):                                                                                                                                                                                                                         
        b = self._read_byte()                                                                                                                                                                                                               
        code = (b >> 3) & 15                                                                                                                                                                                                                
        self.start = self.end                                                                                                                                                                                                               
        self.end = self.start + (b & 7) + 1                                                                                                                                                                                                 
        if code == PyCodeLocation.INFO_NONE:                                                                                                                                                                                                
            endline = -1                                                                                                                                                                                                                    
            col = -1                                                                                                                                                                                                                        
            endcol = -1                                                                                                                                                                                                                     
        elif code == PyCodeLocation.INFO_LONG:                                                                                                                                                                                              
            self.line += self._read_signed_varint()
            endline = self.line + self._read_varint()
            col = self._read_varint() - 1
            endcol = self._read_varint() - 1
        elif code == PyCodeLocation.INFO_NO_COLUMNS:
            self.line += self._read_signed_varint()
            endline = self.line + self._read_varint()
            col = -1
            endcol = -1
        elif code in (
            PyCodeLocation.INFO_ONE_LINE0,
            PyCodeLocation.INFO_ONE_LINE1,
            PyCodeLocation.INFO_ONE_LINE2,
        ):
            self.line += code - 10
            endline = self.line
            col = self._read_byte()
            endcol = self._read_byte()
        else:
            b2 = self._read_byte()
            assert b2 & 128 == 0
            endline = self.line
            col = (code << 3) | (b2 >> 4)
            endcol = col + (b2 & 15)
        return (endline, col, endcol)
1 Like