Why can I not catch this urllib.request.urlopen exception?

vbrozik · August 10, 2022, 1:36pm

For troubleshooting do not catch the exception. The traceback would be more comprehensible without try-except and your custom exception printing. It would be easier for us to understand what is going on.

For exception handling in production code do not catch whole Exception class. You are effectively making yourself blind for errors. See for example this recent post:

There is also a problem in the urllib code:

github.com

python/cpython/blob/v3.10.6/Lib/urllib/request.py#L1351


      
                  # Proxy-Authorization should not be sent to origin
                  # server.
                  del headers[proxy_auth_hdr]
              h.set_tunnel(req._tunnel_host, headers=tunnel_headers)
          
          try:
              try:
                  h.request(req.get_method(), req.selector, req.data, headers,
                            encode_chunked=req.has_header('Transfer-encoding'))
              except OSError as err: # timeout error
                  raise URLError(err)
              r = h.getresponse()
          except:
              h.close()
              raise
          
          # If the server does not send us a 'Connection: close' header,
          # HTTPConnection assumes the socket should be left open. Manually
          # mark the socket to be closed when this response object goes away.
          if h.sock:
              h.sock.close()

The code raising a new exception should be:

            except OSError as err: # timeout error
                # Here we should probably test if it is really a timeout error?
                raise URLError(err) from err

The current code confusingly writes:

During handling of the above exception, another exception occurred:

The fixed code would write the following text which explains what is going on:

The above exception was the direct cause of the following exception:

I think an issue should be opened, right? The same problem is in the current code: cpython/Lib/urllib/request.py at main · python/cpython · GitHub