ImportError: cannot import name 'urllib3'

Python Help
1

Hello there, fairly new to Python, getting an Import Error the package is in the /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/request.py directory, bit no luck

It is not just this package but all. bs4 for example

PATH=/Library/Frameworks/Python.framework/Versions/3.10/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages

2

Please post the code you’ve written and the exact error message you are receiving.

Also note that it’s not useful to put the site-packages directory into your PATH; it doesn’t contain files that are executable by your shell.

3

from urllib3.request import urlopen
from bs4 import BeautifulSoup4
import random
import re
import datetime
import ssl
import main as main

now = datetime.datetime.now()
ssl._create_default_https_context = ssl._create_unverified_context
scriptname = (main.file)

print(’**********************************’)

print('START OF PROCESS: ', scriptname)

print(’**********************************’)

print (“Current date and time : “)
print (now.strftime(”%Y-%m-%d %H:%M:%S”))

print(’**********************************’)

random.seed(datetime.datetime.now())
def getLinks(articleUrl):
html = urlopen(‘http://en.wikipedia.org{}’.format(articleUrl))
bs = BeautifulSoup(html, ‘html.parser’)
return bs.find(‘div’, {‘id’:‘bodyContent’}).find_all(‘a’, href=re.compile(’^(/wiki/)((?!:).)*$’))

links = getLinks(’/wiki/Kevin_Bacon’)
while len(links) > 0:
newArticle = links[random.randint(0, len(links)-1)].attrs[‘href’]
print(newArticle)
links = getLinks(newArticle)

ERROR *****
Traceback (most recent call last):
File “/Users/seanmontgomery/python/scripts/webscraping/crawling/bacon_random_walk.py”, line 1, in
from urllib3.request import urlopen
ImportError: cannot import name ‘urlopen’ from ‘urllib3.request’ (/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/request.py)

4

I don’t see this in the urllib3 documentation as something that is available. urlopen is a method in some of the urllib3 classes, it’s not a class or function of its own as best I can tell.

5

Thanks Kevin.

if I try to import beautiful soup for example it throws the same error, so It seems none of the packages are seen or what

I am also using PyCharm if that helps in debugging anything

from: can’t read /var/mail/urllib3.request
$ python bacon_random_walk.py
Traceback (most recent call last):
File “/Users/seanmontgomery/python/scripts/webscraping/crawling/bacon_random_walk.py”, line 1, in
from bs4 import BeautifulSoup4
ImportError: cannot import name ‘BeautifulSoup4’ from ‘bs4’ (/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bs4/init.py)

6

How did you install the bs4 package?

7

pip3 install

8

But you are running the script using ‘python’, not ‘python3’. What does python --version report? What about python3 --version?

9

Python 3.10.2

10

For both?

What does python -m pip list report? Does it include ‘bs4’?

11

$ pip list
Package Version

beautifulsoup4 4.10.0
bs4 0.0.1
pip 22.0.3
setuptools 58.1.0
soupsieve 2.3.1
urllib3 1.26.8

$ pip3 list
Package Version

beautifulsoup4 4.10.0
bs4 0.0.1
pip 22.0.3
setuptools 58.1.0
soupsieve 2.3.1
urllib3 1.26.8

12

Well, all of that looks to be correct. Someone with more knowledge of the Python import system will have to help you figure out why Python can’t import packages that appear to be properly installed.

13

Thanks Kevin, I appreciate your looking. I cannot find a solution despite exhausting every google search

14

Sean’s code has:

random.seed(datetime.datetime.now())

You don’t need to seed the random number generator, it is automatically seeded for you, and using a much more random seed than the current time.

The only good reason to use random.seed is to use a known seed, so you can reproduce the same sequence of random numbers later.

newArticle = links[random.randint(0, len(links)-1)].attrs['href']

If you want a half-open interval starting from zero, you should use

random.randrange(len(links))

instead of randint.

15

You have:

from bs4 import BeautifulSoup4

but I believe that should be from bs4 import BeautifulSoup.

If that doesn’t solve your issue, here are some more questions.

What is your PYTHONPATH environment variable?

Does PyCharm have an option to view these?

  1. the Python executable it is using;
  2. the location of the site-packages it is using.

Confirm that they are the same as the Python executable and site-packages you are using outside of PyCharm.

E.g. compare the PyCharm interpreter with which python in the shell.

It may help to do a locate site-packages to ensure everything is correct.

What OS are you running? From the path starting with /Users, I’m going to guess it is Mac OS. Is that correct?

16

Thank you Steven

Here is my .bash_profile

Setting PATH for Python 3.10

The original version is saved in .zprofile.pysave

PATH="/Library/Frameworks/Python.framework/Versions/3.10/bin:${PATH}:/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages"

export PYTHONPATH=/usr/local/lib/python/site-packages:$PYTHONPATH

export PATH

Python Interpretor - Python 3.10

which -
Python 3.10.2

macOS

17

As I mentioned above, there is no need to put any site-packages directory into PATH. Python does not use PATH when searching for modules to import, it uses PYTHONPATH.

18

HELLO Kevin
I was able to resolve. Removed all references to anything in a python path and left the standard Path usr/local/bin etc etc

1 Like
19

Thanks a million for helping out. I also resolved the issue
in PyCharm…

20

PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin