Linker error related to OpenSSL when Python is modified to use OpenSSL in the tokenizer

Core Development
1

I created the C language class method of openssl rsa,
Modified Makefile.pre.in to make it compile to xxx.o.
But when compiling and installing in redhat, the error as shown in the figure was finally reported

QQęˆŖ图20201210212428
QQęˆŖ图202012102124281073Ɨ317 80.9 KB

       RE_ BUILTIN  -c ./Modules/hashtable.c -o Modules/hashtable.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_ BUILD_ CORE_ BUILTIN  -c ./Modules/symtablemodule.c -o Modules/symtablemodule.o
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_ BUILD_ CORE_ BUILTIN  -c ./Modules/xxsubtype.c -o Modules/xxsubtype.o
    gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_ BUILD_ CORE \
    -DGITVERSION="\"`LC_ ALL=C git --git-dir ./.git rev-parse --short HEAD`\"" \
    -DGITTAG="\"`LC_ ALL=C git --git-dir ./.git describe --all --always --dirty`\"" \
    -DGITBRANCH="\"`LC_ ALL=C git --git-dir ./.git name-rev --name-only HEAD`\"" \
    -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
    gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_ BUILD_ CORE -o Python/frozen.o Python/frozen.c
    rm -f libpython3.7m.a
    ar rcs libpython3.7m.a Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Parser/ykkjrsa.o Objects/abstract.o Objects/accu.o Objects/boolobject.o  Objects/bytes_ methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o  Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/capsule.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o  Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_ warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/ast_ opt.o Python/ast_ unparse.o Python/bltinmodule.o Python/ceval.o Python/compile.o Python/codecs.o Python/dynamic_ annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/peephole.o Python/pyarena.o  Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/context.o Python/hamt.o Python/pythonrun.o Python/pytime.o Python/bootstrap_ hash.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_ unicode.o Python/fileutils.o Python/dynload_ shlib.o    Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_ sre.o  Modules/_ codecsmodule.o  Modules/_ weakref.o  Modules/_ functoolsmodule.o  Modules/_ operator.o  Modules/_ collectionsmodule.o  Modules/_ abc.o  Modules/itertoolsmodule.o  Modules/atexitmodule.o  Modules/signalmodule.o  Modules/_ stat.o  Modules/timemodule.o  Modules/_ threadmodule.o  Modules/_ localemodule.o  Modules/_ iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o  Modules/zipimport.o  Modules/faulthandler.o  Modules/_ tracemalloc.o Modules/hashtable.o  Modules/symtablemodule.o  Modules/xxsubtype.o Python/frozen.o
    gcc -pthread    -fprofile-generate -Xlinker -export-dynamic -o python Programs/python.o libpython3.7m.a -lcrypt -lpthread -ldl  -lutil   -lm
    Libpython3.7m. A (ykkjrsa. O): in the function "RSA"_ In "encode":
    /Yunku / python-3.7.7-rsa / parser / ykkjrsa. C: 188: for 'bio'_ new_ mem_ BUF 'undefined reference
    Libpython3.7m. A (ykkjrsa. O): in the function "RSA"_ In "encode":
    Ykkjrsa. C: (. Text + 0x72): for "PEM"_ read_ bio_ RSA_ Undefined reference of pubkey '
    Ykkjrsa. C: (. Text + 0x8e): for "bio"_ free_ Undefined reference
    Ykkjrsa. C: (. Text + 0x9e): for "RS"
2

Hello! Are you adding a new function to a module in the standard library?

Have you followed the development instructions in the devguide?

Next time, please copy-paste text rather than an image, and set LANG=C so that everyone can read output and error messages.

3

When I use make & make install to compile and install python, the error printed on the console.
These references use openssl

RE_BUILTIN  -c ./Modules/_tracemalloc.c -o Modules/_tracemalloc.o
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_BUILD_CORE_BUILTIN  -c ./Modules/hashtable.c -o Modules/hashtable.o
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_BUILD_CORE_BUILTIN  -c ./Modules/symtablemodule.c -o Modules/symtablemodule.o
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_BUILD_CORE_BUILTIN  -c ./Modules/xxsubtype.c -o Modules/xxsubtype.o
gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_BUILD_CORE \
      -DGITVERSION="\"`LC_ALL=C git --git-dir ./.git rev-parse --short HEAD`\"" \
      -DGITTAG="\"`LC_ALL=C git --git-dir ./.git describe --all --always --dirty`\"" \
      -DGITBRANCH="\"`LC_ALL=C git --git-dir ./.git name-rev --name-only HEAD`\"" \
      -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fprofile-generate  -I. -I./Include    -DPy_BUILD_CORE -o Python/frozen.o Python/frozen.c
rm -f libpython3.7m.a
ar rcs libpython3.7m.a Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Parser/ykkjrsa.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/capsule.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/ast_opt.o Python/ast_unparse.o Python/bltinmodule.o Python/ceval.o Python/compile.o Python/codecs.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/peephole.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/context.o Python/hamt.o Python/pythonrun.o Python/pytime.o Python/bootstrap_hash.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o    Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/_weakref.o  Modules/_functoolsmodule.o  Modules/_operator.o  Modules/_collectionsmodule.o  Modules/_abc.o  Modules/itertoolsmodule.o  Modules/atexitmodule.o  Modules/signalmodule.o  Modules/_stat.o  Modules/timemodule.o  Modules/_threadmodule.o  Modules/_localemodule.o  Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o  Modules/zipimport.o  Modules/faulthandler.o  Modules/_tracemalloc.o Modules/hashtable.o  Modules/symtablemodule.o  Modules/xxsubtype.o Python/frozen.o
gcc -pthread    -fprofile-generate -Xlinker -export-dynamic -o python Programs/python.o libpython3.7m.a -lcrypt -lpthread -ldl  -lutil   -lm  

libpython3.7m.a(ykkjrsa.o)ļ¼šIn the function ā€˜rsa_encodeā€™:
ykkjrsa.c:(.text+0x72)ļ¼šUndefined reference to ā€˜PEM_read_bio_RSA_PUBKEYā€™
ykkjrsa.c:(.text+0x8e)ļ¼šUndefined reference toā€˜BIO_free_allā€™ 
ykkjrsa.c:(.text+0x9e)ļ¼šUndefined reference toā€˜RSA_sizeā€™ 
ykkjrsa.c:(.text+0xcd)ļ¼šUndefined reference toā€˜RSA_public_encryptā€™
4

I donā€™t think libpython references OpenSSL anywhere - thatā€™s all kept in a separate module (_ssl) - so youā€™ll either have to add your function to that module or else add a reference to OpenSSL into the main lib.

5

I modified tokenizer.c in decode_str method function used RSA_ public_ Encrypt, how can I add a reference to OpenSSL in the main library.

6

No idea, Iā€™m afraid. Iā€™m not familiar with the Linux build system, only the Windows one.

Iā€™d recommend moving your modification to the _ssl module, as it already has the reference.

2 Likes
7

Also, it would be really nice if someone edited the subject of this topic, since itā€™s not really about running ā€˜makeā€™, itā€™s about modifying the CPython code and getting it to build.

8

Which OS is that? RHEL? If yes, which RHEL version? How did you get Python? Which commands did you type to build Python? Which Python version are you trying to build (full version with the 3 numbers please: Python 3.7.x: what is x)?

What are your changes? Why did you modify this file?

It seems like you are using OpenSSL functions in Python, but you didnā€™t link libpython to OpenSSL. You need to add $OPENSSL_LIBS to $LIBS in Makefile.pre.in in this case: tokenize.c is part of ā€œlibpythonā€.

9

I donā€™t think that you injected your code at the right place. The tokenizer is too deep into Python internals. I would suggest you to add an import hook or replace existing import hook, to decrypt your binary files on import.

10

I modified the topic. The linker error is caused by changes in the code base.

1 Like