Why string_at function in python3 gives different result for ubuntu linux, MacOs, Amazon Linux?

python 3.8
$ uname -a
Darwin MBIN0077.local 20.6.0 Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64 x86_64

if value packed in byte obj is 0x88 0xDE it becomes 0Xe4 0X8D on MacOs, Amazon Linux but remains as 0x88 0xDE on ubuntu linux

it seems if prepare docker image on host OS A & deploy on Host OS B is creating issue, pls support for it
?
ex:
buf_kb = ctypes.string_at(raw_data,sizeofrawData)

P.S.: raw_data is byte object returned from function in .so <unsigned char* in c & LP c_ubyte in python 3.8>

Sample code for ideation:

[1]
<main.py>


import ctypes
import os


def main():
    print("inside main")
    test_lib = ctypes.cdll.LoadLibrary("/src/testlib.so")
    a= (ctypes.c_ubyte * 2)(0x1B, 0x26)
    b= (ctypes.c_ubyte * 2)(0xE4, 0x1B)
    result = test_lib.calc(ctypes.byref(a),ctypes.byref(b))
    buf_kb = ctypes.string_at(result, 2)

    print("b4 hex::",buf_kb)

    json = {
        "block": buf_kb.hex()

    }

    print("json is::",json)
if __name__ == '__main__':
    main()

<main.cpp>

#include <iostream>
#include <cstring>
using namespace std;
extern "C" unsigned char*  calc(unsigned char* a,unsigned char* b)
    {
        cout<<"Hello";

unsigned char* result = NULL;
memcpy(result,a,2);
            return result;
    }

Expected output: json should print 0x1B, 0x26 but its different on Ubuntu , AWS fargate.

<Generate .so lib with below command>

g++ -fPIC -Wall -O3 -shared test.cpp -o testlib.so

[2]

FROM ubuntu:latest
COPY . /src
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -y python3
CMD ["python3", "/src/main.py"]

Kindly help to resolve the same asap.

Thank you very much.

Regards,
Gaurav Jain

Please enclose your code in </> preformatted text blocks.

Are all the systems 64bit little endian?

ctypes requires that you figure out ABI details that come from C compiler 32bit vs 64bit.

In the C++ code, you call memcpy(result, a, 2). This attempts to write 2 bytes at address 0, since result is a null pointer. This should crash on a segfault. The calc() function should either take a buffer as an output parameter or malloc() the result buffer. The former approach is generally preferred.

In the Python code, you use test_lib.calc() without declaring the return type of calc(). The default return type is c_int, which is a 32-bit signed integer. Thus a 64-bit pointer result will be truncated to 32-bit, which potentially points at the wrong address. You need to declare the function prototype using the restype and argtypes attributes. For example:

import ctypes
test_lib = ctypes.CDLL("/src/testlib.so")

LP_c_ubyte = ctypes.POINTER(ctypes.c_ubyte)
test_lib.calc.restype = LP_c_ubyte
test_lib.calc.argtypes = (LP_c_ubyte, LP_c_ubyte)

Hi Scott,

Thanks for your reply.

I used multiline comments to highlight sample src code, hope its acceptable.

All systems used are 64 bit little endian.
Data inconsistency happen if prepare docker image on host OS A & deploy on Host OS B . If use same OS for preparing Docker image & deployment , issue didnt observed.
Curious to know what & where is the fix to be applied?

Hi Eryk,

Thanks for highlighting memcpy stuff. Actually i shown sample code to depict scenario. in Actual code i took care for it so no segfault observed.
Also i took care for argtype & restype as below due to pointer:

test_lib.calc.argtypes = [ ctypes.POINTER(ctypes.c_ubyte2), ctypes.POINTER(ctypes.c_ubyte2)]
test_lib.calc.restype = ctypes.POINTER(ctypes.c_ubyte)

Data inconsistency happen if prepare docker image on host OS A & deploy on Host OS B . If use same OS for preparing Docker image & deployment , issue didn’t observed.
Curious to know what & where is the fix to be applied?

Thanks in advance!

Please provide a minimal example that you have confirmed reproduces the problem exactly as written.

What I mean is put the code inside the code fences using the </> button or
by typing like this:

```
Your code here
    with indent
```

Are you really expecting us to debug your code, which we cannot see, by sending us different code, full of other, unrelated bugs?