How to write binary file in python and read it in C++

Python Help
1

Hi,

I have some C++ code that reads binary data from a file. When I use a C++ code to write to it, everything works. But I need a python code to write the file. It works for the first word that I write (using “`fout.write(struct.pack('@Bx', size))”), (the C++ code reads it correctly), but as soon as I try to write more values to the file, the C++ code reads the wrong values. When I use the C++ writing code, it works fine. I guess what I need is the correct translation from the C++ writing code to python.

C++ writing code:

#include <iostream>
#include <fstream>
using namespace std;
int main()
{
 std::ofstream outfile("binary.dat_cpp", std::ios::binary | std::ios::out );
 unsigned int size = 3;
 unsigned int key = 8;
 outfile.write((char*) &size, sizeof(size));
 outfile.write((char*) &key, sizeof(key));
 outfile.flush();
 outfile.close();
 return 1;
}

C++ reading code:

#include <iostream>
#include <fstream>
using namespace std;
int main()
{
 std::ifstream infile("binary.dat", std::ios::binary | std::ios::in );
 if (!infile.is_open())
 {
  cout << "FILE not open" << endl;
  return 1;
 }
 unsigned int size;
 unsigned int key;
 infile.read((char*) &size, sizeof(size));
 cout << "size: " << size << endl;
 infile.read((char*) &key, sizeof(key));
 cout << "event: " << key << endl;
 return 0;
}

python writing code:

import struct
fout = open('binary.dat_py', 'wb')
size = 13
key = 18
fout.write(struct.pack('@Bx', size))
# without the next line, I can read the file correctly
# with the next line, even reading 'size' will go wrong
fout.write(struct.pack('@Bx', key))	
# This will give me:
#	size: 1179661
#	event: 0
# Without second write, I get:
# 	size: 13
# 	event: 0
fout.flush()
fout.close()
2

You’re using unsigned ints (4 bytes) but writing out two bytes. I would suggest struct.pack('I', size), to match your reader’s types.

Note that struct.pack can take multiple arguments, so you can use fout.write(struct.pack('II', size, key)), rather than calling pack and write twice.

3

It is advisable to use a with block to open the file in Python to ensure proper cleanup, like so:

with open('binary.dat_py', 'wb') as fout:
    ... # your code that uses fout
4

Dear Chris,

Thank you, this solves my problem. My actual code is a bit more complicated, but it works just as well by using “I”. I had tried using “I” before, however, I used it like this “>I” (struct.pack(“>I”, size)) and that doesn’t work. Could you explain why it doesn’t work when using “>I”?

5

>I is big-endian, and you’re casting directly as the native data type on the C++ side. Most processors are little-endian. You can use @I if you prefer to be explicit.

6

Thank you Jonathan. I thought that using “fout.flush” and “fout.close” was enough to ensure proper cleanup, but I will follow your advice.

7

Thank you for that clarification, it has been very helpful. I am still a beginner with python, but this has been very clarifying.

8

Flushing and closing is enough for proper cleanup, but you aren’t handling the case where exceptions could be raised, which would look like:

import struct

size = 13
key = 18

fout = open('binary.dat_py', 'wb')
try:
    fout.write(struct.pack('@II', size, key))
finally:
    fout.flush()
    fout.close()

Instead, you can write

import struct

size = 13
key = 18

with open('binary.dat_py', 'wb') as fout:
    fout.write(struct.pack('@II', size, key))

Since file objects know how to clean themselves up, you can now devote your lines of code to the interesting problems, not the book-keeping.

A good resource on context managers is https://realpython.com/python-with-statement/.

If you’re more familiar with C++, December 2000 C++ Experts Forum/Generic<Programming> is a relevant article dealing with the same problems.