Python requests module with json data some fields are binary date

Hello

In Python requests module how to send json data, some fields are a binary data. And doesn’t wants to convert base64 or file as well.

Do we have any other options for sending json data along with binary data by using requests module.

Please let’s know if we do have any other option.

Thanks

Are you asking why json module does not automatically use base64 for objects it does not understand?

I think the answer is that base64 is not the only choice and only the programmer will know the correct encoding to use.

You will need to do the base64 encoding yourself for objects where that makes sense for your application.

Thank you @barry-scott

The reason behind in my application making call downstream API which is exepected binary data in part of json[request [body]. So, why we need to convert base64 encoding and decoding that’s why I ask you.

I did RND there is module urllib3 support binary format as well but thing is they have own implemetation of requests module. I don’t want refectory my entire application.

Like, literal unencoded binary data inside a JSON string? Then that API is broken and doesn’t follow the json spec, which is going to cause major problems all throughout the system.

I think we are going to have an easier time helping you if you show concrete examples of what data structures you want to send and what exactly the other API is expecting.

Because JSON is a Unicode text format. See JSON for the technical specification. Therefore if you want to encode something that is not already supported you have to design your own object-to-text translation.

1 Like

If you simply want to understand why Base64 encoding is used in JSON files, it is trivial to do that web search. Using the standard UTF-8 encoding, there are some specific byte values that are literally not allowed to appear in the file at all, and many more that will get corrupted by the encoding/decoding process or which cannot be placed freely.

When you say “some fields” it’s hard to understand what you are starting with. What are they fields of? Python objects? Rows in a database? Something else?

When you say “downstream API”, I assume you mean that somebody else wrote the other program and offered the API for you to use. In this case, you must encode the data in the way that the API documentation tells you to encode it. The other code is responsible for making sure that this is a legal and possible technique in JSON.

If you are trying to understand how to encode the data, then we need to see what the data looks like, and what the API expects.

A side note to that is “why not base 85”, to which the answer is usualy “because base 64 is faster”.

1 Like