# Converting decimal to byte (little endian): what's wrong with 205?

Hello everybody!

That’s my first message here, so I start by asking apologies for any mistakes.

To the problem…

I have a routine that converts decimal numbers to bytes (little endian) that is working Ok (or so I guessed…). It is like this:

``````_value = int(_amount * 100)
self.ssp_command.CommandData[1] = _value       & 0xff
self.ssp_command.CommandData[2] = (_value>>8)  & 0xff
self.ssp_command.CommandData[3] = (_value>>16) & 0xff
self.ssp_command.CommandData[4] = (_value>>24) & 0xff
``````

It gets a value and multiply by 100 before converting. So:

For a value like “4.05 → 405”, it converts correctly to 95 01 00 00

For a value like “3.05 → 305”, it converts correctly to 31 01 00 00

but…

For a value like “2.05 → 205”, it converts wrongly to CC 00 00 00, which is 204; the correct is CD 00 00 00

Question: how is that possible?

I’m using Python 3.9.2 on a Raspberry Pi 4, with Debian 11 (kernel 6.1.21-v8+)

What does that have to do with bytes? Doesn’t `int(2.05 * 100)` give you 204 already?

1 Like

Don’t test two seperate conversions at the same time. What is going wrong here is the step `_amount``_value`, not `_value` → bytes. `_value` isn’t `205`, it’s `204` because of how floating point numbers work.

1 Like

`round` would be better to use here than `int`, as floating-point errors shouldn’t be large enough to affect rounding to the nearest integer. (`int` itself does not round a floating-point number, but simply takes the integer portion, i.e. drops the fractional part.)

``````>>> round(2.05*100)  # rounds 204.99999999999997
205
``````

Depending on your inputs, there is one thing to be aware of with `round`: it uses “banker’s rounding”, where a result is rounded to the nearest even integer if the argument is half-way in between:

``````>>> round(2.5)
2
>>> round(3.5)
4
``````
3 Likes

Got it.

I fixed by using math.ceil() instead of int(). This gives the correct value and the rest is working fine.

Regards.

So `math.ceil(2.45 * 100)` doesn’t give you 246?

2 Likes

Oh my…

You’re correct. My quick and dirty “solution” is flawed.

The right way is to create a system that preserves the necessary integer value the entire time. For example, by using the desired integer equivalent for `_amount`, and altering other calculations that use `_amount` to divide by 100.

2 Likes

At the end of the day, I just need to convert 2.05 to 205 and 2.45 to 245 and 1.3 to 130 and so on.

So I decided to “convert” the float value to string before the final transformation. Now it’s working fine.

I’m aware that’s not the pythonic or elegant way to solve the issue, but at least it’s a way to circumvent all the complexities of numerical analyses quickly.

What was wrong with @chepner’s suggestion of using `round`?

That’s what you said about the previous attempt, too.

2 Likes

It’s convoluted and probably not efficient, but at least it gives the correct answer:

``````>>> var = 2.05
>>> int(var * 100)
204
>>> from decimal import Decimal as D
>>> D(str(var))
Decimal('2.05')
>>> D(str(var)) * 100
Decimal('205.00')
>>> int(D(str(var)) * 100)
205
``````
1 Like

The `decimal` module provides better ways to convert `float` to `Decimal` values than relying on `str` to produce floating-point literals.

``````# Demonstrate the problem with 2.05 in the first place
>>> decimal.Decimal.from_float(2.05)
Decimal('2.04999999999999982236431605997495353221893310546875')

# A solution that explicitly spells out the desired outcome.
>>> decimal.Context(prec=3).create_decimal_from_float(2.05)
Decimal('2.05')
``````
2 Likes

@chepner How is it better? Please show an example where it is. If I change the number to 12.05, you produce 12.1, that’s worse.

I did; it doesn’t rely on an intermediate conversion to a `str` value. (Or for all I know, it does, and this details are hidden in the implementation of `create_decimal_from_float`. Point is, I’m not re-inventing the wheel.)

No you didn’t. You just got the same correct result. That’s not better, that’s just equally good. And for 12.05, yours is worse.

The point is, I am using a method specifically designed to convert a `float` value to a `Decimal` value.

The problem with 12.05 is that correct `prec` argument depends on the magnitude of the value to convert. Perhaps the `quantize` method is better:

``````>>> [Decimal(v).quantize(Decimal('0.01')) for v in [1.05, 2.05, 3.05, 4.05, 12.05]]
[Decimal('1.05'), Decimal('2.05'), Decimal('3.05'), Decimal('4.05'), Decimal('12.05')]
``````

But my main goal here is not to solve the problem, but to point out that arbitrary string conversions are not the solution.

It’s a redundant alternative constructor, nothing special.

With your prec/quantize attempts it rather looks like you’re the one trying to reinvent the wheel of float’s string conversion.

How is the way via string not the solution? You still haven’t shown a case where it’s wrong.

I’m not trying to do string conversions at all. OP is trying to round (a multiple of) a float to an integer. Somebody else though converting a float to a string was a good way to do that.

I don’t have to show that it is wrong, because I’m not contesting a valid claim that it’s right in the first place. I’m pointing out that, if one chooses to use `Decimal`, there are methods already provided for handling `float` conversions that don’t involve converting it to a string in the first place.

The string conversion is just another way to get to decimal and round the value. Unclear why you have an issue with that and why you think your way is better.