# Add basic boolean indexing to list

**URL:** <https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146>\
**Category:** Ideas\
**Created:** [April 18, 2022, 6:23pm UTC](https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146 "2022-04-18T18:23:21Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![chalggg](https://avatars.discourse-cdn.com/v4/letter/c/6a8cbe/32.png) [@chalggg](https://discuss.python.org/u/chalggg)\
**Post date:** [April 18, 2022, 6:23pm UTC](https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146/1 "2022-04-18T18:23:21Z")

</div>

Many Python libraries related to data science support concept of boolean indexing ([Indexing on ndarrays — NumPy v2.3 Manual](https://numpy.org/doc/stable/user/basics.indexing.html#boolean-array-indexing)). I would like to propose limited version of this concept that could be useful to regular Python programmers.

Sometimes one doesn’t want to (or can’t because of business requirements) import libraries like pandas or numpy with all their bells and whistles, but still wants performant and concise way of filtering lists.

# Idea

My idea is to allow indexing lists using lists of booleans of the same length.

To give an example:

```python
a=[1,2,3]
b=[True,False,True]

print(a[b])#output: [1,3]

```

Same would also work for assignments. Assigned list length would need to be equal to count of True in indices:

```python
a=[1,2,3]
b=[True,False,True]
a[b]=[-1,-2]

print(a)#output: [-1,2,-2]

```

I wrote a short sample [patch](https://gist.github.com/chalggg/7ef059f2492a2b242fec3ed91530b3fc) specifically for lists. I guess concept might be extensible to other sequences, but wanted to see if idea gets any traction first.

# Benchmark

Using the included patch I wrote short benchmarks;

1. “getitem”

2. “setitem”

My code is proof of concept so I expect final version would be slower due to safety checks, yet to me conciseness and performance benefits look interesting.

If there is interest in this concept I could spend some time on implementation. Let me know what you think.

---

<div class="post-metadata">

**Author:** ![malemburg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/malemburg/32/50_2.png) [@malemburg](https://discuss.python.org/u/malemburg)\
**Post date:** [April 18, 2022, 7:19pm UTC](https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146/2 "2022-04-18T19:19:08Z")

</div>

Doesn’t [itertools.compress()](https://docs.python.org/3/library/itertools.html#itertools.compress)  
already provide this ?

---

<div class="post-metadata">

**Author:** ![chalggg](https://avatars.discourse-cdn.com/v4/letter/c/6a8cbe/32.png) [@chalggg](https://discuss.python.org/u/chalggg)\
**Post date:** [April 18, 2022, 7:33pm UTC](https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146/3 "2022-04-18T19:33:46Z")

</div>

This one is nice, thanks for providing link. It also has great performance almost identical to my custom solution.

However I think it does not work for scenario where indexing is done on the left side of assignment, allowing to replace only some elements.

Python already has similar notation for slices:

```python
a=[1,2,3,4]
a[2:6]=[0,0,0,0]
print(a) # output: [1, 2, 0, 0, 0, 0]

```

so I thought extending it would feel quite natural even for people who are not daily pandas/numpy/etc. users.

---

<div class="post-metadata">

**Author:** ![Gouvernathor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/gouvernathor/32/7957_2.png) [@Gouvernathor](https://discuss.python.org/u/Gouvernathor)\
**Post date:** [July 8, 2022, 1:14am UTC](https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146/4 "2022-07-08T01:14:04Z")

</div>

Maybe allowing `slice` to be subclassed or imitated somehow ?  
That is, having a method (` __slice__ ` ?) of the slice object return an ordered set of indices, and that method could be overridden. Natively it would be something like `return [k for k in range(self.start, self.end, self.step)]`. And objects implementing that method, or possibly only subclasses of `slice`, would be used that way by native types.

---

<div class="post-metadata">

**Author:** ![guido](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/guido/32/21_2.png) [@guido](https://discuss.python.org/u/guido)\
**Post date:** [July 10, 2022, 10:25pm UTC](https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146/5 "2022-07-10T22:25:54Z")

</div>

Since `bool` is a subclass of `int` I think we’re on very thin ice if you want `a[[1, 3, 5]]` to have one meaning and `a[[True, True, False]]` a different meaning. The latter would be expected to behave the same as `a[[1, 1, 0]]`. Even if numpy/pandas make this distinction, this is pretty fundamental to Python’s `bool` – we cannot easily stop it from subclassing `int` without breaking vast amounts of user code.

---

<div class="post-metadata">

**Author:** ![chalggg](https://avatars.discourse-cdn.com/v4/letter/c/6a8cbe/32.png) [@chalggg](https://discuss.python.org/u/chalggg)\
**Post date:** [July 11, 2022, 7:41pm UTC](https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146/6 "2022-07-11T19:41:30Z")

</div>

That’s a good point. As a person that heavily uses numpy/pandas daily I never thought of it this way. Yet speed benefits seem pretty significant to me. What about creating a separate function (or list’s method) that works like itertools.compress, but for assignments?

Sth like this:

```auto
a=[1,2,3,4]
b=[True,False,True,False]
replacement_values=[0,0]
a.boolean_replace(b,replacement_values)
print(a) #output: [0,2,0,4]

```

(the name of course is a material for separate discussion).

---

<div class="post-metadata">

**Author:** ![guido](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/guido/32/21_2.png) [@guido](https://discuss.python.org/u/guido)\
**Post date:** [July 11, 2022, 8:29pm UTC](https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146/7 "2022-07-11T20:29:51Z")

</div>

Is there a benefit to making it a method on mutable sequences? That would imply that each mutable sequence implementation would have to implement its own version of this. If it was a helper function (taking a mutable sequence as argument) generic support would be easier obtained. The helper function could still have a special case for lists for performance, if needed.

I do doubt that this would get much use – numpy arrays are just different from lists – so I am not excited about it, and I don’t see much excitement in this thread either.

---

<div class="post-metadata">

**Author:** ![CAM-Gerlach](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cam-gerlach/32/3688_2.png) [@CAM-Gerlach](https://discuss.python.org/u/CAM-Gerlach)\
**Post date:** [July 24, 2022, 6:47pm UTC](https://discuss.python.org/t/add-basic-boolean-indexing-to-list/15146/8 "2022-07-24T18:47:46Z")

</div>

Personally, I’m not sure I see much of a use case that isn’t already handled by Numpy/pandas/xarray in the types of real-world use that would need this, and they already have their own solutions (which I use heavily). But as a start, you could write a helper function along the lines of what @guido suggested, publish that as a package on PyPI, and see what sort of uptake/feedback you get.