Why not make str.join() coerce the items in its iterables

Definitely, one of the most boring and repetitive patterns I find myself typing is
", ".join(str(i) for i in mything). If str.join would call __str__ in each element in the iterator it is given, this kind of code could be simply ", ".join(mything).

I don’t see this change breaking anything, and I see a lot of real-world code getting rid of a pain to get a pretty output.

Also, I can’t see this having any compatibility issues, since, up to know everyone has to painstakingly iterate over their sequence and manually call str() on it.

What do you say?

7 Likes

This has been suggested in the past and rejected because it will hide bugs I recall.

You cannot know how to convert all the objects into strings with the correct formatting in the general case. Objects that do not have a __str__, for example, fall back to __repr__ that is often not what is expected.

1 Like

This seems a weak counter-argument.
It would still work as expected in 99% of the cases - for both novice and expert Python programmers (experts will know and be expecting __repr__ to be called, actually).
The effect, anyway, is not different from what one gets when printing an arbitrary object,
and we don’t see people being hurt by that behavior.

And still, it just make a shorter, garbage free, version of the same expression possible -
one is still free to implement a generator expression if the string conversion is to be
more complicated than calling str() on the object.

Maybe, an optional flag “strict” defaulting to False, that would cause it to type-error
as the current implementation does, just in case someone’s code depends
on it failing like that.

Do you have links to where the discussion has ocured before?
I can dig there and see if there are any actual arguments against this

3 Likes

This was most recently proposed just two weeks ago on this very forum:

2 Likes

And you would save an incredible five characters! What a difference that makes! Right?

Or you can write a helper function once, and save about 20 characters: join(mything).

What you are proposing is a form of weak typing. Python is a strongly-typed language, with very little weak typing, and (almost?) all of that is in automatic coercion of different numeric types.

It is never automatic coercion of strings to ints, or ints to strings (for example). Weak typing has the benefit of saving keystrokes. Strong typing has the benefit of preventing bugs. Some languages are even stronger than Python, and don’t allow automatic numeric coercion.

In the last few weeks, this same issue has come up at least twice, and I responded both times:

Unless you have a new and better argument than “save some keystrokes”, I don’t think there is much to discuss here that hasn’t already been discussed before.

  • this proposal introduces weak typing;
  • its not clear that we should want str rather than repr or ascii or something else;
  • string joining is a low-level API, and should be explicit;
  • writing your own convenience function is literally two lines of code.

If you do have a strong argument for weak typing (heh pun intended) please tell us.

11 Likes

Oh well, the recetn discussion leads here - Make str.join auto-convert inputs to strings. · Issue #87701 · python/cpython · GitHub - where some people seems to be pretty strong opinionated that Python is now a “grown up language” no longer fit for “consenting adults”, where typing should be explicit.

Pythonic behaviors seem reserved to legacy APIs.

It does not look like we are heading into a fun place.

Serious - not trying to argue with people that come up saying “I am -10 on this because it could…print a wrong string on someone’s screen one day”. :frowning:

6 Likes

I agree this is a great idea. Besides making the code nicer, we can also provide a speedup over user code to coerce the input to strings. FWIW, I had run a Twitter poll on the subject. 3,370 voted and 78.1% want to have this change approved.

Regarding worries about explicit vs implicit, experience with print() has shown this to be non-issue. One respondent did a nice job of summarizing why:

Given that I have already chosen that the final output should be a string, the explicitness criterion is already fulfilled. Thus, there is no reason to jump all the arguments through a hoop.

13 Likes

I don’t see where you’re getting this from. Are you saying that Python’s philosophy has changed from how it was in some distant past when it was a toy language, and now that it’s a real language, … no, I can’t figure out what you mean by the “consenting adults” part.

There’s a major difference between print() and str.join(). The latter is a low level operation on arbitrary sequences/iterables, which are expected to provide string elements (not providing these is an error, which code may very well rely on to prevent data corruption). The former explicitly asks to stringify the arguments, so that they can be printed to stdout.

The argument would work, if we’d add a new builtin or some helper concat() or perhaps a new constructor str.concat() which also explicitly asks for stringification of the arguments, in the same way print() works.

It would also make a lot of sense, to have it use almost the same signature as print() does, i.e.

concat(*objects, sep=' ', end='')

returning the concatenated string sep.join([str(x) for x in objects]) + end, only faster :slight_smile:

As a side effect, the function would also resolve the weirdness many people find in having to write ', '.join(list) to build a string from several other strings.

8 Likes

Is it wise to support idioms where devs rely on others to to raise their exceptions? I would rather encourage the devs explicitly check input, or not care about it on the first place.


It’s far more common to have an iterable of objects to string-concatenate. Perhaps a single-argument form as well?

concat(iter, sep=' ', end='')

I would also prefer on of the names cat or strcat.

For what it’s worth, the cleaner way to write this is ', '.join(map(str, iterable))

7 Likes

For the (also in my experience vanishingly scarce) use case where you want to ensure that all input to str.join are strings, you could add a strict=True argument.

1 Like

This was @malemburg’s point – it’s a fine idea for print() becasue that is exactly about what might end up on a screen.

I rarely use str.join() to print it to the screen – I use it to generate formatted data to write to files, or …

All that being said, I agree with @rhettinger: it’s a string method, you are already explicitly wanting strings.

Of course, if a change is made, the type-checking folks will need to decide how to type it – and they could be more strict if they want …

Hmm – I can’t find str in typeshed – but I’ll bet it’s typed: Iterable[str]

I do a lot of building HTTP headers in python for teest code and a bytes version of concat (strcat) would be a nice to have function.

concat((b'Header: value', b'Header2: value2'), sep=b'\r\n', end=b'\r\n\r\n')
1 Like

There are lots of string methods where we explicitly want strings. Should we auto-convert all of the others too?


address = "156 Main Street"

address.startswith(1)  # returns True

address.replace(56, 9*11)  # returns "199 Main Street"

Other languages do this sort of thing all the time. Weak typing like this is a perfectly cromulent design for programming languages, but does it fit Python? I don’t think it does.

3 Likes

It’s a judgement call – I’m not going to push for it.

But a flag seems harmless and convenient:

def join(self, iterable, coerce_to_strings=False)
    # Default to False for backward compat
    # flag name to be bikeshed
    if coerce_to_strings:
        iterable = (str[i] for i in iterable)
    ...
", ".join(an_iterable, True)
1 Like

I already see people writing ", ".join(a, b) expecting to get a and b joined with , often enough that linters will start requiring the optional argument to the join() method to be specified by name. Thus we are going from the fairly self explanatory and concise

", ".join(map(str, an_iterable))

to

", ".join(an_iterble, coerce_to_strings=True)

I don’t really see this as an improvement.

4 Likes

This is a very common path for proposals to take.

  1. “Hey, we should make this big change because it’d be convenient in a few places!”
  2. “That’s a bad idea, and it would break existing code.”
  3. “Then let’s make a boolean flag for it!”

Adding a flag does mean that the change won’t break existing code, but it’s also a VERY ugly way to do things. As soon as you add this flag to one method, you’re going to get people asking for it on every other method as well. It’s not even clear that this functionality is worth having, much less that it’s worth polluting the API with a kwonly parameter all over the place.

Is it really THAT hard to write your own strjoin function that maps str over the iterable and then joins the results?

3 Likes

I’m not a big fan of slippery slope arguments – but I don’t care much about this idea, either. I’m done.

1 Like