Passing a string containing keyword arguments to a function, combining into list comprehension

Python Help
1

Having encountered the difficulty of passing a string of keyword arguments into a function I have found the solution below by converting the string into a dict. However I am having some difficulty combining the for...in loop in the str2dict() function into a one line list comprehension, if that is even possible?

import ast

def myFunc(arg1=None, arg2=None, arg3=None, arg4=None):
    print(f"{arg1=}, {arg2=}, {arg3=}, {arg4=}")

def str2dict(str_in):
    dict_out = {}
    for pair in str_in.split(','):
        k, v = (pair.strip().split('=', maxsplit=1))
        dict_out[k.strip()] = ast.literal_eval(v)
    return dict_out

str_in = "arg1 = 'var1', arg2 = 2, arg3 = True, arg4='var4'"
dict_in = str2dict(str_in) # {'arg1': 'var1', 'arg2': 2, 'arg3': True, 'arg4': 'var4'}
myFunc(**dict_in)
2

Not that its advisable, here’s a list dictionary comprehension version

def str2dict(str_in):
    return {
        k.strip(): ast.literal_eval(v)
        for pair in str_in.split(',')
        for k, v in [pair.strip().split('=', maxsplit=1)]
    }
3

That’s no list comprehension.

4

Thanks @inventshah that’s what I have spent the past two days trying to get to. Looks like I had the two for loops the wrong way around and missing the list [] brackets.

I know ast.literal_eval is not preferred but it is the only way, in the current version of Python, I have found to achieve this.

5

Why would you have a string of keyword arguments? I’m curious to know your use case.

6

The function call is to mysql.connector.connect() but the arguments come from a user configuration file elsewhere in the form of:

db_config = "pool_name = '<name>', pool_size = 3, host='<hostName>', database = '<dbName>', user = '<userName>', password = '<password>' "

Turns out that what I thought should be the simple task of passing a string of keyword arguments into a function in Python is a lot more difficult than I had realised.

7

What you have isn’t really a problem of passing keyword arguments to a function, it’s a problem of defining the structure of that database configuration line. Do you have any influence over that? Can you mandate ANYTHING about that string? Use that to make your life easier. If you can’t, try to at least figure out what the rules are.

And of course, if you have the option to split that into several separate fields, your life will be far easier.

8

It all works fine now using the str2dict() function above, I’m just surprised at how difficult this has been to do in Python.

9

Why? You are trying to implement your own parser for configuration files. It would be much simpler to use existing tools, like configparser or tomllib from the standard library for this purpose.

Just for completeness: The code presented above by inventshah is not production quality. Just try to have a password that contains a comma.

10

Oh well… back to the drawing board then!

Does configparser allow commas in value strings?

I wonder if there is some way I can escape commas or perhaps replace them with another character, then replace that back again at the end.

Edit:
I could be wrong but it seems that configparser cannot decode a comma separated string into a list.

11

Makes me wonder how Python parses keyword function arguments itself?

12

It uses a PEG parser on the grammar described at 10. Full Grammar specification — Python 3.14.7 documentation.

See Home - pegen documentation.

13

Yes, but before parsing using the PEG parser, the source is decomposed into separate tokens. This feature is available using the tokenize module:

>>> import tokenize
>>> s="""user='myself', password='1,2'"""
>>> [t.string for t in tokenize.generate_tokens(s.splitlines().__iter__().__next__)]
['user', '=', "'myself'", ',', 'password', '=', "'1,2'", '', '']
>>>

That module expects that the input string is valid Python code, so the shown code is actually not guaranteed to work.

14

In it’s simplest form all I really want is for that Python PEG parser to remove the two outer quotes before then parsing the string in exactly the same way as it does for the text in between brackets.

Edit:
Something which in another language might be described as Macro Substitution.

15

So on the face of it tokenize appears to work ok with commas:
my test string of:
str_in = "arg1 = 'var1', arg2 = 2, arg3 = True, arg4='va,r4'"
becomes:
['arg1', '=', "'var1'", ',', 'arg2', '=', '2', ',', 'arg3', '=', 'True', ',', 'arg4', '=', "'va,r4'", '', '']

Question now is how do I pass this string through the PEG Parser.

16

Is your goal to convert a string which represents arguments to a Python function into “real” arguments? So like foo(x=1) -> "x=1" then back to something where we can call foo? Here’s an idea, but I suspect this isn’t the best path forward

str_in = "arg1 = 'var1', arg2 = 2, arg3 = True, arg4='va,r4'"
def argskwargs(*args, **kwargs): return args, kwargs
args, kwargs = eval(f"argskwargs({str_in})", {"argskwargs": argskwargs})
myFunc(*args, **kwargs)
17

Anything involving eval and an external source of text is unlikely to be the best path forward, yeah.

18

As does:

str_in = "arg1 = 'var1', arg2 = 2, arg3 = True, arg4='va,r4'"
eval("myFunc(" + str_in + ")")

but uses the dreaded eval() which I am advised is the work of the devil.