How can I format a user selection matrix?

I apologize for being python ignorant – a small push would help.

Suppose there’s a python video processing function, Super().
configSuper, a string, is one of a number of Super() call parameters.
I seek to make configuring configSuper a user-selection matrix, sort of like this:

configSuper = "{"
# To configure, add/move/remove lefthand '#'s"
# +   "gpu:1,full:true,"        #           Use GPU; set precision to whole pixels.
# +   "gpu:1,full:false,"       #           Use GPU;                  whole pixels; limits memory demand when processing very large (e.g. 4K) frames.
# +   "gpu:0,pel:1,"            #                                     whole pixels.
# +   "gpu:0,pel:1,full:true,"  #                                     whole pixels.
# +   "gpu:0,pel:1,full:false." #                                     whole pixels; limits memory demand when processing very large (e.g. 4K) frames.
  +   "gpu:0,pel:2,scale:{"     # -default-                            half pixels (i.e. upscale clipSuper to clipSource via 1x1-to-2x2 pixel duplication).
# +   "gpu:0,pel:4,scale:{"     #                                   quarter pixels (i.e. upscale clipSuper to clipSource via 1x1-to-4x4 pixel duplication); not recommended.
# +       "up:0,"               #           Upscale interim clip method: Bilinear.
# +       "up:1,"               #                                        Bicubic, 4 tap Catmull-Rom.
  +       "up:2,"               # -default-                              Bicubic, 6 tap Wiener -- like Lanczos; sharper.
# +       "down:0},"            #           Downscale result method: Simple 4-pixel average -- like AviSynth's unfiltered SimpleResize; minimum anti-aliasing.
# +       "down:1},"            #                                    Triangle (shifted)     -- like ReduceBy2;                         moderate anti-aliasing.
# +       "down:2},"            #                                    Triangle               -- like BilinearResize;                        more anti-aliasing.
# +       "down:3},"            #                                    Quadratic              --                                        even more anti-aliasing.
  +       "down:4},"            # -default-                          Cubic                  -- like BicubicResize(b=1,c=0);             maximum anti-aliasing.
  +   "rc:0}"                   #           The 'rc' attribute is required by Super(); this merely creates it.

But of course that doesn’t work because of varying indentation (that throws exceptions) and line-spanning concatination (that throws exceptions).
As you can see by the console session below, I can craft something that works and produces the right result (bottom line), but, oh my, it is ugly.
How can I craft a selection matrix that isn’t so ugly? Got any bright ideas/tricks?

>>> configSuper = "{"
>>> #To configure, add/move/remove lefthand '#'s"
>>> #configSuper = configSuper +   "gpu:1,full:true,"        #           Use GPU; set precision to whole pixels.
>>> #configSuper = configSuper +   "gpu:1,full:false,"       #           Use GPU;                  whole pixels; limits memory demand when processing very large (e.g. 4K) frames.
>>> #configSuper = configSuper +   "gpu:0,pel:1,"            #                                     whole pixels.
>>> #configSuper = configSuper +   "gpu:0,pel:1,full:true,"  #                                     whole pixels.
>>> #configSuper = configSuper +   "gpu:0,pel:1,full:false." #                                     whole pixels; limits memory demand when processing very large (e.g. 4K) frames.
>>> configSuper = configSuper +   "gpu:0,pel:2,scale:{"     # -default-                            half pixels (i.e. upscale clipSuper to clipSource via 1x1-to-2x2 pixel duplication).
>>> #configSuper = configSuper +   "gpu:0,pel:4,scale:{"     #                                   quarter pixels (i.e. upscale clipSuper to clipSource via 1x1-to-4x4 pixel duplication); not recommended.
>>> #configSuper = configSuper +       "up:0,"               #           Upscale interim clip method: Bilinear.
>>> #configSuper = configSuper +       "up:1,"               #                                        Bicubic, 4 tap Catmull-Rom.
>>> configSuper = configSuper +       "up:2,"               # -default-                              Bicubic, 6 tap Wiener -- like Lanczos; sharper.
>>> #configSuper = configSuper +       "down:0},"            #           Downscale result method: Simple 4-pixel average -- like AviSynth's unfiltered SimpleResize; minimum anti-aliasing.
>>> #configSuper = configSuper +       "down:1},"            #                                    Triangle (shifted)     -- like ReduceBy2;                         moderate anti-aliasing.
>>> #configSuper = configSuper +       "down:2},"            #                                    Triangle               -- like BilinearResize;                        more anti-aliasing.
>>> #configSuper = configSuper +       "down:3},"            #                                    Quadratic              --                                        even more anti-aliasing.
>>> configSuper = configSuper +       "down:4},"            # -default-                          Cubic                  -- like BicubicResize(b=1,c=0);             maximum anti-aliasing.
>>> configSuper = configSuper +   "rc:0}"                   #           The 'rc' attribute is required by Super(); this merely creates it.
>>> configSuper
'{gpu:0,pel:2,scale:{up:2,down:4},rc:0}'

Use a triple-quoted string:

configSuper = """{
# To configure, add/move/remove lefthand '#'s
# gpu:1,full:true,        # Use GPU; set precision to whole pixels.
# gpu:1,full:false,       # Use GPU; ...
# gpu:0,pel:1,            # ...
gpu:0,pel:2,scale:{...
}
"""

Or you can build a list of lines, and then combine them:

configSuper == [
    "{",
    "# To configure, ...",
    "# gpu: 1, full:true, ...",
    "gpu:0, pel:2, scale: ...",
    ]
configSuper = '\n'.join(configSuper)

A third option would be to put the config into a text file and read it 
in as needed.

with open(‘myconfig.txt’) as f:
configSuper = f.read()

Or you can put the expression in parentheses:

configSuper = (
   "{"
    + "Python doesn't"
           + " care about"
       + " newlines & indentation here"
               + " (but people will so don't be this messy)"
    +"}"
)

Thanks Steven, Petr,

>>> x="""
... { # To configure, add/remove lefthand '#'s
... # 1
...   2
... }
... """
>>> x
"\n{ # To configure, add/remove lefthand '#'s\n# 1\n  2\n}\n"

The above requires a regexp filter to parse out commented lines.

>>> x=[
... "{ # To configure, add/remove lefthand '#'s"
... "# 1"
... "  2"
... "}"
... ]
>>> x='\n'.join(x)
>>> x
"{ # To configure, add/remove lefthand '#'s# 1  2}"

Above: With ‘\n’ removed, parsing is impossible.

>>> x=(
... "{ # To configure, add/remove lefthand '#'s"
... "# 1"
... "  2"
... "}"
... )
>>> x
"{ # To configure, add/remove lefthand '#'s# 1  2}"

Above: With ‘\n’ removed, parsing is impossible.

>>> x=[
... { # To configure, add/remove lefthand '#'s
... # 1
...   2
... }
... ]
>>> x='\n'.join(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: sequence item 0: expected str instance, set found
>>> x
[{2}]

The above produces the desired result but as an array (?), and it throws an exception.

As I hack through python, it would help if there was a way for me to query object methods and attributes without knowing in advance what those methods and attribute names are. Is there a way to do that? All the tutorials I’ve found show how to use methods and attributes, but they don’t show how to query without knowing in advance.

PS: For example, in the method that throws the exception above, I suspect that [{2}] is not a string. It needs to be a string (and without the brackets).

PSS: I guess what I’m asking is: Is there a way to enumerate methods and attributes?

You’re right, [{2}] is not a string, which you can tell from the fact that it’s not surrounded by quotes.
But how do you figure out what it, or any object, is?

>>> thing = [{2}]

The type function[*] will give you the type of an object

>>> type(thing)
<class 'list'>

It’s a list!

That’s what the dir function does.

>>> dir(thing)
['__add__', '__class__', '__class_getitem__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']

Another useful function when you’re exploring in the REPL is help

>>> help(thing)

(opens some documentation on list objects)

Though usually the reference documentation on the web (docs.python.org) is more useful.

Now that you know you’re dealing with a list, you can of course explore further, since you’ll no doubt know (or be able to find out) how to access the elements of a list

>>> thing[0]
{2}
>>> type(thing[0])
<class 'set'>

[*] the “function” type is actually a type

>>> type(type)
<class 'type'>

but that’s really not important right now. dir is very much a function no matter how pedantic you want to be about the word:

>>> type(dir)
<class 'builtin_function_or_method'>

Absolutely!

Keep the Python interactive interpreter, or REPL, open whenever you
work. At the prompt, you can use:

dir(obj)

to get a list of method and attribute names of obj. (“obj” can be any
expression that evaluates to an object.)

You can use type(obj) to see the class or type of the object, and
help(obj) to read some help about the object (sometimes rather
verbose).

If you know the method, you can use that directly:

help(str.upper)  # notice there are no parentheses after upper()

For keywords and operators, you need to wrap them in quotes:

help("==")
help("with")

Or you can use a plain help() with no arguments and follow the
prompts.

Happy hacking!

All works as you describe, Steven, except…

C:\>python
Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32
>>> help
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'help' is not defined
>>> x=1
>>> help(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'help' is not defined
>>> help("==")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'help' is not defined
>>> import help
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'help'

You seem to have a weirdly broken Python implementation. I don’t know
why your help function isn’t working.

Can you check what quit and exit say?

If you use quit() with parentheses, it should actually quit the
interpreter. If you leave the parens out, you should see a message
something like:

Use quit() or Ctrl-D (i.e. EOF) to exit

If you get a NameError, then something very strange is going on. Can you
try this:

import builtins
print(dir(builtins))

and copy and paste the output?

Then:

import site

and if there are no errors, try help again.

Is there any chance you have disabled the site module? Normally you
would do that by running:

python3 -S

but I see you just ran python. On Linux, I would ask you to run

which python

to see exactly what command that corresponds to, but I don’t know what
the Windows equivalent would be.

I submit this triple-quoted string:

  configSuper="""
  { # To configure, add/remove lefthand '#'s
  # gpu:1,full:true,              #           Use GPU; set precision to whole pixels.
  # gpu:1,full:false,             #           Use GPU;                  whole pixels; limits memory demand when processing very large (e.g. 4K) frames.
  # gpu:0,pel:1,                  #                                     whole pixels.
  # gpu:0,pel:1,full:true,        #                                     whole pixels.
  # gpu:0,pel:1,full:false,       #                                     whole pixels; limits memory demand when processing very large (e.g. 4K) frames.
    gpu:0,pel:2,scale:{           # -default-                            half pixels (i.e. upscale clipSuper to clipSource via 1x1-to-2x2 pixel duplication).
  # gpu:0,pel:4,scale:{           #                                   quarter pixels (i.e. upscale clipSuper to clipSource via 1x1-to-4x4 pixel duplication); not recommended.
  #                    up:0,      #           Upscale interim clip method: Bilinear.
  #                    up:1,      #                                        Bicubic, 4 tap Catmull-Rom.
                       up:2,      # -default-                              Bicubic, 6 tap Wiener -- like Lanczos; sharper.
  #                    down:0},   #           Downscale result method: Simple 4-pixel average -- like AviSynth's unfiltered SimpleResize; minimum anti-aliasing.
  #                    down:1},   #                                    Triangle (shifted)     -- like ReduceBy2;                         moderate anti-aliasing.
  #                    down:2},   #                                    Triangle               -- like BilinearResize;                        more anti-aliasing.
  #                    down:3},   #                                    Quadratic              --                                        even more anti-aliasing.
                       down:4},   # -default-                          Cubic                  -- like BicubicResize(b=1,c=0);             maximum anti-aliasing.
    rc:0}                         #           The 'rc' attribute is required by Super(); this merely creates it."""

followed by this:

  import re

Now, if python was javascript, I’d submit this:

  configSuper=configSuper.replace(/#.*| |\n/g,"");

And the result would be this:

  {gpu:0,pel:2,scale{up:2,down:4},rc:0}

But python isn’t javascript, so how do I do such a regexp in python?

PS: In fact, I think the javascript syntax also works in ‘C’.

PPS: Yes, I see the ‘sub()’ method, but it doesn’t make sense to me.

>>> quit
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'quit' is not defined
>>>
C:\>python
Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32
>>> import builtins
>>> print(dir(builtins))
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'WindowsError', 'ZeroDivisionError', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'breakpoint', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']
>>> import site
>>> help
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'help' is not defined
>>>
C:\>python
Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32
>>> print(dir(builtins))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'builtins' is not defined
>>>

On this page:
https://docs.python.org/3/library/re.html
is found this:

" re.sub(pattern, repl, string, count=0, flags=0)
"
"    Return the string obtained by replacing the leftmost non-overlapping occurrences of pattern in string by the replacement repl.
If the pattern isn’t found, string is returned unchanged.
repl can be a string or a function; if it is a string, any backslash escapes in it are processed.
That is, \n is converted to a single newline character, \r is converted to a carriage return, and so forth.
Unknown escapes of ASCII letters are reserved for future use and treated as errors.
Other unknown escapes such as \& are left alone.
Backreferences, such as \6, are replaced with the substring matched by group 6 in the pattern.
For example:
"    >>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
"    ...        r'static PyObject*\npy_\1(void)\n{',
"    ...        'def myfunc():')
"    'static PyObject*\npy_myfunc(void)\n{'"

The explanation paragraph is superfluous and just confuses.
The example…
Line 1 appears to be the definition of a pattern. I understand that ‘r’ in r’<< s t r i n g >>’ means raw text (but I don’t really know what that means). I don’t know what “def” is all about. I have no clue what the example is trying to illustrate.
Line 2 is a complete mystery.
Line 3 is a complete mystery.
Line 4, beginning with “'static”, is completely mystifying.
It is documentation written by codesmiths for codesmiths.

Looking over what I’ve written here it occurs to me that perhaps python is not so object-oriented as I thought. In an object oriented language, a regular expression is a property of a method of a string object, thusly: myString.replace() – a method – can have a regular expression as a property, thusly: myString.replace(/…/, myReplacementString). It appears that in python, I have to take the string object to a regular expression function, thusly: re.sub(myString…).

Is that correct? I can live with it, but it seems strange and ‘non-object’.

A lot of the Python standard library is in a more functional style (e.g. the length of a list is len(my_list), not my_list.length). In some cases, like re, there is also a more object-oriented style using compiled regular expression objects.

pattern = re.sub(r'...')
result = pattern.sub(replacement, string)

However, the real difference between Python and JavaScript here is that in Python Regular Expression support is not part of the string class, but of a separate module. There’s no reason a Python string should know anything about the existence of regular expressions, so it doesn’t.

raw strings avoid parsing escape sequences. The string “\n” is a newline character. The string r"\n" is a backslash followed by a letter n. These can be useful since the regular expression syntax often uses backslashes.

Line 1 is the pattern that it tries to match.
Line 2 is the replacement string that will replace the match
Line 3 is the string the pattern is run against
Line 4 is the (string) output of the function.

Thank you BowlOfRed – note: I don’t know how to reply specifically to your reply or how to quote your text, so this will have to do.

What is the example trying to show? I have no idea. In other words, what is the use case?

Thomas,

Thank you. So python is not an object language. It’s functions. That’s okay. I can live with that. I have to live with that: I’m using a video processing system, Vapoursynth, with a python run-time environment supporting python scripts.

PS: I have about 400 lines of Windows cmd script (aka batch language) that I’d really like to turn into python. So, I really need to learn python.

(I seem to have 2 separate threads now, running in the space of one thread)

In the example

      I have no clue why 'def' is there
     /  One or more spaces
    /  /                      One alpha or underscore followed by zero or more alpha or num or underscore
   /  /                      /
def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):
Are those parentheses capturing? Why is there a colon?

Oh, dear, the spaces in what is supposed to be unformatted text are not preserved, so constructing what I have in the box was difficult. I hope it displays okay on your computer.

@BowlOfRed,

Nope. Lines 2 3 & 4 just look like jibberish to me.

It’s an extended regular expression. It’s supposed to look like jibberish, and all the symbols mean what you’d expect them to mean in an extended regular expression. d matches a literal d, (...) captures a group, : matches a literal :, and so on. Just like in Perl or JavaScript.

If you want to understand the example in the re docs, I suggest you try to reproduce the example in the REPL, and make little changes to the arguments. See what the effect of these changes is, and you should get a better feeling of what’s going on.

If you want to solve your original problem, I commend to you Petr’s answer: