python - argparse: flatten the result of action='append'

Question

I'd like to make a script that supports an argument list of the form

./myscript --env ONE=1,TWO=2 --env THREE=3

Here's my attempt:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
    '--env',
    type=lambda s: s.split(','),
    action='append',
)
options = parser.parse_args()
print options.env

$ ./myscript --env ONE=1,TWO=2 --env THREE=3
[['ONE=1', 'TWO=2'], ['THREE=3']]

Sure I can fix this in postprocessing:

options.env = [x for y in options.env for x in y]

but I'm wondering if there's some way to get the flattened list directly from argparse, so that I don't have to maintain a list of "things I need to flatten afterwards" in my head as I'm adding new options to the program.

The same question applies if I were to use nargs='*' instead of type=lambda....

import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
    '--env',
    nargs='+',
    action='append',
)
options = parser.parse_args()
print options.env

$ ./myscript --env ONE=1 TWO=2 --env THREE=3
[['ONE=1', 'TWO=2'], ['THREE=3']]

Answer 1

Edit: Since Python 3.8, the "extend" is available directly in stdlib. If you only have to support 3.8+ then defining it yourself is no longer required. Usage of stdlib "extend" action is exactly the same way as this answer originally described:

>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> _ = parser.add_argument('--env', nargs='+', action='extend')
>>> parser.parse_args(["--env", "ONE", "TWO", "--env", "THREE"])
Namespace(env=['ONE', 'TWO', 'THREE'])

---

Unfortunately, there isn't an extend action provided in ArgumentParser by default. But it's not too hard to register one:

import argparse

class ExtendAction(argparse.Action):

    def __call__(self, parser, namespace, values, option_string=None):
        items = getattr(namespace, self.dest) or []
        items.extend(values)
        setattr(namespace, self.dest, items)


parser = argparse.ArgumentParser()
parser.register('action', 'extend', ExtendAction)
parser.add_argument('--env', nargs='+', action='extend')

args = parser.parse_args()
print(args)

Demo:

$ python /tmp/args.py --env one two --env three
Namespace(env=['one', 'two', 'three'])

The lambda you have in your example is somewhat outside the intended use-case of the type kwarg. So, I would recommend instead to split on whitespace, because it will be a pain to correctly handle the case where , is actually in the data. If you split on space, you get this functionality for free:

$ python /tmp/args.py --env one "hello world" two --env three
Namespace(env=['one', 'hello world', 'two', 'three'])