Python etc

Python 3 allows you to make some function arguments keyword-only, meaning they must be passed as (arg=value) rather than just (value). It may be useful to prevent function calls like this: grep(text, pattern, True, False, True), where True, False, True actually means ignore case, don't invert match, pattern is Perl regexp. It would be nice to force the only reasonable form of this call:

grep(text, pattern,
    ignore_case=True,
    perl_regexp=True)

To achieve this result you should place the keyword-only arguments after varargs argument (aka *args):

def grep(
    text, pattern, *args,
    ignore_case=False,
    invert_match=False,
    perl_regexp=False,
):
    pass

If you don't need *args (like in the example), just replace it with a bare asterisk:

def grep(
    text, pattern, *,
    ignore_case=False,
    invert_match=False,
    perl_regexp=False,
):
    pass

If you want to catch both IndexError and KeyError, you may and should use LookupError, their common ancestor. It proved to be useful while accessing complex nested data:

try:
    db_host = config['databases'][0]['hosts'][0]
except LookupError:
    db_host = 'localhost'

Redirects cheatsheet

Imagine you are moving your web-API from HTTP to HTTPS. How do you handle all requests from clients who are not aware they should use HTTPS? You set up redirection rules. What HTTP status code should you use? The choice is usually between 301 Moved Permanently and 302 Found. The first one is permanent (as the status name states) and the second one is one-off and never cached. Moving to HTTPS is usually permanent, so the choice is obvious, it's 301 Moved Permanently. The problem with both 301 and 302 is that they work properly only for HEAD and GET requests. Though all other methods (such as POST) should work as well according to RFC, they don't. A lot of modern HTTP-clients (your favorite browser probably included) make GET requests after the redirection despite the original request method. That became so usual that RFC now explicitly says, that you couldn't rely on the client persisting the method. To fight that problem two other codes were introduced: 303 See Other and 307 Temporary Redirect. 303 says use GET for the new request and 307 means use the same method for the new request. So basically most of the clients do 303 instead of 302 while they should do 307. Sadly, both 303 and 307 are temporary. To make a redirect that both method-persisting and permanent one can use 308 Permanent Redirect, but that code is still experimental. So the correct solution for our HTTP to HTTPS migration is to use 307 Temporary Redirect. 308 is even better, but can't be relied on. Mind that human users usually start an interaction by sending GET request, so the problem with 301 only applies to robots.

>>> exit
Use exit() or Ctrl-D (i.e. EOF) to exit

Ever wonder why is this message displayed once you try to exit interactive Python with just exit or quit? The solution is quite unexpected yet graceful. It's not a special case for interactive shell, it just shows a representation of every result evaluated, and this line is just a representation of exit function. Strictly speaking, you should not use exit in your ever day projects since it was created specifically for interactive shell. Use sys.exit() instead.

Even if you use identifier with brackest as a decorator (@atomic()), it will be called again with a function as an argument: query = atomic(skip_errors=True)(query). There are tons of examples out there on how to create parametrized decorators, let's just move on and write a decorator that has no parameters, but can be called with emptry brackets ().

def atomic(func=None):
    if func is None:
        return atomic
    @functools.wraps(func)
    def wrapped():
        print('BEGIN')
        func()
        print('COMMIT')

    return wrapped

@atomic()
def query():
    print('q')

You can even extract that logic by decorating a decorator:

def unparameterized_decorator(decorator):
    @functools.wraps(decorator)
    def wrapped(func=None):
        if func is None:
            return decorator
        return decorator(func)

    return wrapped

@unparameterized_decorator
def atomic(func):
    @functools.wraps(func)
    def wrapped():
        print('BEGIN')
        func()
        print('COMMIT')

    return wrapped

@atomic()
def query():
    print('q')

Famous Python decorator syntax (@this_one) is a way to call higher-order function. Back then people had to do it manually:

# prior to Python 2.4
def query():
    pass
query = atomic(query)

# now
@atomic
def query():
    pass

Basically, the idenitifer after @ is what to be called. You also can use identifier with brackets (@atomic(skip_errors=True)), that is usually used for parameterized decorators. Something like @decorators.db.atomic(True) also works. Looks like you use any kind of expression as a decorator, but that is not true. @ must be followed by one “dotted named” (meaning something like decorators.atomic) and optionally by one pair of brackets with arguments (just like a function call). So, no @decorators[2] for you. Here is a line from Python grammar:

decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE

iPython supports a number of magic commands that can make your life easier. There are two types of them: line magics and cell magics. Line magics start with % sign, %timeit is a good example:

In [1]: %timeit sum(x**2 for x in range(1000))
243 µs ± 2.31 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Cell magics start with double % sign, look at %%ruby:

In [2]: %%ruby
   ...: 3.times do |x|
   ...:   puts x
   ...: end
   ...:
0
1
2

You can even define custom magics. This is an example magic that helps you ignore an expression result except the very end:

In [3]: from IPython.core.magic import register_line_magic

In [4]: @register_line_magic
   ...: def tail(line):
   ...:     result = repr(eval(line))
   ...:     if len(result) > 100:
   ...:         return '... {}'.format(result[-100:])
   ...:

In [5]: %tail list(range(1000))
Out[5]: '...  980, 981, 982, 983, 984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995, 996, 997, 998, 999]'

All that is also true for Jupyter Notebook.

If dict remembers the order of elements in Python3.6+, why do you need collections.OrderedDict anymore? That's why:

>>> OrderedDict(a=1, b=2) == OrderedDict(b=2, a=1)
False
>>> dict(a=1, b=2) == dict(b=2, a=1)
True

In Python 3 keys, values and items methods of dicts return view objects. They returned lists back in Python 2. The main difference is views don't store all items in memory, but yield them as long as they are requested. It works just fine as long as you are trying to iterate over keys (which you usually are), but you can't access elements by index anymore.

TypeError: 'dict_keys' object does not support indexing

You can argue that you don't really need indexing keys since their order is random, but it's not completely true. First of all, d.keys()[0] can be a proper way to get any key (use next(d.keys()) in Python 3). Second, since Python 3.6 dicts are insertion ordered in CPython and that will be a language feature since Python 3.7.

The thing you usually don't care about is loops and if-blocks don't create scopes in Python (as well as try-blocks, with-block etc.). Because if they were, you won't be able to reassign variables inside the block:

max = 0
for x in lst:
  if x > max:
    max = x  # reassigned

But you do care about it if you try to create some closures since nothing really closures until a scope ends.