Python etc

​​The script pydoc can be used to see documentationand docstrings from the console:

$ pydoc3 functools.reduce | cat
Help on built-in function reduce in functools:

functools.reduce = reduce(...)
    reduce(function, sequence[, initial]) -> value

    Apply a function of two arguments cumulatively to the items of a sequence,
    ...

Also, you can specify a port with -p flag, and pydoc will serve the HTML documentation browser on the given port:

$ pydoc3 -p 1234
Server ready at http://localhost:1234/
Server commands: [b]rowser, [q]uit
server>

__slots__ can be used to save memory. You can use any iterable as __slots__ value, including dict. AND Starting from Python 3.8, you can use dict to specify docstrings for slotted attributes __slots__:

class Channel:
  "Telegram channel"
  __slots__ = {
    'slug': 'short name, without @',
    'name': 'user-friendly name',
  }
  def __init__(self, slug, name):
    self.slug = slug
    self.name = name

inspect.getdoc(Channel.name)
# 'user-friendly name'

Also, help(Channel) lists docs for all slotted attributes:

class Channel(builtins.object)
 |  Channel(slug, name)
 |  
 |  Telegram channel
 |  
 |  Methods defined here:
 |  
 |  __init__(self, slug, name)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  name
 |      user-friendly name
 |  
 |  slug
 |      short name, without @

As we said, comprehensions compiled into functions. That means, we can take a types.CodeType object for a comprehension, wrap it into types.FunctionType and get a function.

import types

def make():
    [x*2 for x in _]

code = make.__code__.co_consts[1]
func = types.FunctionType(code, globals())

# call the function!
func(iter(range(5)))
# [0, 2, 4, 6, 8]

The module codecs provides encode and decode function to encode and decode (wow!) text in different encodings, like UTF8, CP1251, Punycode, IDNA, ROT13, execute escape sequences, etc.

codecs.encode('hello, @pythonetc', 'rot13')
# 'uryyb, @clgubargp'

codecs.encode('\n', 'unicode_escape')
# b'\\n'

codecs.encode('привет, @pythonetc', 'punycode')
# b', @pythonetc-nbk5b4b7gra3b'

codecs.encode('привет, @pythonetc', 'idna')
# b'xn--, @pythonetc-nbk5b4b7gra3b'

codecs.encode('привет, @pythonetc', 'cp1251')
# b'\xef\xf0\xe8\xe2\xe5\xf2, @pythonetc'

We can offer big data, highload, modern development techniques, bleeding-edge data science technologies and a really good team (and I mean it). We expect you to know Python and be smart and cunning. The team is based in Moscow but remote work is fully supported. Russian is required though. If you can recommend your friend, we are ready to pay 100 000 roubles if he is hired. Contact @pushtaev for any details.

Hi there! My team and I are looking for new developers once again. We are making a voice assistant, which is currently growing rapidly. Also our first smart speaker is already released:

Docstring is a string that goes before all other statements in the function body (comments are ignored):

def f(): 'a'
f.__doc__  # 'a'

def f(): r'a'
f.__doc__  # 'a'

It must be a static unicode string. F-strings, byte-strings, variables, or methods can't be used:

def f(): b'a'
f.__doc__  # None

def f(): f'a'
f.__doc__  # None

a = 'a'
def f(): a
f.__doc__  # None

def f(): '{}'.format('a')
f.__doc__  # None

Of course, you can just set __doc__ attribute:

def f(): pass
f.__doc__ = f'{"A!"}'
f.__doc__ # 'A!'

Accidentally, yield can be used in generator expressions and comprehensions:

[(yield i) for i in 'ab']
# <generator object <listcomp> at 0x7f2ba1431f48>

list([(yield i) for i in 'ab'])
# ['a', 'b']

list((yield i) for i in 'ab')
# ['a', None, 'b', None]

This is because yield can be used in any function (turning it into a generator) and comprehensions are compiled into functions:

>>> dis.dis("[(yield i) for i in range(3)]")                                                                                                                                             
0 LOAD_CONST     0 (<code object <listcomp> ...>)
2 LOAD_CONST     1 ('<listcomp>')
4 MAKE_FUNCTION  0
...

This produces a warning in Python 3.7 and will raise SyntaxError in python 3.8+. However, yield inside lambda still can be used:

a = lambda x: (yield x)
list(a(1))
# [1]

Some functions can accept as an argument value of any type or no value at all. If you set the default value to None you can't say if None was explicitly passed or not. For example, the default value for argparse.ArgumentParser.add_argument. For this purpose, you can create a new object and then use is check:

DEFAULT = object()

def f(arg=DEFAULT):
  if arg is DEFAULT:
      return 'no value passed'
  return f'passed {arg}'

f()     # 'no value passed'
f(None) # 'passed None'
f(1)    # 'passed 1'
f(object()) # 'passed <object object at ...>'

The module unittest.mock provides a sentinel registry to create unique (by name) objects for the testing purpose:

sentinel.ab.name # 'ab'
sentinel.ab is sentinel.ab  # True
sentinel.ab is sentinel.cd  # False

Creation of class instance is done by __call__ method of object class (provided by metaclass type) and practically includes only 2 steps: 1. Call the __new__ method to create an instance. 2. Call the __init__ method to set up the instance.

class A:
  def __new__(cls, *args):
    print('new', args)
    return super().__new__(cls)

  def __init__(self, *args):
    print('init', args)

A(1)
# new (1,)
# init (1,)

A.__call__(1)
# new (1,)
# init (1,)

So, if you want to create an instance without executing __init__, just call __new__:

A.__new__(A, 1)                                                                         
# new (1,)

Of course, that's a bad practice. The good solution is to avoid a heavy logic in __init__ so nobody wants to avoid calling it.

The module zipapp can pack a python module into a zip archive that can be executed directly by a Python interpreter. It is a good way to ship CLI tools:

$ mkdir example
$ echo 'print("hello, @pythonetc!")' > example/__main__.py
$ python3 -m zipapp example
$ python3 example.pyz      
hello, @pythonetc!

types.DynamicClassAttribute is a decorator that allows having a @property that behaves differently when it's called from the class and when from the instance.

from types import DynamicClassAttribute

class Meta(type):
    @property
    def hello(cls):
        return f'hello from Meta ({cls})'

class C(metaclass=Meta):
    @DynamicClassAttribute
    def hello(self):
        return f'hello from C ({self})'

C.hello
# "hello from Meta (<class '__main__.C'>)"

C().hello
# 'hello from C (<__main__.C object ...)'

Practically, it is used only in enum to provide name and value properties for instances while still allowing to have name and value class members:

import enum

class E(enum.Enum):
    value = 1

E.value
# <E.value: 1>

E.value.value
# 1

The module enum provides a way to build an enumerable class. It is a class with a predefined list of instances, and every instance is bound to a unique constant value.

from colorsys import rgb_to_hls
from enum import Enum

class Color(Enum):
    RED = (1, 0, 0)
    GREEN = (0, 1, 0)
    BLUE = (0, 0, 1)

    @property
    def hls(self):
        return rgb_to_hls(*self.value)

Color
# <enum 'Color'>

Color.RED
# <Color.RED: (1, 0, 0)>

Color.RED.name
# 'RED'

Color.RED.value
# (1, 0, 0)

Color.RED.hls
# (0.0, 0.5, 1.0)

type(Color.RED) is Color
# True

Python 3.9.0 is released! Notable features: + Type hints for built-in generic types + ast.unparse + Relaxing grammar restrictions on decorators + Merging dicts + str.removeprefix + Meta information in type hints

Context manager contextlib.nullcontext is helpful when a block of code not always should be executed in a context. A good example is a function that works with a database. If a session is passed, the function will use it. Otherwise, it creates a new session, and does it in a context to guarantee fallback logic to be executed:

from contextlib import nullcontext

def get_user(id, session=None):
    if session:
        context = nullcontext(session)
    else:
        context = create_session()
    with context as session:
        ...

Another example is optional suppressing errors:

from contextlib import suppress

def do_something(silent=False):
    if silent:
        context = suppress(FileNotFoundError)
    else:
        context = nullcontext()
    with context:
        ...

It was added in Python 3.7. For earlier Python versions DIY:

from contextlib import contextmanager

@contextmanager
def nullcontext(value=None):
    yield value

Another option is to use ExitStack.

The point of the post above was that for some simple tasks there are many ways to do it (and some ways are good only in some cases). Also, the number of possible solutions grows as the language evolves. Another good example is concatenation. You can join 2 strings with f-strings, str.format, str.join, +, and so on. Thinking about these ways, even not suitable for daily usage, helps better language understanding. Our amazing subscribers decided to take the challenge. Below are some not suitable for work but fan solutions how to convert int to str. @dedefer:

import io
with io.StringIO() as f:
    print(n, end='', file=f)
    n_str = f.getvalue()

Evgeny:

import ctypes
ctypes.cdll.LoadLibrary('libc.so.6').printf(b'%d', n)

If you're going to use solutions below on the production, keep in mind that it doesn't work with negative numbers. @oayunin:

from math import log10
''.join(['0123456789'[(n // 10 ** i) % 10] for i in range(int(log10(n)), -1, -1)])

A similar solution from @apatrushev:

''.join(chr(t + 48) for t in (n // 10**x % 10 for x in reversed(range(int(math.log(n,10)) + 1))) if t)

A similar solution with lambdas from @antonboom:

(lambda n:
  ''.join(
    chr(ord('0') + (n // (10**i)) % 10)
    for i in range(math.ceil(math.log(n, 10)) - 1, -1, -1)
  )
)(n)

One more from @oayunin:

from subprocess import check_output
with open('/tmp/tmp.txt', 'w') as f:
  for x in range(n):
    f.write(' ')
check_output(['wc', '-c', '/tmp/tmp.txt']).decode().split()[0]

@orsinium:

import sys
import subprocess
cmd = [sys.executable, '-c', 'print(len("' + '*' * n + '"))']
subprocess.run(cmd, capture_output=True).stdout.strip().decode()

@maxvyaznikov:

chars = []
while n > 0:
    digit = n % 10
    n = int(n / 10)
    chars.append(chr(ord('0') + digit))
print(''.join(chars[::-1]))