Python etc
Regular tips about Python and programming in general Owner — @pushtaev The current season is run by @orsinium Tips are appreciated: https://ko-fi.com/pythonetc / https://sobe.ru/na/pythonetc © CC BY-SA 4.0 — mention if repost
إظهار المزيد6 179
المشتركون
-324 ساعات
-127 أيام
-4830 أيام
- المشتركون
- التغطية البريدية
- ER - نسبة المشاركة
جاري تحميل البيانات...
معدل نمو المشترك
جاري تحميل البيانات...
Great news everyone! We extracted all our recent posts as Markdown, organized them, and made them more accessible. Now we have:
* 🌐 Website: pythonetc.orsinium.dev
* 📢 RSS: pythonetc.orsinium.dev/index.xml
* 🧑💻️ GitHub: github.com/life4/pythonetc
If you want to write a guest post, just send us a PR on GitHub. The README tells what you can write about and how. Thank you all for staying with us all these years ❤️
👍 158
In addition to typing.ParamSpec, PEP 612 introduced typing.Concatenate that allows describing decorators that accept fewer or more arguments that the wrapped function:
from typing import Callable, Concatenate, ParamSpec, TypeVar P = ParamSpec('P') R = TypeVar('R') class User: ... class Request: ... class Response: ... def with_user( f: Callable[Concatenate[User, P], R], ) -> Callable[P, R]: def inner(*args: P.args, **kwargs: P.kwargs) -> R: user = User() return f(user, *args, **kwargs) return inner @with_user def handle_request( user: User, request: Request, ) -> Response: ... request = Request() response = handle_request(request)
👍 42👎 2
Let's say, you have a typical decorator that returns a new function. Something like this:
def debug(f): name = f.__name__ def inner(*args, **kwargs): print(f'called {name} with {args=} and {kwargs=}') return f(*args, **kwargs) return inner @debug def concat(a: str, b: str) -> str: return a + b concat('hello ', 'world') # called concat with args=('hello ', 'world') and kwargs={}If you check the type of
concat
using reveal_type, you'll see that its type is unknown because of the decorator:
reveal_type(concat) # Revealed type is "Any"So, we need to properly annotate the decorator. But how? This is not precise enough (type errors like
x: int = concat(1, 2)
won't be detected):
from typing import Callable def debug(f: Callable) -> Callable: ...This is slightly better but function arguments are still untyped:
from typing import TypeVar T = TypeVar('T') def debug( f: Callable[..., T], ) -> Callable[..., T]: ...This is type-safe but it requires the decorated function to accept exactly 2 arguments:
A = TypeVar('A') B = TypeVar('B') R = TypeVar('R') def debug( f: Callable[[A, B], R], ) -> Callable[[A, B], R]: ...This is type-safe and works on any function but it will report a type error because
inner
is not guaranteed to have the same type as the passed callable (for example, someone might pass a class that is callable but we return a function):
F = TypeVar('F', bound=Callable) def debug(f: F) -> F: ...PEP 612 (landed in Python 3.10) introduced typing.ParamSpec which solves exactly this problem. You can use it to tell type checkers that the decorator returns a new function that accepts exactly the same arguments as the wrapped one:
from typing import Callable, TypeVar, ParamSpec P = ParamSpec('P') R = TypeVar('R') def debug( f: Callable[P, R], ) -> Callable[P, R]: def inner( *args: P.args, **kwargs: P.kwargs, ) -> R: ... return f(*args, **kwargs) return inner @debug def concat(a: str, b: str) -> str: ... reveal_type(concat) # Revealed type is "def (a: str, b: str) -> str"
👍 74👎 4
Let's say, you have the following mock:
from unittest.mock import Mock user = Mock() user.name = 'Guido'You fully specified all attributes and methods it should have, and you pass it into the tested code, but then that code uses an attribute that you don't expect it to use:
user.age # <Mock name='mock.age' id='...'>Instead of failing with an
AttributeError
, the mock instead will create a new mock when its unspecified attribute is accessed. To fix it, you can (and should) use the unittest.mock.seal function (introduced in Python 3.7):
from unittest.mock import seal seal(user) user.name # 'Guido' user.occupation # AttributeError: mock.occupation
👍 83👎 2
Daylight saving time (DST) is the practice of advancing clocks (typically by one hour) during warmer months so that darkness falls at a later clock time and then turning it back for colder months. That means, sometimes, once a year the clock shows the same time twice. It can also happen when the UTC shift of the current timezone is decreased.
To distinguish such situations, PEP-495 (landed in Python 3.6) introduce the fold attribute for
datetime
that is 0 or 1 depending if this is the first or the second pass through the given time in the given timezone.
For example, in Amsterdam the time is shifted from CEST (Central European Summer Time) to CET (Central European Time) on the last Sunday of October:
from datetime import datetime, timedelta, timezone from zoneinfo import ZoneInfo ams = ZoneInfo('Europe/Amsterdam') d0 = datetime(2023, 10, 29, 0, 0, tzinfo=timezone.utc) for h in range(3): du = d0 + timedelta(hours=h) dl = du.astimezone(ams) m = f'{du.time()} UTC is {dl.time()} {dl.tzname()} (fold={dl.fold})' print(m)This code will print:
00:00:00 UTC is 02:00:00 CEST (fold=0) 01:00:00 UTC is 02:00:00 CET (fold=1) 02:00:00 UTC is 03:00:00 CET (fold=0)However, you should keep in mind that
fold
is not considered in comparison operations:
d1 = datetime(2023, 10, 29, 2, 0, tzinfo=ams) d2 = datetime(2023, 10, 29, 2, 0, fold=1, tzinfo=ams) d1 == d2 # TrueNow imagine that your system has a bug because of not handling this. That happens once a year. On Sunday. At night 🌚
👍 51👎 1
PEP-615 (landed in Python 3.9) introduced the module zoneinfo. The module provides access to information about time zones. It will try to use the information about time zones provided by the OS. If not available, it falls back to the official Python tzdata package which you need to install separately.
from zoneinfo import ZoneInfo from datetime import datetime ams = ZoneInfo('Europe/Amsterdam') dt = datetime(2015, 10, 21, 13, 40, tzinfo=ams) dt # datetime(2015, 10, 21, 13, 40, tzinfo=ZoneInfo(key='Europe/Amsterdam')) la = ZoneInfo('America/Los_Angeles') dt.astimezone(la) # datetime(2015, 10, 21, 4, 40, tzinfo=ZoneInfo(key='America/Los_Angeles'))You should not use pytz anymore.
👍 71👎 1
In the previous post, we had the following code:
python import asyncio async def child(): ... async def main(): asyncio.create_task(child()) ...Can you spot a bug? Since we don't store a reference to the background task we create, the garbage collector may destroy the task before it finishes. To avoid that, we need to store a reference to the task until it finishes. The official documentation recommends the following pattern:
python bg_tasks = set() async def main(): t = asyncio.create_task(child()) # hold the reference to the task # in a global set bg_tasks.add(t) # automatically remove the task # from the set when it's done t.add_done_callback(t.discard) ...
👍 46👎 7
When talking about asyncio functions, sometimes I used the word "coroutine" and sometimes "task". It's time to tell you the difference:
+
coroutine
is what async function returns. It can be scheduled, switched, closed, and so on. It's quite similar to generators. In fact, await
keyword is nothing more than an alias for yield from
, and async
is a decorator turning the function from a generator into a coroutine.
+ asyncio.Future
is like "promise" in JS. It is an object that eventually will hold a coroutine result when it is available. It has done
method to check if the result is available, result
to get the result, and so on.
+ asyncio.Task
is like if coroutine and future had a baby. This is what asyncio mostly works with. It can be scheduled, switched, canceled, and holds its result when ready.
There is a cool function asyncio.create_task
that can turn a coroutine into a proper task. What's cool about it is that this task immediately gets scheduled. So, if your code later encounters await
, there is a chance your task will be executed at that point.
import asyncio async def child(): print('started child') await asyncio.sleep(1) print('finished child') async def main(): asyncio.create_task(child()) print('before sleep') await asyncio.sleep(0) print('after sleep') asyncio.run(main())Output:
before sleep started child after sleepWhat happened: 1. When
create_task
is called, it is scheduled but not yet executed.
2. When main
hits await
, the scheduler switches to child
.
3. When child
hits await
, the scheduler switches to another task, which is main
4. When main
finished, asyncio.run
returned without waiting for child
to finish. It's dead in space now.
But what if you want to make sure a scheduled task finishes before exiting? You can pass the task into good old asyncio.gather
. And later we'll see some ways to wait for it with timeouts or when you don't care about the result.
task = create_task(...) ... await asyncio.gather(task)
👍 68👎 1
The
asyncio.gather
is the function that you will use the most. You pass to it multiple coroutines, it schedules them, waits for all to finish, and returns the list of results in the same order.
import asyncio URLS = ['google.com', 'github.com', 't.me'] async def check_alive(url): print(f'started {url}') i = URLS.index(url) await asyncio.sleep(3 - i) print(f'finished {url}') return i async def main(): coros = [check_alive(url) for url in URLS] statuses = await asyncio.gather(*coros) for url, alive in zip(URLS, statuses): print(url, alive) asyncio.run(main())Output:
started google.com started github.com started t.me finished t.me finished github.com finished google.com google.com 0 github.com 1 t.me 2That's what happened: 1.
asyncio.gather
schedules all tasks in order as they are passed.
2. We made the first task wait 3 seconds, the second wait 2 seconds, and the last one wait 1 second. And the tasks finished as soon as they could, without making everyone wait for the first task.
3. asyncio.gather
waits for all tasks to finish.
4. asyncio.gather
returns a list of results in the order as the coroutines were passed in it. So, it's safe to zip
results with input values.👍 48👎 2
Your best companion in learning asyncio is
asyncio.sleep
. It works like time.sleep
making the calling code wait the given number of seconds. This is the simplest example of an IO-bound task because while sleeping, your code literally does nothing but wait. And unlike time.sleep
, asyncio.sleep
is async. That means, while the calling task waits for it to finish, another task can be executed.
import asyncio import time async def main(): start = time.time() await asyncio.sleep(2) return int(time.time() - start) asyncio.run(main()) # 2You can't yet see how the code switches to another task while waiting because we have only one task. But bear with me, in the next posts we'll get to it.
👍 53👎 2
اختر خطة مختلفة
تسمح خطتك الحالية بتحليلات لما لا يزيد عن 5 قنوات. للحصول على المزيد، يُرجى اختيار خطة مختلفة.