IT博客汇
  • 首页
  • 精华
  • 技术
  • 设计
  • 资讯
  • 扯淡
  • 权利声明
  • 登录 注册

    A single Python function for both async/sync

    est发表于 2025-05-14 14:35:00
    love 0

    Scenario: I often need to write Python functions like:

    1. take some parameters and format them
    2. call an API with the formatted parameters
    3. parse the result and return chosen values

    There's a huge problem in step #2.

    In today's Python world, troubles arise because async/await are "infectious", In practice this function is splitted - like in Python stdlib, where a vanilla method and its async counterpart amethod often come in pairs. Package authors scramble to provide sync transport and another async transport. I discovered this ugly fact while reading the source code ofredis-py, httpx and elasticsearch-py. Duplicate and lookalike code was always written twice. All it takes is some random async IOs in one place and your code would be forced to change forever.

    Is there a way to write the function in one place, but callable both with async and without?

    I pondered this question for ages, and today I stumbled upon something interesting:

    
      def s1():
        return asyncio.sleep(1)
    
      async def s2():
        return await async.sleep(1)
    

    There's virtually no difference when calling await s1() and await s2()

    I vaguely remembered how Python’s coroutines were designed, and after some tinkering, I came up with this snippet:

    
    import asyncio, types
    
    
    def aa(f):
        """
        decorator to make a function both awaitable and sync
        idk how to property name this. maybe anti-asyncio (aa)?
        """
        def wrapper(func, *args, **kwargs):
            if asyncio.iscoroutinefunction(func):
                return types.coroutine(f)(func, *args, **kwargs)
            else:
                g = f(func, *args, **kwargs)
                # any better way to write this?
                try:
                    while True:
                        next(g)
                except StopIteration as ex:
                    return ex.value
        return wrapper
    
    
    @aa
    def my_func(func, *args, **kwargs):
        # prepare args, kwargs here
        if asyncio.iscoroutinefunction(func):
            # just replace `await` with `yield from` for async calls
            result = yield from func(*args, **kwargs)
        else:
            result = func(*args, **kwargs)
        # handle the result here
        return result
    
    
    import httpx
    
    # async
    async def main():
        # works the same as `await httpx.AsyncClient(timeout=3).get('https://est.im')`
        print(await my_func(httpx.AsyncClient(timeout=3).get, 'https://est.im/'))
    asyncio.run(main())
    
    
    # sync
    print(my_func(httpx.get, 'https://est.im'))
    # works the same as httpx.get('https://est.im')
    
    

    The above shows a single function called my_func, dependency injection of an HTTP get call of either sync/async, allows for customizable pre- and post-processing logic, and returns the result with clean syntax.

    The only mental tax: inside my_func, you have to replace all await keyword with yield from.

    It works for my scenario and I’ve yet to find a simpler solution.



沪ICP备19023445号-2号
友情链接