Async web servers

Why do you need them?

Andrew Svetlov

http://asvetlov.blogspot.com
andrew.svetlov@gmail.com
http://asvetlov.github.io/aiohttp-nizhny-2016/

Bio

Use Python for more than 16 years
Python Core Developer since 2012
asyncio committer
aiohttp maintainer
Author of a dozen libraries under aio-libs umbrella

Why?

It's cool!!!
I'm an author
Websockets out-of-the-box
...

To save memory!

WSGI

500 Mb process
100 ms response time
70% serving requests
7 RPS
5% CPU usage

Async

500 Mb process
103-105 ms response time
100 RPS
70% CPU usage

Failure

Latency: 50 ms → 3-15 sec
Response: 50 ms → 0.5-1 sec

Total Failure

0.01% requests with 1 min latency
Response: 50 ms → 5-15 minutes

Concurrent execution

Naive approach


async def handler(request):
    value1 = await get_part1(request)
    value2 = await get_part2(request)
    return render_response(value1, value2)

Explicit concurrency


async def handler(request):
    value1, value2 = await asyncio.gather(get_part1(request),
                                          get_part2(request))
    return render_response(value1, value2)

Hidden concurrency


async def handler(request):
    value1 = await get_part1(request)
    value2 = await get_part2(request)
    return render_response(value1, value2)

HTTP Keep-Alive

Server-side connections

NGINX
Backend (aiohttp)

Client connections


async def handler(request):
    session = request.app['client_session']
    async with session.get(url) as resp:
        body = yield from resp.json()
    return render_json(body)

DB connection pools


async def handler(request):
    async with request.app['db'] as conn:
        await conn.execute('SELECT * FROM ...')

Timeouts


with asyncio_timeout.timeout(10):
    async with session.get(url) as response:
        assert response.status == 200
        return await response.read()

Performance

JSON requests

External resource

Think about github.com, facebook.com or google.com

Database

Hosted in sibling docker contatiner

Questions?

Andrew Svetlov

http://asvetlov.blogspot.com
andrew.svetlov@gmail.com
http://asvetlov.github.io/aiohttp-nizhny-2016/