Learn about with / async with in Python

ST

Surendra Tamang

3 min read
Python async with context managers

The with statement (and by extension, async with) is all about resource management. Let me break it down:

  1. Basic with example first:
# Without 'with'
file = open('file.txt')
try:
data = file.read()
finally:
file.close()
# With 'with' - same thing but cleaner
with open('file.txt') as file:
data = file.read()
# File automatically closes when we exit the with block
  1. The async with follows the same principle but for asynchronous resources:
# Without async with
actor = await Actor.initialize()
try:
# do stuff with actor
await actor.cleanup()
finally:
await actor.final_cleanup()
# With async with - same thing but cleaner
async with Actor:
# do stuff with actor
# Actor automatically cleans up when we exit the block

Think of with like a sandwich:

  • Opening the with block is like the top slice of bread
  • Your code is the filling
  • Closing the with block is like the bottom slice of bread

The with statement ensures cleanup happens even if your code raises an exception - just like the bread holds the sandwich together even if some filling falls out.

In your Apify code:

async def main() -> None:
async with Actor: # Actor starts up
actor_input = await Actor.get_input() or {}
# ... more code ...
process.start()
# Actor automatically shuts down properly here

The async with Actor ensures that:

  1. The Actor properly initializes at the start
  2. Your code runs in the middle
  3. The Actor properly cleans up at the end, even if your code crashes

Multiple examples of both with and async with to help clarify their usage:

  1. Database Connection Example:
# Without 'with'
connection = create_connection()
try:
cursor = connection.cursor()
cursor.execute("SELECT * FROM users")
results = cursor.fetchall()
finally:
connection.close()
# With 'with' - much cleaner
with create_connection() as connection:
cursor = connection.cursor()
cursor.execute("SELECT * FROM users")
results = cursor.fetchall()
  1. Multiple Resources Example:
# You can manage multiple resources in one with statement
with open('input.txt') as input_file, open('output.txt', 'w') as output_file:
data = input_file.read()
output_file.write(data.upper())
  1. Async Database Example:
import asyncpg
async def get_users():
# Async database connection
async with asyncpg.create_pool(database='mydb') as pool:
async with pool.acquire() as connection:
async with connection.transaction():
# Do database operations
return await connection.fetch('SELECT * FROM users')
  1. Custom Context Manager Example:
from contextlib import contextmanager
import time
@contextmanager
def timer():
start = time.time()
yield # This is where the code inside the 'with' block runs
end = time.time()
print(f"Operation took {end - start} seconds")
# Using it
with timer():
# Some time-consuming operation
time.sleep(2)
  1. Async Resource Lock Example:
import asyncio
async def worker(lock, number):
async with lock: # Only one worker can enter at a time
print(f"Worker {number} starting...")
await asyncio.sleep(1)
print(f"Worker {number} finished!")
async def main():
lock = asyncio.Lock()
workers = [worker(lock, i) for i in range(3)]
await asyncio.gather(*workers)
  1. Custom Async Context Manager:
from contextlib import asynccontextmanager
@asynccontextmanager
async def managed_resource():
print("Starting up")
try:
yield "resource" # This is what the 'as' variable receives
finally:
print("Shutting down")
async def use_resource():
async with managed_resource() as resource:
print(f"Using {resource}")
  1. Real-world Redis Example:
import aioredis
async def cache_operation():
async with aioredis.from_url('redis://localhost') as redis:
# Set value
await redis.set('key', 'value')
# Get value
value = await redis.get('key')
return value
  1. HTTP Session Example:
import aiohttp
async def fetch_data(urls):
async with aiohttp.ClientSession() as session:
tasks = []
for url in urls:
async with session.get(url) as response:
data = await response.text()
tasks.append(data)
return tasks

These context managers are handling important setup and cleanup tasks like:

  • Opening and closing files
  • Starting and committing/rolling back database transactions
  • Acquiring and releasing locks
  • Opening and closing network connections
  • Resource cleanup
  • Performance measurement
  • Error handling

The beauty of with and async with is that they guarantee proper resource management even if exceptions occur in your code. This prevents resource leaks and ensures clean shutdown of your applications.