The with
statement (and by extension, async with
) is all about resource management. Let me break it down:
- Basic
with
example first:
# Without 'with'file = open('file.txt')try: data = file.read()finally: file.close()
# With 'with' - same thing but cleanerwith open('file.txt') as file: data = file.read()# File automatically closes when we exit the with block
- The
async with
follows the same principle but for asynchronous resources:
# Without async withactor = await Actor.initialize()try: # do stuff with actor await actor.cleanup()finally: await actor.final_cleanup()
# With async with - same thing but cleanerasync with Actor: # do stuff with actor# Actor automatically cleans up when we exit the block
Think of with
like a sandwich:
- Opening the
with
block is like the top slice of bread - Your code is the filling
- Closing the
with
block is like the bottom slice of bread
The with
statement ensures cleanup happens even if your code raises an exception - just like the bread holds the sandwich together even if some filling falls out.
In your Apify code:
async def main() -> None: async with Actor: # Actor starts up actor_input = await Actor.get_input() or {} # ... more code ... process.start() # Actor automatically shuts down properly here
The async with Actor
ensures that:
- The Actor properly initializes at the start
- Your code runs in the middle
- The Actor properly cleans up at the end, even if your code crashes
Multiple examples of both with
and async with
to help clarify their usage:
- Database Connection Example:
# Without 'with'connection = create_connection()try: cursor = connection.cursor() cursor.execute("SELECT * FROM users") results = cursor.fetchall()finally: connection.close()
# With 'with' - much cleanerwith create_connection() as connection: cursor = connection.cursor() cursor.execute("SELECT * FROM users") results = cursor.fetchall()
- Multiple Resources Example:
# You can manage multiple resources in one with statementwith open('input.txt') as input_file, open('output.txt', 'w') as output_file: data = input_file.read() output_file.write(data.upper())
- Async Database Example:
import asyncpg
async def get_users(): # Async database connection async with asyncpg.create_pool(database='mydb') as pool: async with pool.acquire() as connection: async with connection.transaction(): # Do database operations return await connection.fetch('SELECT * FROM users')
- Custom Context Manager Example:
from contextlib import contextmanagerimport time
@contextmanagerdef timer(): start = time.time() yield # This is where the code inside the 'with' block runs end = time.time() print(f"Operation took {end - start} seconds")
# Using itwith timer(): # Some time-consuming operation time.sleep(2)
- Async Resource Lock Example:
import asyncio
async def worker(lock, number): async with lock: # Only one worker can enter at a time print(f"Worker {number} starting...") await asyncio.sleep(1) print(f"Worker {number} finished!")
async def main(): lock = asyncio.Lock() workers = [worker(lock, i) for i in range(3)] await asyncio.gather(*workers)
- Custom Async Context Manager:
from contextlib import asynccontextmanager
@asynccontextmanagerasync def managed_resource(): print("Starting up") try: yield "resource" # This is what the 'as' variable receives finally: print("Shutting down")
async def use_resource(): async with managed_resource() as resource: print(f"Using {resource}")
- Real-world Redis Example:
import aioredis
async def cache_operation(): async with aioredis.from_url('redis://localhost') as redis: # Set value await redis.set('key', 'value') # Get value value = await redis.get('key') return value
- HTTP Session Example:
import aiohttp
async def fetch_data(urls): async with aiohttp.ClientSession() as session: tasks = [] for url in urls: async with session.get(url) as response: data = await response.text() tasks.append(data) return tasks
These context managers are handling important setup and cleanup tasks like:
- Opening and closing files
- Starting and committing/rolling back database transactions
- Acquiring and releasing locks
- Opening and closing network connections
- Resource cleanup
- Performance measurement
- Error handling
The beauty of with
and async with
is that they guarantee proper resource management even if exceptions occur in your code. This prevents resource leaks and ensures clean shutdown of your applications.