Learn about with / async with in Python

The with statement (and by extension, async with) is all about resource management. Let me break it down:

1. Basic with example first:

# Without 'with'
file = open('file.txt')
try:
    data = file.read()
finally:
    file.close()

# With 'with' - same thing but cleaner
with open('file.txt') as file:
    data = file.read()
# File automatically closes when we exit the with block

2. The async with follows the same principle but for asynchronous resources:

# Without async with
actor = await Actor.initialize()
try:
    # do stuff with actor
    await actor.cleanup()
finally:
    await actor.final_cleanup()

# With async with - same thing but cleaner
async with Actor:
    # do stuff with actor
# Actor automatically cleans up when we exit the block

Think of with like a sandwich:

• Opening the with block is like the top slice of bread
• Your code is the filling
• Closing the with block is like the bottom slice of bread

The with statement ensures cleanup happens even if your code raises an exception - just like the bread holds the sandwich together even if some filling falls out.

In your Apify code:

async def main() -> None:
    async with Actor:  # Actor starts up
        actor_input = await Actor.get_input() or {}
        # ... more code ...
        process.start()
    # Actor automatically shuts down properly here

The async with Actor ensures that:

1. The Actor properly initializes at the start
2. Your code runs in the middle
3. The Actor properly cleans up at the end, even if your code crashes

Multiple examples of both with and async with to help clarify their usage:

1. Database Connection Example:

# Without 'with'
connection = create_connection()
try:
    cursor = connection.cursor()
    cursor.execute("SELECT * FROM users")
    results = cursor.fetchall()
finally:
    connection.close()

# With 'with' - much cleaner
with create_connection() as connection:
    cursor = connection.cursor()
    cursor.execute("SELECT * FROM users")
    results = cursor.fetchall()

2. Multiple Resources Example:

# You can manage multiple resources in one with statement
with open('input.txt') as input_file, open('output.txt', 'w') as output_file:
    data = input_file.read()
    output_file.write(data.upper())

3. Async Database Example:

import asyncpg

async def get_users():
    # Async database connection
    async with asyncpg.create_pool(database='mydb') as pool:
        async with pool.acquire() as connection:
            async with connection.transaction():
                # Do database operations
                return await connection.fetch('SELECT * FROM users')

4. Custom Context Manager Example:

from contextlib import contextmanager
import time

@contextmanager
def timer():
    start = time.time()
    yield  # This is where the code inside the 'with' block runs
    end = time.time()
    print(f"Operation took {end - start} seconds")

# Using it
with timer():
    # Some time-consuming operation
    time.sleep(2)

5. Async Resource Lock Example:

import asyncio

async def worker(lock, number):
    async with lock:  # Only one worker can enter at a time
        print(f"Worker {number} starting...")
        await asyncio.sleep(1)
        print(f"Worker {number} finished!")

async def main():
    lock = asyncio.Lock()
    workers = [worker(lock, i) for i in range(3)]
    await asyncio.gather(*workers)

6. Custom Async Context Manager:

from contextlib import asynccontextmanager

@asynccontextmanager
async def managed_resource():
    print("Starting up")
    try:
        yield "resource"  # This is what the 'as' variable receives
    finally:
        print("Shutting down")

async def use_resource():
    async with managed_resource() as resource:
        print(f"Using {resource}")

7. Real-world Redis Example:

import aioredis

async def cache_operation():
    async with aioredis.from_url('redis://localhost') as redis:
        # Set value
        await redis.set('key', 'value')
        # Get value
        value = await redis.get('key')
        return value

8. HTTP Session Example:

import aiohttp

async def fetch_data(urls):
    async with aiohttp.ClientSession() as session:
        tasks = []
        for url in urls:
            async with session.get(url) as response:
                data = await response.text()
                tasks.append(data)
        return tasks

These context managers are handling important setup and cleanup tasks like:

• Opening and closing files
• Starting and committing/rolling back database transactions
• Acquiring and releasing locks
• Opening and closing network connections
• Resource cleanup
• Performance measurement
• Error handling

The beauty of with and async with is that they guarantee proper resource management even if exceptions occur in your code. This prevents resource leaks and ensures clean shutdown of your applications.