On this page
AWS Lambda Event-Driven Architecture
The Mental Model Shift
Traditional infrastructure starts with capacity planning: How many servers? What instance types? Serverless inverts this model entirely. You pay only for actual execution time measured in milliseconds, not idle capacity.| Traditional | Serverless |
|---|---|
| Provision servers | Write functions |
| Pay for uptime (24/7) | Pay per invocation (ms) |
| Manual scaling | Auto-scales to 1000+ concurrent |
| Patch OS, runtime | AWS manages everything |
| Plan capacity | Infinite scale (within limits) |
Cold Start vs Warm Start
Lambda Function Example: API Backend
Python
import json
import boto3
from decimal import Decimal
# Initialize outside handler (runs once per container)
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('users')
def lambda_handler(event, context):
"""API Gateway proxy integration."""
# Parse request
http_method = event['httpMethod']
path = event['path']
if http_method == 'GET' and path == '/users':
# Query DynamoDB
response = table.scan(Limit=100)
return {
'statusCode': 200,
'headers': {'Content-Type': 'application/json'},
'body': json.dumps(response['Items'], default=str)
}
elif http_method == 'POST' and path == '/users':
# Create user
body = json.loads(event['body'])
table.put_item(Item={
'user_id': body['email'],
'name': body['name'],
'created_at': context.request_id
})
return {
'statusCode': 201,
'headers': {'Content-Type': 'application/json'},
'body': json.dumps({'message': 'User created'})
}
return {
'statusCode': 404,
'body': json.dumps({'error': 'Not found'})
}S3 Event Processing
Python
import boto3
from PIL import Image
import io
s3 = boto3.client('s3')
def lambda_handler(event, context):
"""Triggered by S3 object creation."""
for record in event['Records']:
bucket = record['s3']['bucket']['name']
key = record['s3']['object']['key']
# Download image
response = s3.get_object(Bucket=bucket, Key=key)
img = Image.open(response['Body'])
# Create thumbnail (200x200)
img.thumbnail((200, 200))
# Upload thumbnail
buffer = io.BytesIO()
img.save(buffer, format='JPEG')
buffer.seek(0)
thumb_key = f"thumbnails/{key}"
s3.put_object(
Bucket=bucket,
Key=thumb_key,
Body=buffer,
ContentType='image/jpeg'
)
print(f"Created thumbnail: {thumb_key}")
return {'statusCode': 200}SQS Queue Processing with DLQ
Python
import json
import boto3
from botocore.exceptions import ClientError
ses = boto3.client('ses')
def lambda_handler(event, context):
"""Process SQS messages with error handling."""
for record in event['Records']:
try:
# Parse message
message = json.loads(record['body'])
# Send email
ses.send_email(
Source='noreply@example.com',
Destination={'ToAddresses': [message['email']]},
Message={
'Subject': {'Data': message['subject']},
'Body': {'Text': {'Data': message['body']}}
}
)
print(f"Email sent to {message['email']}")
except ClientError as e:
# Log error - message will retry or go to DLQ
print(f"Error: {e}")
raise # Fail this message
return {'batchItemFailures': []} # All succeededRDS Proxy Pattern
Python
import pymysql
import os
# Connection is reused across invocations
connection = None
def get_connection():
global connection
if connection is None or not connection.open:
# Connect via RDS Proxy (connection pooling)
connection = pymysql.connect(
host=os.environ['RDS_PROXY_ENDPOINT'],
user=os.environ['DB_USER'],
password=os.environ['DB_PASSWORD'],
database=os.environ['DB_NAME'],
cursorclass=pymysql.cursors.DictCursor
)
return connection
def lambda_handler(event, context):
conn = get_connection()
with conn.cursor() as cursor:
sql = "SELECT * FROM orders WHERE user_id = %s"
cursor.execute(sql, (event['user_id'],))
results = cursor.fetchall()
return {
'statusCode': 200,
'body': json.dumps(results)
}Event Sources Comparison
| Source | Invocation | Use Case | Retry |
|---|---|---|---|
| API Gateway | Synchronous | REST/WebSocket APIs | Client retry |
| S3 | Asynchronous | File processing | Auto (2x) |
| SQS | Poll-based | Async jobs, decoupling | Configurable + DLQ |
| SNS | Asynchronous | Fanout, notifications | Auto (3x) |
| DynamoDB Streams | Stream | CDC, replication | Until success |
| Kinesis | Stream | Real-time analytics | Until success |
| EventBridge | Asynchronous | Scheduled, custom events | Auto (185x over 24h) |
Best Practices
- Initialize outside handler: Reuse connections, SDKs across invocations
- Use environment variables: Configuration without code changes
- Enable X-Ray tracing: Distributed tracing for debugging
- Set appropriate timeouts: Default 3s, max 15 minutes
- Right-size memory: More memory = more CPU (test 512MB-3GB)
- Use provisioned concurrency: For latency-sensitive apps
- Implement idempotency: Handle duplicate invocations
- Use layers for dependencies: Share code, reduce package size
- Monitor CloudWatch metrics: Duration, errors, throttles
- Use RDS Proxy: For relational databases (connection pooling)
When to Use Lambda vs Containers
| Use Lambda | Use ECS/EKS |
|---|---|
| Event-driven workloads | Long-running processes (> 15 min) |
| Sporadic traffic | Consistent high traffic |
| Microservices | Monolithic apps |
| Auto-scaling needs | Need full OS control |
| < 10 GB memory | > 10 GB memory needed |
| Stateless functions | Stateful services |
Cost Optimization
- Right-size memory: Test different sizes (128MB-10GB) for cost/performance
- arm64 architecture: 20% better price/performance vs x86
- Reduce cold starts: Smaller packages, fewer dependencies
- Use reserved concurrency wisely: Prevent cost spikes
- Batch processing: Process multiple items per invocation
- Monitor unused functions: Delete or archive
References
Discover more from C4: Container, Code, Cloud & Context
Subscribe to get the latest posts sent to your email.