Django Under Hood 04: Django’s Signal Dispatch — The Observer Pattern You’re Using Wrong
Part 4 of the “Django Under the Hood ” series — deep dives into Django’s internals, edge cases, and the mechanics that separate production-grade applications from tutorial code.
from django.db.models.signals import post_save
from django.dispatch import receiver
@receiver(post_save, sender=User)
def create_user_profile(sender, instance, created, **kwargs):
if created:
Profile.objects.create(user=instance)
This is the canonical Django signal example. It’s in every tutorial.
It’s also a time bomb.
One day, this handler will silently stop running. No errors. No logs. No indication anything is wrong. Users will sign up, and profiles won’t be created. The cause? Garbage collection.
Django’s signal system uses weak references by default. If nothing else holds a reference to your handler, Python’s garbage collector deletes it. The signal keeps firing, but your handler is gone.
Understanding signal internals isn’t academic. It’s the difference between code that works reliably and code that fails mysteriously in production.
Let’s look inside.
The Signal Class
# django/dispatch/dispatcher.py
class Signal:
def __init__(self, use_caching=False):
self.receivers = []
self.lock = threading.Lock()
self.use_caching = use_caching
self.sender_receivers_cache = {} if use_caching else None
A signal is just a list of receivers with thread-safe access. When you call signal.send(), it iterates through receivers and calls each one.
# Simplified send() implementation
def send(self, sender, **named):
responses = []
for receiver in self._live_receivers(sender):
response = receiver(signal=self, sender=sender, **named)
responses.append((receiver, response))
return responses
The interesting part is _live_receivers() — it handles weak references and sender filtering.
Connecting Receivers: The Two Methods
Method 1: signal.connect()
def my_handler(sender, instance, **kwargs):
print(f"Saved: {instance}")
post_save.connect(my_handler, sender=User)
What happens internally:
# django/dispatch/dispatcher.py
class Signal:
def connect(self, receiver, sender=None, weak=True, dispatch_uid=None):
# Create lookup key
if dispatch_uid:
lookup_key = (dispatch_uid, _make_id(sender))
else:
lookup_key = (_make_id(receiver), _make_id(sender))
# Wrap in weak reference (by default!)
if weak:
ref = weakref.ref
receiver_object = receiver
# Handle bound methods specially
if hasattr(receiver, '__self__'):
ref = weakref.WeakMethod
receiver = ref(receiver)
with self.lock:
# Check for duplicates
for r_key, _ in self.receivers:
if r_key == lookup_key:
break
else:
self.receivers.append((lookup_key, receiver))
The critical line: receiver = ref(receiver) — your handler is wrapped in a weak reference.
Method 2: @receiver decorator
@receiver(post_save, sender=User)
def my_handler(sender, instance, **kwargs):
print(f"Saved: {instance}")
The decorator just calls connect():
# django/dispatch/dispatcher.py
def receiver(signal, **kwargs):
def decorator(func):
if isinstance(signal, (list, tuple)):
for s in signal:
s.connect(func, **kwargs)
else:
signal.connect(func, **kwargs)
return func
return decorator
Weak References: The Silent Killer
A weak reference doesn’t prevent garbage collection:
import weakref
def my_function():
print("Called!")
# Strong reference - function stays alive
ref = my_function
# Weak reference - function can be garbage collected
weak_ref = weakref.ref(my_function)
del my_function # Remove strong reference
del ref # Remove another strong reference
# Now the function might be garbage collected!
print(weak_ref()) # Returns None if collected
When Handlers Get Garbage Collected
Scenario 1: Lambda functions
# THIS HANDLER WILL DISAPPEAR
post_save.connect(lambda sender, **kwargs: print("Saved!"), sender=User)
# The lambda has no other reference. After this line executes,
# it's eligible for garbage collection.
Scenario 2: Handlers defined in functions
def setup_signals():
def my_handler(sender, **kwargs):
print("Saved!")
post_save.connect(my_handler, sender=User)
# my_handler's only reference is local to this function
setup_signals()
# Function returns, local variables are garbage collected
# my_handler is gone!
Scenario 3: Method handlers without persistent instance
class MyHandler:
def handle(self, sender, **kwargs):
print("Saved!")
post_save.connect(MyHandler().handle, sender=User)
# The MyHandler instance has no reference
# It gets garbage collected, taking the method with it
The Fix: strong=True or Module-Level Functions
Option 1: Disable weak references
post_save.connect(my_handler, sender=User, weak=False)
Option 2: Module-level functions (recommended)
# signals.py
from django.db.models.signals import post_save
from django.dispatch import receiver
@receiver(post_save, sender=User)
def create_user_profile(sender, instance, created, **kwargs):
if created:
Profile.objects.create(user=instance)
# This function is defined at module level
# The module holds a strong reference to it
# It won't be garbage collected
Option 3: Keep a reference yourself
# If you must use a class
class SignalHandlers:
@staticmethod
def user_saved(sender, instance, **kwargs):
print(f"Saved: {instance}")
# Keep the class around (it's at module level, so it persists)
post_save.connect(SignalHandlers.user_saved, sender=User)
Sender Filtering: How It Really Works
post_save.connect(my_handler, sender=User)
The sender argument filters which signals trigger your handler:
# django/dispatch/dispatcher.py
def _live_receivers(self, sender):
receivers = []
for (lookup_key, receiver) in self.receivers:
lookup_sender = lookup_key[1] # Second part is sender ID
# None means "any sender"
# Otherwise, must match
if lookup_sender is None or lookup_sender == _make_id(sender):
# Resolve weak reference
if isinstance(receiver, weakref.ref):
receiver = receiver()
if receiver is None:
continue # Garbage collected, skip
receivers.append(receiver)
return receivers
The sender=None trap:
# Receives ALL post_save signals for ALL models
@receiver(post_save) # No sender specified
def universal_handler(sender, instance, **kwargs):
print(f"Any model saved: {instance}")
# This fires for User, Profile, Order, LogEntry, Session...
# Everything that saves to the database
In production, this can mean thousands of unintended calls.
dispatch_uid: Preventing Duplicates
# This can register the same handler multiple times
for i in range(3):
post_save.connect(my_handler, sender=User)
# my_handler runs 3 times for each User save!
The dispatch_uid parameter prevents duplicates:
# Only registers once, no matter how many times called
for i in range(3):
post_save.connect(
my_handler,
sender=User,
dispatch_uid='user_saved_handler'
)
How it works:
def connect(self, receiver, sender=None, weak=True, dispatch_uid=None):
if dispatch_uid:
lookup_key = (dispatch_uid, _make_id(sender))
else:
lookup_key = (_make_id(receiver), _make_id(sender))
# Check if this key already exists
for r_key, _ in self.receivers:
if r_key == lookup_key:
break # Already registered, don't add again
else:
self.receivers.append((lookup_key, receiver))
When to use dispatch_uid:
- Signals connected in
AppConfig.ready()(can be called multiple times in tests) - Dynamic signal connection in loops
- Anywhere the connection code might run multiple times
Signal Execution: send() vs send_robust()
send(): Exceptions Propagate
# django/dispatch/dispatcher.py
def send(self, sender, **named):
responses = []
for receiver in self._live_receivers(sender):
response = receiver(signal=self, sender=sender, **named)
responses.append((receiver, response))
return responses
If any receiver raises an exception, it propagates immediately. Other receivers don’t run.
@receiver(post_save, sender=User)
def handler_1(sender, instance, **kwargs):
raise ValueError("Oops!")
@receiver(post_save, sender=User)
def handler_2(sender, instance, **kwargs):
print("This never runs!")
send_robust(): Exceptions Captured
def send_robust(self, sender, **named):
responses = []
for receiver in self._live_receivers(sender):
try:
response = receiver(signal=self, sender=sender, **named)
except Exception as err:
response = err # Capture, don't propagate
responses.append((receiver, response))
return responses
All receivers run, even if some fail:
responses = my_signal.send_robust(sender=self)
for receiver, response in responses:
if isinstance(response, Exception):
logger.error(f"Handler {receiver} failed: {response}")
Django’s built-in signals use send(), so one failing handler can break your entire save operation.
Built-in Model Signals: Deep Dive
pre_save and post_save
# django/db/models/base.py
class Model:
def save(self, ...):
# pre_save fires here
signals.pre_save.send(
sender=self.__class__,
instance=self,
raw=raw,
using=using,
update_fields=update_fields,
)
# Actual database operation
self._save_table(...)
# post_save fires here
signals.post_save.send(
sender=self.__class__,
instance=self,
created=created,
raw=raw,
using=using,
update_fields=update_fields,
)
Arguments you receive:
pre_delete and post_delete
# django/db/models/base.py
def delete(self, using=None, keep_parents=False):
signals.pre_delete.send(
sender=self.__class__,
instance=self,
using=using,
)
# Actual deletion
collector.delete()
signals.post_delete.send(
sender=self.__class__,
instance=self,
using=using,
)
Warning: post_delete fires after the database delete. The instance still exists in memory, but instance.pk might be None.
m2m_changed
The most complex signal:
@receiver(m2m_changed, sender=Article.tags.through)
def tags_changed(sender, instance, action, pk_set, **kwargs):
# action is one of:
# "pre_add", "post_add"
# "pre_remove", "post_remove"
# "pre_clear", "post_clear"
if action == "post_add":
print(f"Added tags: {pk_set}")
Request Signals
# django/core/handlers/base.py
class BaseHandler:
def get_response(self, request):
# Fires at start of request
signals.request_started.send(sender=self.__class__, environ=environ)
try:
response = self._get_response(request)
finally:
# Fires at end of request
signals.request_finished.send(sender=self.__class__)
return response
Important: request_finished fires even if the view raises an exception.
@receiver(request_finished)
def cleanup_resources(sender, **kwargs):
# This runs after every request
# Good for cleanup, but be careful about performance
pass
Signal Performance Considerations
The Cost of Signals
Every signal check iterates through all receivers:
def _live_receivers(self, sender):
receivers = []
for (lookup_key, receiver) in self.receivers: # O(n) iteration
# ...
With many receivers, this adds up:
# 100 handlers connected to post_save
# Every model save iterates through all 100 to find matches
Caching for Performance
Some signals enable caching:
# django/db/models/signals.py
pre_save = Signal(use_caching=True)
post_save = Signal(use_caching=True)
With caching:
class Signal:
def _live_receivers(self, sender):
if self.use_caching:
# Check cache first
sender_id = _make_id(sender)
if sender_id in self.sender_receivers_cache:
return self.sender_receivers_cache[sender_id]
# ... find receivers ...
if self.use_caching:
self.sender_receivers_cache[sender_id] = receivers
return receivers
The cache is invalidated when receivers connect/disconnect.
Common Patterns and Anti-Patterns
Anti-Pattern: Heavy Work in Signals
# ❌ DON'T: Slow operations in post_save
@receiver(post_save, sender=Order)
def send_order_email(sender, instance, created, **kwargs):
if created:
# This blocks the request!
send_mail(
'Order Confirmation',
'Your order is confirmed...',
'from@example.com',
[instance.user.email],
)
✅ Fix: Defer to background task
@receiver(post_save, sender=Order)
def queue_order_email(sender, instance, created, **kwargs):
if created:
# Returns immediately, email sent async
send_order_confirmation.delay(instance.id)
Anti-Pattern: Circular Signals
@receiver(post_save, sender=User)
def create_profile(sender, instance, created, **kwargs):
if created:
Profile.objects.create(user=instance)
@receiver(post_save, sender=Profile)
def update_user(sender, instance, **kwargs):
instance.user.last_profile_update = timezone.now()
instance.user.save() # Triggers User post_save again!
✅ Fix: Use update_fields or flags
@receiver(post_save, sender=Profile)
def update_user(sender, instance, **kwargs):
User.objects.filter(pk=instance.user_id).update(
last_profile_update=timezone.now()
)
# .update() doesn't trigger signals!
Anti-Pattern: Assuming Transaction State
@receiver(post_save, sender=Order)
def notify_warehouse(sender, instance, **kwargs):
# This might run BEFORE the transaction commits!
# If the transaction rolls back, the order doesn't exist,
# but the warehouse was already notified.
warehouse_api.notify(instance.id)
✅ Fix: Use transaction.on_commit()
from django.db import transaction
@receiver(post_save, sender=Order)
def notify_warehouse(sender, instance, **kwargs):
transaction.on_commit(
lambda: warehouse_api.notify(instance.id)
)
# Only runs after transaction successfully commits
Creating Custom Signals
# signals.py
from django.dispatch import Signal
# Define signal with argument documentation
order_completed = Signal() # Django 3.0+ doesn't need providing_args
# Send signal
order_completed.send(
sender=Order,
order=order,
total=order.total,
)
# Receive signal
@receiver(order_completed)
def handle_order_completed(sender, order, total, **kwargs):
print(f"Order {order.id} completed: ${total}")
Debugging Signals
List All Receivers
from django.db.models.signals import post_save
# See all receivers for a signal
for lookup_key, receiver in post_save.receivers:
print(f"Key: {lookup_key}")
# Resolve weak reference
if hasattr(receiver, '__call__'):
actual = receiver
else:
actual = receiver() # Call weakref
if actual:
print(f" Handler: {actual.__module__}.{actual.__name__}")
else:
print(f" Handler: GARBAGE COLLECTED!")
Trace Signal Execution
# Temporarily wrap send() to debug
original_send = Signal.send
def debug_send(self, sender, **named):
print(f"Signal sent: sender={sender}, kwargs={named.keys()}")
result = original_send(self, sender, **named)
print(f" {len(result)} receivers called")
return result
Signal.send = debug_send
Check If Handler Is Connected
from django.db.models.signals import post_save
def is_connected(handler, signal, sender=None):
lookup_key = (_make_id(handler), _make_id(sender))
return any(key == lookup_key for key, _ in signal.receivers)
print(is_connected(my_handler, post_save, User))
What’s Next
This was signal dispatch — weak references, sender filtering, and execution mechanics.
Next in the series: Template Engine Compilation — how Django transforms template syntax into Python bytecode, the template lexer and parser, and why some template operations are surprisingly expensive.
Series: Django Under the Hood
- What Actually Happens When a Request Hits Your Server
- The ORM Query Compiler
- Connection Management and the Database Wrapper
- Signal Dispatch Internals ← You are here
- Template Engine Compilation (coming next)
- Form and Validation Pipeline
- Authentication Backend Chain
- Static Files and WhiteNoise Internals
- Migration System Deep Dive
- Test Client and Request Factory Mechanics