Python: When should I use generators vs list comprehensions for memory efficiency?
I'm working on a Python application and running into an issue with Python performance. Here's the problematic code:
# Current implementation
def fibonacci(n):
if n <= 1:
return n
return fibonacci(n-1) + fibonacci(n-2)
# This causes RecursionError for large n
result = fibonacci(1000)
The error message I'm getting is: "ValueError: invalid literal for int() with base 10: 'abc'"
What I've tried so far:
- Used pdb debugger to step through the code
- Added logging statements to trace execution
- Checked Python documentation and PEPs
- Tested with different Python versions
- Reviewed similar issues on GitHub and Stack Overflow
Environment information:
- Python version: 3.11.0
- Operating system: Ubuntu 22.04
- Virtual environment: venv (activated)
- Relevant packages: django, djangorestframework, celery, redis
Any insights or alternative approaches would be very helpful. Thanks!
2 Answers
Here's a comprehensive approach to implementing JWT authentication in Django REST Framework:
# settings.py
INSTALLED_APPS = [
'rest_framework',
'rest_framework_simplejwt',
]
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework_simplejwt.authentication.JWTAuthentication',
),
'DEFAULT_PERMISSION_CLASSES': [
'rest_framework.permissions.IsAuthenticated',
],
}
from datetime import timedelta
SIMPLE_JWT = {
'ACCESS_TOKEN_LIFETIME': timedelta(minutes=60),
'REFRESH_TOKEN_LIFETIME': timedelta(days=7),
'ROTATE_REFRESH_TOKENS': True,
}
# urls.py
from rest_framework_simplejwt.views import (
TokenObtainPairView,
TokenRefreshView,
)
urlpatterns = [
path('api/token/', TokenObtainPairView.as_view()),
path('api/token/refresh/', TokenRefreshView.as_view()),
]
# Custom serializer for additional user data
from rest_framework_simplejwt.serializers import TokenObtainPairSerializer
class CustomTokenObtainPairSerializer(TokenObtainPairSerializer):
@classmethod
def get_token(cls, user):
token = super().get_token(user)
token['username'] = user.username
token['email'] = user.email
return token
Comments
michael_code: This Python memory optimization technique reduced my application's RAM usage by 60%. Brilliant! 1 week, 4 days ago
To optimize Django QuerySets and avoid N+1 problems, use select_related() for ForeignKey and OneToOneField, and prefetch_related() for ManyToManyField and reverse ForeignKey:
# Bad: N+1 query problem
for book in Book.objects.all():
print(book.author.name) # Each iteration hits the database
# Good: Use select_related for ForeignKey
for book in Book.objects.select_related('author'):
print(book.author.name) # Single query with JOIN
# Good: Use prefetch_related for ManyToMany
for book in Book.objects.prefetch_related('categories'):
for category in book.categories.all():
print(category.name) # Optimized with separate query
You can also use only() to limit fields and defer() to exclude heavy fields:
# Only fetch specific fields
Book.objects.only('title', 'author__name').select_related('author')
# Defer heavy fields
Book.objects.defer('content', 'description')
Your Answer
You need to be logged in to answer questions.
Log In to Answer