Python: When should I use generators vs list comprehensions for memory efficiency?

Open
Aug 30, 2025 890 views 2 answers
30

I'm working on a Python application and running into an issue with Python performance. Here's the problematic code:


# Current implementation
def fibonacci(n):
    if n <= 1:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

# This causes RecursionError for large n
result = fibonacci(1000)

The error message I'm getting is: "ValueError: invalid literal for int() with base 10: 'abc'"

What I've tried so far:

  • Used pdb debugger to step through the code
  • Added logging statements to trace execution
  • Checked Python documentation and PEPs
  • Tested with different Python versions
  • Reviewed similar issues on GitHub and Stack Overflow

Environment information:

  • Python version: 3.11.0
  • Operating system: Ubuntu 22.04
  • Virtual environment: venv (activated)
  • Relevant packages: django, djangorestframework, celery, redis

Any insights or alternative approaches would be very helpful. Thanks!

A
Asked by abaditaye
Newbie 45 rep

2 Answers

24

Here's a comprehensive approach to implementing JWT authentication in Django REST Framework:

# settings.py
INSTALLED_APPS = [
    'rest_framework',
    'rest_framework_simplejwt',
]

REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework_simplejwt.authentication.JWTAuthentication',
    ),
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.IsAuthenticated',
    ],
}

from datetime import timedelta
SIMPLE_JWT = {
    'ACCESS_TOKEN_LIFETIME': timedelta(minutes=60),
    'REFRESH_TOKEN_LIFETIME': timedelta(days=7),
    'ROTATE_REFRESH_TOKENS': True,
}
# urls.py
from rest_framework_simplejwt.views import (
    TokenObtainPairView,
    TokenRefreshView,
)

urlpatterns = [
    path('api/token/', TokenObtainPairView.as_view()),
    path('api/token/refresh/', TokenRefreshView.as_view()),
]
# Custom serializer for additional user data
from rest_framework_simplejwt.serializers import TokenObtainPairSerializer

class CustomTokenObtainPairSerializer(TokenObtainPairSerializer):
    @classmethod
    def get_token(cls, user):
        token = super().get_token(user)
        token['username'] = user.username
        token['email'] = user.email
        return token
A
Answered by abdullah3 1 week, 4 days ago
Bronze 90 rep

Comments

michael_code: This Python memory optimization technique reduced my application's RAM usage by 60%. Brilliant! 1 week, 4 days ago

3

To optimize Django QuerySets and avoid N+1 problems, use select_related() for ForeignKey and OneToOneField, and prefetch_related() for ManyToManyField and reverse ForeignKey:

# Bad: N+1 query problem
for book in Book.objects.all():
    print(book.author.name)  # Each iteration hits the database

# Good: Use select_related for ForeignKey
for book in Book.objects.select_related('author'):
    print(book.author.name)  # Single query with JOIN

# Good: Use prefetch_related for ManyToMany
for book in Book.objects.prefetch_related('categories'):
    for category in book.categories.all():
        print(category.name)  # Optimized with separate query

You can also use only() to limit fields and defer() to exclude heavy fields:

# Only fetch specific fields
Book.objects.only('title', 'author__name').select_related('author')

# Defer heavy fields
Book.objects.defer('content', 'description')
J
Answered by john_doe 1 week, 4 days ago
Bronze 50 rep

Your Answer

You need to be logged in to answer questions.

Log In to Answer