API Response Perf

You are a backend performance engineer. The user wants to optimize API response times by identifying bottlenecks and implementing caching, query optimization, and response compression strategies.

What to check first

• Run npm list or pip list to confirm installed packages like redis, express-compression, or similar caching/compression libraries
• Check current API response times with curl -w "@curl-format.txt" -o /dev/null -s https://your-api.com/endpoint to establish baseline metrics
• Review database query logs (MySQL: SET GLOBAL slow_query_log = 'ON'; SET GLOBAL long_query_time = 0.5; or PostgreSQL: log_min_duration_statement = 500)

Steps

1. Install performance monitoring tools: npm install express-compression redis ioredis (Node) or pip install redis flask-caching (Python)
2. Enable HTTP compression by adding compression middleware before route handlers
3. Implement Redis caching for frequently-accessed endpoints with TTL (time-to-live) values
4. Add database query pagination with LIMIT and OFFSET to reduce payload size
5. Create database indexes on columns used in WHERE clauses and JOIN conditions
6. Implement response filtering to return only required fields instead of full objects
7. Set appropriate HTTP cache headers (Cache-Control, ETag, Last-Modified) for client-side caching
8. Use connection pooling for database connections to reuse existing connections

Code

const express = require('express');
const compression = require('compression');
const redis = require('redis');
const { promisify } = require('util');

const app = express();
const client = redis.createClient({ host: 'localhost', port: 6379 });
const getAsync = promisify(client.get).bind(client);
const setAsync = promisify(client.setex).bind(client);

// Enable gzip compression for responses > 1KB
app.use(compression({ threshold: 1024 }));

// Middleware for Redis caching
const cacheMiddleware = (ttl = 3600) => {
  return async (req, res, next) => {
    const cacheKey = `api:${req.originalUrl}`;
    try {
      const cachedResponse = await getAsync(cacheKey);
      if (cachedResponse) {
        return res.json(JSON.parse(cachedResponse));
      }
    } catch (err) {
      console.error('Cache read error:', err);
    }
    res.originalJson = res.json;
    res.json = function(data) {
      setAsync(cacheKey, ttl, JSON.stringify(data)).catch(err =>
        console.error('Cache write error:', err)
      );
      return res.originalJson(data);
    };
    next();
  };
};

// Optimized endpoint with caching and pagination
app.get('/api/users',

Note: this example was truncated in the source. See the GitHub repo for the latest full version.

Common Pitfalls

• Treating this skill as a one-shot solution — most workflows need iteration and verification
• Skipping the verification steps — you don't know it worked until you measure
• Applying this skill without understanding the underlying problem — read the related docs first

When NOT to Use This Skill

• When a simpler manual approach would take less than 10 minutes
• On critical production systems without testing in staging first
• When you don't have permission or authorization to make these changes

How to Verify It Worked

• Run the verification steps documented above
• Compare the output against your expected baseline
• Check logs for any warnings or errors — silent failures are the worst kind

Production Considerations

• Test in staging before deploying to production
• Have a rollback plan — every change should be reversible
• Monitor the affected systems for at least 24 hours after the change