Streaming Response

You are a backend developer implementing Server-Sent Events (SSE) streaming responses. The user wants to create endpoints that send data to clients in real-time chunks using HTTP streaming.

What to check first

• Verify your framework supports streaming responses (Express, Fastify, Node.js http module all do)
• Check that clients support EventSource API or fetch with ReadableStream for consuming streams
• Ensure you're not using response compression middleware that buffers data before sending

Steps

1. Create an HTTP endpoint that sets response headers to Content-Type: text/event-stream and Cache-Control: no-cache
2. Set Connection: keep-alive header to prevent the connection from closing after the first flush
3. Write data in SSE format: data: {content}\n\n (two newlines required to signal event end)
4. Use response.write() to send chunks without ending the response immediately
5. Call response.flush() if available (Express/Fastify) or rely on buffering to push data to client
6. Handle client disconnections by listening to close or end events on the response object
7. Implement exponential backoff or heartbeat by sending comment lines (:heartbeat\n\n) every 30 seconds to keep connection alive
8. End the stream with response.end() when all data is sent or on error

Code

// Express/Node.js SSE streaming endpoint
import express from 'express';

const app = express();

app.get('/stream', (req, res) => {
  // Set SSE headers
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  
  // Optional: tell proxies not to buffer
  res.setHeader('X-Accel-Buffering', 'no');

  // Simulate data source (replace with real data)
  let count = 0;
  const dataInterval = setInterval(() => {
    if (count < 10) {
      const data = { timestamp: Date.now(), message: `Event ${count}` };
      res.write(`data: ${JSON.stringify(data)}\n\n`);
      count++;
    } else {
      clearInterval(dataInterval);
      res.write('event: done\ndata: Stream ended\n\n');
      res.end();
    }
  }, 1000);

  // Send heartbeat every 30 seconds to keep connection alive
  const heartbeatInterval = setInterval(() => {
    res.write(':heartbeat\n\n');
  }, 30000);

  // Handle client disconnect
  req.on('close', () => {
    clearInterval(dataInterval);
    clearInterval(heartbeatInterval);
    res.end();
  });

  // Handle errors
  res.on('error', (err) => {

Note: this example was truncated in the source. See the GitHub repo for the latest full version.

Common Pitfalls

• Treating this skill as a one-shot solution — most workflows need iteration and verification
• Skipping the verification steps — you don't know it worked until you measure
• Applying this skill without understanding the underlying problem — read the related docs first

When NOT to Use This Skill

• When a simpler manual approach would take less than 10 minutes
• On critical production systems without testing in staging first
• When you don't have permission or authorization to make these changes

How to Verify It Worked

• Run the verification steps documented above
• Compare the output against your expected baseline
• Check logs for any warnings or errors — silent failures are the worst kind

Production Considerations

• Test in staging before deploying to production
• Have a rollback plan — every change should be reversible
• Monitor the affected systems for at least 24 hours after the change