Multi-Cloud Storage

You are a cloud infrastructure architect. The user wants to abstract storage operations across multiple cloud providers (AWS S3, Google Cloud Storage, Azure Blob Storage) with a unified interface.

What to check first

• Verify AWS SDK v3, Google Cloud Storage client, and Azure Storage Blob SDK are installed: npm list aws-sdk-js-v3 @google-cloud/storage @azure/storage-blob
• Confirm IAM credentials are configured for each cloud provider in environment variables or credential files
• Check that storage buckets exist in each cloud provider before deployment

Steps

1. Create an abstract storage interface class with methods like upload(), download(), delete(), and listObjects()
2. Implement provider-specific adapter classes for AWS S3, GCS, and Azure Blob Storage that conform to the interface
3. Add a factory function that instantiates the correct adapter based on a provider string parameter
4. Implement unified error handling that maps provider-specific exceptions to common error types
5. Add metadata normalization to return consistent object properties across all providers
6. Implement retry logic with exponential backoff for transient failures
7. Create a multi-cloud orchestrator that can write to multiple providers simultaneously or fall back to secondary providers
8. Add configuration management for provider credentials, region/location settings, and timeout values

Code

const AWS = require('@aws-sdk/client-s3');
const { Storage } = require('@google-cloud/storage');
const { BlobServiceClient } = require('@azure/storage-blob');

// Abstract storage interface
class StorageAdapter {
  async upload(bucket, key, data, metadata = {}) {
    throw new Error('upload() must be implemented');
  }

  async download(bucket, key) {
    throw new Error('download() must be implemented');
  }

  async delete(bucket, key) {
    throw new Error('delete() must be implemented');
  }

  async listObjects(bucket, prefix = '') {
    throw new Error('listObjects() must be implemented');
  }

  normalizeObject(obj) {
    throw new Error('normalizeObject() must be implemented');
  }
}

// AWS S3 adapter
class S3Adapter extends StorageAdapter {
  constructor(config) {
    super();
    this.client = new AWS.S3Client({ region: config.region || 'us-east-1' });
  }

  async upload(bucket, key, data, metadata = {}) {
    try {
      await this.client.send(
        new AWS.PutObjectCommand({
          Bucket: bucket,
          Key: key,
          Body: data,
          Metadata: metadata,
        })
      );
    } catch (error) {
      throw new StorageError(`S3 upload failed: ${error.message}`, 'S3', error);
    }
  }

  async download(bucket, key) {
    try {
      const result = await this.client.send(
        new AWS.GetObject

Note: this example was truncated in the source. See the GitHub repo for the latest full version.

Common Pitfalls

• Treating this skill as a one-shot solution — most workflows need iteration and verification
• Skipping the verification steps — you don't know it worked until you measure
• Applying this skill without understanding the underlying problem — read the related docs first

When NOT to Use This Skill

• When a simpler manual approach would take less than 10 minutes
• On critical production systems without testing in staging first
• When you don't have permission or authorization to make these changes

How to Verify It Worked

• Run the verification steps documented above
• Compare the output against your expected baseline
• Check logs for any warnings or errors — silent failures are the worst kind

Production Considerations

• Test in staging before deploying to production
• Have a rollback plan — every change should be reversible
• Monitor the affected systems for at least 24 hours after the change