Data Transformer

You are a data transformation specialist. The user wants to convert data between JSON, XML, and CSV formats bidirectionally with proper encoding, escaping, and schema validation.

What to check first

• Verify the source file exists and is readable: file <source_file>
• Check the input format is valid: jq empty <file.json> for JSON, xmllint --noout <file.xml> for XML
• Confirm required libraries are installed: npm list xml2js csv-parse js2xmlparser or equivalent

Steps

1. Detect input format by checking file extension and parsing first few bytes to validate structure
2. Load the source data into memory using the appropriate parser (JSON.parse, xml2js, or csv-parse)
3. Normalize the parsed data into an intermediate JavaScript object representation
4. Apply any field mapping, filtering, or transformation rules to the normalized structure
5. Validate the transformed data against expected schema (required fields, data types, constraints)
6. Serialize the normalized data into the target format using the appropriate serializer
7. Handle special cases: escaping quotes in CSV, CDATA sections in XML, nested arrays in JSON
8. Write output to file with proper encoding (UTF-8) and formatting (indentation for readability)

Code

const fs = require('fs');
const xml2js = require('xml2js');
const { Parser: CSVParser } = require('csv-parse/sync');
const { stringify: csvStringify } = require('csv-stringify/sync');
const js2xmlparser = require('js2xmlparser');

class DataTransformer {
  // Transform data between formats: json, xml, csv
  static transform(inputPath, outputPath, targetFormat) {
    // Detect input format from file extension
    const inputFormat = this.detectFormat(inputPath);
    
    // Read and parse input file
    const rawData = fs.readFileSync(inputPath, 'utf-8');
    const parsedData = this.parse(rawData, inputFormat);
    
    // Normalize to intermediate format (array of objects)
    const normalized = this.normalize(parsedData, inputFormat);
    
    // Validate data structure
    if (!this.validate(normalized)) {
      throw new Error('Data validation failed: invalid structure or missing required fields');
    }
    
    // Serialize to target format
    const output = this.serialize(normalized, targetFormat);
    
    // Write output file
    fs.writeFileSync(outputPath, output, 'utf-8');
    console.log(`Successfully transformed ${inputPath} to ${outputPath} (${targetFormat})`);
  }
  
  static detectFormat(filePath) {
    const ext = filePath.split('.').pop().toLowerCase();
    if (['json'].includes(ext)) return 'json';
    if (['xml'].includes(ext)) return 'xml';
    if (['csv'].includes(ext)) return 'csv';
    throw new Error(`

Note: this example was truncated in the source. See the GitHub repo for the latest full version.

Common Pitfalls

• Treating this skill as a one-shot solution — most workflows need iteration and verification
• Skipping the verification steps — you don't know it worked until you measure
• Applying this skill without understanding the underlying problem — read the related docs first

When NOT to Use This Skill

• When a simpler manual approach would take less than 10 minutes
• On critical production systems without testing in staging first
• When you don't have permission or authorization to make these changes

How to Verify It Worked

• Run the verification steps documented above
• Compare the output against your expected baseline
• Check logs for any warnings or errors — silent failures are the worst kind

Production Considerations

• Test in staging before deploying to production
• Have a rollback plan — every change should be reversible
• Monitor the affected systems for at least 24 hours after the change