JSON API Gateway: Request Transformation, Schema Validation & Caching

Last updated:

An API gateway sits between clients and your backend services, handling cross-cutting concerns that would otherwise be duplicated across every microservice: authentication, rate limiting, CORS, structured logging, and — critically for JSON APIs — request/response transformation and schema validation. Unlike a reverse proxy (Nginx, HAProxy) which forwards HTTP traffic without understanding the payload, an API gateway is payload-aware: it can inspect, validate, transform, and cache the JSON bodies flowing through it. This guide focuses on the JSON-specific features that most API gateway articles skip: field-level request transformation, gateway-level JSON schema validation, response caching strategies tuned for JSON endpoints, structured JSON rate-limit error responses, AWS API Gateway mapping templates for Lambda integrations, and Kong declarative YAML configuration for all major plugins.

What an API Gateway Does for JSON APIs

A JSON API gateway consolidates six concerns that would otherwise be implemented separately in every backend service: routing, authentication, rate limiting, request/response transformation, schema validation, and caching. Each concern has a JSON-specific dimension that generic gateway documentation often omits.

Routing at an API gateway can be based on JSON body content, not just URL path. Advanced gateways support routing to different upstreams based on a JSONPath expression evaluated against the request body — for example, routing {"type": "payment"} to the payments service and {"type": "refund"} to the refunds service, even on the same endpoint URL.

Authentication is the most universally implemented gateway concern. JWT verification at the gateway means your backend receives only pre-authenticated requests — no need for JWT parsing libraries or verification middleware in each service. The decoded claims can be injected as request headers (e.g., X-User-Id, X-User-Role) so backends receive structured context without touching the Authorization header.

Transformation is where API gateways provide the most JSON-specific value. Field remapping between client-facing and internal API contracts, stripping internal fields from responses before they reach clients, injecting metadata fields, and converting between JSON structures for versioned APIs — all handled at the gateway without modifying backend code.

# The six concerns an API gateway centralizes for JSON APIs
# Without a gateway: each microservice implements all of these

┌─────────────────────────────────────────────────────────┐
│                    API Gateway                          │
│                                                         │
│  1. Routing       — URL path, method, body JSONPath     │
│  2. Auth          — JWT verify, API key, OAuth2         │
│  3. Rate Limiting — per-consumer token bucket / window  │
│  4. Transformation— JSON field map, add/remove/rename   │
│  5. Validation    — JSON Schema body validation         │
│  6. Caching       — Cache-Control, ETag, TTL per route  │
└─────────────────────────────────────────────────────────┘
         │                          │
   ┌─────▼──────┐            ┌──────▼──────┐
   │ Service A  │            │  Service B  │
   │ (no auth   │            │  (no rate   │
   │  code)     │            │   limiting) │
   └────────────┘            └─────────────┘

# Without a gateway — cross-cutting code per service:
# Each service adds ~40% boilerplate for auth, rate limiting,
# CORS, error formatting, structured logging, and health checks.
# A gateway eliminates this duplication.

# JSON-specific gateway capabilities most articles skip:
# - Route by request body field value (content-based routing)
# - Validate request body against JSON Schema (reject before backend)
# - Strip internal fields from JSON responses (e.g., internalId, cost)
# - Inject decoded JWT claims as JSON headers for downstream services
# - Cache JSON responses with ETag and Vary header awareness
# - Return structured JSON 4xx/5xx errors in your API's error format

The 40% code reduction figure comes from eliminating common middleware patterns — JWT verification, rate limit counters, CORS header injection, structured error formatting, and health check endpoints — from each service. These are policy concerns, not business logic. Moving them to the gateway makes each service smaller, easier to test, and deployable independently of policy changes.

JSON Request/Response Transformation

JSON transformation at the gateway serves two primary use cases: API versioning (mapping client-facing v1 fields to backend v2 field names) and contract isolation (preventing internal field names, database IDs, and cost data from leaking to external clients). Both cases are handled without modifying backend code.

# ── Kong request-transformer plugin ──────────────────────────────────
# Rename, add, remove fields in the JSON request body before
# forwarding to the upstream service.

# Declarative YAML config (kong.yaml):
plugins:
  - name: request-transformer
    config:
      # Rename client field name to backend field name
      rename:
        body:
          - "userId:user_id"        # renames "userId" -> "user_id"
          - "orderId:order_id"

      # Add fields to the forwarded request
      add:
        body:
          - "source:gateway"        # injects constant value
          - "api_version:2"
        headers:
          - "X-Gateway-Version:2"

      # Remove fields before forwarding (client-only fields)
      remove:
        body:
          - "clientMetadata"        # strip fields the backend doesn't need
          - "uiSessionId"
        headers:
          - "X-Debug-Mode"          # never forward debug headers

# ── Kong response-transformer plugin ─────────────────────────────────
# Strip internal fields from the JSON response before returning
# to the client. Prevents data leakage.

plugins:
  - name: response-transformer
    config:
      remove:
        body:
          - "internalCost"          # never expose cost data
          - "supplierId"            # internal supplier reference
          - "debugTrace"
        headers:
          - "X-Internal-Service"
      rename:
        body:
          - "user_id:userId"        # normalize to camelCase for clients
          - "created_at:createdAt"
      add:
        headers:
          - "X-API-Version:v2"

# ── AWS API Gateway mapping template (VTL) ───────────────────────────
# Integration Request mapping template (application/json)
# Transforms the client request before sending to Lambda:

{
  "user_id":  "$input.path('$.userId')",
  "action":   "$input.path('$.action')",
  "amount":   $input.path('$.amount'),
  "metadata": {
    "request_id": "$context.requestId",
    "source_ip":  "$context.identity.sourceIp",
    "stage":      "$context.stage"
  }
}

# Integration Response mapping template
# Transforms the Lambda response before returning to client:
#set($body = $input.path('$'))
{
  "id":        "$body.internal_id",
  "status":    "$body.status",
  "createdAt": "$body.created_at"
}

# ── jq-style transformation logic ────────────────────────────────────
# For complex reshaping outside VTL, use a Lambda or serverless
# function as a transformer. Example: restructure nested array:

# Input (from client):
# { "items": [{"product_id": 1, "qty": 2}] }

# Target (backend expects):
# { "line_items": [{"sku": 1, "quantity": 2}] }

# Lambda transformer (Node.js):
export const handler = async (event) => {
  const body = JSON.parse(event.body);
  const transformed = {
    line_items: body.items.map(item => ({
      sku:      item.product_id,
      quantity: item.qty,
    })),
  };
  return {
    statusCode: 200,
    body: JSON.stringify(transformed),
  };
};

The critical distinction between Kong's request-transformer and AWS VTL mapping templates: Kong's plugin uses a declarative field-list configuration that covers the majority of real-world cases without code. VTL gives full programmatic control at the cost of a bespoke template language that is difficult to test locally. For teams on AWS, the Lambda proxy integration pattern (skip mapping templates, handle transformation in Lambda) is often more maintainable despite the additional function hop.

JSON Schema Validation at the Gateway

Validating JSON schema at the gateway is a correctness and availability control. Invalid requests are rejected in under 1ms before any backend compute is consumed — no Lambda invocations, no database connections, no application error handling. On public APIs that receive significant invalid traffic (mobile clients with stale app versions, third-party integrations), this can reduce backend load by 20–60%.

# ── Kong request-validator plugin ────────────────────────────────────
# Validates request body against a schema before forwarding.
# Returns 400 if validation fails — upstream never receives the request.

plugins:
  - name: request-validator
    config:
      body_schema:
        - name: userId
          type: string
          required: true
        - name: amount
          type: number
          required: true
        - name: currency
          type: string
          required: true
          # Kong Enterprise supports "one_of" validation:
          # one_of: ["USD", "EUR", "GBP"]
        - name: items
          type: array
          required: false
      # Reject requests where Content-Type is not application/json:
      content_type_parameter_validation: true
      # Return verbose validation errors (useful for development):
      verbose_response: true

# Error response when validation fails (Kong default):
# HTTP 400
# {
#   "message": "request body validation failed, ...",
#   "name": "schema violation",
#   "fields": { "userId": "required field missing" }
# }

# ── AWS API Gateway model-based validation ────────────────────────────
# Step 1: Create a Model (JSON Schema draft-04)

POST /restapis/{restapi_id}/models
{
  "name": "CreateOrderRequest",
  "contentType": "application/json",
  "schema": {
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "required": ["userId", "amount", "currency"],
    "properties": {
      "userId":   { "type": "string", "minLength": 1 },
      "amount":   { "type": "number", "minimum": 0.01 },
      "currency": { "type": "string", "enum": ["USD", "EUR", "GBP"] },
      "items": {
        "type": "array",
        "items": {
          "type": "object",
          "required": ["productId", "qty"],
          "properties": {
            "productId": { "type": "string" },
            "qty":       { "type": "integer", "minimum": 1 }
          }
        }
      }
    },
    "additionalProperties": false
  }
}

# Step 2: Enable validation on the method request
# In API Gateway console: Method Request -> Request Validator
# Set to "Validate body" or "Validate body, query string parameters,
# and headers"

# When validation fails, AWS API Gateway returns:
# HTTP 400
# {
#   "message": "Invalid request body"
# }

# Customize the error body via Gateway Responses -> BAD_REQUEST_BODY:
{
  "error": "validation_error",
  "message": "$context.error.validationErrorString",
  "requestId": "$context.requestId"
}

# ── Performance: validation before backend ────────────────────────────
# Without gateway validation — invalid request lifecycle:
# Client -> Gateway (0ms) -> Backend (5ms) -> DB query (10ms)
#        -> Validation error (15ms) -> 400 response
# Total: 15-30ms, backend compute consumed

# With gateway validation — invalid request lifecycle:
# Client -> Gateway validation (<1ms) -> 400 response
# Total: <5ms, zero backend compute consumed

AWS API Gateway model validation supports JSON Schema draft-04, which covers required, properties, type, enum, minimum, maximum, minLength, maxLength, pattern, and additionalProperties. Draft-07 features like if/then/else, anyOf, and $defs are not supported. For full draft-07 validation, use Kong's request-validator (Enterprise) or implement validation in a Lambda authorizer before the main handler. See the JSON Schema validation guide for application-layer validation with Ajv and Zod.

JSON Response Caching Strategies

Caching JSON responses at the gateway is one of the highest-leverage optimizations for read-heavy APIs. A cached response requires zero backend computation, zero database queries, and returns in under 5ms from gateway memory or a shared cache store. The critical requirement: cache keys must correctly incorporate all dimensions that affect the response — different userId values must not share the same cache entry.

# ── Kong proxy-cache plugin ──────────────────────────────────────────
# Caches upstream JSON responses in memory (default) or Redis.
# Reduces backend load by up to 80% for cacheable endpoints.

plugins:
  - name: proxy-cache
    config:
      # Which HTTP methods to cache
      request_method:
        - "GET"
        - "HEAD"
      # Which response status codes to cache
      response_code:
        - 200
        - 301
        - 404
      # Which Content-Types to cache
      content_type:
        - "application/json"
        - "application/json; charset=utf-8"
      # Cache TTL in seconds
      cache_ttl: 300          # 5 minutes default
      # Cache storage backend
      strategy: memory        # or: redis

      # With Redis backend (shared across Kong nodes):
      # strategy: redis
      # redis:
      #   host: redis.internal
      #   port: 6379
      #   timeout: 2000

# ── Cache key dimensions ──────────────────────────────────────────────
# By default, Kong cache key = method + URL + query string + Vary headers
# For per-user caching, add Authorization or X-User-Id to Vary:

routes:
  - name: user-profile
    paths:
      - "/api/v1/users/me"
    plugins:
      - name: proxy-cache
        config:
          cache_ttl: 60          # 1 minute for user-specific data
          vary_headers:
            - "Authorization"    # separate cache entry per token
            # Or inject the decoded user ID as a header via JWT plugin:
            # - "X-User-Id"

# ── Cache-Control header strategy ────────────────────────────────────
# The upstream service controls cacheability via Cache-Control.
# Kong respects Cache-Control: no-store and no-cache by default.

# Backend response headers for a cacheable product listing:
Cache-Control: public, max-age=300, stale-while-revalidate=60
ETag: "a1b2c3d4e5"
Vary: Accept-Encoding, Accept-Language

# Backend response headers for a user-specific response:
Cache-Control: private, max-age=60
Vary: Authorization

# Backend response for a never-cache endpoint:
Cache-Control: no-store

# ── ETag-based conditional requests ──────────────────────────────────
# Gateway serves ETag in first response.
# Client sends If-None-Match on subsequent requests.
# If content unchanged, gateway returns 304 Not Modified (no body).

# First request:
GET /api/products/123
# Response:
HTTP/1.1 200 OK
ETag: "abc123"
Cache-Control: public, max-age=300
{ "id": 123, "name": "Widget", "price": 19.99 }

# Subsequent request (client sends If-None-Match):
GET /api/products/123
If-None-Match: "abc123"
# Response (unchanged):
HTTP/1.1 304 Not Modified
ETag: "abc123"
# No body — client uses its cached copy

# ── Per-route TTL configuration ───────────────────────────────────────
# Different TTLs for different endpoint characteristics:

# Static reference data: long TTL
/api/v1/countries         -> cache_ttl: 86400   # 24 hours
/api/v1/currencies        -> cache_ttl: 3600    # 1 hour
/api/v1/product-catalog   -> cache_ttl: 300     # 5 minutes

# User-specific data: short TTL
/api/v1/users/me          -> cache_ttl: 60      # 1 minute
/api/v1/orders            -> cache_ttl: 30      # 30 seconds

# Real-time data: no cache
/api/v1/inventory         -> Cache-Control: no-store
/api/v1/pricing/realtime  -> Cache-Control: no-store

The most common caching mistake with JSON APIs is failing to set the Vary header correctly on user-specific responses. Without Vary: Authorization, a cached response for user A can be served to user B — a serious data leakage bug. Always test caching behavior with two different authenticated users before deploying. See the JSON caching strategies guide for application-layer caching patterns with Redis.

Rate Limiting JSON APIs at the Gateway

Rate limiting at the gateway prevents individual consumers from overwhelming your backend services. Centralizing this at the gateway means backend services never handle throttled requests — the 429 response is returned by the gateway without any upstream call. The JSON error format returned on 429 should match your API's standard error schema so clients can handle it consistently.

# ── Kong rate-limiting plugin ─────────────────────────────────────────
# Multiple window sizes simultaneously. Per-consumer or per-IP.

plugins:
  - name: rate-limiting
    config:
      # Limit 100 req/minute AND 10 req/second (both enforced)
      second: 10
      minute: 100
      hour: 2000
      day: 20000

      # Identifier for rate limiting (who to limit):
      limit_by: consumer       # or: ip, credential, service, header, path
      # For header-based: limit_by: header + header_name: "X-API-Key"

      # Algorithm: fixed_window (default) or sliding_window
      policy: local            # or: redis (shared across Kong nodes)

      # Fault tolerance: if Redis is down, allow traffic through?
      fault_tolerant: true

      # Custom error response on 429:
      error_code: 429
      error_message: "rate limit exceeded"

# Standard rate-limit response headers (Kong injects these):
# X-RateLimit-Limit-Second: 10
# X-RateLimit-Limit-Minute: 100
# X-RateLimit-Remaining-Second: 7
# X-RateLimit-Remaining-Minute: 94
# RateLimit-Reset: 1706361600    (Unix timestamp, window reset)
# Retry-After: 45                (seconds until client may retry)

# ── JSON error response format on 429 ────────────────────────────────
# Default Kong 429 body:
{ "message": "API rate limit exceeded" }

# Customize to match your API error schema using Kong's response
# for RATE_LIMITED error type or a custom serverless-function plugin:

# Target error format:
{
  "error": {
    "code": "rate_limit_exceeded",
    "message": "You have exceeded 100 requests per minute.",
    "retry_after_seconds": 45,
    "limit": {
      "window": "minute",
      "max_requests": 100
    }
  }
}

# ── Per-consumer rate limits ──────────────────────────────────────────
# Different limits for different consumer tiers:

consumers:
  - username: free-tier-user
    plugins:
      - name: rate-limiting
        config:
          minute: 60         # 1 req/second average
          day: 1000

  - username: pro-tier-user
    plugins:
      - name: rate-limiting
        config:
          minute: 1000       # 16 req/second average
          day: 100000

  - username: enterprise-user
    plugins:
      - name: rate-limiting
        config:
          second: 100        # 100 req/second
          # No daily limit for enterprise

# ── AWS API Gateway usage plans + throttling ──────────────────────────
# Usage plan: attach to API stage, link to API keys
# Throttle: burst capacity + steady-state rate

aws_usage_plan:
  throttle:
    burstLimit: 500          # max concurrent requests (token bucket burst)
    rateLimit:  100          # steady-state requests per second
  quota:
    limit:  10000            # total requests
    period: DAY              # DAY, WEEK, or MONTH

# AWS 429 response (default):
# { "message": "Too Many Requests" }

# Customize via Gateway Response (THROTTLED type):
# { "error": "throttled", "message": "$context.error.message",
#   "requestId": "$context.requestId" }

# ── Token bucket vs sliding window ───────────────────────────────────
# Token bucket (AWS API Gateway default):
#   - Allows burst up to burstLimit even if rate exceeds rateLimit
#   - Good for APIs with bursty but well-behaved clients
#   - Risk: a client can consume the entire burst allowance instantly

# Sliding window (Kong rate-limiting-advanced plugin):
#   - Smoothly distributes limit over the window
#   - No burst allowance — 100 req/min = max 1.67 req/second always
#   - Better for protecting backends from sudden load spikes

For distributed Kong deployments (multiple Kong nodes), set policy: redis so rate limit counters are shared across all nodes. Without Redis, each node maintains its own counter, effectively multiplying the per-node limit by the number of nodes — a consumer can make 100 requests per minute to each of 3 Kong nodes for a total of 300 requests/minute against your backend. For JSON rate limiting patterns in application code, see the linked guide.

AWS API Gateway JSON Integration

AWS API Gateway has two integration modes with significantly different JSON handling: Lambda proxy integration (raw pass-through, Lambda controls the response structure) and non-proxy integration (VTL mapping templates transform request and response bodies). Most teams default to Lambda proxy integration for its simplicity; non-proxy integration is necessary when you need server-side JSON transformation without Lambda code changes.

# ── Lambda proxy integration (most common) ───────────────────────────
# API Gateway passes the raw request as a structured event object.
# Lambda must return a specific response structure.

# Lambda receives this event (proxy integration):
{
  "httpMethod": "POST",
  "path": "/orders",
  "headers": {
    "Content-Type": "application/json",
    "Authorization": "Bearer eyJ..."
  },
  "body": "{"userId":"u_123","amount":99.99}",  # body is a string
  "isBase64Encoded": false,
  "queryStringParameters": null,
  "requestContext": {
    "requestId": "abc-123",
    "identity": { "sourceIp": "1.2.3.4" }
  }
}

# Lambda must return this structure:
export const handler = async (event) => {
  const body = JSON.parse(event.body);  // parse the string body

  const result = await processOrder(body);

  return {
    statusCode: 201,
    headers: {
      "Content-Type": "application/json",
      "Cache-Control": "no-store",
    },
    body: JSON.stringify({          // body must be serialized string
      id: result.orderId,
      status: "created",
    }),
  };
};

# ── Non-proxy integration with VTL mapping templates ─────────────────
# Transforms the request before Lambda and the response after Lambda.
# Lambda receives the already-transformed body directly (no JSON.parse
# of the outer event structure needed).

# Integration Request mapping template (application/json):
# Transforms client request body before sending to Lambda:
{
  "user_id":    "$input.path('$.userId')",
  "amount":     $input.path('$.amount'),
  "currency":   "$input.path('$.currency')",
  "request_id": "$context.requestId",
  "ip_address": "$context.identity.sourceIp"
}

# Lambda receives this directly (non-proxy):
{
  "user_id": "u_123",
  "amount": 99.99,
  "currency": "USD",
  "request_id": "abc-123",
  "ip_address": "1.2.3.4"
}

# Integration Response mapping template:
# Transforms Lambda output before returning to client.
# $input.path('$') accesses the Lambda return value.
#set($result = $input.path('$'))
{
  "orderId":   "$result.internal_order_id",
  "status":    "$result.status",
  "createdAt": "$result.created_ts"
}

# ── API Gateway request/response models ──────────────────────────────
# Models define JSON Schema for request body validation and
# response documentation. Create via AWS console or CLI:

aws apigateway create-model   --rest-api-id "abc123"   --name "CreateOrderRequest"   --content-type "application/json"   --schema '{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "required": ["userId", "amount"],
    "properties": {
      "userId":   { "type": "string" },
      "amount":   { "type": "number", "minimum": 0 },
      "currency": { "type": "string", "enum": ["USD","EUR","GBP"] }
    }
  }'

# Enable validation on the method request:
aws apigateway update-method   --rest-api-id "abc123"   --resource-id "xyz789"   --http-method "POST"   --patch-operations     op=replace,path=/requestValidatorId,value={validatorId}

# ── HTTP API (API Gateway v2) — simpler, faster, cheaper ─────────────
# No VTL support — uses parameter mapping for headers/query/path only.
# Body transformation requires Lambda or a dedicated transformer.

# serverless.yml (AWS SAM / Serverless Framework):
functions:
  createOrder:
    handler: src/orders.handler
    events:
      - httpApi:
          path: /orders
          method: POST
          # No body transformation on HTTP APIs
          # Handle validation inside the Lambda function

AWS API Gateway charges per million API calls ($3.50/million for REST APIs, $1.00/million for HTTP APIs) plus data transfer. The 5–20ms gateway overhead is fixed regardless of backend execution time. For latency-sensitive APIs, use HTTP APIs (API Gateway v2) which add 2–10ms; for transformation-heavy APIs that need VTL, REST APIs are the only option. Lambda cold starts (100–3000ms) dominate latency for infrequently called endpoints — use Provisioned Concurrency to eliminate cold starts on critical paths.

Kong Gateway JSON Configuration

Kong supports declarative configuration via a kong.yaml file (deck CLI) — the entire gateway configuration is expressed as YAML and applied atomically. This is the recommended approach over the Admin API for production deployments, as it enables version control, code review, and GitOps workflows. Below is a complete declarative configuration for a JSON API with authentication, rate limiting, transformation, caching, and CORS.

# kong.yaml — declarative configuration (deck sync)
_format_version: "3.0"
_transform: true

# ── Services: upstream backends ───────────────────────────────────────
services:
  - name: orders-service
    url: http://orders.internal:8080
    connect_timeout: 5000
    read_timeout: 30000
    write_timeout: 30000
    retries: 3

  - name: products-service
    url: http://products.internal:8080
    connect_timeout: 5000
    read_timeout: 10000

# ── Routes: incoming request matching ─────────────────────────────────
routes:
  - name: create-order
    service: orders-service
    methods: ["POST"]
    paths: ["/api/v1/orders"]
    strip_path: false

  - name: get-products
    service: products-service
    methods: ["GET"]
    paths: ["/api/v1/products"]
    strip_path: false

# ── Plugins: JSON-specific cross-cutting concerns ─────────────────────
plugins:

  # 1. JWT Authentication (global — applies to all routes)
  - name: jwt
    config:
      claims_to_verify:
        - exp                    # verify token not expired
      key_claim_name: kid        # which claim holds the key ID
      secret_is_base64: false

  # 2. Rate Limiting (global default)
  - name: rate-limiting
    config:
      minute: 200
      hour: 5000
      policy: redis
      redis_host: redis.internal
      redis_port: 6379
      fault_tolerant: true

  # 3. CORS (global — required for browser clients)
  - name: cors
    config:
      origins:
        - "https://app.example.com"
        - "https://staging.example.com"
      methods:
        - GET
        - POST
        - PUT
        - DELETE
        - OPTIONS
      headers:
        - Authorization
        - Content-Type
        - X-Request-ID
      exposed_headers:
        - X-RateLimit-Remaining-Minute
        - X-RateLimit-Limit-Minute
        - Retry-After
      credentials: true
      max_age: 3600

  # 4. Request validation (on create-order route)
  - name: request-validator
    route: create-order
    config:
      body_schema:
        - name: userId
          type: string
          required: true
        - name: amount
          type: number
          required: true
        - name: currency
          type: string
          required: true

  # 5. Request transformation (on create-order route)
  - name: request-transformer
    route: create-order
    config:
      rename:
        body:
          - "userId:user_id"
          - "orderId:order_id"
      add:
        headers:
          - "X-Gateway-Version:2"
      remove:
        body:
          - "uiSessionId"

  # 6. Response caching (on get-products route)
  - name: proxy-cache
    route: get-products
    config:
      request_method: ["GET", "HEAD"]
      response_code: [200]
      content_type: ["application/json"]
      cache_ttl: 300
      strategy: redis
      redis:
        host: redis.internal
        port: 6379

  # 7. Response transformation (on get-products route)
  - name: response-transformer
    route: get-products
    config:
      remove:
        body:
          - "internalCost"
          - "supplierId"
      add:
        headers:
          - "Cache-Control:public, max-age=300"

# ── Consumers: per-consumer config ───────────────────────────────────
consumers:
  - username: mobile-app
    jwt_secrets:
      - key: mobile-app-key
        secret: "{{ env 'MOBILE_APP_JWT_SECRET' }}"
    plugins:
      - name: rate-limiting
        config:
          minute: 500           # higher limit for first-party app
          policy: redis
          redis_host: redis.internal

  - username: partner-api
    jwt_secrets:
      - key: partner-api-key
        secret: "{{ env 'PARTNER_JWT_SECRET' }}"
    plugins:
      - name: rate-limiting
        config:
          minute: 100
          day: 10000

# ── Apply configuration with deck ────────────────────────────────────
# deck sync --kong-addr http://kong-admin:8001 --state kong.yaml
# deck diff --kong-addr http://kong-admin:8001 --state kong.yaml

The deck sync command applies the declarative configuration atomically — it diffs the desired state against current Kong state and applies only the changes. Use deck diff in CI to preview changes before applying to production. Secrets are injected via environment variable substitution ({{ env 'SECRET_NAME' }}) rather than hardcoded in the YAML file. Store kong.yaml in version control, reference secrets from your secrets manager, and run deck sync as a deployment step. See the JSON API design guide for upstream service contract conventions that pair well with this gateway configuration.

Key Terms

API Gateway
An application-layer intermediary that sits between clients and backend services, handling cross-cutting concerns: routing, authentication, rate limiting, JSON transformation, schema validation, and response caching. Unlike a reverse proxy (which is payload-agnostic), an API gateway is payload-aware — it can inspect, validate, and transform JSON request and response bodies. Common implementations include Kong (open-source and Enterprise), AWS API Gateway, Google Apigee, Azure API Management, and Traefik. An API gateway centralizes concerns that would otherwise require separate middleware implementations in each microservice, typically eliminating ~40% of boilerplate infrastructure code from backend services.
Request Transformation
The process of modifying a JSON request body (field renaming, addition, removal, or restructuring) at the gateway before forwarding to the upstream backend service. Request transformation enables contract isolation between public API fields (camelCase, versioned) and internal service fields (snake_case, implementation-specific), API versioning compatibility (mapping v1 client fields to v2 backend fields), and injection of gateway-level metadata (request ID, source IP, decoded JWT claims). In Kong, the request-transformer plugin handles declarative field-level configuration. In AWS API Gateway, Velocity Template Language (VTL) mapping templates provide full programmatic control over request reshaping.
Response Transformation
The process of modifying a JSON response body at the gateway before returning to the client. Response transformation prevents data leakage by stripping internal fields (cost data, supplier IDs, internal database identifiers) that should never be exposed to external clients. It also supports API versioning by renaming response fields to match the client-facing contract while the backend uses its own naming convention. In Kong, the response-transformer plugin mirrors the request-transformer interface. In AWS API Gateway, Integration Response mapping templates transform the Lambda return value. Response transformation runs after the upstream responds and before the gateway sends the reply, adding minimal latency (0.5–3ms depending on payload size).
Mapping Template
A Velocity Template Language (VTL) document used in AWS API Gateway to transform JSON request or response bodies. Integration Request mapping templates run before the request reaches the Lambda function or HTTP backend; Integration Response mapping templates run after the backend responds. VTL provides variables for accessing the request body ($input.path('$.field')), context ($context.requestId), stage variables ($stageVariables.version), and conditional logic. Mapping templates are only available in AWS REST APIs (API Gateway v1) — HTTP APIs (v2) do not support VTL body transformation. Mapping templates are evaluated server-side by API Gateway without invoking Lambda, adding no Lambda compute cost for transformation.
Plugin (Kong)
A modular extension to Kong Gateway that adds specific functionality to the request/response lifecycle. Plugins execute in defined phases: rewrite (early URL rewriting), access (authentication, rate limiting, validation, transformation), header_filter (modify response headers), body_filter (modify response body in chunks), and log (access logging, analytics). Plugins can be applied globally (all services and routes), per service, per route, or per consumer. Core open-source plugins relevant to JSON APIs include: jwt, rate-limiting, cors, request-validator, request-transformer, response-transformer, and proxy-cache. Kong Enterprise adds advanced versions: rate-limiting-advanced (sliding window, Redis Cluster), request-transformer-advanced (Lua expressions), and oas-validation (OpenAPI Schema validation).
Lambda Proxy Integration
An AWS API Gateway integration mode where the entire HTTP request is passed to a Lambda function as a structured event object, and the Lambda function is responsible for returning a complete HTTP response including status code, headers, and a JSON-serialized body string. In proxy mode, no VTL mapping templates are applied — the Lambda function receives the raw request body as a string in event.body and must call JSON.parse(event.body) to access the JSON data. The Lambda must return { statusCode, headers, body: JSON.stringify(result) }. Proxy integration is simpler to implement but requires the Lambda to handle all response formatting. Non-proxy (custom) integration supports VTL transformation but requires maintaining mapping templates separately from application code.
ETag Caching
An HTTP caching mechanism where the server returns an ETag header with each response — a hash or version identifier for the specific response content. On subsequent requests, the client sends the ETag value in an If-None-Match header. If the resource has not changed, the server (or gateway cache) returns HTTP 304 Not Modified with no body, saving bandwidth. If the resource changed, the server returns the full 200 response with a new ETag. At the API gateway level, Kong's proxy-cache plugin and AWS API Gateway caching both participate in ETag validation. For JSON APIs, ETags should be computed from the response content — typically a hash of the serialized JSON body. ETags work alongside Cache-Control headers: max-age controls how long clients cache without re-validating, while ETags enable efficient revalidation after the max-age expires.

FAQ

What is the difference between an API gateway and a reverse proxy for JSON APIs?

A reverse proxy (Nginx, HAProxy) is payload-agnostic — it routes HTTP traffic based on URL path or hostname without understanding or modifying the request or response body. An API gateway is payload-aware: it understands the semantics of JSON API requests and can inspect, validate, transform, and cache JSON bodies. For JSON APIs specifically, an API gateway validates incoming JSON against a schema and rejects malformed payloads in <1ms before backend processing, transforms request and response fields (renaming, adding, removing) without modifying backend code, applies per-consumer rate limits and returns structured JSON 429 error responses, verifies JWTs and injects decoded claims as headers, caches JSON responses with correct ETag and Vary header handling, and returns JSON-formatted 4xx/5xx errors matching your API's error schema. The practical consequence: centralizing these concerns at the gateway eliminates approximately 40% of boilerplate middleware code from each backend service — no auth middleware, rate limiting logic, CORS headers, or structured error formatting needs to be duplicated per service.

How do I transform JSON request/response bodies in AWS API Gateway?

AWS API Gateway uses Velocity Template Language (VTL) mapping templates to transform JSON bodies in non-proxy integrations. In the Integration Request, add an application/json mapping template: use $input.path('$.fieldName') to read a field from the incoming body and compose the transformed JSON that your Lambda or HTTP backend receives. For example: {"user_id": "$input.path('$.userId')", "source": "gateway"} renames the field and injects a constant. In the Integration Response, add a mapping template that reads from the Lambda output via $input.path('$') and reshapes it. VTL supports conditional logic with #if, loops with #foreach, and string manipulation. For Lambda proxy integrations (the most common setup), no mapping templates are applied — the Lambda receives the raw event and controls its own JSON response structure. HTTP APIs (API Gateway v2) do not support VTL body transformation at all; use a Lambda function to handle reshaping. For complex transformations on REST APIs, consider a thin Lambda transformer instead of VTL, as VTL is harder to test locally and lacks good tooling.

Can I validate JSON schema at the API gateway level before hitting my backend?

Yes, and it is one of the most valuable optimizations for public JSON APIs. AWS API Gateway supports native request body validation: create a Model (JSON Schema draft-04 definition), assign it to the method request, and enable "Validate body" — the gateway returns 400 Bad Request with a validation error message without invoking your Lambda. Kong provides the request-validator plugin, which accepts a body_schema array defining expected fields and types and rejects non-conforming requests with a configurable error message. The performance benefit is significant: gateway-level validation rejects invalid requests in under 1ms before any backend compute is consumed. On APIs that receive significant invalid traffic — mobile clients with outdated app versions, third-party webhook integrations, public APIs — gateway validation can reduce backend load by 20–60%. AWS API Gateway model validation covers JSON Schema draft-04 vocabulary: required, type, enum, minimum, maximum, pattern, and additionalProperties. For draft-07 features like anyOf or if/then/else, implement validation in a Lambda authorizer before the main handler, or use Kong Enterprise's OAS Validation plugin.

How does Kong API Gateway handle JSON request transformation?

Kong's request-transformer plugin uses declarative field-list configuration to add, remove, replace, and rename fields in the JSON request body before forwarding to the upstream service. You configure rename.body as an array of "oldName:newName" mappings, add.body for injecting constant values, remove.body for stripping fields, and replace.body for overwriting existing field values. Kong handles JSON parsing and re-serialization automatically. For complex transformations — computing derived values, conditional field mapping, restructuring nested arrays — Kong Enterprise's request-transformer-advanced plugin accepts Lua expressions for each field operation, and the serverless-functions plugin executes arbitrary Lua code in any request phase. The response-transformer plugin mirrors this interface for outbound responses: strip internal fields (cost data, supplier IDs) before returning to clients, inject metadata headers, or normalize field names for API versioning. Kong processes transformation in the access phase for requests and body_filter phase for responses. The body_filter phase buffers the full response body before modification, adding memory overhead proportional to response size — consider chunked streaming for very large JSON responses.

What is the performance overhead of an API gateway for JSON APIs?

Latency overhead varies by gateway product and configuration. AWS API Gateway REST APIs add approximately 5–20ms per request in us-east-1 (TLS termination, routing, optional VTL evaluation, integration call). AWS HTTP APIs (v2) are faster at 2–10ms. Kong adds roughly 1–5ms on a well-tuned instance with minimal plugins; each active plugin adds 0.1–2ms depending on complexity, and body transformation plugins add 0.5–3ms depending on payload size. JSON schema validation adds less than 1ms. The largest latency variable is Lambda cold starts for AWS integrations: a cold Lambda invocation adds 100–3000ms on the first request after an idle period — this is a Lambda cost, not a gateway cost. Kong's proxy-cache plugin can eliminate backend latency entirely for cacheable endpoints, serving cached JSON responses in under 5ms regardless of backend response time — for read-heavy APIs, this more than compensates for any gateway overhead. To minimize gateway latency: use HTTP/2 or HTTP/3 between client and gateway, keep plugin chains short, enable keep-alive connections to upstream services, and use caching aggressively for stable data.

How do I configure rate limiting with JSON error responses at the API gateway?

Kong's rate-limiting plugin enforces multiple window sizes simultaneously (second, minute, hour, day) and returns HTTP 429 with headers X-RateLimit-Limit-Minute, X-RateLimit-Remaining-Minute, and Retry-After. The default 429 body is {"message": "API rate limit exceeded"}. To customize this to match your API's error schema, use a Kong serverless-function plugin in the header_filter phase to intercept 429 responses and rewrite the body, or use the error-transformer plugin (Kong Enterprise). For distributed Kong deployments, set policy: redis so counters are shared across all nodes — without Redis, per-node limits multiply by node count. AWS API Gateway integrates with AWS WAF for IP-level rate limiting and usage plans with API keys for per-consumer throttling. When throttled, AWS returns {"message": "Too Many Requests"}; customize via Gateway Responses for the THROTTLED type. The rate limit headers to include in all 429 responses are: X-RateLimit-Limit (max requests in window), X-RateLimit-Remaining (requests left), X-RateLimit-Reset (Unix timestamp of window reset), and Retry-After (seconds until retry). Including these headers prevents clients from hammering the gateway with retries.

Should I do JSON validation at the gateway or in the application?

Both — each layer serves a distinct purpose. Gateway-level JSON schema validation is a security and availability control: it rejects structurally malformed requests (wrong field types, missing required fields, invalid enum values) in under 1ms before consuming any backend compute, database connections, or triggering error paths in application code. It protects your infrastructure from bad payloads regardless of their origin. Gateway validation should enforce the structural contract: required fields present, field types correct, string formats valid (UUID, email, ISO 8601 date). Application-level validation is a business logic control: it enforces domain rules that require context the gateway does not have — the userId must exist in the database, the quantity must not exceed available inventory, the scheduled date must be in the future, the combination of fields must satisfy business invariants. These validations require database lookups, service calls, or application state that is meaningless to a stateless gateway. The practical recommendation: define a JSON Schema for each endpoint and enforce the structural subset at the gateway, then apply the full schema plus business rules in the application. Gateway catches 80–90% of invalid requests cheaply; the application handles domain-specific validation with full context. Libraries like Zod (TypeScript) and Pydantic (Python) can export JSON Schema for gateway use, keeping the two layers in sync from a single schema definition.

Further reading and primary sources