JSON in AWS: IAM Policies, Lambda Events & CloudFormation

IAM Policy JSON Structure

IAM policy documents are JSON objects with a top-level Statement array, where each statement contains at minimum 3 keys: Effect ("Allow" or "Deny"), Action (a list of service:operation strings), and Resource (an ARN or "*"). The policy language version is set with "Version": "2012-10-17" — always include this or AWS defaults to the 2008 version, which lacks policy variables. A single policy document can contain up to 10 statements. IAM evaluates all applicable policies and applies a default deny; an explicit Deny always overrides any number of Allows.

The following is a minimal read-only S3 policy granting 2 actions on a specific bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::my-bucket",
        "arn:aws:s3:::my-bucket/*"
      ]
    }
  ]
}

Identity-based policies (attached to users, roles, or groups) omit the Principal key — the principal is the entity the policy is attached to. Resource-based policies (S3 bucket policies, SQS queue policies, Lambda resource policies) require a Principal key specifying which account, user, or service is granted access. The optional Condition block adds constraints such as IP range, MFA requirement, or tag matching. Deny always overrides Allow; an explicit Deny beats 1000 Allows — this is the most important IAM evaluation rule. Use JSON Schema validation to verify policy document structure before deploying.

Lambda Event Schemas

Lambda receives a JSON event object whose shape is defined entirely by the trigger source — there is no single universal schema. Lambda supports over 20 event sources, and the 5 most common each have a distinct top-level structure. The Lambda context object (containing request ID, function name, remaining time) is a separate second parameter to the handler, not part of the event JSON. Lambda can receive up to 6 MB of payload from synchronous invocations and up to 256 KB from asynchronous invocations.

The 5 most common trigger schemas are:

// S3 trigger — records[].s3.bucket.name and records[].s3.object.key
{
  "Records": [{
    "eventName": "ObjectCreated:Put",
    "s3": {
      "bucket": { "name": "my-bucket" },
      "object": { "key": "uploads%2Ffile.json", "size": 1024 }
    }
  }]
}

// API Gateway v2 (HTTP API)
{
  "requestContext": { "http": { "method": "POST", "path": "/users" } },
  "body": "{"name":"Alice"}",
  "isBase64Encoded": false,
  "queryStringParameters": { "page": "1" }
}

// SQS trigger — body is the raw message string (JSON-parse it again)
{
  "Records": [{
    "body": "{"orderId":"abc123","amount":49.99}",
    "messageId": "msg-001",
    "receiptHandle": "AQ..."
  }]
}

// SNS trigger — Sns.Message is a string
{
  "Records": [{
    "Sns": {
      "Message": "{"event":"user.created","userId":"u-42"}",
      "MessageAttributes": { "type": { "Type": "String", "Value": "user" } }
    }
  }]
}

// DynamoDB Streams — NewImage uses DynamoDB typed JSON
{
  "Records": [{
    "eventName": "INSERT",
    "dynamodb": {
      "NewImage": {
        "userId": { "S": "u-42" },
        "score":  { "N": "100" }
      }
    }
  }]
}

For DynamoDB Streams, use the unmarshall utility from @aws-sdk/util-dynamodb to convert the typed DynamoDB JSON into plain JavaScript objects. For SQS and SNS, the message body arrives as a string — call JSON.parse(record.body) a second time if the message payload is itself JSON. See REST API JSON responses for patterns on structuring the JSON you return from Lambda-backed API endpoints.

CloudFormation JSON Templates

CloudFormation JSON templates define infrastructure as a JSON document with up to 6 root-level keys. The only required key is Resources — all others are optional. Templates can be up to 51,200 bytes when passed inline to the API, or up to 460,800 bytes when uploaded to S3 and referenced by URL. A single template can define up to 500 resources. CloudFormation JSON supports all 11 intrinsic functions for dynamic value resolution at deploy time.

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "Example S3 bucket with versioning",
  "Parameters": {
    "BucketName": {
      "Type": "String",
      "Default": "my-app-data"
    }
  },
  "Resources": {
    "DataBucket": {
      "Type": "AWS::S3::Bucket",
      "DeletionPolicy": "Retain",
      "Properties": {
        "BucketName": { "Ref": "BucketName" },
        "VersioningConfiguration": { "Status": "Enabled" }
      }
    }
  },
  "Outputs": {
    "BucketArn": {
      "Value": { "Fn::GetAtt": ["DataBucket", "Arn"] }
    }
  }
}

The 3 most commonly used intrinsic functions are: {"Ref": "LogicalId"} returns the resource's primary identifier or a parameter value; {"Fn::Sub": "arn:aws:s3:::${BucketName}/*"} performs string substitution using parameter and resource values; {"Fn::If": ["ConditionName", "TrueVal", "FalseVal"]} conditionally selects a value based on a named condition. The DeletionPolicy: Retain attribute on a resource instructs CloudFormation to leave the resource in place when the stack is deleted — critical for S3 buckets and RDS instances that contain data. Validate template JSON structure with JSON Schema validation before running aws cloudformation deploy.

AWS CLI JSON Patterns

The AWS CLI outputs JSON by default and provides 2 built-in mechanisms to filter and reshape that output: the --query flag (JMESPath) and the --output flag. On the input side, --cli-input-json reads a complete JSON request document from a file, replacing dozens of individual flags with a single maintainable file. These patterns make the CLI scriptable and composable with standard Unix tools.

# Download a file from S3
aws s3api get-object --bucket my-bucket --key file.json output.json

# --output json is the default; --output text strips quotes for shell scripts
aws s3api list-buckets --output json

# --query uses JMESPath to filter output
aws s3api list-objects --bucket my-bucket --query 'Contents[].Key'

# Filter by value — objects larger than 1 MB
aws s3api list-objects --bucket my-bucket \
  --query 'Contents[?Size > `1048576`].{Key:Key,Size:Size}'

# --cli-input-json reads a full JSON request document
aws s3api put-bucket-policy --cli-input-json file://bucket-policy.json

# Deploy a CloudFormation stack with parameter overrides
aws cloudformation deploy \
  --template-file template.json \
  --stack-name my-stack \
  --parameter-overrides BucketName=my-prod-bucket

# Pipe to jq for complex transformations
aws ec2 describe-instances | \
  jq '.Reservations[].Instances[].PublicIpAddress'

The --query flag runs server-side filtering in most services, reducing the amount of JSON transferred. Use --output text to strip JSON quoting for use in shell variable assignments. The --no-cli-pager flag disables the default pager (less) in scripts. For repeatable API calls with complex request bodies, prefer --cli-input-json file://request.json over long flag lists — the JSON file can be version-controlled alongside your infrastructure code. Use Jsonic's JSON formatter to pretty-print and validate CLI input JSON files before running commands.

S3, SQS, and SNS Event JSON

S3, SQS, SNS, and EventBridge each deliver events to Lambda with a distinct JSON envelope. Understanding the exact field paths is essential because Lambda does not normalize these across sources — you must handle each trigger type separately. All 4 services wrap their payloads in a Records array (except EventBridge), allowing batching of up to 10,000 records in a single Lambda invocation for SQS.

// S3 notification event
// eventName: "ObjectCreated:Put", "ObjectRemoved:Delete", etc.
// s3.object.key is URL-encoded — decode with decodeURIComponent()
{
  "Records": [{
    "eventName": "ObjectCreated:Put",
    "s3": {
      "bucket": { "name": "my-bucket", "arn": "arn:aws:s3:::my-bucket" },
      "object": { "key": "uploads%2Fphoto.jpg", "size": 204800 }
    }
  }]
}

// SQS event — body is a raw string; JSON.parse again if it contains JSON
{
  "Records": [{
    "messageId": "abc-123",
    "body": "{"event":"order.placed","orderId":"O-999"}",
    "attributes": { "ApproximateReceiveCount": "1" }
  }]
}

// SNS event — Sns.Message is a string; Sns.MessageAttributes is a map
{
  "Records": [{
    "Sns": {
      "TopicArn": "arn:aws:sns:us-east-1:123456789012:my-topic",
      "Message": "{"event":"user.created","userId":"u-7"}",
      "MessageAttributes": {
        "eventType": { "Type": "String", "Value": "user.created" }
      }
    }
  }]
}

// EventBridge event — standard envelope with source, detail-type, detail
{
  "source": "com.myapp.orders",
  "detail-type": "OrderPlaced",
  "detail": {
    "orderId": "O-999",
    "amount": 49.99,
    "currency": "USD"
  }
}

Key implementation details: S3 object keys are URL-encoded — always call decodeURIComponent(record.s3.object.key) before using the key to fetch the object. SQS body is always a string — if your producers publish JSON messages, you must call JSON.parse(record.body) inside the Lambda handler. SNS wraps SQS when you subscribe an SQS queue to an SNS topic — the Lambda event then has an outer SQS envelope with the SNS message as the body string, requiring 2 levels of JSON parsing. EventBridge events use detail (not body) for the payload object. Consider using NDJSON format for high-volume streaming of events to S3 via Kinesis Firehose.

Key Terms

IAM Policy

A JSON document attached to an AWS identity or resource that specifies which actions are allowed or denied on which resources, evaluated by AWS on every API call.

ARN (Amazon Resource Name)

A globally unique string identifier for every AWS resource, formatted as arn:partition:service:region:account-id:resource, used as the value for the Resource field in IAM policy statements.

Lambda Event Object

The JSON object passed as the first argument to a Lambda handler function, whose schema is determined by the event source (S3, API Gateway, SQS, SNS, DynamoDB Streams, etc.) rather than by Lambda itself.

CloudFormation Intrinsic Function

A built-in JSON key recognized by CloudFormation at deploy time to compute dynamic values — for example, Ref returns a resource ID, Fn::Sub performs string substitution, and Fn::If selects between two values based on a condition.

DynamoDB JSON

A typed JSON encoding used by DynamoDB where each value is wrapped in an object with a type descriptor key, such as {"S":"hello"} for strings and {"N":"42"} for numbers, required when reading DynamoDB Streams events directly.

JMESPath

A query language for JSON supported by the AWS CLI --query flag, allowing users to filter, project, and transform JSON API responses without piping to external tools like jq.

Service Control Policy (SCP)

A JSON policy document applied at the AWS Organizations level that sets maximum permission boundaries for all accounts in an organizational unit, and whose Deny statements override any Allow in account-level IAM policies.

Frequently asked questions

What does "Version": "2012-10-17" mean in an IAM policy?

It specifies the IAM policy language version. Always use "2012-10-17" — it's the only version that supports policy variables like ${aws:username}. The older "2008-10-17" lacks these features. If you omit Version, AWS assumes the 2008 version, which silently disables policy variables and other modern constructs. The version string does not refer to a calendar date of the policy itself; it is a fixed language-version identifier. You should treat it as mandatory boilerplate at the top of every IAM policy document. Including the wrong version or omitting it entirely is a common source of subtle permission bugs where variable substitution fails silently and the literal string ${aws:username} is evaluated instead of the actual username. Validate your policy JSON with the JSON formatter to catch structural errors before deploying.

How do I parse the Lambda event body from API Gateway?

API Gateway sends the HTTP body as a base64-encoded string when isBase64Encoded: true, or a raw string when false. Parse it with JSON.parse(event.body) in JavaScript for plain JSON payloads. If the body is binary (for example, a file upload), decode it first with Buffer.from(event.body, 'base64') before processing. For API Gateway v2 (HTTP API), the event shape uses requestContext.http.method, queryStringParameters as a flat map, and body as a string. For API Gateway v1 (REST API), the structure differs slightly — multiValueQueryStringParameters is available separately. Always check isBase64Encoded before parsing, and wrap JSON.parse in a try/catch to handle malformed bodies gracefully and return a 400 response rather than letting Lambda crash.

What is the difference between CloudFormation JSON and YAML?

They are functionally identical — any template can be converted between the 2 formats without losing functionality. YAML is generally more readable, supports multiline strings without escaping, and requires less punctuation. JSON is stricter and easier to validate programmatically with JSON Schema tools. The main syntax difference is in intrinsic functions: {"Fn::Sub": "..."} in JSON becomes !Sub "..." in YAML, and {"Ref": "LogicalId"} becomes !Ref LogicalId. YAML also supports comments (# comment) while JSON does not. Teams working with infrastructure-as-code often prefer YAML for human-authored templates, while programmatically generated templates tend to use JSON since most languages have built-in JSON serialization. Both formats support the same maximum template size of 51,200 bytes when passed inline.

Why is my IAM action being denied even with an Allow policy?

A Deny statement in any policy always overrides Allow, including from an explicit deny in a Service Control Policy (SCP) or resource-based policy. AWS evaluates policies in a specific order: an explicit Deny anywhere in the chain wins, regardless of how many Allow statements exist. Also check that the action string matches exactly — it is case-insensitive but must include the service prefix, for example s3:GetObject not just GetObject. Also verify the Resource ARN — a wildcard "*" matches all resources, but a specific ARN like arn:aws:s3:::my-bucket only covers the bucket object, not the objects inside it. You need arn:aws:s3:::my-bucket/* for object-level operations like s3:GetObject. Use the IAM Policy Simulator in the AWS console to test policies against specific actions and resources.

How do I read JSON output from AWS CLI?

Use --output json (the default) and pipe to jq or use the built-in --query flag with JMESPath syntax. For example: aws ec2 describe-instances --query 'Reservations[*].Instances[*].InstanceId' --output text returns a flat list of IDs. The --query flag supports filtering, projections, and multi-select. For complex transformations, pipe to jq: aws ec2 describe-instances | jq '.Reservations[].Instances[] | {id: .InstanceId, state: .State.Name}'. Use --no-cli-pager to prevent output from being paginated through less in scripts. Store frequently used queries in shell aliases. You can also use Jsonic's JSON formatter to explore and pretty-print CLI JSON output.

What is DynamoDB JSON format?

DynamoDB uses a typed JSON format where each value is an object with a type key: {"S":"hello"} for string, {"N":"42"} for number (always stored as a string representation), {"BOOL":true} for boolean, {"L":[...]} for list, and {"M":{...}} for map. This format appears in the raw DynamoDB API responses and in DynamoDB Streams event records under the NewImage and OldImage keys. The AWS SDK v3 DocumentClient automatically marshals and unmarshals between DynamoDB JSON and plain JavaScript objects, so most application code never sees the typed format directly. When working with DynamoDB Streams in Lambda, use the unmarshall utility from @aws-sdk/util-dynamodb to convert NewImage from DynamoDB JSON to plain objects before processing.