AWS services security event logs - A Reference

Properties of AWS Services logs with examples and resources

25 October 2021 3505 words 17 minutes

/posts/aws-services-security-event-logs-reference/assets/aws-logs-cover.png

Contents

This post is always in progress

This blog post was initially written on 2021 and AWS continues to add features continuously. If you notice something wrong or obsolete, share it to me!

Why this repository

AWS lets customers to generate and integrate some services security logs. However, each log source has its own configuration making understanding quite difficult for new customers.

This post references main AWS services logging sources with its properties, format(s), some examples and resource to go further.

Services grouped by log types

API Calls
Network
- Flow Logs
- Route 53 Resolver
- Route 53 Public zone
- Network Load Balancer
Web access
Security & compliance
Other applications
- RDS

Help - Repository guide

Property definitions are available in the last section.

CloudTrail logs

CloudTrail logs most of API call events made on AWS Control plane and few ones on Data plane.

There are 3 methods to record CloudTrail events described below:

Log	CloudTrail Event History
Content	Last 90 days of AWS API call details for Management events. It’s enabled by default and cannot be disabled.Included events types: Read and Write Management event (control plane). Formerly known as API Activity History.
Format	JSON
Delivery	Between 1 and 15 min (smaller delivery time since September 2021)
Output	Only available trough the Console or API (must be queried)
Custom/Filter	Console: apply only one filter at a time
Scope	Account
ID	None
Sharing	No
Regional	Yes. IAM Service events in us-east-1 only
Cost	Free
Availability	API Activity History: Mars 2015 (last 7 days on limited services and regions).Event History: August 2017 (GA, all regions), June 2018 (All Management events and last 90 days)

Log	CloudTrail trail
Content	A trail records most of AWS API call details.Events types can be selected: Read and Write Management events (Control plane), Data events (Data plane) or Insights (unusual volume of management events)
Format	JSON
Delivery	15 min
Output	S3 bucket (objects name format AccountID_CloudTrail_RegionName_YYYYMMDDTHHmmZ_UniqueString.json.gz) and/or a CloudWatch Logs log group
Custom/Filter	Event type (Management, Data, Insight), Action type (Read, Write) and region (Current, All). KMS events can be excluded. A Organization trail can be created on the Management Account applied on all Accounts and regions.
Scope	Trail per Account or per Organization (current and future Accounts)
Regional	Yes. IAM Service events in us-east-1 only
ID	ARN: `arn:${Partition}:cloudtrail:${Region}:${Account}:trail/${TrailName}`
Sharing	No
Cost	The first trail of Management events is free, the following are paying as well as other events types: Data and Insights. Indirect charges: CloudWatch/S3 charges
Availability	November 2013 (GA), November 2016 (S3 Data Events), November 2019 (Insights)

Log	EventBridge default bus
Content	AWS API call details (recorded by CloudTrail) can be caught in the EventBridge default event bus of the Account with a rule. Only Write Management event (control plane) are available. EventBridge is formerly known as CloudWatch Events (same API with improvements).
Format	JSON
Delivery	Near-real time (1 sec)
Output	EventBridge Rule target
Custom/Filter	Highly granular, any JSON event pattern
Scope	Account
ID	ARN: `arn:${Partition}:events:${Region}:${Account}:rule/[${EventBusName}/]${RuleName}`
Sharing	Yes by using another Account bus as Rule target
Regional	Yes. IAM Service events in us-east-1 only
Cost	Free. Indirect: EventBridge rule target charges
Availability	January 2016

Format

All CloudTrail logs are generated in JSON. Content could be complex to parse because JSON structure depends of each service and action details (What). However details about the Principal (Who) and Request Context (When and How) have the same structure.
AWS documents the record contents.

Samples

Sign-in events on Console

All these events are recorded with "eventType": "AwsServiceEvent".

Existing IAM User - Successful MFA Console Login

    {
        "eventVersion": "1.05",
        "userIdentity": {
            "type": "IAMUser",
            "principalId": "AIDAXY6RSLOFBLXXXXXXX",
            "arn": "arn:aws:iam::111111111111:user/user-name-1",
            "accountId": "111111111111",
            "userName": "user-name-1"
        },
        "eventTime": "2020-03-29T13:49:13Z",
        "eventSource": "signin.amazonaws.com",
        "eventName": "ConsoleLogin",
        "awsRegion": "us-east-1",
        "sourceIPAddress": "1.2.3.4",
        "userAgent": "Mozilla/5.0 ... Firefox/73.0",
        "requestParameters": null,
        "responseElements": {
            "ConsoleLogin": "Success"
        },
        "additionalEventData": {
            "LoginTo": "https://console.aws.amazon.com/console/home?state=hashArgs%23&isauthcode=true",
            "MobileVersion": "No",
            "MFAUsed": "Yes"
        },
        "eventID": "45fadea2-caac-4bd4-859a-48a9762c66a5",
        "eventType": "AwsConsoleSignIn",
        "recipientAccountId": "111111111111"
    }

Existing IAM User - Failed Console Login

    {
        "eventVersion": "1.05",
        "userIdentity": {
            "type": "IAMUser",
            "principalId": "AIDAXY6RSLOFBLXXXXXXX",
            "accountId": "111111111111",
            "accessKeyId": "",
            "userName": "user-name-1"
        },
        "eventTime": "2020-03-29T15:53:13Z",
        "eventSource": "signin.amazonaws.com",
        "eventName": "ConsoleLogin",
        "awsRegion": "us-east-1",
        "sourceIPAddress": "1.2.3.4",
        "userAgent": "Mozilla/5.0 ... Firefox/73.0",
        "errorMessage": "Failed authentication",
        "requestParameters": null,
        "responseElements": {
            "ConsoleLogin": "Failure"
        },
        "additionalEventData": {
            "LoginTo": "https://console.aws.amazon.com/console/home?state=hashArgs%23&isauthcode=true",
            "MobileVersion": "No",
            "MFAUsed": "Yes"
        },
        "eventID": "ee74b50a-1272-4e87-907f-e6094bd00e84",
        "eventType": "AwsConsoleSignIn",
        "recipientAccountId": "111111111111"
    }

No-existing IAM User - Failed Console Login

Therefore AWS replace the username with HIDDEN_DUE_TO_SECURITY_REASONS.

    {
        "eventVersion": "1.05",
        "userIdentity": {
            "type": "IAMUser",
            "accountId": "111111111111",
            "accessKeyId": "",
            "userName": "HIDDEN_DUE_TO_SECURITY_REASONS"
        },
        "eventTime": "2020-03-29T15:37:18Z",
        "eventSource": "signin.amazonaws.com",
        "eventName": "ConsoleLogin",
        "awsRegion": "us-east-1",
        "sourceIPAddress": "1.2.3.4",
        "userAgent": "Mozilla/5.0 ... Firefox/73.0",
        "errorMessage": "No username found in supplied account",
        "requestParameters": null,
        "responseElements": {
            "ConsoleLogin": "Failure"
        },
        "additionalEventData": {
            "LoginTo": "https://console.aws.amazon.com/console/home?state=hashArgs%23&isauthcode=true",
            "MobileVersion": "No",
            "MFAUsed": "No"
        },
        "eventID": "c053eda0-fda0-4bd2-b9e9-cc9517556f8a",
        "eventType": "AwsConsoleSignIn",
        "recipientAccountId": "111111111111"
    }

Successful Console Login with STS tokens

Federation Console link is generated with STS temporary tokens.

    {
        "eventVersion": "1.05",
        "userIdentity": {
            "type": "AssumedRole",
            "principalId": "AIDAXY6RSLOFBLXXXXXXX:Session_AWS-CLI_1234",
            "arn": "arn:aws:sts::111111111111:assumed-role/user-name-2/Session_AWS-CLI_1234",
            "accountId": "111111111111",
            "sessionContext": {
                "attributes": {
                    "mfaAuthenticated": "false",
                    "creationDate": "2020-03-26T21:58:31Z"
                },
                "sessionIssuer": {
                    "type": "Role",
                    "principalId": "AIDAXY6RSLOFBLXXXXXXX",
                    "arn": "arn:aws:iam::111111111111:role/user-name-2",
                    "accountId": "111111111111",
                    "userName": "user-name-2"
                }
            }
        },
        "eventTime": "2020-03-26T22:02:47Z",
        "eventSource": "signin.amazonaws.com",
        "eventName": "ConsoleLogin",
        "awsRegion": "us-east-1",
        "sourceIPAddress": "1.2.3.4",
        "userAgent": "Mozilla/5.0 ... Firefox/73.0",
        "requestParameters": null,
        "responseElements": {
            "ConsoleLogin": "Success"
        },
        "additionalEventData": {
            "MobileVersion": "No",
            "MFAUsed": "No"
        },
        "eventID": "cb31c487-7527-4540-8b62-d2e294c5db83",
        "eventType": "AwsConsoleSignIn",
        "recipientAccountId": "111111111111"
    }

AWS SSO - Successful Console Login

    {
        "eventVersion": "1.08",
        "userIdentity": {
            "type": "AssumedRole",
            "principalId": "AROAZZTXDSP5VXXXXXXXX:sso-username-1",
            "arn": "arn:aws:sts::111111111111:assumed-role/AWSReservedSSO_sso-role_720f8b25b6806XXX/sso-username-1",
            "accountId": "111111111111",
            "sessionContext": {
                "sessionIssuer": {
                    "type": "Role",
                    "principalId": "AROAZZTXDSP5VXXXXXXXX",
                    "arn": "arn:aws:iam::111111111111:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_sso-role_720f8b25b6806XXX",
                    "accountId": "111111111111",
                    "userName": "AWSReservedSSO_sso-role_720f8b25b6806XXX"
                },
                "webIdFederationData": {},
                "attributes": {
                    "creationDate": "2021-10-28T20:13:23Z",
                    "mfaAuthenticated": "false"
                }
            }
        },
        "eventTime": "2021-10-28T20:13:23Z",
        "eventSource": "signin.amazonaws.com",
        "eventName": "ConsoleLogin",
        "awsRegion": "us-east-1",
        "sourceIPAddress": "1.2.3.4",
        "userAgent": "Mozilla/5.0 ... Chrome/95.0.4638.54 Safari/537.36",
        "requestParameters": null,
        "responseElements": {
            "ConsoleLogin": "Success"
        },
        "additionalEventData": {
            "MobileVersion": "No",
            "MFAUsed": "No"
        },
        "eventID": "d09430a8-d1be-4d72-9a22-c833817e5cd0",
        "readOnly": false,
        "eventType": "AwsConsoleSignIn",
        "managementEvent": true,
        "recipientAccountId": "111111111111",
        "eventCategory": "Management"
    }

Others

Cloudwatch Log stream created by the CloudWatch Agent on EC2 instance

The EC2 instance use his instance role to push logs to CloudWatch Logs ; the first time a Log Stream is created.

    {
        "eventVersion": "1.08",
        "userIdentity": {
            "type": "AssumedRole",
            "principalId": "AROA4HGNPFE42IQKZGFHF:i-0bb6317a82af461f3",
            "arn": "arn:aws:sts::111111111111:assumed-role/ec2_CloudWatchAgent/i-0bb6317a82af461f3",
            "accountId": "111111111111",
            "accessKeyId": "ASIA4HGNPFE4UFRDB77L",
            "sessionContext": {
                "sessionIssuer": {
                    "type": "Role",
                    "principalId": "AROA4HGNPFE42IQKZGFHF",
                    "arn": "arn:aws:iam::111111111111:role/ec2_CloudWatchAgent",
                    "accountId": "111111111111",
                    "userName": "ec2_CloudWatchAgent"
                },
                "webIdFederationData": {},
                "attributes": {
                    "creationDate": "2021-10-31T12:54:25Z",
                    "mfaAuthenticated": "false"
                },
                "ec2RoleDelivery": "2.0"
            }
        },
        "eventTime": "2021-10-31T13:19:39Z",
        "eventSource": "logs.amazonaws.com",
        "eventName": "CreateLogStream",
        "awsRegion": "us-east-1",
        "sourceIPAddress": "54.91.187.20",
        "userAgent": "CWAgent/1.247347.4 (go1.15.8; linux; arm64) 2021-02-24T00:14:10+0000 inputs:(mem disk logfile) outputs:(cloudwatch cloudwatchlogs)",
        "requestParameters": {
            "logGroupName": "instance-log",
            "logStreamName": "i-0bb6317a82af461f3"
        },
        "responseElements": null,
        "requestID": "44b30da7-ec0c-4c98-b4d0-f77606665c41",
        "eventID": "403e18e9-555f-4bdb-9f2e-e5af82e47a4d",
        "readOnly": false,
        "eventType": "AwsApiCall",
        "apiVersion": "20140328",
        "managementEvent": true,
        "recipientAccountId": "111111111111",
        "eventCategory": "Management",
        "tlsDetails": {
            "tlsVersion": "TLSv1.2",
            "cipherSuite": "ECDHE-RSA-AES128-GCM-SHA256",
            "clientProvidedHostHeader": "logs.us-east-1.amazonaws.com"
        }
    }

Resources

API Gateway - REST API

https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-logging.html

API Gateway - HTTP API

https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-logging.html

Route 53 Resolver DNS queries

Log	Route 53 Resolver Query Logging
Content	All DNS queries received by a VPC DNS resolver
Format	JSON
Delivery	10-20 sec
Output	CloudWatch Logs group, S3 bucket or Kinesis Data Firehose delivery stream
Custom/Filter	No
Scope	VPC
Regional	Yes
ID	ARN: `arn:aws:route53resolver:<region>:<acountID>:resolver-query-log-config/rqlc-<16digit>`
Sharing	Yes with another AWS Accounts, OU/whole Organization through Resource Access Manager
Cost	Free. Indirect: CloudWatch/S3/Kinesis charges
Availability	August 2020

The Resolver Query Logging Configuration can only have one output and cannot be edited after creation.

Format

See below samples, JSON structure details format.

Sample

Real samples with a hosted private zone mhg.vpc and a public zone.

Private zone - Existing record

``` json
{
  "version": "1.000000",
  "account_id": "1111222233334444",
  "region": "eu-west-3",
  "vpc_id": "vpc-5717186e",
  "query_timestamp": "2020-11-01T09:25:02Z",
  "query_name": "asset1.mhg.vpc.",
  "query_type": "A",
  "query_class": "IN",
  "rcode": "NOERROR",
  "answers": [
    {
      "Rdata": "10.5.0.34",
      "Type": "A",
      "Class": "IN"
    }
  ],
  "srcaddr": "172.31.24.160",
  "srcport": "41366",
  "transport": "UDP",
  "srcids": {
    "instance": "i-061fb32d2b51f86a3"
  }
}
```

Private zone - No-existing record

``` json
{
  "version": "1.000000",
  "account_id": "1111222233334444",
  "region": "eu-west-3",
  "vpc_id": "vpc-5717186e",
  "query_timestamp": "2020-11-01T09:32:05Z",
  "query_name": "test2.mhg.vpc.",
  "query_type": "A",
  "query_class": "IN",
  "rcode": "NXDOMAIN",
  "answers": [],
  "srcaddr": "172.31.24.160",
  "srcport": "37654",
  "transport": "UDP",
  "srcids": {
    "instance": "i-061fb32d2b51f86a3"
  }
}
```

Private zone - Existing public record

``` json
{
    "version": "1.000000",
    "account_id": "1111222233334444",
    "region": "eu-west-3",
    "vpc_id": "vpc-5717186e",
    "query_timestamp": "2020-11-01T10:06:16Z",
    "query_name": "mhg.ovh.",
    "query_type": "A",
    "query_class": "IN",
    "rcode": "NOERROR",
    "answers": [
        {
            "Rdata": "13.32.145.91",
            "Type": "A",
            "Class": "IN"
        },
        {
            "Rdata": "13.32.145.81",
            "Type": "A",
            "Class": "IN"
        },
        {
            "Rdata": "13.32.145.126",
            "Type": "A",
            "Class": "IN"
        },
        {
            "Rdata": "13.32.145.44",
            "Type": "A",
            "Class": "IN"
        }
    ],
    "srcaddr": "172.31.24.160",
    "srcport": "58050",
    "transport": "UDP",
    "srcids": {
        "instance": "i-061fb32d2b51f86a3"
    }
}
```

Route 53 Public DNS queries

Log	Public DNS queries (public zone)
Content	DNS queries of the NS servers (resolvers, not final clients)
Format	Line with values separated by space
Delivery	2-3 min
Output	CloudWatch Logs group
Custom/Filter	No
Scope	Route 53 public zone
Regional	No. Stored in us-east-1 only
ID	UUIDv4 (no ARN)
Sharing	Yes with another AWS Accounts, OU/whole Organization through Resource Access Manager
Cost	Free. Indirect: CloudWatch Logs charges
Availability	September 2017

CloudWatch Logs allows you to export streams to a S3 bucket, however you have to call yourself the CloudWatch API to initiate this asynchronous task. a scheduled Lambda function can be used to do the job.

Following samples come from S3 and includes the CloudWatch Logs timestamp because the Export Task API call task adds it in the S3 object.

Format

CloudWatch-timestamp Log-format-version Query-timestamp Hosted-zone-ID Query-name Query-type Response-code Layer-4-protocol Route-53-edge-location Resolver-IP-address EDNS-client-subnet

Sample

AAAA DNS request on mhg.ovh

2020-03-06T00:31:09.000Z 1.0 2020-03-06T00:31:09Z Z3M111CF8036L2 mhg.ovh AAAA NOERROR UDP VIE50 37.187.141.25 -

Flow Logs

Log	Flow Logs
Content	IP Flow metadatas (ISO Layer 4) similar to NetFlow or IPFIX.
Format	Line with values separated by space
Delivery	1 or 10 minutes (aggregation interval)
Output	CloudWatch Logs, S3 Bucket
Custom/Filter	Custom format available (fields and order)
Scope	VPC, Subnet or network interface (ENI).
Regional	Yes
ID	fl-<17charcaters> (No ARN)
Sharing	No
Cost	Free. Indirect: CloudWatch Logs/S3 charges
Availability	June 2015 (GA), August 2018 (S3 target)

Format

Custom format with all fields

${version} ${account-id} ${vpc-id} ${subnet-id} ${instance-id} ${interface-id} ${type} ${srcaddr} ${dstaddr} ${pkt-srcaddr} ${pkt-dstaddr} ${srcport} ${dstport} ${protocol} ${tcp-flags} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}

Samples

Custom format - Accept and Reject streams

Version number is 3 because of custom format.

Accepted: 3 013206742943 vpc-603f3c09 subnet-dc5cd691 i-00bc69353a1229ec3 eni-0cb464188f367fbc2 IPv4 52.95.155.48 172.31.40.102 52.95.155.48 172.31.40.102 443 37318 6 19 15577 21652528 1577191414 1577191424 ACCEPT OK

Rejected: 3 013206742943 vpc-603f3c09 subnet-dc5cd691 i-00bc69353a1229ec3 eni-0cb464188f367fbc2 IPv4 216.218.206.85 172.31.40.102 216.218.206.85 172.31.40.102 52862 5900 6 2 1 40 1577191398 1577191404 REJECT OK

CloudFront Access Logs

Log	Cloudfront standard logs
Content	Web/RTMP access log of the CDNs similar to reverse proxy logs.
Format	Line with values separated by space
Delivery	Up to several times an hour. Can sometimes be delayed by up to 24 hours
Output	S3 Bucket
Custom/Filter	No
Scope	Distribution
Regional	No, stored in us-east-1 only
ID	None
Sharing	?
Cost	Free. Indirect: S3 charges
Availability	May 2009 (GA), December 2019 (new fields)

Log	Real-time log configuration
Content	Web/RTMP access log of the CDN, similar to reverse proxy logs.
Format	Line
Delivery	Near real-time (1 sec)
Output	Kinesis data stream
Custom/Filter	Sampling rate (1-100%), fields (among 40) and Cache behavior
Scope	Attach configuration to Distribution(s)
Regional	No, stored in us-east-1 only
ID	ARN: `arn:aws:cloudfront::<accountID>:realtime-log-config/<configID>`
Sharing	To an external Kinesis data stream
Cost	Volume of log. Indirect: Kinesis
Availability	August 2020

Format

Standard log

#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type cs-protocol-version fle-status fle-encrypted-fields c-port time-to-first-byte x-edge-detailed-result-type sc-content-type sc-content-len sc-range-start sc-range-end

Sample

Standard log

2019-12-26 09:12:26    FRA50-C1    3198    1.2.3.4   GET djm1t0idtm8px.cloudfront.net    /about.html 200 https://mhg.ovh/    Mozilla/5.0%2520(Macintosh;%2520Intel%2520Mac%2520OS%2520X%252010.15;%2520rv:71.0)%2520Gecko/20100101%2520Firefox/71.0  -   -   Miss    BCFMyYSZyL1u6ZixT2CLGNnFBcVg7P2Y-Rr93r68KwxN2d6OoFyVfw==    mhg.ovh https   253 0.078   -   TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 Miss    HTTP/2.0    -   -   50024   0.078   Miss    text/html   -   -   -

GuardDuty findings

Log	GuardDuty finding S3 Export
Content	Security alerts
Format	JSON (details)
Delivery	5 min for new finding and 15min/1h/6h for next occurrences of the same issue (finding aggregation)
Output	S3 bucket with a mandatory KMS CMK key.
Custom/Filter	No
Scope	GuardDuty Detector
Regional	Yes
ID	32 characters (no ARN)
Sharing	Possible to a external and unique S3 bucket for multiple Export configuration
Cost	Free. Indirect: S3 and KMS CMK charges
Availability	November 2019

Log	GuardDuty finding through EventBridge
Content	GuardDuty findings (Security alerts) can be caught in the EventBridge default event bus of the Account with a rule.
Format	JSON (details)
Delivery	5 min for new finding and 15min/1h/6h for next occurrences of the same issue (finding aggregation)
Output	EventBridge event bus (Account default or custom)
Custom/Filter	Rule pattern
Scope	Account (GuardDuty Detector)
Regional	Yes
ID	ARN: `arn:${Partition}:events:${Region}:${Account}:rule/[${EventBusName}/]${RuleName}`
Sharing	Possible with Rule target
Cost	Free. Indirect: EventBridge
Availability	November 2017

Remarks

If you use the Multi-Account/Organization feature in a GuardDuty administrator account, all findings related to member Accounts will be exported to the S3 bucket configured in the Administrator account.

Format

Please refer to the AWS documentation.

Samples

IAMUser/RootCredentialUsage finding

Policy:IAMUser/RootCredentialUsage finding: An API was invoked using root credentials.

    {
        "schemaVersion": "2.0",
        "accountId": "111111111111",
        "region": "us-east-1",
        "partition": "aws",
        "id": "aab8f74669d66b7115893a8a87779XXX",
        "arn": "arn:aws:guardduty:eu-west-3:111111111111:detector/caFC17dEAbDBCe5CA3EeEeEBaEXXXXXXX/finding/aab8f74669d66b7115893a8a8XXXXXXX",
        "type": "Policy:IAMUser/RootCredentialUsage",
        "resource":
        {
            "resourceType": "AccessKey",
            "accessKeyDetails":
            {
                "accessKeyId": "ASIA2MSOSYTBPZMUXXXX",
                "principalId": "111111111111",
                "userType": "Root",
                "userName": "Root"
            }
        },
        "service":
        {
            "serviceName": "guardduty",
            "detectorId": "caFC17dEAbDBCe5CA3EeEeEBaEEcEXXX",
            "action":
            {
                "actionType": "AWS_API_CALL",
                "awsApiCallAction":
                {
                    "api": "CreateServiceLinkedRole",
                    "serviceName": "iam.amazonaws.com",
                    "callerType": "Remote IP",
                    "remoteIpDetails":
                    {
                        "ipAddressV4": "72.21.217.15",
                        "organization":
                        {
                            "asn": "16509",
                            "asnOrg": "AMAZON-02",
                            "isp": "Amazon.com",
                            "org": "Amazon.com"
                        },
                        "country":
                        {
                            "countryName": "United States"
                        },
                        "city":
                        {
                            "cityName": "Ashburn"
                        },
                        "geoLocation":
                        {
                            "lat": 39.0481,
                            "lon": -77.4728
                        }
                    },
                    "affectedResources":
                    {}
                }
            },
            "resourceRole": "TARGET",
            "additionalInfo":
            {},
            "evidence": null,
            "eventFirstSeen": "2020-05-07T09:34:32Z",
            "eventLastSeen": "2020-05-07T09:35:35Z",
            "archived": false,
            "count": 2
        },
        "severity": 2,
        "createdAt": "2020-05-07T09:52:18.860Z",
        "updatedAt": "2020-05-07T09:52:18.860Z",
        "title": "API CreateServiceLinkedRole was invoked using root credentials.",
        "description": "API CreateServiceLinkedRole was invoked using root credentials from IP address 72.21.217.15."
    }

Security Hub

Log	Security Hub finding trough EventBridge
Content	Alerts received from other security tools
Format	JSON, AWS Security Finding Format (ASFF)
Delivery	Near real-time (1sec) ?
Output	EventBridge Account default bus
Custom/Filter	Rule pattern and 3 event types: All findings, Findings for custom actions, Insight results for custom actions
Scope	Security Hub
Regional	Yes
ID	EventBridge
Sharing	Another Account with EventBridge target
Cost	Free. Indirect: EventBridge
Availability	November 2018

Format

AWS published a standardized format, AWS Security Finding Format (ASFF), detailed on the documentation.

Samples

GuardDuty findings - S3.BucketAnonymousAccessGranted

    {
      "SchemaVersion": "2018-10-08",
      "Id": "arn:aws:guardduty:eu-west-3:111122223333:detector/fcb76996ec7afe8c7fbef8e5c08eXXXX/finding/94b9ef45f2f0f4b2dc00006d77b8XXXX",
      "ProductArn": "arn:aws:securityhub:eu-west-3::product/aws/guardduty",
      "GeneratorId": "arn:aws:guardduty:eu-west-3:111122223333:detector/fcb76996ec7afe8c7fbef8e5c08eXXXX",
      "AwsAccountId": "111122223333",
      "Types": [
        "Software and Configuration Checks/Policy:S3.BucketAnonymousAccessGranted"
      ],
      "FirstObservedAt": "2020-08-11T17:09:59Z",
      "LastObservedAt": "2020-08-11T17:09:59Z",
      "CreatedAt": "2020-08-11T17:22:57.121Z",
      "UpdatedAt": "2020-08-11T17:22:57.121Z",
      "Severity": {
        "Product": 8,
        "Label": "HIGH",
        "Normalized": 80
      },
      "Title": "Amazon S3 Public Anonymous Access was granted for S3 bucket bucket1.",
      "Description": "The Amazon S3 bucket bucket1 was granted public anonymous access by Role1 calling PutBucketPolicy. If this behavior is not expected, it may indicate a configuration mistake or that your credentials are compromised.",
      "SourceUrl": "https://eu-west-3.console.aws.amazon.com/guardduty/home?region=eu-west-3#/findings?macros=current&fId=94b9ef45f2f0f4b2dc00006d77b8XXXX",
      "ProductFields": {
        "aws/guardduty/service/archived": "false",
        "aws/guardduty/service/action/awsApiCallAction/remoteIpDetails/organization/asnOrg": "XXXX",
        "aws/guardduty/service/action/awsApiCallAction/remoteIpDetails/organization/org": "XXXX",
        "aws/guardduty/service/additionalInfo": "",
        "aws/guardduty/service/resourceRole": "TARGET",
        "aws/guardduty/service/action/awsApiCallAction/remoteIpDetails/organization/isp": "XXXX",
        "aws/guardduty/service/action/awsApiCallAction/remoteIpDetails/geoLocation/lat": "X.X",
        "aws/guardduty/service/count": "1",
        "aws/guardduty/service/action/awsApiCallAction/remoteIpDetails/ipAddressV4": "1.1.1.1",
        "aws/guardduty/service/action/awsApiCallAction/remoteIpDetails/country/countryName": "XXXX",
        "aws/guardduty/service/action/awsApiCallAction/callerType": "Remote IP",
        "aws/guardduty/service/action/awsApiCallAction/serviceName": "s3.amazonaws.com",
        "aws/guardduty/service/action/awsApiCallAction/remoteIpDetails/city/cityName": "XXXX",
        "aws/guardduty/service/action/awsApiCallAction/api": "PutBucketPolicy",
        "aws/guardduty/service/serviceName": "guardduty",
        "aws/guardduty/service/evidence": "",
        "aws/guardduty/service/action/awsApiCallAction/remoteIpDetails/geoLocation/lon": "X.X",
        "aws/guardduty/service/detectorId": "fcb76996ec7afe8c7fbef8e5c08eXXXX",
        "aws/guardduty/service/action/awsApiCallAction/remoteIpDetails/organization/asn": "XXXX",
        "aws/guardduty/service/eventFirstSeen": "2020-08-11T17:09:59Z",
        "aws/guardduty/service/action/awsApiCallAction/affectedResources/AWS::S3::Bucket": "bucket1",
        "aws/guardduty/service/eventLastSeen": "2020-08-11T17:09:59Z",
        "aws/guardduty/service/action/actionType": "AWS_API_CALL",
        "aws/securityhub/FindingId": "arn:aws:securityhub:eu-west-3::product/aws/guardduty/arn:aws:guardduty:eu-west-3:111122223333:detector/fcb76996ec7afe8c7fbef8e5c08eXXXX/finding/94b9ef45f2f0f4b2dc00006d77b8XXXX",
        "aws/securityhub/ProductName": "GuardDuty",
        "aws/securityhub/CompanyName": "Amazon"
      },
      "Resources": [
        {
          "Type": "AwsIamAccessKey",
          "Id": "AWS::IAM::AccessKey:ASIA4HGNPFE47Q4MXXXX",
          "Partition": "aws",
          "Region": "eu-west-3",
          "Details": {
            "AwsIamAccessKey": {
              "PrincipalId": "AROA4HGNPFE4XI5FLXXXX:user1",
              "PrincipalType": "AssumedRole",
              "PrincipalName": "Role1"
            }
          }
        }
      ],
      "WorkflowState": "NEW",
      "Workflow": {
        "Status": "NEW"
      },
      "RecordState": "ACTIVE"
    }

Resources

CloudWatch Event setup

S3 logs

Multiple methods exist to record events on Simple Storage Service (S3):

Log	Server access logging
Content	S3 Access logs records every call made on a S3 Bucket, including unauthenticated/anounimous access. It’s similar to Web server logs.
Format	Line with values separated by space (details)
Delivery	Few hours or less
Output	S3 bucket: the bucket source itself (not recommended) or any one the same Account and Region. File name frmat: TargetPrefixYYYY-mm-DD-HH-MM-SS-UniqueString/
Custom/Filter	No
Scope	S3 bucket
Regional	Yes
ID	None
Sharing	No
Cost	Free. Indirect: S3
Availability	Before 2009 (GA), March 2019 (6 new fields)

Log	S3 CloudTrail Data events
Content	CloudTrail can API call details made on bucket-level (Management event) and/or object-level (Data event).Please refer to the CloudTrail chapter for all details.
Format	-
Delivery	-
Output	-
Custom/Filter	-
Scope	-
Regional	-
ID	-
Sharing	-
Cost	-
Availability	-

AWS recommends that you use AWS CloudTrail data events instead of Amazon S3 access logs. CloudTrail data events are easier to set up and contain more information. Source

Format

Please refer to the AWS documentation

Samples

CloudTrail Data event - Athena reads an object

Object event trough Cloudtrail: Athena Service reads an object on behalf of an IAM User that assumed a Role in the Account’s bucket. The example show a bucket with Scott Piper’s flaws.cloud CloudTrail logs.

    {
      "eventVersion": "1.07",
      "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AROA4HGNPFE4XI5FLXXXX:user1",
        "arn": "arn:aws:sts::111122223333:assumed-role/OrganizationAccountAccessRole/user1",
        "accountId": "111122223333",
        "accessKeyId": "ASIAIAH3T5UEST3NXXXX",
        "sessionContext": {
          "sessionIssuer": {
            "type": "Role",
            "principalId": "AROA4HGNPFE4XI5FLXXXX",
            "arn": "arn:aws:iam::111122223333:role/Role1",
            "accountId": "111122223333",
            "userName": "Role1"
          },
          "attributes": {
            "creationDate": "2020-10-31T16:41:18Z",
            "mfaAuthenticated": "true"
          }
        },
        "invokedBy": "athena.amazonaws.com"
      },
      "eventTime": "2020-10-31T17:05:36Z",
      "eventSource": "s3.amazonaws.com",
      "eventName": "GetObject",
      "awsRegion": "us-east-1",
      "sourceIPAddress": "athena.amazonaws.com",
      "userAgent": "athena.amazonaws.com",
      "requestParameters": {
        "bucketName": "xxxx-flaws-cloud-cloudtrail",
        "Host": "xxxx-flaws-cloud-cloudtrail.s3.amazonaws.com",
        "key": "flaws_cloudtrail09.json.gz"
      },
      "responseElements": null,
      "additionalEventData": {
        "SignatureVersion": "SigV4",
        "CipherSuite": "ECDHE-RSA-AES128-GCM-SHA256",
        "bytesTransferredIn": 0,
        "AuthenticationMethod": "AuthHeader",
        "x-amz-id-2": "YVT7gnHZDu7agzbOV+t8GECNPiJ31IErBUGuTEFkJyxvQ4HYylq5w0SBQOogF9u9Q+iee54wXXXX",
        "bytesTransferredOut": 10977945f
      },
      "requestID": "DE670E80ED3590E0",
      "eventID": "b02731a7-41b4-4cd5-aabf-858d447bXXXX",
      "readOnly": true,
      "resources": [
        {
          "type": "AWS::S3::Object",
          "ARN": "arn:aws:s3:::xxxx-flaws-cloud-cloudtrail/flaws_cloudtrail09.json.gz"
        },
        {
          "accountId": "111122223333",
          "type": "AWS::S3::Bucket",
          "ARN": "arn:aws:s3:::xxxx-flaws-cloud-cloudtrail"
        }
      ],
      "eventType": "AwsApiCall",
      "managementEvent": false,
      "recipientAccountId": "111122223333",
      "eventCategory": "Data"
    }

WAF logs

TODO

Application Load Balancer

TODO

https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html

Network Load Balancer

https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-access-logs.html

RDS

TODO

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Procedural.UploadtoCloudWatch.html

VPC Lattice

Log	VPC Lattice service network / service access log
Content	HTTP(S) access logs of VPC Lattice service network or service.
Format	JSON
Delivery	Between 2 and 6 min (smaller after first requests in CloudWatch)
Output	CloudWatch Logs group, S3 bucket or Kinesis Data Firehose
Custom/Filter	No
Scope	Service network (all linked services) or individual service
ID	None
Sharing	No
Regional	Yes
Cost	Free. Indirect: CloudWatch/S3/Kinesis charges
Availability	March 2023 (service GA)

Format

AWS documentation details JSON content.

Samples

VPC Lattice - Anonymous access allowed

The log below was sent to CloudWatch Logs and is for an allowed HTTPS request from an EC2 instance to a Lambda function in the same region and the same AWS account.
The auth policy (resource-based policy) was set as None on the service network.
The log recorded at the service network or service level is identical (tested).

{
    "startTime": "2023-04-01T11:16:05Z",
    "requestMethod": "GET",
    "requestPath": "/",
    "protocol": "HTTP/2",
    "responseCode": 200,
    "bytesReceived": 283,
    "bytesSent": 2281,
    "duration": 290,
    "userAgent": "-",
    "hostHeader": "lambda",
    "targetIpPort": "44.192.251.238:443",
    "targetGroupArn": "arn:aws:vpc-lattice:us-east-1:840097016121:targetgroup/tg-040ede59f30d6a02a",
    "sourceIpPort": "172.31.30.232:60894",
    "serverNameIndication": "s-test-1-01ede641cc872f98c.7d67968.vpc-lattice-svcs.us-east-1.on.aws",
    "sourceVpcId": "vpc-e532159f",
    "destinationVpcId": "-",
    "serviceArn": "arn:aws:vpc-lattice:us-east-1:840097016121:service/svc-01ede641cc872f98c",
    "serviceNetworkArn": "arn:aws:vpc-lattice:us-east-1:840097016121:servicenetwork/sn-0bd03117fade6adb5",
    "requestToTargetDuration": 9,
    "responseFromTargetDuration": 0,
    "sslCipher": "ECDHE-RSA-AES128-GCM-SHA256",
    "tlsVersion": "TLSv1.2",
    "resolvedUser": "-",
    "authDeniedReason": "-"
}

VPC Lattice - Anonymous access denied

The log below was sent to CloudWatch Logs and is for an denied HTTPS request from an EC2 instance to a Lambda function in the same region and the same AWS account.
The auth policy (resource-based policy) was set as IAM on the service network and allows only IAM authenticated access.

{
    "startTime": "2023-04-01T15:48:06Z",
    "requestMethod": "GET",
    "requestPath": "/",
    "protocol": "HTTP/2",
    "responseCode": 502,
    "bytesReceived": 81,
    "bytesSent": 219,
    "duration": 114,
    "userAgent": "curl/7.88.1",
    "hostHeader": "s-test-2-03ba044479164d14a.7d67968.vpc-lattice-svcs.us-east-1.on.aws",
    "targetIpPort": "-",
    "targetGroupArn": "arn:aws:vpc-lattice:us-east-1:111122223333:targetgroup/tg-040ede59f30d6a02a",
    "sourceIpPort": "172.31.30.232:58938",
    "serverNameIndication": "s-test-2-03ba044479164d14a.7d67968.vpc-lattice-svcs.us-east-1.on.aws",
    "sourceVpcId": "vpc-e532159f",
    "destinationVpcId": "-",
    "serviceArn": "arn:aws:vpc-lattice:us-east-1:111122223333:service/svc-03ba044479164d14a",
    "serviceNetworkArn": "arn:aws:vpc-lattice:us-east-1:111122223333:servicenetwork/sn-0bd03117fade6adb5",
    "requestToTargetDuration": 0,
    "responseFromTargetDuration": 0,
    "sslCipher": "ECDHE-RSA-AES128-GCM-SHA256",
    "tlsVersion": "TLSv1.2",
    "resolvedUser": "Anonymous",
    "authDeniedReason": "Network"
}

VPC Lattice - Authenticated access allowed

The log below was sent to CloudWatch Logs and is for an authenticated (signed) HTTPS request from an EC2 instance to a Lambda function in the same region and the same AWS account.
The auth policy (resource-based policy) was set as IAM on the service network and allows only IAM authenticated access.

{
    "startTime": "2023-04-01T16:51:29Z",
    "requestMethod": "GET",
    "requestPath": "/",
    "protocol": "HTTP/1.1",
    "responseCode": 502,
    "bytesReceived": 81,
    "bytesSent": 1968,
    "duration": 105,
    "userAgent": "python-requests/2.25.1",
    "hostHeader": "s-test-2-03ba044479164d14a.7d67968.vpc-lattice-svcs.us-east-1.on.aws",
    "targetIpPort": "-",
    "targetGroupArn": "arn:aws:vpc-lattice:us-east-1:111122223333:targetgroup/tg-040ede59f30d6a02a",
    "sourceIpPort": "172.31.30.232:56822",
    "serverNameIndication": "s-test-2-03ba044479164d14a.7d67968.vpc-lattice-svcs.us-east-1.on.aws",
    "sourceVpcId": "vpc-e532159f",
    "destinationVpcId": "-",
    "serviceArn": "arn:aws:vpc-lattice:us-east-1:111122223333:service/svc-03ba044479164d14a",
    "serviceNetworkArn": "arn:aws:vpc-lattice:us-east-1:111122223333:servicenetwork/sn-0bd03117fade6adb5",
    "requestToTargetDuration": 0,
    "responseFromTargetDuration": 0,
    "sslCipher": "ECDHE-RSA-AES128-GCM-SHA256",
    "tlsVersion": "TLSv1.2",
    "resolvedUser": "arn:aws:sts::111122223333:assumed-role/ec2_SSM_SessionManager/i-021dd3f494cca8b54",
    "authDeniedReason": "-"
}

IAM Access Analyzer

TO DO

Other types of logs (out of scope)

AWS provides tools to integrate with your own workloads. These capabilities are categorized as out of scope because it’s not generated by an AWS service and may not be security event logs.

EC2 instances logs with SSM agent / CloudWatch Log Agent

SSM agent or CloudWatch Agent can collect specific log file(s) from EC2 instances and on-premises servers, running either Linux or Windows Server ; the agent software is available on GitHub.
The installed agent natively streams log(s) content to CloudWatch Logs in near-real time (2-5 sec). Agent configuration file allows to select specific files.

Help - Repository guide

Summary table description (template)

For each Service, the following summary will give you a clear overview of the log, its capabilities and limits.

Log	Log source name (with link to the doc)
Content	Nature of event
Format	File format, value separator, possible custom format
Delivery	Frequency of sending logs
Output	Where we can stream log to another service/target
Filter	Possibility to filter log content
Scope	AWS context of regarding logs
Regional	AWS Region linking of the log source
ID	The Amazon Resource Name (ARN) or ID format of the log resource (if exists)
Sharing	Native sharing capabilities with other AWS Accounts
Cost	Direct and indirect cost (without price)
Availability	Launch date