Amazon S3 monitoring integration | New Relic Documentation

Important

Enable the AWS CloudWatch Metric Streams integration to monitor all CloudWatch metrics from your AWS services, including custom namespaces. Individual integrations are no longer our recommended option.

New Relic infrastructure integrations include an integration for reporting your Amazon S3 data to New Relic. This document explains how to activate the integration and describes the data reported.

Features

Amazon Simple Storage Service (Amazon S3), provides developers and IT teams with secure, durable, highly-scalable cloud storage.

With New Relic's Amazon S3 integration, data reported includes S3 bucket size, bucket object counts, GET requests, POST requests, and other metrics. S3 data is available in pre-built dashboards and you can also create custom queries and charts in New Relic One. You can also create alert conditions to notify you of changes in S3 data.

Activate integration

Important

Request and Data Transfer metrics are premium metrics and paid for separately through AWS. For Cloudwatch pricing information, see Amazon's S3 enhanced monitoring documentation.

To enable this integration follow standard procedures to Connect AWS services to New Relic.

Configuration and polling

You can change the polling frequency and filter data using configuration options.

Default polling information for the Amazon S3 integration:

New Relic polling intervals:
- S3 buckets: 1 hour
- S3 requests: 5 minutes
Amazon CloudWatch polling intervals:
- S3 buckets: 24 hours
- S3 requests: 1 minute

Note that changing the polling interval for this integration will only affect data collection for S3 request metrics.

Find and use data

To find your integration data, go to one.newrelic.com > Infrastructure > AWS and select one of the S3 integration links.

You can query and explore your data using the DatastoreSample event type, with a provider value of S3Bucket.

For more on how to use your data, see Understand integration data.

Metric data

This integration collects the following Amazon S3 metrics. For more details about these metrics, see Amazon's S3 documentation.

Amazon S3 data for buckets

Metric	Description
`BucketSizeBytes`	The amount of data in bytes stored in a bucket including Standard Storage, Reduced Redundancy Storage, Infrequent Access Storage (IAS), One zone IAS, and Glacier Storage including overheads.
`NumberOfObjects`	The total number of objects stored in a bucket for all storage classes except for the `GLACIER` storage class.

Amazon S3 data for request metrics

To collect these metrics, you must enable request metrics for your S3 bucket:

Metric	Description
`AllRequests`	The total number of HTTP requests made to an Amazon S3 bucket, regardless of type. If you’re using a metrics configuration with a filter, then this metric only counts HTTP requests from objects which meet the filter's requirements.
`GetRequests`	The number of HTTP GET requests made for objects in an Amazon S3 bucket. This doesn't include list operations.
`PutRequests`	The number of HTTP PUT requests made for objects in an Amazon S3 bucket.
`DeleteRequests`	The number of HTTP DELETE requests made for objects in an Amazon S3 bucket. This also includes Delete Multiple Objects requests. This metric shows the number of requests, not the number of objects deleted.
`HeadRequests`	The number of HTTP HEAD requests made to an Amazon S3 bucket.
`PostRequests`	The number of HTTP POST requests made to an Amazon S3 bucket.
`ListRequests`	The number of HTTP requests that list the contents of a bucket.
`BytesDownloaded`	The number bytes downloaded for requests made to an Amazon S3 bucket, where the response includes a body. Valid statistics: Average (bytes per request), Sum (bytes per period), Sample Count, Min, Max
`BytesUploaded`	The number bytes uploaded that contain a request body, made to an Amazon S3 bucket. Valid statistics: Average (bytes per request), Sum (bytes per period), Sample Count, Min, Max
`4xxErrors`	The number of HTTP 4xx client error status code requests made to an Amazon S3 bucket with a value of either `0` or `1`. The `average` statistic shows the error rate, and the `sum` statistic shows the count of that type of error, during each period. Valid statistics: Average (reports per request), Sum (reports per period), Min, Max, Sample Count
`5xxErrors`	The number of HTTP 5xx server error status code requests made to an Amazon S3 bucket with a value of either `0` or `1`. The `average`statistic shows the error rate, and the `sum` statistic shows the count of that type of error, during each period. Valid statistics: Average (reports per request), Sum (reports per period), Min, Max, Sample Count
`FirstByteLatency`	The per-request time, in milliseconds, from the time the S3 bucket received a complete request to the time the response started to be returned. Valid statistics: Average, Sum, Min, Max, Sample Count
`TotalRequestLatency`	The elapsed per-request time, in milliseconds, from the first byte received to the last byte sent to an Amazon S3 bucket. This includes the time taken to receive the request body and send the response body, which is not included in `FirstByteLatency`. Valid statistics: Average, Sum, Min, Max, Sample Count

Inventory data

EOL NOTICE

After March 2022, we're discontinuing support for several capabilities, including inventory data for cloud integrations. For more details, including how you can easily prepare for this transition, see our Explorers Hub post.

This integration collects the following inventory data. For more information, see our documentation about integration data.

In order to fetch inventory data for the following items, you must have extended inventory collection turned on:

/bucket/acl
/bucket/loggingConfiguration
/bucket/notificationConfiguration/configurations
/bucket/policy
/bucket/replicationConfiguration
/bucket/versioningConfiguration
/bucket/websiteConfiguration
/bucket/lifecycleConfiguration
/bucket/crossOriginConfiguration

/bucket

Name	Description
`region`	The AWS region this bucket is provisioned in.
`name`	Name of the S3 bucket.

/bucket/acl

Name	Description
`grantList`	This property holds the list of accounts that have been granted access according to the S3 Access Control List (ACL) for the bucket. JSON format. For more about ACLs, see Amazon's ACL documentation.
`isRequesterCharged`	True if the requestor pays, false if not. For more information, see Amazon's documentation on requester-pays buckets.

/bucket/acl/owner

Name	Description
`displayName`	Name of the account that owns this bucket according to the S3 Acccess Control list (ACL). JSON format. For more about ACLs, see Amazon's ACL documentation.

/bucket/crossOriginConfiguration

Name	Description
`rules`	Cross origin resource sharing (CORS) rules defining what domains may share the data in the bucket. JSON format. For more about CORS, see Amazon's CORS documentation.

/bucket/lifecycleConfiguration

Name	Description
`rules`	Lifecycle configuration rules for data storage management. JSON format. For more about life-cycle configuration, see Amazon's lifecycle configuration documentation.

/bucket/loggingConfiguration

Name	Description
`rules`	Access logging configuration in JSON format. Access log records provide information about each access request, including the requester, bucket name, request time, and error code, if any. For more about logging configuration, see Amazon's server access logging documentation.

/bucket/notificationConfiguration/configurations

Name	Description
`events`	The bucket event for which to send notifications, for a certain notification configuration.
`*Arn`	Resource ARN that Amazon S3 will use when it detects events of the specified type (this can be an SNS topic, an SQS queue, or a Lambda function).

/bucket/policy

Name	Description
`policyText`	Bucket policy as well as user policy are two access policy options that you can use to grant permission to your Amazon S3 resources. JSON format. For more about bucket policy, see Amazon's bucket policy documentation.

/bucket/replicationConfiguration

Name	Description
`rules`	Replication configuration of the bucket in JSON format.

/bucket/taggingConfiguration

Name	Description
`tagSets`	Tags can be used to organize your AWS billing to reflect your own cost structure. JSON format. For more about tags, see Amazon's tag documentation.

/bucket/versioningConfiguration

Name	Description
`status`	This indicates if versioning of the data in the S3 bucket is "Enabled" or "Suspended" For more about versioning, see Amazon's versioning documentation.

/bucket/websiteConfiguration

Name	Description
`indexDocumentSuffix`	When you configure your bucket as a website, you should provide the name of the index document. `indexDocumentSuffix` is the suffix appended to a request for a directory on the website endpoint. (For example, if the suffix is `index.html` and you make a request to `samplebucket/images/`, the data returned will be for the object with the key name `images/index.html`) The suffix must not be empty and must not include a slash character. For more on configuring a bucket as a static website, see Amazon's example of setting up a static website.
`routingRules`	`RedirectRule` is a property of the Amazon S3 website configuration routing rules property that describes how requests are redirected. You can specify a different error code to return in the event an error occurs.

Name

Description

indexDocumentSuffix

When you configure your bucket as a website, you should provide the name of the index document. indexDocumentSuffix is the suffix appended to a request for a directory on the website endpoint. (For example, if the suffix is index.html and you make a request to samplebucket/images/, the data returned will be for the object with the key name images/index.html) The suffix must not be empty and must not include a slash character. For more on configuring a bucket as a static website, see Amazon's example of setting up a static website.

routingRules

RedirectRule is a property of the Amazon S3 website configuration routing rules property that describes how requests are redirected. You can specify a different error code to return in the event an error occurs.