AWS Incomplete S3 Multipart Uploads Detector
Expose abandoned, invisible multipart upload chunks in S3 buckets quietly accumulating standard S3 storage fees.
1. Detection Scope & Operational Boundaries
Every ArthaOps detector enforces strict epistemic honesty. We clearly separate what can be mathematically proven via agentless telemetry from what requires contextual human judgement.
What This Detector Identifies
- Multipart upload parts that were initiated but never completed or aborted, lingering for $\ge 7$ days.
- Incomplete upload parts invisible to standard `aws s3 ls` queries and the default AWS Management Console bucket object listing.
- Buckets lacking an explicit `AbortIncompleteMultipartUpload` lifecycle policy rule.
What It Cannot Establish Alone
- Whether a large, multi-day data migration job (e.g. multi-terabyte database transfer over a constrained connection) is still actively transmitting parts.
- The exact application component responsible for the broken upload script without CloudTrail Data Events enabled.
2. Telemetry & Evidence Requirements
ArthaOps operates strictly via Plane 1 (Read-Only Discovery). The following CloudFormation / IAM permissions and CloudWatch metrics are evaluated:
AWS::S3::BucketVerification Protocol: Tested against synthetic buckets with multi-part uploads initiated and interrupted via AWS CLI and boto3.
Validation Context: Validated in production data engineering and backup buckets discovering terabytes of ghost multipart storage.
3. Reproducible AWS CLI Verification Recipe
List in-flight multipart uploads in a target S3 bucket to inspect initiated dates and keys.
# List incomplete multipart uploads in a bucket
aws s3api list-multipart-uploads \
--bucket production-backups-bucket \
--query "Uploads[*].{Key:Key,UploadId:UploadId,Initiated:Initiated,StorageClass:StorageClass}" \
--output table
# Check if the bucket has an active lifecycle configuration
aws s3api get-bucket-lifecycle-configuration \
--bucket production-backups-bucket \
--query "Rules[?AbortIncompleteMultipartUpload!=null]" \
--output json--------------------------------------------------------------------------------------------------------------------------
| ListMultipartUploads |
+----------------------+-------------------------------------------------------+---------------+-------------------------+
| Initiated | Key | StorageClass | UploadId |
+----------------------+-------------------------------------------------------+---------------+-------------------------+
| 2026-07-04T02:11:00Z | database-dumps/postgres-full-2026-07-04.tar.gz | STANDARD | 2Yx9QpL0m...abcd12345 |
+----------------------+-------------------------------------------------------+---------------+-------------------------+💡 Incomplete uploads initiated weeks or months ago represent broken processes. Their uploaded parts occupy full S3 storage and cannot be downloaded as objects.
4. Transparent Economic Model
- Stored parts are charged at standard regional S3 storage rates.
- Monthly waste scales linearly with total gigabytes of uncompleted parts.
- Zero API request cost included in storage calculation.
- Data retrieval fees and cross-region replication fees do not apply to uncompleted parts.
5. False-Positive Scenarios & Operational Risks
Never delete or terminate cloud resources blindly. These validated architectural scenarios must be reviewed before initiating remediation:
Documented False-Positive Scenarios
- In-Flight Large Ingestion: A multi-terabyte data migration streaming over a slow WAN link may legitimately require 48–72 hours to finalize all parts.
- Resumable Upload Workflows: Specialized desktop or mobile client upload protocols that pause and resume over multiple days.
Blast Radius & Remediation Risks
- Upload Termination: Aborting an in-progress multipart upload immediately purges all transmitted parts. The uploading client must re-transmit from part 1.
- Mitigation: Configure a 7-day grace window before automated deletion to protect active transfers.
6. Remediation Procedure & Rollback Runbook
Follow this ordered sequence of verified AWS CLI actions to remediate the resource with complete rollback availability:
Abort Specific Orphaned Uploads
Immediately purge parts for an abandoned upload using its Key and UploadId.
aws s3api abort-multipart-upload --bucket production-backups-bucket --key database-dumps/postgres-full-2026-07-04.tar.gz --upload-id 2Yx9QpL0m...abcd12345Deploy Universal 7-Day Lifecycle Rule
Apply an automated bucket lifecycle policy that instructs S3 to clean up uncompleted parts after 7 days automatically.
cat << 'EOF' > s3-lifecycle.json
{
"Rules": [
{
"ID": "AutoAbortIncompleteMultipartUploadsAfter7Days",
"Status": "Enabled",
"Filter": {},
"AbortIncompleteMultipartUpload": {
"DaysAfterInitiation": 7
}
}
]
}
EOF
aws s3api put-bucket-lifecycle-configuration --bucket production-backups-bucket --lifecycle-configuration file://s3-lifecycle.jsonForensic Rollback Procedure
Aborted parts cannot be recovered. If a client was actively transmitting, the client application simply initiates a new multipart upload.
🔒 ArthaOps Assurance: ArthaOps recommends applying a declarative bucket lifecycle policy with a 7-day retention buffer rather than manual ad-hoc part deletions.
Aborted parts are immediately deleted from S3 storage and cannot be recovered. Ensure active transfers are protected via the 7-day retention grace threshold.
How This Appears in the ArthaOps Control Plane
12.6 TB of orphaned multipart upload parts detected across 42 abandoned upload sessions with no lifecycle policy configured.
Test Your Infrastructure in 180 Seconds
Explore our zero-setup interactive sample report covering real Mumbai findings, or launch a 100% read-only agentless scan across your AWS environment.