AWS S3 Automation Using Python Boto3 – Create Buckets, Upload Files, Manage Objects, and Host Static Websites

Are you still uploading files to Amazon S3 manually through the AWS Console?

As cloud environments continue to grow, manually managing buckets and objects becomes inefficient. Using Python and Boto3, we can automate S3 operations, improve productivity, and integrate storage management into DevOps workflows.

In this article, we will explore the most common Amazon S3 operations using Python Boto3, including bucket creation, file uploads, object management, and static website hosting.

What We'll Learn

By the end of this article, you will be able to:

• Create Amazon S3 buckets using Python
• Upload files and objects to S3
• Delete objects from buckets
• List bucket contents programmatically
• Retrieve all S3 buckets in your AWS account
• Understand S3 access control concepts
• Learn S3 automation best practices
• Prepare S3 buckets for static website hosting
• Build reusable cloud storage automation scripts

Why Automate Amazon S3?

Amazon S3 (Simple Storage Service) is one of the most widely used AWS services for storing application data, backups, logs, static websites, and media content.

Using Python Boto3, DevOps Engineers and Cloud Administrators can automate repetitive storage operations instead of relying on manual console activities.

Benefits of S3 Automation

• ✅ Reduce manual effort
• ✅ Improve operational consistency
• ✅ Automate backups and log archival
• ✅ Integrate storage operations into CI/CD pipelines
• ✅ Improve governance and compliance
• ✅ Scale storage management across multiple environments

Prerequisites

Before running the examples in this article, ensure that:

• Python 3 is installed
• Boto3 package is installed
• AWS CLI is configured
• IAM user or role has appropriate S3 permissions

1. How to create S3 Bucket using Python3 Boto3?

To create a bucket, we use the create_bucket() method.

Important Parameters

Parameter	Description
Bucket	Unique bucket name
ACL	Access level (private/public)
CreateBucketConfiguration	AWS region for bucket creation dependent on the region

Example code:

import boto3
s3_resource = boto3.resource('s3')
bucket = s3_resource.create_bucket(ACL='private',
			Bucket='vybhava2023demo.com',
			CreateBucketConfiguration={
			'LocationConstrating': 'us-west-2'
			}) 
			
print("Successfully created bucket:", bucket)

Note

Bucket names must be globally unique across AWS.

Examples:

my-company-backups
devops-training-2026
vybhava-demo-storage

2. How to update the objects into the S3 Bucket using Boto3

Once the bucket is created, we can upload files using the put_object() method.

When you want to add file from local system we can use Python file methods and have a object reference here f is the reference to the file opened for read operation.

import boto3

with open('greet.txt', 'rb') as f:

    s3_client = boto3.client('s3')

    response = s3_client.put_object(
        ACL='private',
        Body=f,
        Bucket='vybhava2023demo-com',
        Key='greet.txt'
    )

print("File uploaded successfully")

Benefits:

• Multipart uploads
• Better performance
• Automatic retries

3. Delete object from S3 Bucket

Delete object from S3 bucket 

Object : client 

method : delete_object()

import boto3

s3_client = boto3.client('s3')

response = s3_client.delete_object(
    Bucket='vybhava2023demo-com',
    Key='greet.txt'
)

print("Object deleted successfully")

4. List all content objects in a Bucket

List all the contents of a Bucket this may be file objects those are put into the Bucket earlier.

Object: client

method: list_objects

import boto3

s3_client = boto3.client('s3')

response = s3_client.list_objects_v2(
    Bucket='vybhava2023demo-com'
)

for content in response.get('Contents', []):
    print(content['Key'])

Sample Output

greet.txt
backup.tar.gz
application.log

Why Use list_objects_v2?

AWS recommends list_objects_v2() because it supports: Better pagination Improved performance Large bucket scalability

5. List all S3 Buckets

Get the s3_client object 

method: list_buckets : 

On the aws-cli run the command:

aws s3 ls

# File: list-s3.py 
# Description: This script will list all s3 buckets using client interface

import boto3

s3_client = boto3.client('s3')

response = s3_client.list_buckets()

for bucket in response['Buckets']:
    print(bucket['Name'])
 

Sample Output

dev-backups
production-logs
terraform-state-files
vybhava2023demo-com

6. Download Files from S3

Downloading files is another common automation task. Example

import boto3

s3_client = boto3.client('s3')

s3_client.download_file(
    'vybhava2023demo-com',
    'greet.txt',
    './downloads/greet.txt'
)

print("Download completed")

7. Upload Multiple Files to S3

Many DevOps teams upload logs, backups, reports, and build artifacts automatically. Example

import os
import boto3

s3_client = boto3.client('s3')

for file in os.listdir('./uploads'):

    s3_client.upload_file(
        f'./uploads/{file}',
        'vybhava2023demo-com',
        file
    )

print("All files uploaded")
  

8. Host a Static Website Using Amazon S3

Amazon S3 can host static websites without requiring EC2 instances.

Common use cases:

• Portfolio websites
• Documentation portals
• Training websites
• Landing pages

Benefits

• Low cost
• High availability
• No server management
• Easy integration with CloudFront

Recommended Architecture

Users
   |
CloudFront CDN
   |
Amazon S3 Website Bucket

This approach provides better security, caching, and performance.

---

AWS Security Best Practices

When automating S3 operations, follow these recommendations:

Use IAM Roles

Avoid hardcoding AWS Access Keys inside scripts.

Enable Bucket Versioning

Protect against accidental file deletion.

Enable Server-Side Encryption

SSE-S3
SSE-KMS

Block Public Access Only allow public access when explicitly required. 

Use Bucket Policies Grant least-privilege permissions. 

Enable Logging Monitor access through: AWS CloudTrail S3 Access Logs AWS Config

Real-World DevOps Use Cases

• CI/CD Artifact Storage Store application packages generated by Jenkins or GitLab pipelines. 
• Backup Automation Upload database dumps and configuration backups automatically. 
• Log Archiving Archive application logs for troubleshooting and compliance. 
• Static Website Hosting Serve HTML, CSS, JavaScript, and documentation directly from S3. Disaster Recovery Replicate critical files across AWS Regions.

Reader Challenge

Try extending the examples with the following features: 

Beginner 

Upload files based on file extension 

Create folders automatically 

Intermediate 

Upload files using tags 

Generate S3 inventory reports 

Enable bucket versioning

Advanced 

Integrate S3 uploads with AWS Lambda 

Trigger notifications through SNS 

Replicate files across Regions Automate lifecycle policies 

Conclusion

Amazon S3 is one of the most powerful and cost-effective storage services in AWS. By combining S3 with Python Boto3, DevOps Engineers can automate bucket management, file uploads, downloads, backups, reporting, and website hosting operations.

Whether you are building cloud-native applications, automating infrastructure, or managing enterprise storage, learning S3 automation is a valuable skill that improves efficiency, reliability, and scalability.

Reader Question

How are you currently using Amazon S3 in your organization—backups, CI/CD artifacts, log storage, static websites, or something else?

Share your use case in the comments and help fellow cloud engineers learn from real-world implementations.