How to Create Subfolders in S3 Bucket using AWS CLI

The AWS Simple Storage Service (S3) is a cloud service provided by Amazon Web Services (AWS) to store your data securely. There are different approaches to storing and retrieving data from AWS S3; one of them is using the AWS CLI provided by Amazon Web Services. In this article, we will provide you with step-by-step instructions on how to use AWS CLI to create subfolders in an AWS S3 bucket.

What Is Amazon S3?

Amazon S3 is a simple storage service provided by Amazon Web Services. It is highly durable and provides security and scalability for any kind of data. This data is stored as objects within resources called buckets. Amazon S3 is used for various purposes in the cloud because of its robust features for scaling and securing data. It helps people with all kinds of use cases from fields such as mobile and web applications, Big data, Machine Learning and many more.

Adding a Folder in Amazon S3 Bucket

Create an AWS account and an IAM role. Identity and Access Management (IAM) helps you secure the services you want to use by providing authentication and authorization as per your team’s requirements. Use the IAM role for accessing the S3 bucket data instead of directly using your main or root account.

Step 1: Log in to the Amazon S3 Console

Login to the amazon console using your credentials and navigate to S3.

Step 2: Create An Amazon S3 Bucket

• After Sign in to your Amazon console and navigate to Amazon S3. Click on Create bucket to proceed.

Step 3: Configure The Amazon S3 Bucket

• Select an AWS Region and enter a unique bucket name to create a new bucket.

Step 4: Select the Bucket

• Upon successful creation, you can see the list of buckets in the AWS Management Console.

Step 5: Create Folder on your S3 Bucket

• Click on upload and create a new folder on the S3 bucket.
• On successful creation, you’ll see the folder in your bucket’s object list.

Step 6: Add Files to your Folder

• Add few files to your folder by clicking on the upload button.
• You can see 3 files are added below.

Using AWS CLI to Add a Folder in Amazon S3 Bucket

Step 1: Install and Configure AWS CLI

• Install AWS CLI on your command line using the below command.

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg AWSCLIV2.pkg -target /

• Verify successful installation by checking the version of aws using below command.

aws --version

• Configure your IAM user on the AWS CLI using below command

aws configure

• You’ll have to enter your AWS Access Key ID, AWS Secret Access Key and default region name.

Step 2: List the Buckets

• Execute below command to list the buckets. Or If bucker is not exist create the bucket.

aws s3 ls

Step 3: Create Sub Folder in AWS S3

• Create a folder on AWS S3
  • Execute the below command to create a folder on the AWS S3 bucket.
  • Provide bucket name and the new folder’s name you want to create on the S3.

aws s3api put-object --bucket <bucket_name> --key <new_folder_name>/ --content-length 0

• You’ll get the below output on executing above command.

• Check if folder is created on AWS S3 console.

• Create a subfolder on AWS S3
  • Execute the below command to create a sub folder on the AWS S3 bucket.
  • Provide bucket name and the new sub folder’s name you want to create on the S3 along with the path.

aws s3api put-object --bucket <bucket_name> --key <path_to_sub_folder>

• You’ll get the below output on executing above command.

Step 4: Verify the Folder

• Verify if sub folder is generated on the existing folder on AWS S3 console.

• We can also verify this using AWS CLI by executing below command.

aws s3 ls s3://<bucket_name> --recursive

In this way, you can create sub folder on AWS S3 using AWS CLI.

AWS CLI S3 Get Bucket Location

To get the bucket location by using the below command. Here you can replace the bucket name. That will displays the bucket location using the command line interface.

aws s3api get-bucket-location --bucket YOUR_BUCKET_NAME

How to create folder on S3 from Ec2 instance

The AWS CLI or SDK can be used to create a “folder” (really, an S3 prefix) from an EC2 instance. Using a key that ends in a slash (“/”), use the aws s3api put-object command to construct the folder-like structure in your S3 bucket.

aws s3api put-object --bucket YOUR_BUCKET_NAME --key foldername/

Common Errors and Solutions

Error 1: Access Denied

• Error Message: When you try to access an AWS S3 bucket without the required authorization, you get this error.
• Possible Causes: It is possible that the AWS credentials being used lack the necessary authorization. One possible reason for this could be inadequate or absent IAM (Identity and Access Management) policies linked to the role or user.
• Solution: Examine the IAM policies linked to the AWS login credentials being utilized. Make that the policies provide the rights required for the S3 operations you are trying to carry out. Moreover, confirm that the access keys you are using are current and genuine.

Error 2: NoSuchBucket

• Error Message: This error suggests that there is not an S3 bucket with the supplied name.
• Possible Causes: It is possible that the bucket name is spelled incorrectly or has not been created yet.
• Solution: Verify the bucket name one more time for any typos. Use the AWS Management Console or the AWS CLI (Command Line Interface) to create the bucket if it does not already exist. Make that the bucket name is unique for all AWS accounts and complies with S3 naming guidelines.

Error 3: InvalidAccessKeyId

• Error Message: This error occurs when the provided AWS Access Key Id does not match any records.
• Possible Causes: The access key might be incorrect or outdated.
• Solution: Check the access key that is being used. Create a new access key from the AWS Management Console and amend your login information if it is inaccurate or out-of-date. Make sure the new access key is kept in a safe place.

Error 4: BucketAlreadyExists

• Error Message: This error occurs when attempting to create a bucket with a name that’s already in use by another AWS user.
• Possible Causes: The chosen bucket name might already be taken.
• Solution: Select a bucket name that is distinct and has not been used by another AWS customer. To make the bucket name stand out, think about using a prefix or suffix. Before attempting to create the desired bucket name, make sure it is available.

Create Folders in S3 bucket using AWS CLI – FAQs

How do I create a folder and upload files on my S3?

When transferring files to S3 via the AWS CLI or SDKs, all you have to do is include the target folder path in the object key. As a case study:

aws s3 cp FILE_PATH s3://YOUR_BUCKET_NAME/FOLDER_NAME/

Which AWS CLI command is used to create a new S3 bucket?

The AWS CLI command aws s3api create-bucket is used to create a new S3 bucket. An example usage: aws s3api create-bucket --bucket YOUR_BUCKET_NAME --region YOUR_REGION

What is the CLI command to list S3 buckets?

The command to list S3 buckets in the AWS CLI is:

aws s3 ls

How to access S3 bucket from aws cli?

Use commands like aws s3 ls s3://bucket-name to list contents or aws s3 cp file.txt s3://bucket-name to upload and download files to and from the bucket to access an S3 bucket from the AWS CLI.

How to create sub folders in S3 bucket?

Although there are not actual “folders” in S3, you can give the impression that there are by adding slashes (“/”) to object keys. Uploading an object with a key that has the desired path, such as “folder/subfolder/object.txt,” is all that is needed to create a “subfolder.”