AWS Credential Report:

AWS credential report can be generated from AWS Console or using SDK's which will contain the list of all users in your account and the status of their various credentials, including passwords, access keys, and MFA devices.

Now, what if you need to create this file programmatically and automate this generation and save the file to S3 instead of creating it manually from AWS Console. That is what we are going to see in this blog.

Requirements:

1. AWS Account (Create one or use your existing one)
2. Install AWS CLI
3. Configure AWS CLI
4. Install Python
5. Install Boto3
6. Install Pandas
7. VS Code (Any code editor as per your preference)

Let's Start:

I won't be going into creating AWS account, installing AWS CLI, installing python and its dependencies, and installing VS Code. Please see the attached links for doing those.

Now let us configure AWS CLI. This is required if we need to run the script we are going to create in our local machine as AWS IAM Credentials is required to access AWS S3.

Open your terminal or open VS Code and open terminal from there by the shortcut ctrl+` and type the below command

aws configure

You will be greeted with,

1. AWS Access Key ID [None]:, provide your AWS Access Key ID. Click Enter.
2. AWS Secret Access Key [None]:, provide your AWS Access Key. Click Enter.
3. Default region name [None]:, provide the region code ([e.g. us-west-2]). Click Enter.
4. Default output format [None]:, provide json as the value. Click Enter.

AWS Access Key can be generated from AWS IAM Management Console.

Now let's us create a python script for creating the credential report and pushing that report to AWS S3.

Required Packages:

import os
import boto3
import time
import csv
import pandas as pd
import requests

Generating and Fetching the report:

client = boto3.client('iam')
response = client.generate_credential_report()
time.sleep(10)
cred_repot = client.get_credential_report()

Decoding because of byte type and converting to basic python dictionary:

The response we get from the above client.get_credential_report() will contain many values. We only need the value of Content key from it. And this value will be in byte type, which cannot be directly converted into Python Dictionary type. So we need to decode it.

content = cred_repot["Content"].decode("utf-8")   
content_lines = content.split("\n")
creds_reader = csv.DictReader(content_lines, delimiter=",")
creds_dict = dict(enumerate(list(creds_reader)))

Finding and creating the header row and appending the value rows in seperate list:

header_names = []
creds_list = []
iter_count = 0

for i in creds_dict:
    if (iter_count == 0):
        header_names = list(creds_dict[i].keys())
        creds_list.append(creds_dict[i])
        iter_count += 1
    else:
        creds_list.append(creds_dict[i])
        iter_count += 1

Saving the report into external .csv file:

Change the path in the below code to your S3 Bucket path which contains a CSV file. I have created one with the name of Status_report.csv in my S3 Bucket.

s3 = boto3.resource('s3')
bucket = s3.Bucket('test-cred-report')

with requests.Session() as s:
    getfile = s.get('https://path/to/your/S3_bucket/which_contains/CSV_file')
with open('/tmp/Status_report.csv', 'w' ) as csvfile:
    writer = csv.DictWriter(csvfile, fieldnames = header_names)
    writer.writeheader()
    writer.writerows(creds_list)

Removing empty rows:

There may be empty rows in the CSV file that was generated. We need to remove those to make the report look concise. There are many ways to do this and you can search about those methods, but I am using pandas library for this. Please note that the cells that contains the value 'N/A' will be converted into empty cells.

df = pd.read_csv('/tmp/Status_report.csv')
df.to_csv('/tmp/Status_report.csv', index=False)

Pushing the file to AWS S3 Bucket:

key = 'Status_report.csv'
bucket.upload_file('/tmp/Status_report.csv', key)

Complete Python Function:

The finished python script will look like below. I have created this as a function in-order to deploy it in lambda.

import os
import boto3
import time
import csv
import pandas as pd
import requests

client = boto3.client('iam')
def lambda_handler(event, context):

    response = client.generate_credential_report()
    time.sleep(10)
    cred_repot = client.get_credential_report()

    content = cred_repot["Content"].decode("utf-8")   
    content_lines = content.split("\n")
    creds_reader = csv.DictReader(content_lines, delimiter=",")
    creds_dict = dict(enumerate(list(creds_reader)))

    header_names = []
    creds_list = []
    iter_count = 0

    for i in creds_dict:
        if (iter_count == 0):
            header_names = list(creds_dict[i].keys())
            creds_list.append(creds_dict[i])
            iter_count += 1
        else:
            creds_list.append(creds_dict[i])
            iter_count += 1

    s3 = boto3.resource('s3')
    bucket = s3.Bucket('test-cred-report')

    with requests.Session() as s:
        getfile = s.get('https://path/to/your/S3_bucket/which_contains/CSV_file')
    with open('/tmp/Status_report.csv', 'w' ) as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames = header_names)
        writer.writeheader()
        writer.writerows(creds_list)

    df = pd.read_csv('/tmp/Status_report.csv')
    df.to_csv('/tmp/Status_report.csv', index=False)

    key = 'Status_report.csv'
    bucket.upload_file('/tmp/Status_report.csv', key)
    return {'Status': 200}

Deploying the function in AWS Lambda:

1. Search for Lambda in AWS search bar and click on Functions. Then create a new function using Create Function button on the top-right.
2. Select Author from scratch in the next screen.
3. Given a function name and select the runtime as Python 3.9 (Version may vary).
4. Don't change anything in the architecture.
5. Click on Create Function.
6. You will see a Code Source screen, and it will contain a text editor. Copy paste the above code in there.

7. Now, in local machine, we would just install pandas dependency by using pip install pandas, but in-order to add that dependency in lambda, go-to Function Overview section at the top of the same page and click on Layers. This will show a screen area which will have Add Layer button. Click on that button. Now choose as per below,

   • Layer Source -> AWS Layers
   • AWS Layers -> AWSSDKPandas-Python39
   • Version -> 1

8. Click Add button on the bottom right side. This will add pandas dependency to our lambda function.

9. Now, click on Test button in Code Source screen area and give an event name. Click on Save button.
10. Now click on Deploy button. You can see your AWS Credentails Report generated in the AWS Bucket you have specified.

Voila! You have created your AWS Lambda function to programmatically create AWS Credential Report and save it to AWS S3.

Do let me know in the comments for any further improvements.