PYTHON: To download AWS ELB access log files from S3 bucket based on last modified time

python boto3 s3 aws 4 years, 2 months ago
import boto3 import datetime from datetime import datetime,timedelta,date import os client = boto3.client('s3') s3 = boto3.resource('s3') iam = boto3.resource('iam') account_id = iam.CurrentUser().arn.split(':')[4] buckername=input("Enter S3 bucket name:") region=input("Enter Region:") now=input("Format Y-M-D H:M(2020-01-03 13:50)\nEnter the error occured time(UTC):") s=datetime.strptime(now,'%Y-%m-%d %H:%M') sdate_time= s - timedelta(minutes = 10) edate_time= s + timedelta(minutes = 10) s_time=sdate_time.strftime('%Y-%m-%d %H:%M') e_time=edate_time.strftime('%Y-%m-%d %H:%M') y= str(s_time[:4]) m= str(s_time[5:7]) d=str(s_time[8:10]) prefix= "AWSLogs/"+account_id+"/elasticloadbalancing/"+region+"/"+y+"/"+m+"/"+d objects = client.list_objects(Bucket=buckername,Prefix=prefix+'/', Delimiter='/') i=1 for o in objects["Contents"]: key=(o["Key"]) time=str(o["LastModified"]) cd=time[:16] tt = datetime.strptime(str(time[:10]),'%Y-%m-%d') d1 = datetime.strptime(s_time, '%Y-%m-%d %H:%M') d2 = datetime.strptime(e_time, '%Y-%m-%d %H:%M') c1 = datetime.strptime(str(cd), '%Y-%m-%d %H:%M') if(d1<=c1<=d2): s3.meta.client.download_file(Bucket=buckername,Key=key,Filename='file_'+str(i)+'.gz') i=i+1
1832
Instructions

Before using this script, you need to set up AWS CLI first as below:

ubuntu@ip-172-31-46-2:~$ aws configure
AWS Access Key ID [None]: <access_key>
AWS Secret Access Key [None]: <secret_key>
Default region name [None]: <region>
Default output format [None]: <eg:JSON>

Posted By
Python Script to create AWS beanstalk
#!/usr/bin/python
  
import boto
python aws beanstalk
sandeep sandeep
List all files and folders using python os mo
import os

def list_files_folders(path):
python python-os
kishore_kumar
Get current environment variables in python
import os
env = os.environ

python python-os
kishore_kumar
Get os details using python os
import os
print os.uname()
# Don't use os.system('uname -a'), its j
python python-os
kishore_kumar
Get stats ( lines, words, char count ) of fil
def file_stats(path):
    f = open(path, 'r')
    lines = f.readlines()
python
kishore_kumar
Use map function in python
def get_double(num):
    return num * 2

python
kishore_kumar
Python sample codes for beginners
print "Welcome to python"
python
gaya38 gaya38
Python program for even number checking
a=input("Enter a value:")
if (a%2==0):
    print "The given number is even numb
python
gaya38 gaya38
Python program for prime number check
a=input("Enter a value:")
k=0
b=(a/2)+1
python
gaya38 gaya38
Pass command line arguments in python
import sys
x=len(sys.argv)
a=[]
python
gaya38 gaya38
Python program for the largest number in an a
a = [1,43,98,5]#Dummy data
for l in range(len(a)-1):
        if (a[l]>a[l+1]):
python
gaya38 gaya38
print list of even numbers within a range
n=100
a=[10,20,30,40,50]
b=[60,70,80,90]
python
gaya38 gaya38
generate fibonacci series in python
n=input("Enter the constraint to print n
m=input("Enter the maximum value to prin
a=0
python
gaya38 gaya38
Generate Random number within the range in py
import random
print random.uniform(10,500)
python
gaya38 gaya38
Shuffle list elements in python
import random;
z = [1,90,4,2]
z = random.shuffle(z)
python
gaya38 gaya38
use python requests to get contents of url (
import requests

req = requests.get("https://httpbin.org/
python python-requests
kishore_kumar
how to iterate or get values in python dictio
sample_dict = { "number": 1, "fruits": [

for key in sample_dict:
python
kishore_kumar
create matrix and multiply using numpy in pyt
import numpy as np

matrix = [[1,2,3], [4,5,6], [7,8,9]]
python numpy
kishore_kumar
generate random numbers matrix with numpy pyt
import numpy as np

random_arr = np.random.randint(1,50,9)
python numpy
kishore_kumar
Find min , max and mean for numpy arrays
import numpy as np

random_arr = np.random.randint(1,50,9)
python numpy
kishore_kumar