AWS Manage S3 Buckets using Python Boto3

 In this post, I will show you 

how to create S3 bucket, 

how to put objects into the bucket, 

how to upload multiple objects in s3, 

how to download multiple objects, 

how to control access policy, and 

how to host a static website in S3. 

1. How to create S3 Bucket using Python3 Boto3?

Object : resource method: create_bucket 

important method  parameters: 

 ACL : private or public 

 Bucket - name of the bucket name this should be unique for each bucket 

 CreateBucketConfiguration - have the LocationConstrating that is region on which you want to host your s3 bucket.

import boto3
s3_resource = boto3.resource('s3')
bucket = s3_resource.create_bucket(ACL='private',
			Bucket='vybhava2023demo.com',
			CreateBucketConfiguration={
			'LocationConstrating': 'us-west-2'
			}) 
			
print("Successfully create bucket:", bucket)

2. How to put the objects into the S3 Bucket using Boto3

When you want to add file from local system we can use Python file methods and have a object reference here f is the reference to the file opened for read operation.

f=open('greet.txt').read()
s3_client =boto3.client('s3')
respose = s3_client.put_object(
	ACL='private',
	Body=f,
	Bucket='vybhava2023demo.com',
	Key='greet.txt'
	)

3. Delete object from S3 Bucket

Delete object from S3 bucket 

Object : client 

method : delete_object()

s3_client = boto3.client('s3')
response = s3_client.delete_object(
	Bucket='vybhava2023demo.com',
	Key='greet.txt'	
)

4. List all content objects in a Bucket

List all the contents of a Bucket this may be file objects those are put into the Bucket earlier.

Object: client

method: list_objects

s3_client = boto3.client('s3')
response = s3_client.list_objects(
	Bucket='vybhava2023demo.com'
	)

for content in response['Contents']:
	print(content['Key'])

5. List all S3 Buckets

Get the s3_client object 

method: list_buckets : 

On the aws-cli run the command: aws s3 ls

# File: list-s3.py 
# Description: This script will list all s3 buckets using client interface

s3_client = boto3.client('s3')
list_buckets=s3_client.list_buckets()
print(list_buckets)

# run 2
for b in list_buckets['Buckets']:
	print(b['Name'])

Hope you enjoyed this post!! Please write back your errors and exceptions when you run the Boto3 programs for AWS services and resources.

Manage AWS EC2 Instances using Python Boto3 script

Hey Welcome! back to Automations with Python for AWS!! 

Now IT market says AWS is the top number one Cloud Computing platform. That is why I've selected this AWS automations using Python Boto3.

In this post we will be exploring the AWS EC2 Instance related operations, and manage them in a reusable form.

• Create EC2 instance using Python Boto3
• Launch AWS EC2 instance using Python Boto3 script
• Stop AWS EC2 instance using Python Boto3 script
• Start AWS EC2 Instance using Python Boto3 script
• Terminate AWS EC2 instance using Python Boto3 script
• Fetching Public IP of given instance-id

How do you Create EC2 instance using Python3 Boto3? 

 Creating EC2 instance using Boto3 Python code

#=============================================
# File : create_ec2.py
# Description: Create EC2 instance by Boto3

import boto3
ec2 = boto3.resource('ec2')

instances = ec2.create_instances(
        ImageId="ami-0dafa01c8100180f8",
        MinCount=1,
        MaxCount=1,
        InstanceType="t2.micro",
        KeyName="KeyPair1"
    )
    

Launch instance

import boto3
ec2_client = boto3.client('ec2')

# This function will requires 
# image_id [Required] based on the region this will be changing
# instance_type [optional] default t2.micro type otherwise you can provide
# max [optional] default as 1, you can provide maximum number of instances
def launch_instance(image_id, instance_type='t2.micro', max=1):
	resource = ec2_client.run_instances(ImageId=image_id,
					InstanceType=instance_type,
					MinCount=1, MaxCount=max)
									
	for instance in resource['Instances']:
		print(instance['InstanceId'])

# main program
launch_instance('yourami-id')

To control the ec2 instances individual AWS cli commsnd  test Start, Stop

ec2.start_instances 

ec2.stop_instances

and terminate the ec2 instance
List AWS EC2 Instances using python boto3 script

import boto3
ec2_client = boto3.client('ec2')
resp = ec2_client.describe_instances()
for reservation in resp['Reservations']:
	for instance in reservation['Instances']:
		print("Running Instance Image ID: {} Running instance Instance Type: {} Running Instance Keyname {}".format(instance['InstanceId'],instance['InstanceType'],instance['KeyName']))

Python code with Menu driven program to manage EC2 instances

import boto3
import time

ec2 = boto3.resource('ec2')
ec2_client = boto3.client('ec2')

## Display all instances
def display_instances():
    for instance in ec2.instances.all():
        print (instance.id , instance.state)

## Stop instance by given instance id
def stopinstance():    
    instanceid=input("Please enter instanceid:")
    response = ec2_client.stop_instances(InstanceIds=[instanceid])
    time.sleep(90)
    display_instances()

## Terminate instance by given instance id
def terminateinstance():    
    instanceid=input("Please enter instanceid:")
    response = ec2_client.terminate_instances(InstanceIds=[instanceid])
    time.sleep(90)
    display_instances()
    
## Start instance by given instance id
def startinstance():    
    instanceid=input("Please enter instanceid:")
    response = ec2_client.start_instances(InstanceIds=[instanceid])
    time.sleep(90)
    display_instances()

## Main Program
def main():
    while True:
        menu_list=["Display All Instance","Stop Instance","Start Instance","Terminate Instance","Exit"]
        i=1
        for item in menu_list:
            print (i,item)
            i=i+1
        choice=int(input("Please enter Menu Choice:"))
        if choice==1:
            display_instances()
        elif choice==2:
            stopinstances()
        elif choice==3:
            startinstances()
        elif choice==4:
            terminateinstance()            
        else:
            exit()

if __name__ == '__main__':
    main()        
 

Enjoy the automations with Python Boto3 for AWS...

Python Interview Questions -Coding Snipets

 This post is dedicated for all DevOps Engineer, Software Engineers who are preparing for Coding Interviews, 

Overview of Coding Interviews

Most Companies looking for People with minimum Coding knowledge. In a coding interview, you will be given a small problem to solve within 10 - 20 minutes online screen or in-person on their system. In the question, you might be having some part of the code framed and you might be asked to write a snippet of code in between. You need to understand the code comments and proceed to build the expected snippet of code.

Bigger companies look for the General purpose questions, where small companies look for specific questions. The General questions would be like this:

1. Determine if the given word is a palindrome or not. (Example madam)
2. Determine given number is prime or not.

How to prepare for a coding interview?

Now we have the flexibility to choose the programming language on which you are comfortable. 

• In general DevOps Infra guys will be asked to code on Shell or Python scripting. 
• Developers expected to write on : Java, C++ or Python code

How do you improve problem-solving skills

• You can practice more problems to solve helps you in code interview.
• Code on paper and try to run it 
• Practice more mock exam questions 

This way you can be ready for the coding interviews. 

Good practice technical coding web sites are:

1. www.careercup.com Cracking the Coding interview 

2. www.leetcode.com online Judge 

What you should know editors?

For python coders, one of the following will be an option.

1. Jypiter notebook 
2. VisualStudio Code 
3. atom 
4. Pycharm. 

Better to know keyboard shortcuts for these editors.

Python Interview Questions

Python Scripting online test questions collected and prepared the code answers.

1. Question: Write a code for chech the entered number is prime or not?

   a=int(input("Please enter number:"))
   if a>1:
      for x in range(2,a):
        if(a%x)==0:
          print("not prime")
          break
      else:
        print("Prime")
   else:
      print("not prime")
   

   Output:

   
   
   
   
   
   

2. Question: Write code concate strings using print command usage with end

   # This Python program must be run with
   # Python 3 as it won't work with 2.7.
   # Usage:  python3 printend.py
   
   # ends the output with '@'
   print("Python" , end = '@')
   print("vybhavatechnologies.com")
   

   Output:

   
   
   

3. Question: Write bubblesorting code in Python

   # File: bubblesort.py
   # Usage: python bubblesort.py
   #
   def bubblesort(a):      # a = name of list
      b=len(a)-1    # minus 1 because we always compare 2 adjacent values
   
      for x in range(b):
        for y in range(b-x):
          if a[y] > a[y+1]:
            a[y],a[y+1]=a[y+1],a[y]
      return a
   
   #==== main ==========
   
   print("before sort:")
   a=[32,5,3,6,70,57,87]
   print(a)
   bubblesort(a)
   print("after sort")
   print(a)
   

   Output:

   
   
   

4. Question: Write code for printing fibinacci series

   # File: fibonacci.py
   # Description: As per the input number fibonnacci series
   # Enter number of terms needed         #0,1,1,2,3,5....
   #
   a=int(input("Enter the terms:"))
   f=0                    #first element of series
   s=1                    #second element of series
   if a <= 0:
      print("The requested series is ",f)
   else:
      print(f,s,end = ' ')
      for x in range(2,a):
        next=f+s
        print(next,end = ' ')
        f=s
        s=next
   
   ===============
   vagrant@dockerhost:~/pylab$ python3 fibonacci.py
   Enter the terms:10
   0 1 1 2 3 5 8 13 21 34 vagrant@dockerhost:~/pylab$
   vagrant@dockerhost:~/pylab$ python3 fibonacci.py
   Enter the terms:0
   The requested series is  0
   vagrant@dockerhost:~/pylab$ python3 fibonacci.py
   Enter the terms:-5
   The requested series is  0
   

5. Question: Can you write the Python code to shufle the list of names?

   # File: name_shufle.py
   # Execution: python name_shufle.py
   from random import shuffle
   
   names = ['Vybhav', 'Jahnavi', 'Vaishnavi', 'Vasanthika', 'Sanjay', 'Viswasri']
   shuffle(names)
   print("After shuffle randomized list")
   print(names)
   
   	

   

   Shaffle of list

6. Question: Write a Python code - Simple and smart way of Fibonacci series up to 100

   # Fibonacci
   print("initialized")
   
   a,b=0,1
   print("Fibonacci series:")
   while b < 100:
       print(b)
       a,b=b,a+b
   Python code for Fibonacci series
   

7. Question: Write Python code for the following instructions

   print 1-100 numbers
   All 3 multiples should print 'Vybhava'
   All 5 multiples should print 'Technologies'
   The number which is 3 multiple and 5 multiple should print 'vybhavatechnologies'
   

   Code :

   # code test
   # print 1-100 numbers
   # All 3 multiples should print 'Vybhava'
   # All 5 multiples should print 'Technologies'
   # The number which is 3 multiple and 5 multiple should print 'vybhavatechnologies'
   #
   for i in range(1,100):
       if i%3==0 and i%5 == 0:
           print("VybhavaTechnologies")
       elif i%3==0:
           print("Vybhava",end=' ')
       elif i%5==0:
           print("Technologies",end=' ')
       else:
           print(i,end=' ')
   
   print()
   	

   

   print the series with conditional 3 multiples 5 multiples

8. Question:Give a example for dictionaries in python

           CM={'Telangana':'KCR','Andhra Pradesh':'Jagan','Tamil Nadu':'MK Stalin'}
           print (CM['Telangana'])
   
           print (CM['Andhra Pradesh'])
   
           print (CM['Tamil Nadu'])
       

   Output :

   

   
   
   
   
   

9. Question: How to remove an element from an array

           import array as arr
           a=arr.array('d', [1.1, 1.2, 28.22, 33.1, 38.7, 108.2, 420.6])
           print(a.pop()) #removes last element
           print(a.pop(3)) #removes element with index 3
           a.remove(1.1)#removes the desired element
           print(a)
           
       

   Output :

   
   
   
   
   
   
   

10. Question: How to import a module

            import numpy           #importing using the original module name
            import numpy as np    # importing using an alias name
            from numpy import *    #imports everything present in the numpy module
            
        

11. Question: How to create a class

            class person:
                def __init__(self, name):
                    self.name = name
    
            p1=person("Pavan")
            print(p1.name)
            
        

    Output :

    
    
    

12. Question: Write a program in Python to produce Star triangle.

    1.             def triangle(r):
                       for x in range(r):
                           print(' '*(r-x-1)+'*'*(2*x+1))    
                   triangle(5)
                   
               

       Output :

       
       
       

    2.             def triangle(r):
                       l=range(r)
                       for x in l[::-1]:
                           print(' '*(r-x-1)+'*'*(2*x+1))    
                   triangle(5)
                   

       Output:

       
       
       

    3.             def triangle(r):
                       for i in range(r+1):
                           print('* '*i)
                   triangle(5)
                           
                                      
               

       Output :

       
       
       

13. Question: How To Save An Image Locally Using Python Whose URL Address I Already Know?

                import urllib.request
                urllib.request.urlretrieve("https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhv-SDScZLOymVHqnfeVCEf0C7SUfEywSAf7nTboSI6UJ0h3rOV9aR5lbnH52spuE6MsWS0oDoghsbfisAadgfhQBV6ZZBaWraZFnmbqEpceDaZkuEPG_Lmxt78smPFqqZ_TfUqZe1UPVg/s771/py-interview-questions.JPG", "img.jpg")
               

    Output: before executing code:

    

    after executing code:

    

14. Question: How To check whether the given sequence is a palindrome or not

                a=input("enter sequence: ")
                b=a[::-1]
                if a==b:
                  print("palindrome")
                else:
                  print("Not a Palindrome")
                
            

    Output :

    

    
    
    
    
    
    

    
    
    

15. Question: Write a one-liner that will count the number of capital letters in a file. Your code should work even if the file is too big to fit in memory.

    multiple lines way:

                with open(SOME_LARGE_FILE) as fh:
                    count = 0
                    text = fh.read()
                    for character in text:
                        if character.isupper():
                    count += 1
                
            

     one liner way:

                count sum(1 for line in fh for character in line if character.isupper())
            

    Output :

16. Question: Write a sorting algorithm for a sequence.

                seq = ["19", "21", "0", "5", "35"]
                seq = [int(i) for i in seq]
                seq.sort()
                print (seq)
                
            

    Output :

    
    
    

17. Question: You are required to scrap data from IMDb top 250 movies page. It should only have fields movie name, year, and rating.

                from bs4 import BeautifulSoup
     
                import requests
                import sys
                 
                url = 'http://www.imdb.com/chart/top'
                response = requests.get(url)
                soup = BeautifulSoup(response.text)
                tr = soup.findChildren("tr")
                tr = iter(tr)
                next(tr)
                 
                for movie in tr:
                    title = movie.find('td', {'class': 'titleColumn'} ).find('a').contents[0]
                    year = movie.find('td', {'class': 'titleColumn'} ).find('span', {'class': 'secondaryInfo'}).contents[0]
                    rating = movie.find('td', {'class': 'ratingColumn imdbRating'} ).find('strong').contents[0]
                    row = title + ' - ' + year + ' ' + ' ' + rating
                    
                    print(row)
            

    
    

18. Question: How do you calculate percentiles with Python/ NumPy?

                import numpy as np
                a = np.array([1,2,3,4,5])
                p = np.percentile(a, 50) #Returns 50th percentile, e.g. median
                print('the 50 percentile of the given data is:',p)            
            

    Output :

    

    
    
    
    
    

19. Question: Write a code for multiplying two numbers and giving the output using f strings.

                num1 = 6
                num2= 5
                print(f'the result of {num1}x{num2} is {num1*num2}')                        
            

    Output :

    
    
    

20. Question: Convert the following number into international system.

                number=30122005
                print(f'{number:,}')
            

    Output :

    
    
    

21. Question: Check an if clause within a 'f' string

                x=300
                print(f"let's test the if clause: {True if x ==300 else False }")
            

    Output :

    
    
    

22. Question: Find the timestamp in a log file howmany times a timestamp occured?:

    Logic :
    1. Read the log file
    2. Input the timestamp
    3. Using count function on str to get the number of times it occured.
    

    Sample logfile wls.log:

    "    
        
        
        "
    

    Now here is the sample solution for finding how manytimes the given timestamp occured in the above WebLogic log file.

                f=open(r"C:\Users\L440\Desktop\wls.log","r")
                text=f.read()
                timestamp=input("Enter the timestamp:")
                print("Nubmer of times occured: ", text.count(timestamp))
            

    Output :

    

    Log file count of Timestamp frequency

    
    
    

Operators in Python

Hello Pythonist, here in this post I wiould like to share you about the all Python operators how they can coperates on the literals and variables at different sinarios. Unlike any other Programming languages,  Python supports wide variety of operators.

1. Logical operators
2. Comparison operators
3. Bitwise operators

Let's explore more ...

Logical Operators

The Python Logical operators are simple English words

•  and
•  or
•  not

All these above operators returns boolean values based on the combination of operands.

Comparison operators

There are several comparison operators as >, <, >=, <=, ==, != 

Also in operators

Bitwise Operators

Binary number 0 and 1 will be used as operands bitwise operators & | ^ and ~ allow us to manipulate single bits of data and they returns 0 or 1 based on the data bit values passed.

Python Bit-wise Operators

Bit shift operators

• Left bit-shift operator <<
• Right bit-shift operator >>

Installation of Python 3 in Linux

Beliefs

As our belief, every Linux flavor has Python as one of the Shell. but that shell was created long back based on the OS release time repositories. In the most common situation where built-in Python might be at Python 2.7.x version but the latest version is on Python 3.7.x. The big challenge here is 'how do I upgrade or install Python3 on Linux?' So I've chased this challenge and completed the latest version installed on my Oracle Linux box.

Python 3 installation on RHELflavors

How do I install Python3 latest version on Linux?

Python Programming is simple and kids can learn by doing! It's capabilities to interact with machine internals and the simple structure makes easy to write and understand it. The latest Python version for download you can find from the Python official site.

Here I'm installing Python3 on the Oracle Linux same steps will be followed on any RHEL flavors as well.

yum -y groupinstall development
yum -y install zlib-devel

Note: If you are working behind a proxy then enable them by exporting the HTTP, HTTPS proxy environment variables.

wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz

#Extract the downloaded file
tar xJf Python-3.7.3.tar.xz

cd Python-3.7.3
./configure
or 
./configure --enable-optimizations # if you want stable optimizations
make
make install

This binary installation might take a few minutes, you can relax and then resume back to check the installation is showing python3. Remember this, When you run the make command it will build the Python3 installer by compiling it with a local 'gcc' compiler because Python libraries and dependencies are written in C and C++.

Good References links:

1. Python Source installation
2. Python FTP link

Control your PC from Python Code

Hey guys, One of the easiest programming languages in the World is our Python. Here this post is intended for those who want to learn the hacking techniques! Here I would like to share simple 4lines of Python hack tricks that make shutdown or restart your PC. The logic here is that have the function contains the CMD Command line 'shutdown' with forceful(/s) timeout as 1 minute.

Shutdown logic is here...

import os
def shutdown():
    os.system("shutdown /s /t 1")
shutdown()

Restart logic is here

import os
def restart():
    os.system("shutdown /r /t 1")
restart()

Jump to Linux VM and same thought to execute the same script then it failed! "Must be root." The solution there you need to be superuser to shutdown your Linux machine.

To have better understand about the shutdown command in Linux operating system, used the --help option.

[vagrant@mydev ~]$ sudo shutdown --help
shutdown [OPTIONS...] [TIME] [WALL...]

Shut down the system.

     --help      Show this help
  -H --halt      Halt the machine
  -P --poweroff  Power-off the machine
  -r --reboot    Reboot the machine
  -h             Equivalent to --poweroff, overridden by --halt
  -k             Don't halt/power-off/reboot, just send warnings
     --no-wall   Don't send wall message before halt/power-off/reboot
  -c             Cancel a pending shutdown

On Linux the script will be as follows:

import os
def shutdown():
    os.system("shutdown -h")
shutdown()

The output is as follows for the Linux platform.

Linux Python Execution

Enhancement of the script

import os
import sys

def shutdown(platform):
    if platform=='linux2':
        os.system("shutdown -s")
    elif platform=='win32':
        os.system("shutdown /s /t 1")

def restart(platform):
    if platform=='linux2':
        os.system("shutdown -r")
    elif platform=='win32':
        os.system("shutdown /r /t 1")

# The main program starts here
platform=sys.platform
print platform
opt=sys.argv[1]
if opt=='s':
    print 'Shutting down your PC!!'
    shutdown(platform)
elif opt=='r':
    print 'Rebooting...'
    restart(platform)
else:
    print 'Invalid option'