Know Thy Instance - Lighting the LAMPS

One of the first things you do after signing up with AWS is creating an EC2 instance. In simpler terms, this is the machine that you want to put up on the cloud. The AWS user interface is fairly straight forward and within minutes you can get your first cloud server up and running. However, I think that it’s best to understand what choices we have when choosing an instance…

I use the acronym LAMPS to easily remember important Instance choices:

1. L (Location)
2. A (AMI)
3. M (Machine aka Hardware)
4. P (Pricing)
5. S(Storage)

The visualization and the following table will help you understand the LAMPS in better detail:

(Click on the image to get the larger version)

 

Your Choice	What it defines…	Description
Location	Where will your machine be located?	AWS has it’s data centers in US, EU, APAC and Brazil. When you create a virtual machine, you can specify which data center should host your machine. This is basically your “Region”. A region is comprised of a set of what are called “availability zones”. More than one zone can be leveraged for disaster recovery purposes i.e. if one of your instance fails then another instance in a different availability zone can take over. <<Availability zones cannot span multiple regions.>>
AMI	What is the OS and applications you want to run?	AMI is the acronym for Amazon Machine Image. Think of it as a box that holds your OS and required applications. You can simply unpack the box and start running it. AMIs can be obtained from AWS Marketplace (Paid) or from the AWS Community (Free). You can also create your own AMI and share it with the community.
Machine	What kind of usage are you planning?	AWS provides you machines with pre-configured hardware taking into account the typical uses. The options with examples are: General purpose – Web servers Compute optimized – Computational Fluid Dynamics Memory optimized- Analytics applications GPU - Video Processing Storage Optimized – Big Data For a particular type of machine you can also select the size – ranging from micro, small, medium large to 2xlarge, 4xlarge or 8xlarge. Note: Not all sizes are available for all machine types. <<Can custom machines be created?>>
Pricing	How much will it cost?	Few of the instances can be used for free. For others, Amazon offers the following payment options - On-demand instances – Pay as you use Reserved instances – 1 to 3 year commitment, some upfront payment may be required. Spot instances - “Bidding” for Amazon instances that are unused. Be careful here because if anybody else bids a higher amount, your instance will be terminated immediately and provided to the highest bidder.
Storage	Where to store all your application data?	So what do you do when you have the AMI and the hardware? Start running it! AWS by default creates a temporary storage to run the AMI. This is called the EC2 instance store. You don’t need to pay any additional cost for this storage. However, all data in an instance store will be lost when the instance terminates. A better solution is to use something called the Elastic Block Store (EBS). You need to pay for using EBS. However, EBS provides permanent storage with the ability to provision IOPS (Input/Output Operations per seconds) according to your application’s needs. The third type of storage is called Simple Storage Service (S3). You do not run your AMIs on S3 as it would be too slow. Think of S3 as a repository to hold your data. A good use case for S3 would be to use it as a content delivery network (CDN).

 

Now that you have chosen the instance, it’s time to understand how it can be configured or in other words it’s time to Unpack the KITS!

DevOps

Desired Skills and Experience

Requirements

• Computer science graduate.
• 5 to 8 years’ experience in systems engineering managing and automating large scale systems.
• Must have worked with the Agile/Scrum methodology.
• Hands-on experience in Linux/CentOS/RHEL system administration.
• Expertise and Hands-on experience in any one Puppet/Chef/Dockers.
• Experience in deployment automation tools like Glu, Capistrano, Rundeck.
• Good knowledge on Jboss, WebLogic, Apache, Tomcat, Oracle, Big Data, NoSQL, DNS, Firewall, LDAP, SAN Storage and Other servers.
• Understanding of Nagios, Zabbix, and other monitoring tools.
• Understanding of SVN, Git, Jenkins and Maven/Graddle.
• Experience in Quality control tools like sonar.
• Hands-on experience in Virtualization & Cloud Computing (Hypervisors, VMware, AWS, EC2, S3, Google App Engine, Security Groups, Profiles, Images, Open Stack etc.,)
• Expertise and hands-on in any scripting (Shell, Perl, Python etc.,)
• Experience in implementing continuous delivery is a big plus.

Desirable: (Any of the following)

• Linux Administration Certification.
• Cloud Computing Certification.
• CompTIA Server+ / Storage+ / Network+ / Security+

Using PuTTY to access Vagrant VMs from Windows

 
If you are running Vagrant on a Windows host, connecting to the Vagrant managed machines is not very straight forward. This is so because Windows does not natively support SSH. I usually use one of these options to interact with my Vagrant VMs from Windows:

1. Get SSH for Windows
   You can do this by installing Cygwin or Git for Windows. After installation, set the location of ssh.exe in your PATH Environment variable. Lastly, just type “vagrant ssh <<machine name or IP>>” from the Windows command prompt.
2. Use WinSCP
   WinSCP is useful to transfer files between your host and the VMs. Of course, you can also use Shared Folders feature of VirtualBox which is also easily configurable through Vagrant.
3. Use an SSH Client like PuTTY  -->  the focus of this article.

Decide on the Host Name and the Port

When using PuTTY, you can either connect directly to IP address of the VM or through the localhost (via port forwarding):

SSH Port of Virtual Machine Just give the configured IP of the VM and connect over port 22 (the standard SSH port). If you do not want to remember the VM’s IP then add a hostname entry for the IP addres in the %systemroot%\system32\drivers\etc\hosts file.

Forwarded port of Host Machine Forwarded ports can be of two types. First is the default port that Vagrant opens on the host machine. You can find out the default forwarded port by running the “vagrant ssh-config” command. The second is an explicitly specified forwarded port in the VagrantFile: config.vm.network "forwarded_port", guest: 22, host: 8000  This indicates that any traffic to port 22 of your VM will also be forwarded to port 8000 of your host machine.

Option 1 - Login with username/password

Once you connect, you will get the login prompt. The default username/password is vagrant/vagrant

 

Option 2 - Login with SSH Keys

An easier way to connect than using passwords is using SSH keys. So how does this work?
When you manage a VM through Vagrant, it automatically does the following things on the host:

1. Create a public-private key pairs
2. Copy the keys into the ~vagrant/.ssh directory
3. Adds the public key to the ~vagrant/.ssh/authorized_keys file

The private key generated is then copied on to the local machine to enable automatic connection through SSH. In this section we’ll see how to find out the location of the VM’s private key in your host and then use it to connect to the VM via PuTTY.

Step 1 - Identify the location of Vagrant’s private key for your VM

To find out the location of the private key go to the host directory containing your VagrantFile and type “vagrant ssh-config”. This command will give you the entire SSH configuration.You will get the private key location in the “IdentityFile” field. To run this command you need the VM to be up and running.

Alternately, you can just drill down in to the directory containing the VagrantFile and locate the file named “private_key”. For example, in my case the VagrantFile is present in “F:\Virtual Machines\ELK1” directory. I found the private key file in “F:\Virtual Machines\ELK1\.vagrant\machines\logstash-server\virtualbox\private_key”.

 

Step 2 - Use PuTTYgen to convert the private key to PPK format

Vagrant stores the private key in a OpenSSH format. However PuTTY only supports keys that are in PPK format (PuTTY Private Key). So now we will have to convert the private key from OpenSSH format to PPK format.
There is a tool called PuTTYgen to do this. The steps to be followed are given below.
(Note: You can optionally choose a passphrase for your private key. I am not doing so because I am just running a home lab here.)

 

Step 3 - Configure PuTTY to use the private key

Now that you have the private key in PPK format, configure PuTTY to use the same:

 

Step 4 - Just connect!

Everything done right, you can now use PuTTY to connect directly to your Vagrant VM without any username and password!