Tuesday, 6 October 2015

5 things to know before you “vagrant up”

With it’s simplicity of use and the elegance it provides, Vagrant proves to be an interesting tool for a security professional. In this article I’ll give a basic overview so that you can start using it. So what is exactly Vagrant? Think of it as a wrapper around your virtualization provider like VMware or Virtual Box. By using Vagrant you get greater control and flexibility in creating, and managing multiple virtual machines at the same time.

To understand better let’s take an use case. Imagine that you are building an IDS lab. You would typically require 3 machines – Server, Sensor and Monitoring machine. Now to create this setup in Virtual Box you need to perform a series of repetitive tasks - installing the operating system, applications, define networking etc. Once the setup is done, you need to bring up and manage these machines individually. All these activities kind of take your focus away from the core of what you might have wanted– like testing some Snort rules or understanding how Bro IDS works. Wouldn’t it be great if these repetitive tasks can be automated, so that you can focus on stuff that matters?

This is where Vagrant comes in. With Vagrant you simply write a configuration file to define these 3 machines. You can subsequently bring them up, install the OS, provision them and manage them using a handful of “vagrant” commands. So with Vagrant, you move from managing individual machines to managing an environment. Here’s a simple visualization of how Vagrant works:

vagrant complete

#1 - The VagrantFile

The VagrantFile is an ASCII based text file where you specify the configuration of the virtual machine you require. If you need 10 machines for your lab then you need to specify them sequentially in the VagrantFile.
For every VM you can specify:

Physical properties of the machine (CPU, RAM etc)
Network Interfaces
Shared Folders (Folders shared between host and the VM)
Hostname/IP Address
Details of software to be installed on the VM – In Vagrant terms this is called provisioning.

Here is a snippet from the VagrantFile, to create a simple Ubuntu virtual machine. The syntax is based on Ruby. But even if you don’t know Ruby at all it is fairly straight to understand and implement.

#- This line indicates the beginning of configuration.  
Vagrant.configure(2) do |config| 
#- Begin configuration for my-ubuntu-machine 
config.vm.define "My Ubuntu Machine" do |my-ubuntu-machine| 
#- Specifying the “box” required for this machine and the URL to get it from.  
my-ubuntu-machine.vm.box="precise64"
my-ubuntu-machine.vm.box_url ="http://files.vagrantup.com/precise64.box"
#- Setting the hostname, IP and SSH port forwarding  
my-ubuntu-machine.vm.hostname="ubuntu-machine"
my-ubuntu-machine.vm.network "private_network", ip: "10.1.1.10"
my-ubuntu-machine.vm.network "forwarded_port", guest: 22, host: 2210  
#- Provisioning or software/config update – I am just running apt-get update to get the latest versions of packages  
my-ubuntu-machine.vm.provision "shell", inline: -SHELL 
#!/usr/bin/env bash 
apt-get update /dev/null 21 
SHELL 
end  
#- Add more VMs as required… 
#- Below line indicates end of configuration. 
end

#2 – The Vagrant Boxes

A box is file that represents a virtual machine. Think of it as an appliance that you can plug in to Vagrant and start running. Boxes are available for download from vagrantbox.es and the Hashicorp site. In addition to downloading boxes you can also create your own boxes and use them.

#3 – Virtualization Providers

This would typically be any software that provides an emulated environment for running different operating systems. Popular options are Vmware Player/Workstation/Fusion, and VirtualBox. Vagrant works free and fine with VirtualBox. However if you want to use it with VMware, you need to purchase additional licenses.

#4 – The Vagrant Software

The Vagrant software:

Reads the VagrantFile,
Downloads the boxes (if not already available)
Installs the required applications – For this it takes the help of
Creates VMs in the Virtualization provider
Provisions the machines as required - For simple software installations you can use shell scripts. Where shell scripts get complicated, Vagrant also provides support for Configuration management tools such as Chef, Puppet, Salt andAnsible.
Provides capabilities for managing the VMs using the “vagrant” command.

#5 – Plug-ins

Vagrant provides extensibility with the use of Plug-ins (not shown in above diagram). As a developer you might be interested to add new vagrant commands, provide easier options to configure a host, add support for provisioners etc. In fact, Vagrant itself ships with a set of core plug-ins that work behind the scenes to configure and bring up your VM. You can find the list of core plug-ins here. Plug-ins are developed using Ruby and are implemented as self-contained packages (Rubygems).

To know more about Vagrant refer the Vagrant documentation or the very readable book by Mitchell Hashimoto – Vagrant Up and Running.

Sunday, 4 October 2015

MFT Analysis using Wireshark

The first step is to obtain the MFT. It can help answer the question – has any file been created on the file system between two times?

Obtain the MFT

The tool that I use for obtaining MFT is RawCopy.exe. It is a command line based tool, which interfaces with low level disk I/O. This allows the tool to even extract NTFS files that are locked by Windows.

Instructions	Screenshots
Download RawCopy.exe from GitHub
You will see the following files in the downloaded ZIP
Open Command prompt and run rawcopy.exe D:0 F:\output_folder The User Access Control popup will be shown. Click “Yes”. In this example, the $MFT file from D:\ partition will be copied under F:\output_folder.

Convert MFT –> BodyFile –> CSV

Instructions	Screenshots
Download and Install Python 2.7 The installation is simply point and click.
Add the path of python.exe to Environment variables This step is required so that you are able to run python from any directory.
Download analyzeMFT.py
You will see the following files after extracting the ZIP
Install analyzeMFT.py python setup.py install
Generate the Body File The body file is an intermediate file when creating a timeline of file activity. It is a pipe ("\|") delimited text file that contains one line for each file. The body file will have the following fields: MD5\|name\|inode\|mode_as_string\|UID\|GID\|size\|atime\|mtime\|ctime\|crtime To generate the body file run: analyzeMFT.py -f MFT1 -b MFT_body --bodystd –bodyfull This will take the $MFT (renamed to MFT1 for simplification), convert it to a body file and write the output to MFT_body.
Convert the BodyFile to CSV

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

#!/usr/bin/python 
############################################################################### 
# This script will parse a timeline file to create a Pcap representation 
# of the timeline for analysis in wireshark using the built-in features 
# such as profiles, colorization, filtering, and annotation. 
# 
# The basic process is to open the timeline file, parse each line, and 
# output a host file and Pcap that can be used together. Each entry in 
# the timeline is processed to create an individual tcp packet. 
# Characteristics of the packet are mapped to the timeline entry information. 
# For instance: 
# Source IP = Hostname 
# Destination IP = Target File 
# Source Port = Unused 
# Destination Port = Timeline Protocol Identifier 
# TCP Flags = MACB Flags 
# 
# This is just an example of the mapping power of using tcp. Other fields 
# of the packet that may be useful are IPID, sequence numbers, options field, and more. 
# 
# Limitations of this tool include: 
# 
# The host file is whitespace delimited so any whitespace characters are 
# interpreted as the end of the host name. This causes file names with white 
# space to be displayed improperly. This is a minor inconvenience as it is 
# remedied by importing the timeline parsing lua dissector which allows the 
# analyst to insert fields into the display to include the correct filename. 
# 
# The maximum payload of a tcp packet is 65,535 bytes. Timeline entries that 
# exceeded this maximum length were not expected but prefetch files have a 
# large amount of metadata included with them. This could be resolved by 
# creating tcp sessions for this information but that paricular solution 
# defeats the purpose of use which is easy display, colorization and marking 
# of timeline data. 
# 
############################################################################# 
# imports for required libraries 
import argparse 
import ntpath 
import logging 
logging.getLogger("Scapy").setLevel(1) 
# import the Scapy module for packet generation 
from scapy.all import *
#from scapy.utils import  
# import datetime and pytz to adjust for timezone offset 
from datetime import datetime, timedelta 
from pytz import timezone 
import pytz 
# Create a parser to read in command line arguments 
parser = argparse.ArgumentParser(description='Convert a timeline file into Pcap') 
  
# Add argument for input timeline file 
parser.add_argument('-f','--file', help='The timeline file to parse') 
# Add argument for time zone 
parser.add_argument('-z','--tzone', help='Timezone offset of the timeline in pytz format') 
# Add argument for output Pcap file 
parser.add_argument('-o','--outfile',help='Output filename for the resulting Pcap',default='./stimeline.Pcap') 
# Add argument for hostname 
parser.add_argument('-hn','--hostname',help='Name of the source host for this evidence',default='computer') 
# Add argument for hostfile 
parser.add_argument('-hf','--hostfile',help='Output filename for the resulting host file',default='./hosts') 
# Parse arguments for use 
args = parser.parse_args() 
# Open the host file with write access 
hostFile = open(args.hostfile, 'w') 
# Set the location of the Pcap file 
PcapFile = args.outfile 
# Set the port to represent timeline protocol traffic 
# this must match the port used in the target lua dissector 
timelinePort = 7143
# Create a timezone object for use in setting packet timestamp 
tz = timezone(args.tzone) 
# Set the starting network address for the packet capture 
# Currently, this packet capture will be able to process 56,634 
# unique files 
network = "172.16."
# Set the value of the third octet 
three = 0
# Set the value of the fourth octet, reserving .1 for the source host 
four = 2
  
# Set the source host information (IP and hostname) 
srcIP = "172.16.0.1"
srcHost = args.hostname 
# Create a packet list object to hold the crafted packets 
pkts = scapy.plist.PacketList() 
# Write the first entry in the host file, identifying the source of evidence 
hostFile.write(srcIP + "\t" + srcHost + "\n") 
#print(srcIP + "\t" + srcHost + "\n") 
# Dictionary to determine file/IP utilization 
files = dict() 
# Helper function for mapping months to month numbers 
def getMonthNumberFromShortMonth(shortMonth): 
    months={ "jan" : 1, "feb" : 2, "mar" : 3, "apr" : 4, "may" : 5, "jun" : 6, "jul" : 7, "aug" : 8, "sep" : 9, "oct" : 10, "nov" : 11, "dec" : 12 } 
    return months[shortMonth.lower()] 
  
#- rampa 
pktdump = PcapWriter(PcapFile,append=True, sync=True) 
  
# Open the timeline file with read permissions for parsing 
with open(args.file, 'r') as file: 
#{ 
    # For each line, parse the individual elements 
    for line in file: 
    #{ 
        elements = line.split(',') 
        date = elements[0] 
        size = elements[1] 
        macb = elements[2] 
        mode = elements[3] 
        uid = elements[4] 
        gid = elements[5] 
        meta = elements[6] 
        fname = elements[7] 
  
        # if the date element is "Date" then we are reading the header, skip 
        if (date.lower() == "date"): 
        #{ 
            continue
        #} 
  
        # Create a short filename by splitting at the last forward slash 
        short_fname_arr = fname.rsplit('/',1) 
  
        # Parse the macb flags and set tcp flags accordingly 
        # M=Push, A=Ack, C=Urg, B=Syn 
        P = "P" if macb[0].upper() == "M" else "" # M 
        A = "A" if macb[1].upper() == "A" else "" # A 
        U = "U" if macb[2].upper() == "C" else "" # C 
        S = "S" if macb[3].upper() == "B" else "" # B 
  
        # Assemble the resulting tcp flags into a single field 
        flags = S + A + P + U 
  
        # Check the length of the short name array, if < 2 then 
        # filename is first element, otherwise second 
        if (len(short_fname_arr) < 2): 
        #{ 
            short_fname = short_fname_arr[0][:-2] 
        #} 
        else: 
        #{ 
            short_fname = short_fname_arr[1][:-2] 
        #} 
  
        # Check to see if we already have an entry for this file, since 
        # multiple files may have the same name, we must use full name 
        if fname in files: 
        #{ 
            # If file already exists, reuse IP address 
            ip = files[fname] 
        #} 
        else: 
        #{ 
            # If not, then grab another IP, add it to the dictionary 
            if (four == 254): 
                if (three == 255): 
                    sys.exit('Timeline exceeded 56,654 entries...') 
                else: 
                    four = 0
                    three += 1
            else: 
                four += 1
  
            ip = network + str(three) + "." + str(four) 
            files[fname] = ip 
            # Add a new entry to the host file 
            hostFile.write(ip + "\t" + short_fname + "\n") 
            #print(ip + "\t" + short_fname + "\n") 
        #} 
  
        # Parse the date from the entry for use in Scapy 
        dateElements = date.split(' ') 
        timeElements = dateElements[4].split(':') 
        # If the timezone parameter is not null then use passed in parameter 
        # otherwise use None 
        if (timezone != None): 
            pktTime = datetime(int(dateElements[3]),getMonthNumberFromShortMonth(dateElements [1]),int(dateElements[2]),int(timeElements[0]),int(timeElements[1]),int (timeElements[2]),0,tzinfo=tz) 
        else: 
            pktTime = datetime(int(dateElements[3]),getMonthNumberFromShortMonth(dateElements [1]),int(dateElements[2]),int(timeElements[0]),int(timeElements[1]),int (timeElements[2]),0,None) 
  
        # Assemble packet payload constructing null terminated string from 
        # timeline file entry data 
        data = date + "\x00" + size + "\x00" + macb + "\x00" + mode + "\x00" + uid + "\x00" + gid + "\x00" + meta + "\x00" + short_fname + "\x00" + fname + "\x00" # Create an IP packet with TCP transport and data payload 
        p=IP(src=srcIP,dst=ip)/TCP(sport=timelinePort,dport=9999,flags=flags)/Raw(load=data) 
        # Set packet timestamp to constructed value 
        p.time = int(pktTime.strftime("%s")) 
  
        # Append packet to packet list 
        pkts.append(p) 
  
        # Write the Pcap file out to disk 
        pktdump.write(pkts) 
        #wrPcap(PcapFile,pkts) 
    #} 
#} 
# Close open resources 
hostFile.close() 
file.close()

Sunday, 16 August 2015

Know Thy Instance – Unpacking the KITS

In the previous article I covered how you can choose the right instances according to your purpose. Once you have chosen the instances it’s time to understand the configuration options available to you. I use the acronym KITS to refer to instance configuration options. This is in no way comprehensive but covers the minimum portion of what you need to know.

K (Key Pairs)
I (IP Addresses)
T (Tags)
S (Security Groups)

#1 - Key Pairs

After you create an instance, how do you connect to it? In the AWS world, you need a couple of things to connect to an instance – IP Address and Key Pairs. We shall look at Key Pairs in this section and IP addresses in the next.

How do Key Pairs work?

To understand Key Pairs we’ll take the example of Alice and Bob who want to communicate securely over email. That is, even if someone intercepts the emails in transit he should not be able to make sense of the contents of the mail. To accomplish this, they use an email program called PGP. This is how it works:

Alice uses PGP to generate her own public and private key. This set of keys is called a Key Pair. The keys are generated and stored as text files.
Alice keeps the private key to herself and distributes her public key to Bob.
When Bob wants to send a message to Alice, he loads Alice’s public key into his PGP program. PGP then uses this key and encrypts Bob’s message.
This message can now be decrypted only by someone who has Alice’s private key – which (hopefully) should be only Alice. So on receipt of Bob’s message, the PGP program running on Alice’s machine decrypts the message using Alice’s private key and gives the clear text message “Hello Alice!”.

In summary, a Key Pair consists of a couple of keys called public and private key. Any information encrypted using the public key can ONLY be decrypted with the private key (and vice-versa).

Now let’s see how these Key pairs are used in AWS.

How are Key pairs used in a Linux instance?

Most of the Linux distributions provide support for a cryptographic protocol called Secure Shell (SSH). SSH is client-server based and allows a user to remotely logon to a Linux system using public/private Key Pairs.

When you want to connect to an EC2 Linux instance, the following things are required:

A public-private key pair - This key-pair can be created by AWS itself or using tools like ssh-keygen or PuTTYgen.
The SSH package – This is available by default on most Linux instances.
The sshd service – This service is the “server” side SSH. It should be running on the instance when it boots up.
The authorized_hosts file on the Linux instance must have the public key added to it.
SSH client – The “client” side of SSH. There are lots of SSH clients available out there. I usually go with PuTTY. The private key must be available with the SSH client to automatically connect to the instance (without using any passwords).

Steps 1 to 4 will typically be taken care by AWS when you bring up an instance.

How are Key pairs used in a Windows instance?

Windows does not support SSH out-of-the-box. So the public-private keys are used in a different way here. The pre-requisites for Windows are:

A public-private key pair - This key-pair can be created by AWS itself or using tools like ssh-keygen or PuTTYgen.
A Windows user account – When you install Windows it automatically creates an Administrator user.
The user account’s password – This will be automatically set by AWS. The password will then be encrypted with the public key. Remember that the password encrypted with the public key can ONLY be decrypted with the private key.
The private key – Will be required to decrypt the password. Loading the private key and decrypting the password can be done via AWS console itself.
A Remote Desktop (RDP) Client - Once you have the password you can use an RDP client and logon to the Windows machine. Here again you have a number of options. I normally use Remote Desktop Services (mstsc) that comes default with Windows.

Steps 1 to 3 will be done automatically by AWS when you bring up an instance.

#2 - IP Addresses

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-ip-addressing.html

What is a VPC?

When talking about IP addresses it’s important to understand a bit about VPC (or) Virtual Private Cloud. A VPC is like a "network container" for your AWS instances. Inside this container you can create your own subnets, assign routes, setup internet gateways, and even create a VPN tunnel to your premises. VPC is just Amazon’s way of making network administrators feel at home when they are configuring cloud instances:)

When you login to your AWS account, you will find that there is a default VPC already created for any region that you select. This is just to make launching instances easier if you don’t want to get into the nitty-gritties of configuring the network. However, you can create your own VPCs in addition to the default one. A region can have a maximum of 5 VPCs. Also, you cannot create a VPC larger than /16.

To explain VPC in a better way take a look at the below example:

What types of IP Addresses exist?

Private IP Address
Private IP addresses are not routable to the internet. The list of these IP addresses are (as per RFC 1918):
10.0.0.0 to 10.255.255.255 (or) 10.0.0.0/8
172.16.0.0 to 172.31.255.255 (or) 172.16.0.0/12
192.168.0.0 to 192.168.255.255 (or) 192.168.0.0/16

Every EC2 instance gets a Private IP assigned to it by default. This is called the Primary Private IP address.

Public IP Address
A public IP address makes an instance accessible from the internet. If you configure an instance with a default subnet then it is automatically assigned a public IP. You can also explicitly override assigning a public IP when launching an instance.

Elastic IP Address
Let’s say your instance has a public IP assigned to it. What happens when you shutdown or restart the instance? You will be assigned a new Public IP from the EC2-VPC public IP address pool. Now this can be a problem especially if you are hosting a website on that IP address.

A solution to this problem is to use what is called an Elastic IP or EIP. An EIP is basically a public IP address that you can tie to an instance. You can also untie it from a running instance and reassign it to another instance. For more specifics on EIP refer AWS documentation.

Note: If you have Elastic IP tied to an instance and the instance is not running then you will be charged for it.This is AWS’ way of ensuring efficient usage of public IP addresses!

#3 – Tags

After Key Pairs and IP Addresses it’s time to move to something easier. It’s time for Tags!
Tags are case-sensitive key/value pairs that you can add to an instance. One instance can have up to 10 tags associated with it. Tags can be useful to:

Group and Manage Instances
Instance Usage Report based on tag keys
Get AWS account bill based on tag keys

You can read more about tags from here.

#4 - Security Groups

A Security Group (SG) is a collection of network access rules. Using SGs you can define Ports, Protocols and Hosts that are allowed to interact with your EC2 instance. A few examples are - “Allow only ICMP Echo Requests”, “Allow SSH traffic only from 192.168.1.1”, “Allow outbound TCP connections over port 80” and so on. You can attach multiple Security Groups to one instance.

An important limitation of Security Groups is that you can only filter allow traffic and not deny traffic. For example, if you want to block a list of known malicious IPs then you cannot do it using Security Groups.

One solution would be to control deny traffic via host based controls like iptables for Linux and WFAS for Windows. Alternately, AWS also provides something called as NACL (Network Access Control Lists) that can help you with deny traffic. A comparison of NACL and Security groups is given here.

AWS currently places the following limits when configuring SGs:

Only 100 Security Groups are allowed per VPC
Each Security Group can have up to 50 Rules
A Network Interface can be assigned to a maximum of 5 Security Groups

For more information on Securing the VPC refer AWS Documentation.

Now that you understand the LAMPS and the KITS, you can be more confident in bringing up your instance in AWS. If you are creating a multi-tiered/clustered environment it helps to explore more on VPC. Thanks for reading!

Friday, 17 July 2015

Know Thy Instance - Lighting the LAMPS

One of the first things you do after signing up with AWS is creating an EC2 instance. In simpler terms, this is the machine that you want to put up on the cloud. The AWS user interface is fairly straight forward and within minutes you can get your first cloud server up and running. However, I think that it’s best to understand what choices we have when choosing an instance…

I use the acronym LAMPS to easily remember important Instance choices:

L (Location)
A (AMI)
M (Machine aka Hardware)
P (Pricing)
S(Storage)

The visualization and the following table will help you understand the LAMPS in better detail:

(Click on the image to get the larger version)

Your Choice	What it defines…	Description
Location	Where will your machine be located?	AWS has it’s data centers in US, EU, APAC and Brazil. When you create a virtual machine, you can specify which data center should host your machine. This is basically your “Region”. A region is comprised of a set of what are called “availability zones”. More than one zone can be leveraged for disaster recovery purposes i.e. if one of your instance fails then another instance in a different availability zone can take over. <<Availability zones cannot span multiple regions.>>
AMI	What is the OS and applications you want to run?	AMI is the acronym for Amazon Machine Image. Think of it as a box that holds your OS and required applications. You can simply unpack the box and start running it. AMIs can be obtained from AWS Marketplace (Paid) or from the AWS Community (Free). You can also create your own AMI and share it with the community.
Machine	What kind of usage are you planning?	AWS provides you machines with pre-configured hardware taking into account the typical uses. The options with examples are: General purpose – Web servers Compute optimized – Computational Fluid Dynamics Memory optimized- Analytics applications GPU - Video Processing Storage Optimized – Big Data For a particular type of machine you can also select the size – ranging from micro, small, medium large to 2xlarge, 4xlarge or 8xlarge. Note: Not all sizes are available for all machine types. <<Can custom machines be created?>>
Pricing	How much will it cost?	Few of the instances can be used for free. For others, Amazon offers the following payment options - On-demand instances – Pay as you use Reserved instances – 1 to 3 year commitment, some upfront payment may be required. Spot instances - “Bidding” for Amazon instances that are unused. Be careful here because if anybody else bids a higher amount, your instance will be terminated immediately and provided to the highest bidder.
Storage	Where to store all your application data?	So what do you do when you have the AMI and the hardware? Start running it! AWS by default creates a temporary storage to run the AMI. This is called the EC2 instance store. You don’t need to pay any additional cost for this storage. However, all data in an instance store will be lost when the instance terminates. A better solution is to use something called the Elastic Block Store (EBS). You need to pay for using EBS. However, EBS provides permanent storage with the ability to provision IOPS (Input/Output Operations per seconds) according to your application’s needs. The third type of storage is called Simple Storage Service (S3). You do not run your AMIs on S3 as it would be too slow. Think of S3 as a repository to hold your data. A good use case for S3 would be to use it as a content delivery network (CDN).

Now that you have chosen the instance, it’s time to understand how it can be configured or in other words it’s time to Unpack the KITS!

Monday, 6 July 2015

DevOps

Desired Skills and Experience

Requirements

Computer science graduate.
5 to 8 years’ experience in systems engineering managing and automating large scale systems.
Must have worked with the Agile/Scrum methodology.
Hands-on experience in Linux/CentOS/RHEL system administration.
Expertise and Hands-on experience in any one Puppet/Chef/Dockers.
Experience in deployment automation tools like Glu, Capistrano, Rundeck.
Good knowledge on Jboss, WebLogic, Apache, Tomcat, Oracle, Big Data, NoSQL, DNS, Firewall, LDAP, SAN Storage and Other servers.
Understanding of Nagios, Zabbix, and other monitoring tools.
Understanding of SVN, Git, Jenkins and Maven/Graddle.
Experience in Quality control tools like sonar.
Hands-on experience in Virtualization & Cloud Computing (Hypervisors, VMware, AWS, EC2, S3, Google App Engine, Security Groups, Profiles, Images, Open Stack etc.,)
Expertise and hands-on in any scripting (Shell, Perl, Python etc.,)
Experience in implementing continuous delivery is a big plus.

Desirable: (Any of the following)

Linux Administration Certification.
Cloud Computing Certification.
CompTIA Server+ / Storage+ / Network+ / Security+

Saturday, 4 July 2015

Using PuTTY to access Vagrant VMs from Windows

If you are running Vagrant on a Windows host, connecting to the Vagrant managed machines is not very straight forward. This is so because Windows does not natively support SSH. I usually use one of these options to interact with my Vagrant VMs from Windows:

Get SSH for Windows
You can do this by installing Cygwin or Git for Windows. After installation, set the location of ssh.exe in your PATH Environment variable. Lastly, just type “vagrant ssh <<machine name or IP>>” from the Windows command prompt.
Use WinSCP
WinSCP is useful to transfer files between your host and the VMs. Of course, you can also use Shared Folders feature of VirtualBox which is also easily configurable through Vagrant.
Use an SSH Client like PuTTY --> the focus of this article.

Decide on the Host Name and the Port

When using PuTTY, you can either connect directly to IP address of the VM or through the localhost (via port forwarding):

SSH Port of Virtual Machine

Just give the configured IP of the VM and connect over port 22 (the standard SSH port).
If you do not want to remember the VM’s IP then add a hostname entry for the IP addres in the %systemroot%\system32\drivers\etc\hosts file.

Forwarded port of Host Machine
Forwarded ports can be of two types.
First is the default port that Vagrant opens on the host machine. You can find out the default forwarded port by running the “vagrant ssh-config” command.
The second is an explicitly specified forwarded port in the VagrantFile:
config.vm.network "forwarded_port", guest: 22, host: 8000 This indicates that any traffic to port 22 of your VM will also be forwarded to port 8000 of your host machine.

Option 1 - Login with username/password

Once you connect, you will get the login prompt. The default username/password is vagrant/vagrant

Option 2 - Login with SSH Keys

An easier way to connect than using passwords is using SSH keys. So how does this work?
When you manage a VM through Vagrant, it automatically does the following things on the host:

Create a public-private key pairs
Copy the keys into the ~vagrant/.ssh directory
Adds the public key to the ~vagrant/.ssh/authorized_keys file

The private key generated is then copied on to the local machine to enable automatic connection through SSH. In this section we’ll see how to find out the location of the VM’s private key in your host and then use it to connect to the VM via PuTTY.

Step 1 - Identify the location of Vagrant’s private key for your VM

To find out the location of the private key go to the host directory containing your VagrantFile and type “vagrant ssh-config”. This command will give you the entire SSH configuration.You will get the private key location in the “IdentityFile” field. To run this command you need the VM to be up and running.

Alternately, you can just drill down in to the directory containing the VagrantFile and locate the file named “private_key”. For example, in my case the VagrantFile is present in “F:\Virtual Machines\ELK1” directory. I found the private key file in “F:\Virtual Machines\ELK1\.vagrant\machines\logstash-server\virtualbox\private_key”.

Step 2 - Use PuTTYgen to convert the private key to PPK format

Vagrant stores the private key in a OpenSSH format. However PuTTY only supports keys that are in PPK format (PuTTY Private Key). So now we will have to convert the private key from OpenSSH format to PPK format.
There is a tool called PuTTYgen to do this. The steps to be followed are given below.
(Note: You can optionally choose a passphrase for your private key. I am not doing so because I am just running a home lab here.)

Step 3 - Configure PuTTY to use the private key

Now that you have the private key in PPK format, configure PuTTY to use the same:

Step 4 - Just connect!

Everything done right, you can now use PuTTY to connect directly to your Vagrant VM without any username and password!

My Journey into Information Security

Search This Blog