Your Environment is Critical: Using the Power of Shell Scripts to Shape Our Process

This is a premium course, which you can purchase below.

When working with data you often hear the terms 'extraction, transformation and loading' or ETL. This is something that data analysis people think about more than say Application Developers - which I think is a bummer. It's important to know how the data in any application is going to be used so you can make sure you collect the right stuff!

We need to setup our dev environment (quickly) with a few bash commands:

mkdir cassini
cd cassini
mkdir csvs
touch csvs/import.sql
touch README.md
createdb cassini

These commands will create our working directories and, most importantly, a document where we can log what we've done and why we did it. That's really important for our future selves and teammates.

To pull out the names of the columns in our CSV we use the head command:

head -1 csvs/master_plan.csv

This will display the first row of our CSV.