Day 14: Fluid Architecture, Part 2
Fluid Architecture: Part 2
==Draft: 2019-01-06==
Yesterday
Last time, we installed Python, updated PIP, and created the directory on our Desktop where we'll be building our project.
Today
Today, we'll learn the fundamentals of Python and Bash syntax and install our basic dependencies.
First Things First
Before we dive in, we'll need to create a virtual environment in our project directory.
A virtual environment acts like a self-contained instance of Python. According to Technopedia, an instance is defined as "a case or occurrence of anything." You can think of a virtual environment as a mini-Python that's just for one project at a time.
Each Python project is different. In Fluid Architecture, we'll be scraping the web for data, saving it to a database, and serving it over the Internet as email, using a Python framework as the technical infrastructure. Other projects, with different specifications, will require different dependencies. They may even need to run on a different version of Python.
We instantiate Python in virtual environments so our projects can manage their own dependencies without touching the system Python.
Modules
To instantiate a virtual environment, we'll need to run a Python module from the command prompt. A module is a compartmentalized bit of Python code, either written by us or someone else, that can be reused by others. Python's modular design makes it an exceptionally easy programming language to learn and use.
Use command + space to open Spotlight search. Type "terminal" and hit enter. At the command prompt, enter the following command:
$ cd ~/Desktop/fluid_architecture
cd stands for "change directory," and the ~ character refers to your Home folder in macOS. This command is the Terminal equivalent of opening your fluid_architecture folder using the Finder, but we can do a lot more from the Terminal console than from a Finder window.
To instantiate our virtual environment, we'll need to run Python's built-in virtualenv module, which accepts one argument. An argument in software is a parameter that is passed to a routine.
Arguments in Python
Let's say we wanted to play a Joni Mitchell song on the guitar. We can think of all the tasks involved in the process of playing the song a routine. As you may know, Joni uses a bunch of funky tunings in her songs. "I Don't Know Where I Stand," for instance, requires that the guitar's strings be tuned to "FFCGAC".
Let's pretend for a moment that the routine of playing a Joni Mitchell song is defined as a function in a Python program. We might define it like this:
def play_song(): pass
As you may have guessed, def is short for "define." play_song is the name of our function, but we tell Python that a name is a function by placing a pair of parentheses afterward. (pass is simply a Python reserved word that means "don't do anything.")
But this function isn't very specific. What song do we want to play? What tuning do we use?
We'll pass those specifics to our function using arguments. One implementation of our function with arguments might look like this:
def play_song("I Don't Know Where I Stand", "FFCGAC"): pass
When arguments are passed to a function this way, they're called positional arguments. We'd have to know, at the outset, that the first argument passed to the function refers to the song title and the second refers to the tuning.
A better implementation might use what are called keyword arguments, or kwargs for short. These arguments allow us to clarify our code by indicating, more specifically, what our functions expect. Here's an implementation of play_song() with kwargs:
def play_song(song="I Don't Know Where I Stand", tuning="FFCGAC"): pass
But this is Python. When we run commands in the Terminal console, we're writing for Bash, which is an acronym for "Bourne Again Shell" (you can read more here if you're curious).
Arguments in Bash
Bash commands also accept arguments, but not with parentheses. All Bash commands are positional, but we can specify how a command runs with what are called flags. The Python interpreter is a great example for this.
Say we're at the Terminal console and we run the following:
$ python3
Bash will look for our system Python 3 interpreter and run it as an executable. An executable is a script, application, or other file that can be run as a program from the Terminal.
In this case, without any arguments passed to python3, Bash will open what's known as the Python REPL. We'll see something like the following:
Python 3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>>
Notice that the command prompt has changed from $ to >>>. What that means is that we're running an instance of Python that will execute our Python statements on-demand. In other words, instead of having to put our statements in a *.py document, we can type them into the Terminal directly. (You can exit the Python REPL by running exit().)
But what if, instead of running the REPL, we want to run a Python module? We'll need to do exactly that to install our virtual environment.
There are two ways to run Python modules: 1. python3 modulename.py, and 2. python3 -m modulename.
In the second implementation, -m is a flag, which is a special type of argument that tells the Python interpreter that you intend to open a module. -m, specifically, is germane to Python.
After the -m flag comes the name of the module, separated by a space; note that you don't need the .py at the end when you refer to the module this way (in fact, Python will throw an exception, or stop the application and report an error, if you add it!).
Installing our Virtual Environment with venv
In this case, we need to open the venv module in Python, which accepts an argument of its own. Run the following Bash command:
$ python3 -m venv venv
This command executes the Python interpreter with python3, tells it that we expect it to open a Python module with -m, and refer to that module with the first venv.
The second venv refers to the directory into which we'll be installing our virtual environment.
After running this command, you won't see anything at the command prompt. However, the venv module has created a new directory at ~/Desktop/fluid_architecture/venv, into which a virtual instance of Python has been installed.
To verify that venv has been installed, run the following:
$ ls
ls tells Bash to list the contents of the current working directory, which, as you may have guessed, refers to the directory in which we're currently working (~/Desktop/fluid_architecture). You should see the following:
venv
Hurray! Our virtual environment is installed. Now we'll need to activate it.
Activating the Virtual Environment
To activate our virtual environment, run the following command at the prompt:
$ source venv/bin/activate
If the virtual environment is set up correctly, our prompt should be preceded by (venv), which indicates we're working from within an installed virtual environment.
To deactivate a virtual environment, simply run:
(venv) ... $ deactivate
The (venv) preceding your usual command prompt will disappear.
Installing our Basic Dependencies
Eventually, Fluid Architecture will do lots of cool things. First of all, though, we need to be able to use Python to get the contents of a website and interpret what we get back.
For that, we'll install two basic dependencies:
requests, Kenneth Reitz's self-described "HTTP for Humans" module, and
Beautiful Soup, a module for turning content from the web into objects that can be manipulated in Python.
To do that, we need to first activate our virtual environment. We do this so these modules are available to our project, but they won't touch our system Python.
Make sure you're working in ~/Desktop/fluid_architecture and run the following:
$ source venv/bin/activate
Your prompt will change, as expected, with (venv) preceding your normal command prompt.
Run this:
(venv) ... $ pip3 install requests
When given this command, PIP will look up the requests module and download it into your virtual environment. You should see something like the following:
Collecting requests Using cached https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl Collecting urllib3<1.25,>=1.21.1 (from requests) Using cached https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl Collecting certifi>=2017.4.17 (from requests) Using cached https://files.pythonhosted.org/packages/9f/e0/accfc1b56b57e9750eba272e24c4dddeac86852c2bebd1236674d7887e8a/certifi-2018.11.29-py2.py3-none-any.whl Collecting chardet<3.1.0,>=3.0.2 (from requests) Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl Collecting idna<2.9,>=2.5 (from requests) Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl Installing collected packages: urllib3, certifi, chardet, idna, requests Successfully installed certifi-2018.11.29 chardet-3.0.4 idna-2.8 requests-2.21.0 urllib3-1.24.1
It worked!
Next, we'll install Beautiful Soup, which is registered with the name bs4.
Run this:
(venv) ... $ pip3 install bs4
You'll see a similar string of lines from PIP that inform you that Beautiful Soup has been installed.
To see what all is installed in your virtual environment, run the following:
(venv) ... $ pip3 freeze
You'll notice that, on our list of installed modules that PIP returns, there are several we didn't install ourselves:
beautifulsoup4==4.7.0 bs4==0.0.1 certifi==2018.11.29 chardet==3.0.4 idna==2.8 requests==2.21.0 soupsieve==1.6.2 urllib3==1.24.1
That's because both requests and Beautiful Soup, like most Python modules, have their own dependencies. PIP keeps track of everything we'll need to install these modules and packages them together.
Recap
We're done for today!
We've installed a virtual environment, which will manage our project's instance of the Python interpreter, and learned how to activate and deactivate it.
We've learned about functions and arguments, both in Python and Bash.
We've learned about code compartmentalization in Python using modules.
We've installed the two modules we'll need to run Fluid Architecture's first set of tasks — requests and Beautiful Soup.
What's To Come
Tomorrow, we'll learn the basics of web scraping, and end the day with Joni Mitchell's entire catalog of lyrics on our computer.
See you tomorrow!











