progress-log

A well-cared for log is a beautiful and useful thing. A project’s long-term success rests (among other things) on its maintainability, and a maintainer has few tools more powerful than his project’s log. It’s worth taking the time to learn how to care for one properly. What may be a hassle at first soon becomes habit, and eventually a source of pride and productivity for all involved. In this post, I am addressing just the most basic element of keeping a healthy commit history: how to write an individual commit message. There are other important practices like commit squashing that I am not addressing here.

In order to create a useful revision history, teams should first agree on a commit message convention that defines at least the following three things:

Style. Markup syntax, wrap margins, grammar, capitalization, punctuation. Spell these things out, remove the guesswork, and make it all as simple as possible. The end result will be a remarkably consistent log that’s not only a pleasure to read but that actually does get read on a regular basis.

Content. What kind of information should the body of the commit message (if any) contain? What should it not contain?

Metadata. How should issue tracking IDs, pull request numbers, etc. be referenced? The seven rules of a great Git commit message

Separate subject from body with a blank line

Limit the subject line to 50 characters

Capitalize the subject line

Do not end the subject line with a period

Use the imperative mood in the subject line

Wrap the body at 72 characters

Use the body to explain what and why vs. how

Firstly, not every commit requires both a subject and a body. Sometimes a single line is fine, especially when the change is so simple that no further context is necessary. For example:

Fix typo in introduction to user guide

Nothing more need be said; if the reader wonders what the typo was, she can simply take a look at the change itself, i.e. use git show or git diff or git log -p. However, when a commit merits a bit of explanation and context, you need to write a body. For example:

Derezz the master control program MCP turned out to be evil and had become intent on world domination. This commit throws Tron's disc into MCP (causing its deresolution) and turns it back into a chess game.

Commit messages with bodies are not so easy to write with the -m option. You’re better off writing the message in a proper text editor.

—

Example:

Summarize changes in around 50 characters or less More detailed explanatory text, if necessary. Wrap it to about 72 characters or so. In some contexts, the first line is treated as the subject of the commit and the rest of the text as the body. The blank line separating the summary from the body is critical (unless you omit the body entirely); various tools like `log`, `shortlog` and `rebase` can get confused if you run the two together. Explain the problem that this commit is solving. Focus on why you are making this change as opposed to how (the code explains that). Are there side effects or other unintuitive consequences of this change? Here's the place to explain them. Further paragraphs come after blank lines. - Bullet points are okay, too - Typically a hyphen or asterisk is used for the bullet, preceded by a single space, with blank lines in between, but conventions vary here If you use an issue tracker, put references to them at the bottom, like this: Resolves: #123 See also: #456, #789

5. Use the Imperative Mood The imperative can sound a little rude; that’s why we don’t often use it. But it’s perfect for Git commit subject lines. One reason for this is that Git itself uses the imperative whenever it creates a commit on your behalf. Writing this way can be a little awkward at first. We’re more used to speaking in the indicative mood, which is all about reporting facts. That’s why commit messages often end up reading like this:

Fixed bug with Y

Changing behavior of X

And sometimes commit messages get written as a description of their contents:

More fixes for broken stuff

Sweet new API methods

To remove any confusion, here’s a simple rule to get it right every time.

A properly formed Git commit subject line should always be able to complete the following sentence:

If applied, this commit will your subject line here

6. Wrap the body at 72 characters

Git never wraps text automatically. When you write the body of a commit message, you must mind its right margin, and wrap text manually.

The recommendation is to do this at 72 characters, so that Git has plenty of room to indent text while still keeping everything under 80 characters overall.

A good text editor can help here. It’s easy to configure Vim, for example, to wrap text at 72 characters when you’re writing a Git commit. Traditionally, however, IDEs have been terrible at providing smart support for text wrapping in commit messages (although in recent versions, IntelliJ IDEA has finally gotten better about this).

In 2005, the relationship between the community that developed the Linux kernel and the company that developed BitKeeper broke down, and the tool’s free-of-charge status was revoked. This prompted the Linux development community (and in particular Linus Torvalds, the creator of Linux) to develop their own tool based on some of the lessons they learned while using BitKeeper. Some of the goals of the new system were as follows: speed, simple design, strong support for non-linear development, fully distributed, able to handle large projects efficiently. Since 2005, Git has evolved to be easy to use, and is amazingly fast, very efficient with large projects, and has an incredible branching system for non-linear development.

Getting-Started-What-is-Git - Chapter 1.3 Snapshots, Not Differences The major difference between Git and any other VCS is the way Git thinks about its data. Conceptually, most other systems store information as a list of file-based changes. They think of the information they store as a set of files and the changes made to each file over time (this is commonly described as delta-based version control). Git doesn’t think of or store its data this way. Instead, Git thinks of its data more like a series of snapshots of a miniature filesystem. Every time you commit, or save the state of your project, Git basically takes a picture of what all your files look like at that moment and stores a reference to that snapshot. If files have not changed, Git doesn’t store the file again, just a link to the previous identical file it has already stored. Git thinks about its data more like a stream of snapshots.

Nearly Every Operation Is Local

Most operations in Git need only local files and resources to operate — generally no information is needed from another computer on your network. If you’re used to a CVCS where most operations have that network latency overhead, this aspect of Git will make you think that the gods of speed have blessed Git. Because the entire history of the project is right there on your local disk, most operations seem almost instantaneous.

Ex. Git doesn’t need to go to the server to get the history — it reads it from your local database, so you see it instantly. If you want to see the changes between the current version and the file a month ago, Git can look up the file a month ago and do a local difference calculation, instead of having to either ask a remote server to do it or pull an older version of the file from the remote server to do it locally.

This also means that there is very little you can’t do if you’re offline or off VPN. If you get on an airplane and want to do a little work, you can commit happily (to your local copy, remember?) until you get to a network connection to upload. ~Checksums + Hash Everything in Git is checksummed before it is stored and is then referred to by that checksum. So it’s impossible to change the contents of any file or directory without Git knowing about it—this functionality is integral to its philosophy. You can’t lose information in transit or get file corruption without Git being able to detect it.

The mechanism that Git uses for checksumming is called a SHA-1 hash. This is a 40-character string composed of hexadecimal characters (0–9 and a–f) and calculated based on the contents of a file or directory structure in Git. A SHA-1 hash example: this:24b9da6552252987aa493b52f8696cd6d3b00373

You will see these hash values everywhere in Git as it stores everything in its database not by file name but by the hash value of its contents. — The Three States

Pay attention now — here is the main thing to remember about Git if you want the rest of your learning process to go smoothly. Git has three main states that your files can reside in: modified, staged, and committed:

Modified means that you have changed the file but have not committed it to your database yet.

Staged means that you have marked a modified file in its current version to go into your next commit snapshot.

Committed means that the data is safely stored in your local database.

This leads us to the three main sections of a Git project:

The working tree is a single checkout of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify.

The staging area is a file, generally in your Git directory, that stores information about what will go into your next commit. Its technical name in Git parlance is the “index”, but the phrase “staging area” works just as well.

The Git directory is where Git stores the metadata and object database for your project. This is the most important part of Git, and it is what is copied when you clone a repository from another computer.

The basic Git workflow goes something like this:

You modify files in your working tree.

You selectively stage just those changes you want to be part of your next commit, which adds only those changes to the staging area.

You do a commit, which takes the files as they are in the staging area and stores that snapshot permanently to your Git directory.

By making branches. Think of your Git commit history as a tree. The trunk is what we refer to as the master branch. In order for teams to avoid stepping on each other’s toes, they need to work in isolation from everyone else (on a feature branch), whilst ultimately still contributing back to the main codebase (the master branch). The general workflow starts with each author branching off master when they’re doing a piece of work (maybe writing a chapter, or editing one). When the rest of the co-authors have approved the changes, the branch gets merged into the master trunk. When a branch gets merged that means its content overrides the master. So any changes to existing content will supersede what was there before, and any new content will be added in too. In essence, when a branch gets merged into master, its commits get added to the top of the master history.

Your remote repository on Github is a mirror of what you have locally. So on your computer you have a local Git repository for that project (i.e.: a folder configured to allow Git commits). Inside this local Git repo (again, a specific Git-enabled folder on your computer), you have all the files that are part of this project. This ultimately syncs up to a location online. However, as we know, there are extra steps involved in the Git/Github workflow. You have to consciously decide to take a “snapshot” of your work as it stands at that moment in time (a commit) and deliberately choose to push that commit up to Github. Only then is the work “synced” to the online location (the Github repo).

So why not automate it? Why not make it like Dropbox and have the file update on Github as you update it locally?

Mainly, because of bugs. And you don’t always want to keep everything you write.

Sometimes you want to experiment, and should that fail, you want an easy way to go back to the last correctly working state. So the minute something is working the way you want it to, commit it before you try to edit it or experiment with a different approach. There’s no harm in committing small chunks of work very frequently—many engineers pride themselves on that doing exactly that. — Eventually, she’s ready for Tom and Adam to review her work and creates a Pull Request, a Github feature that allows her to explain the changes that her branch makes to the master branch. It's also where the co-authors can have a discussion about the branch’s content and request changes before the branch becomes part of the master trunk.

At this point, Sally can move to (or “check out”) the master branch on her local machine and pull down the very commits that were previously isolated in a feature branch. She’s now back at square one, with an updated master, and can proceed to create a new local branch for her next piece of work, helping poor Tom edit his chapter on Goblins. And so, the process would start all over again:

Create local branch

Write some commits within local branch

Push to Github

Create Pull Request explaining changes that branch contains

Merge to master

Pull down new master commits locally

Repeat

This is where Git shines. If Sally had been using Word combined with Git, she could’ve associated all those changes into one nice neat little summary of “Introduction of Elves into the series”. She could’ve seen the changes made across pages, chapters, files, and books, allowing her to truly understand the impact of the Elves on her fantasy series. This “neat little summary” is what we, in Git-land, refer to as a commit.

Recap. Git is a program that allows you to annotate the history of a system (or group) of files via commits, which are “snapshots” of the differences made to the system at a given point in time. Github takes that lovely commit history and hosts it online so that you can access it from any computer. You do this via pushing commits from your local machine (i.e.: the computer you’re currently using) up to Github, and then, from the new/different computer pulling those commits down. Sally author workflow ex. cont. "Sally starts her day working on her home desktop, writes some chapters, does some editing, etc. Along the way, she takes three “snapshots” (Git commits) of her work at strategic points... before she shuts down her home desktop computer, she makes sure to push her current Git commit history up online to Github. Once on Github, the commits live in a remote repository."

remote merely means online (vs local, which as we learned earlier means whichever computer you’re currently using). As for repository (often shorted to “repo”), it’s basically a folder with Git superpowers.

General Syntax of a Shell Command

We have now encountered commands, options, and arguments, but it is perhaps useful to formalise some terminology. Consider the command below as a general example of a command, which we will dissect into its component parts:

ls is the command, with an option-F and an argument/. We’ve already encountered options which either start with a single dash (-), known as short options, or two dashes (--), known as long options. [Options] change the behavior of a command and Arguments tell the command what to operate on (e.g. files and directories). Sometimes options and arguments are referred to as parameters.  Sometimes options are referred to as switches or flags, especially options that take no argument. In this lesson we will stick with using the term option. Note that the sizes returned by ls -s are in blocks. These are defined differently for different operating systems. When options exist as both short and long options: - Use the short option when typing commands directly into the shell to minimize keystrokes and get your task done faster. - Use the long option in scripts to provide clarity. It will be read many times and typed once.

Nelle's Pipeline: Organizing Files

Nelle creates a directory called north-pacific-gyre (to remind herself where the data came from), which will contain her data files and data processing scripts. Each of her physical samples is labelled according to her lab’s convention with a unique ten-character ID, such as ‘NENE01729A’. This ID is what she used in her collection log to record the location, time, depth, and other characteristics of the sample, so she decides to use it within the filename of each data file. All 1520 files will go into the same directory.

If Nelle types part of a directory, then presses Tab, the shell will automatically complete the directory name for her using tab completion. Pressing Tab twice brings up a list of all the files. If Nelle then presses G and then presses Tab again, the shell will append ‘goo’ since all files that start with ‘g’ share the first three characters ‘goo’. To see all of those files, she can press Tab twice more.

Keypoints

The file system is responsible for managing information on the disk.

Information is stored in files, which are stored in directories (folders).

Directories can also store other directories, which then form a directory tree.

pwd prints the user’s current working directory.

ls [path] prints a listing of a specific file or directory; ls on its own lists the current working directory.

cd [path] changes the current working directory.

Most commands take options that begin with a single -.

Directory names in a path are separated with / on Unix, but \ on Windows.

/ on its own is the root directory of the whole file system.

An absolute path specifies a location from the root of the file system.

A relative path specifies a location starting from the current location.

Exploring Other Directories

The file system is the part of the operating system responsible for managing files and directories. It organizes our data into files, which hold information, and directories (also called ‘folders’), which hold files or other directories.

PWD

pwd (which stands for ‘print working directory’) shows you where you are. Directories are like places — at any time while we are using the shell, we are in exactly one place called our current working directory. Commands mostly read and write files in the current working directory, i.e. ‘here’, so knowing where you are before running a command is important.  Here, the computer’s response is /Users/nelle, which is Nelle’s home directory. Let’s have a look at how the file system as a whole is organized.

Like an upside down tree, the topmost directory is the root directory that holds everything else. We refer to it using a slash character, /, on its own; this character is the leading slash in /Users/nelle. Inside that directory are several other directories: bin (which is where some built-in programs are stored), data (for miscellaneous data files), Users (where users’ personal directories are located), tmp (for temporary files that don’t need to be stored long-term), and so on.

The most widely used way to interact with personal computers is called a graphical user interface (GUI). With a GUI, we give instructions by clicking a mouse and using menu-driven interactions. While intuitive to learn, this way of delivering instructions to a computer scales poorly. Imagine: to copy the third line of one thousand text files in one thousand different directories and paste it into a single file [would take] hours with a GUI, and you could commit an error. The Unix shell is both a command-line interface (CLI) and a scripting language, allowing such repetitive tasks to be done automatically and fast. With the proper commands, the shell can repeat tasks with or without some modification as many times as we want, [and complete the task in the example] in seconds. While a GUI presents you with choices to select, CLI choices are not automatically presented to you, so you must learn a few commands. Unlike a spoken language, a small number of “words” (i.e. commands) gets you a long way. Sequences of commands can be written into a script, improving the reproducibility of workflows. As clusters and cloud computing systems become more popular for scientific data crunching, being able to interact with the shell is becoming a necessary skill. We can build on the command-line skills covered here to tackle a wide range of scientific questions and computational challenges. — Most popular shell environments by default put your user name and the host name before the $. Our first command, ls, which is short for listing. This command will list the contents of the current directory. If the shell can’t find a program whose name is the command you typed, it will print an error message such as: ks: command not found

Belle's Pipeline: A Typical Problem

Nelle Nemo has 1520 samples that she needs to run through an imaginary program called goostats.sh. In addition to this huge task, she has to write up results by the end of the month, so her paper can appear in a special issue of Aquatic Goo Letters. If Nelle chooses to run goostats.sh by hand using a GUI, she’ll have to select and open a file 1520 times. If goostats.sh takes 30 seconds to run each file, the whole process will take more than 12 hours of Nelle’s attention. With the shell, Nelle can instead assign her computer this mundane task while she focuses her attention on writing her paper. More specifically, the lessons explain how she can use a command shell to run the goostats.sh program, using loops to automate the repetitive steps of entering file names, so that her computer can work while she writes her paper. As a bonus, once she has put a processing pipeline together, she will be able to use it again whenever she collects more data. In order to achieve her task, Nelle needs to know how to: - navigate to a file/directory - create a file/directory - check the length of a file - chain commands together - retrieve a set of files - iterate over files - run a shell script containing her pipeline

Keypoints

A text editor is by far the most used developer tool regardless of what type of developer you are. A good text editor can help you write better code with real-time code checking, syntax highlighting, and automatic formatting. Plugins, syntax highlighting, auto-closing of brackets and braces, and linting are just a few of the benefits of using a code editor. Visual Studio Code, or just VSCode as it’s commonly referred to, is an excellent free code editor. It has outstanding add-on support and great Git integration. VSCode is the most popular code editor among Odin’s students and moderators, so support is easy to find in the community. Keep in mind that this means you will not be able to get help if you are using a different text editor other than VSCode for the curriculum. [unrelated] As a reminder, if you’re using a virtual machine, you should install VSCode on your VM. You’re welcome to also install it on your host (i.e., your Windows main OS), but you’ll want to be sure that you have this critical tool inside your VM. —

foundations-command-line-basics

Clients and servers

Computers connected to the internet are called clients and servers.

Clients are the typical web user's internet-connected devices (for example, your computer connected to your Wi-Fi, or your phone connected to your mobile network) and web-accessing software available on those devices (usually a web browser like Firefox or Chrome).

Servers are computers that store webpages, sites, or apps. When a client device wants to access a webpage, a copy of the webpage is downloaded from the server onto the client machine to be displayed in the user's web browser.

The other parts of the toolbox

analogy: imagine web = road, client = house, server = store.

In addition to the client and the server, we also need to say hello to:

Your internet connection: Allows you to send and receive data on the web. Like the street between your house and the shop.

TCP/IP: Transmission Control Protocol and Internet Protocol are communication protocols that define how data should travel across the internet. This is like the transport mechanisms that let you place an order, go to the shop, and buy your goods. In our example, this is like a car or a bike (or however else you might get around).

DNS: Domain Name System is like an address book for websites. When you type a web address in your browser, the browser looks at the DNS to find the website's IP address before it can retrieve the website. The browser needs to find out which server the website lives on, so it can send HTTP messages to the right place (see below). This is like looking up the address of the shop so you can access it.

HTTP: Hypertext Transfer Protocol is an application protocol that defines a language for clients and servers to speak to each other. This is like the language you use to order your goods.

Component files: A website is made up of many different files, which are like the different parts of the goods you buy from the shop. These files come in two main types:

Code files: Websites are built primarily from HTML, CSS, and JavaScript, though you'll meet other technologies a bit later.

Assets: This is a collective name for all the other stuff that makes up a website, such as images, music, video, Word documents, and PDFs.

So what happens, exactly?

When you type a web address into your browser (for our analogy that's like walking to the shop):

The browser goes to the DNS server, and finds the real address of the server that the website lives on (you find the address of the shop).

The browser sends an HTTP request message to the server, asking it to send a copy of the website to the client (you go to the shop and order your goods). This message, and all other data sent between the client and the server, is sent across your internet connection using TCP/IP.

If the server approves the client's request, the server sends the client a "200 OK" message, which means "Of course you can look at that website! Here it is", and then starts sending the website's files to the browser as a series of small chunks called data packets (the shop gives you your goods, and you bring them back to your house).

The browser assembles the small chunks into a complete web page and displays it to you (the goods arrive at your door — new shiny stuff, awesome!).

Order in which component files are parsed

When browsers send requests to servers for HTML files, those HTML files often contain <link> elements referencing external CSS stylesheets and <script>elements referencing external JavaScript scripts. It's important to know the order in which those files are parsed by the browser as the browser loads the page:

The browser parses the HTML file first, and that leads to the browser recognizing any <link>-element references to external CSS stylesheets and any <script>-element references to scripts.

As the browser parses the HTML, it sends requests back to the server for any CSS files it has found from <link> elements, and any JavaScript files it has found from <script> elements, and from those, then parses the CSS and JavaScript.

The browser generates an in-memory DOM tree from the parsed HTML, generates an in-memory CSSOM structure from the parsed CSS, and compiles and executes the parsed JavaScript.

As the browser builds the DOM tree and applies the styles from the CSSOM tree and executes the JavaScript, a visual representation of the page is painted to the screen, and the user sees the page content and can begin to interact with it.

DNS explained

Real web addresses aren't the nice, memorable strings you type into your address bar to find your favorite websites. They are special numbers that look like this: 192.0.2.172.

This is called an IP address, and it represents a unique location on the web. However, it's not very easy to remember, is it? That's why the Domain Name System was invented. This system uses special servers that match up a web address you type into your browser (like "mozilla.org") to the website's real (IP) address.

Websites can be reached directly via their IP addresses. You can use a DNS lookup tool to find the IP address of a website.

Packets explained

— web server: a computer that hosts a website on the Internet A web server is a computer hosting one or more websites. "Hosting" means that all the web pages and their supporting files are available on that computer. The web server will send any web page from the website it is hosting to any user's browser, per user request. Don't confuse websites and web servers. "My website is not responding", means the web server is not responding and therefore the website is not available. Since a web server can host multiple websites, the term web server is never used to designate a website, as it could cause great confusion. If we said, "My web server is not responding", that means multiple websites on that web server are not available. What_is_a_web_server: web server can refer to hardware or software

On the hardware side, a web server is a computer that stores web server software and a website's component files (for example, HTML documents, images, CSS stylesheets, and JavaScript files). A web server connects to the Internet and supports physical data interchange with other devices connected to the web.

On the software side, a web server includes several parts that control how web users access hosted files. At a minimum, this is an HTTP server. An HTTP server is software that understands URLs (web addresses) and HTTP(the protocol your browser uses to view webpages). An HTTP server can be accessed through the domain names of the websites it stores, and it delivers the content of these hosted websites to the end user's device.