365Git

Just a short one today.

Say you have a repository that contains a submodule. But, for whatever reason, you want to change the URL that the submodule points to. This could be because that repo moved, or changed it's name. Or maybe you've forked that repository and you want to use your fork as a submodule.

All you need to do is to change the URL in the ".gitmodules" file (I wrote about doing this in my last post). Now, all you need to do is to run the command from the top level of the repository:

git submodule sync

This will update the local repository settings in ".git/config".

Note

If you have more than one submodule, this will update all the submodules in ".git/config" the the URLs in ".gitmodules". If you've set up your local repository with custom URLs (again, which I wrote about last time), this might not be what you want. You can choose to update submodule explicitly with:

git submodule sync -- name

Say you want to add a submodule to your repository, and this submodule is a one to which you have push access. It is frequently useful to be able to publish a repository with the submodule URL as pull-only, while keeping the push-pull URL in your own repository. This way, you can make changes to the submodule repository from within the enclosing repository. This is a simple thing to do in Git once you know how submodules are arranged.

Example

For convenience, let's use a a couple of GitHub repositories as an example. Let's start with the Hello repository, and say I want to add the Just Testing repository as a submodule. That's simple enough; I check out the Hello repository to my local drive:

cd tmp git clone -o github [email protected]:Abizern/Hello.git cd Hello

(You may notice the unfamiliar -o github in the clone command. This names the remote for this clone "github" instead of the default "origin". Git is distributed, and I often have more than one remote for repositories, so I like to name my remotes semantically. Another name I use is "dropbox" for obvious reasons).

Now I'm in the repository I add the remote. This is where I have to make a choice. GitHub gives you a choice of clone URLs:

If I use the Read+Write URLs, for the submodules, and someone else clones my repository, they will be using URLs to which they do not have access. So, cloning the repository will work, but when they go to initialise the submodule, the command will fail because they can't access the repository through that URL. But, If I use the Read-Only URL, I won't be able to push to my submodule repository. That's okay, because I can add the repository with Read+Write access for me but change it for the shared repository. Let's start by adding the submodule to the repository using the Read+Write URL:

git submodule add [email protected]:Abizern/JustTesting.git

This will add a directory called "JustTesting" to my repository and check out the files. You can check the submodule with git submodule status which, in my case, shows this:

a73686133fc66884faac1f8c1d9603b9ffcf09b1 JustTesting (heads/master)

Which shows the sha of the checked out commit of the submodule (there are no - or + adornments so there are no issues) the directory which contains it (JustTesting) and the current description (in this case it just shows the head of the master branch. If there were any tags in the repository it would show the output of git describe).

Checking the status of the repository with git status shows that we have two new files:

JustTesting isn't a new file, it's just the way that Git sees a submodule. What we are interested in is the ".gitmodules" file. You can see that this shows where the the submodule is kept and what it's URL is. Now let's look at the way the submodule is configured:

First I find out what the name of the remote is (yes, I know it's origin) and then I get information about the remote. This shows that it is configured to the Read+Write URL

Let's take a look at the ".git/config" file. This is where the configuration for the local repository is kept:

The interesting section is at the bottom, which has information about the submodule. This looks a lot like the entry in the ".gitmodules" file, except that it doesn't reference the path to the submodule.

Now let's change the URL in the ".gitmodules" file so that it points to the Read-only URL.

So there are two URLs for this submodule. The repository itself contains the Read-Only URL in ".gitmodules" and this is the configuration that is shared among all repositories that clone it. And the private Read+Write module is configured in the ".git/config" file which applies only to this particular repository. Now I just need to commit the changes and I can push this back up to GitHub.

Go and look at the Hello repository for yourself. You can see that JustTesting is a submodule. And if you clone this and update the submodule, it will be configured to point to the Read-Only version of JustTesting:

cd tmp git clone -o github [email protected]:Abizern/Hello.git Hello2 cd Hello2 git submodule update --init

And you can see for yourself that ".gitmodules", ".git/config" are configured for the Read-only URL. I'm not going to show you the output of those because I want to show you something more interesting.

Imagine I am at my other machine and I want to clone this repository but still have Read+Write Access to the submodule. This is how I go about that. I clone the repository as normal but I don't update the submodule in the same way as I just did above

cd tmp git clone -o github [email protected]:Abizern/Hello.git Hello3 cd Hello3

Then – I init the submodule:

git submodule init

And you can see that all this does is register the submodule, but does not actually check out the repository:

Now - I can change my local configuration in ".git/config" so that the submodule URL is the Read+Write one:

And now I update the submodule repository:

git submodule update

And I have Read+Write access from my local submodule:

Explanation

That seems like a lot of steps - but only because I went through it in detail. This is why it works:

URLs for submodules are kept in the ".gitmodules" file. As this is in the top level of the repository, it is shared amongst all repositories that clone from it. When you initially add a submodule to your repository, an entry is added to the ".gitmodules" file and to the local ".git/config" file and the repository is checked out. All in one step (although the changes to the repo have to be committed manually, this allows you to change the public facing configuration).

There is a flag to the git merge command: --squash which at first looks as if it does the same thing as a rebase, but this isn't exactly so. This post attempts to explain the difference.

Consider the following repository structure:

There is a master branch and two feature branches feat1 and feat2. The feat2 branch will introduce a merge conflict, which is why it is highlighted in red.

Say you want to bring the two features into the master branch as a single change. There are, as you would expect with Git, multiple ways of doing this.

Merging

One way to do that is by merging the two branches into the master. From the master branch you can just git merge feat1 feat2 (yes, you can merge multiple branches in one step) You will get an error because of the merge conflict, which you should know how to fix by now, and this will give you this structure:

Rebasing

That's fine if you want to be able to show that the merges have happened, but sometimes, that isn't the case and you want to be able to show a clean history and a single change. To do that, you can use the rebase command. First by rebasing feat1 onto feat2 git checkout feat1; git rebase feat2

And then you can rebase this branch onto the master with git rebase --interactive, where you can squash the three individual commits to a single commit, and resolve the merge conflict introduced by feat2:

And now, switch to the master branch and rebase it onto the feat1 branch (or merge it, the result will be the same as it will resolve as a fast-forward merge). This will give you a single commit with a clean history and all the merged changes from feat1 and feat2:

git merge --squash

Now to the topic of this post; using the --squash flag with git merge. What this does is similar to an interactive rebase where you squash all of the commits, except it doesn't check in the change, it just updates the index (and by extension, the working directory). So, starting from the original repository structure we can get the same result by first git merge --squash feat1 feat2 and fixing the merge conflict. This won't affect the the checked in state of the master branch, but your index will have all the changes:

What's the advantage of this? Well, you can get a clean history by just commiting the changes:

But, even more interestingly, you can now back out the changes from the index (which leaves them in the working directory) and add back selective changes using git add --patch and create an entirely different set of commits.

Which should you use?

For most cases, merging would be the choice of most people. It's simple enough, and if there aren't many micro commits the history will be coherent. In this example it only takes three steps (1. merge 2. fix conflicts 3. commit).

If you are the sort that makes many micro commits (like me) and you then rebase them into larger chunks before merging the changes to another branch, then the rebasing method is the one you are already using, even if you squash all the changes into a single commit. I prefer this method, because you have greater flexibility with what changes you make. I also find it easier to handle merge conflicts this way. But it does take longer (1. rebase feat1 onto feat2 2. Interactive rebase 3. Amend the commit messages 4. fix conflicts 5. commit 6. rebase master onto feat1).

If you have a repository with many submodules, it's easy to update them all at once.

For example: I have my vim dotfiles in a Git repository. Not only does this make it easier for me to keep my vim setup the same across multiple machines, but also lets me add other bundles as submodules which makes it easy to update the bundles and propagate changes.

Once in a while I go into the repository and run:

git submodule foreach 'git pull || :'

The foreach command evaluates a shell command in each of the submodules. In this case git pull. The || : (in bash this is 'or true') stops the foreach command from terminating because of a non-zero return from the shell command.

Or, you could avoid the zero return problem by echoing the return value:

git submodule foreach 'echo `git pull`'

This is just one use of foreach. Since I don't actually develop the submodules, I can keep the repository tidy by running:

I've previously written about how to get the diffs between your working tree, the index and the last commit. But you frequently need to see the difference between the code you're working on and previous commits.

Here's a diagram that summarises the commands:

If you've read the previous article then this should make sense I've used HEAD~ to represent the parent of the tip of the branch you're working on, but this can be replaced with the any of the other methods of referring to commits specified in the gitrevisions documentation.

However, when doing this, you usually only want to see changes between specific files. For this you can optionally pass in the paths that you want the diff between after a pair of dashes --. If you leave out the paths you'll get the complete diff.

git diff --cached HEAD~ -- somefile.py This will show the difference between the version of somefile.py in the index and the version in HEAD~

git diff HEAD~ -- somefile.py This will show the difference between the version of somefile.py in the working tree and the version in HEAD~.

Unsurprisingly, there is a convention for writing Git commit messages. This comes from the submitting patches guidelines for Git itself. In summary:

The first line of the commit message should be a short description (50 characters is the soft limit), and should skip the full stop

The body should provide a meaningful commit message, which:

uses the imperative, present tense: "change" not "changed" or "changes".

includes motivation for the change, and contrasts its implementation with previous behaviour.

Most tools only show the first line of the commit message, which is usually enough to know what it is about. It's like the subject line in an email: you don't need to see the whole message to know whether you are interested in it. The 50 character soft limit is to play nicely with command line displays. If you use vim as a commit message editor it knows about this soft limit, which is why the colour changes after the 50th character. Personally, I go past this if I have to (which is most of the time because of the long names that are used in Cocoa).

The use of the imperative, present tense is one that takes a little getting used to. When I started mentioning it, it was met with resistance. Usually along the lines of "The commit message records what I have done". But, Git is a distributed version control system where there are potentially many places to get changes from. Rather than writing messages that say what you've done; consider these messages as the instructions for what applying the commit will do. Rather than having a commit with the title:

Renamed the iVars and removed the common prefix.

Have one like this:

Rename the iVars to remove the common prefix.

Which tells someone what applying the commit will do, rather than what you did. Also, if you look at your repository history you will see that the Git generated messages are written in this tense as well - "Merge" not "Merged", "Rebase" not "Rebased" so writing in the same tense keeps things consistent. It feels strange at first but it does make sense (testimonials available upon application) and eventually becomes natural.

Having said all that - it's your code, your repository: so set up your own guidelines and stick to them.

After my post about signed tags yesterday Kevin Ballard left a comment about a neat trick that lets you put the gpg public key used to sign tags into the repository itself. This explains how to do just that.

As a practical example I'll use this Hello World repository on GitHub.

What this technique does is to add the gpg public key as a blob to the repository, but not as part of any branch. Normally, this would be cleaned up by git's garbage collection, but by tagging this blob it gets a reference which means it isn't orphaned so it persists.

So the repository will look like this:

It has one branch master and two tags v1.0 that tags version 1 and pubkey that tags the public key.

The project was created and checked in as normal. But how do you add a file by itself without adding it to the project? We do this by adding it as a raw blob to the repository.

Create the tagged public key

First. Lets get the gpg public key that we need. The email I am using for this repository as [email protected] (and I've set this up in the config file) so to get the public key as a plain text file I run:

gpg --armor --export <[email protected]> > pubkey.txt

The actual name of the file doesn't matter because a blob only stores the contents of a file. Now we can turn this file into a blob with:

git hash-object -w pubkey.txt

This will compute the hash of the object and the -w flag gets the object written into the object database. It returns the hash value. you want to copy this. In my case it was

ab147894b92328ffef9db011789e4d0e55652688

So now I can tag this blob, even though it isn't in the index or part of the tree of commits:

git tag -s pubkey ab147894b92328ffef9db011789e4d0e55652688

This creates a signed tag called pubkey that points to the blob that contains the public key. I didn't need to use a signed tag, but it doesn't hurt.

Now, I don't need the pubkey.txt file so I can delete it. Running git status now should show that there is nothing more to do.

You can now tag other commits, as I have done with v1.0, and sign them. Remember when pushing your changes to remote repositories you need to push tags explicitly. You can use git push origin --tags to push all tags or git push origin pubkey to push just the pubkey tag (assuming your remote repository is named origin).

Verify signed tags.

Now that the public key is in the repository, anybody who clones the repository can add the public key to their gpg keychain by:

git cat-file blob pubkey | gpg --import

This gets the contents of the blob pointed to by pubkey which is the text version of the public key formatted for export and imports it to your own gpg keychain.

Now they can verify a signed tag, for example v1.0 by:

git tag -v v1.0

and the output will be something like this:

object d56fe222cd3d5fe5ac1709368c5c09cd496932df type commit tag v1.0 tagger Abizer Nasir <[email protected]> 1295369217 +0000 Tagging version 1.0 gpg: Signature made Tue 18 Jan 16:46:57 2011 GMT using RSA key ID 65875E7F gpg: Good signature from "Abizer Nasir <[email protected]>" gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 9CC3 9E69 683B DD90 36DD 4622 FFD4 A90C 2D9E EF7B Subkey fingerprint: 83B8 FAA4 62BD 4821 49B3 5FE3 4F65 DD8F 6587 5E7F

The important thing here is to see the line that says that this is a good signature. The warning just says that you only have my word that this signature actually belongs to me. Those warnings tie in to the GPG web of trust which is beyond the scope of this site.

Have a look at the example repository and try it for yourself. As an exercise for the reader, why not sign my public key in your keychain so you don't get these warning messages.

FAQ

Why can't I see the pubkey tag in my Git GUI?

Some of the visualisation tools, such as gitk don't show tags that point to objects, only tags that point to commits. Which is why it is useful to use the command git tag to show the tags in a repo.

How do I see what's in a tag?

you can use git cat-file -p <tagname> or git show <tagname> which also shows a diff.

How do I get the SHA1 of the tag?

I've already written about the Git Tag Object and made a passing reference to signed tags.

Git doesn't make any checks about whether the name and email you enter into a git config file really is you, so it's trivial make a check-in that looks like it's been made by someone else. But, if you have some form of GPG installed, you can cryptographically sign a tag object (not a lightweight tag) than can be verified as being created by you.

Since a tag object references a commit, which in turn references another commit, and so on, a tag verifies a whole tree of commits. Changing any of the commits, or the tag, would mean new objects would be created that would break this tree. So by signing a tag, which points to a commit, a whole section of the code history is being validated by someone. And if you trust that person, and you trust his public key, you can verify that the tag was created by that person.

To create a signed tag, just pass the -s flag instead of the -a flag:

git tag -s -m"tagging version 1.0" v1.0

This will create a signed tag object called v1.0 with the passed commit message and it will use the signing identity of the committer (if there is a key for that). The documentation for git-config states that you can set a key for user.signingkey that will be used by default to sign commits, but I've found that I don't need to do this as the correct key is used based on the user.email that I am using. A prompt will come up asking you for your key's passphrase.

If you want to use a particular signing identity when tagging then use then use:

git tag -u <key-id> -m"tagging version 1.0" v1.0

Here, the -u <key-id> specifies the key to use.

Also, as with commit messages, if you leave out the message an editor will be brought up for you to enter the tag message.

The signed tag object is only slightly different from the unsigned tag object in that it has a PGP signature block. To verify the signature use the -v flag of the tag command

git tag -v v1.0

and if you have the correct public key for the user in your GPG Keychain it will let you know if the signature is valid or not.

Depending on how you installed Git, there may or may not be man pages on your system. ##Using manpages [Manpages](http://en.wikipedia.org/wiki/Manpages) are the standard way to get help with command line programs. To see if they are installed, type `man git` into the terminal and you should get the man page for Git.  The space bar will move the page forwards and `q` will get you back to the terminal. If you want to know more about man, type `man man` into the terminal. There are manpages for the the Git commands, but you need to hyphenate the command. e.g. to get help on commits, type `man git-commit`. Actually the full command names are hyphenated, but the git command wraps these up for you. ##Updating manpages It is easy to download the latest source code for Git (using a Git repository, of course) and build the program yourself. But on the Mac it takes more work to build the man pages. However, you can keep up to date with the manpages by downloading prebuilt ones from [http://kernel.org/pub/software/scm/git/](http://kernel.org/pub/software/scm/git/) Installing and downloading them is easy I look on the page to see what the latest version is, and then I run this script in a temporary directory. It downloads and installs the latest manpages and tidies up after itself.

#!/bin/sh VERSION=1.7.1.rc1 curl -O \"http://kernel.org/pub/software/scm/git/git-manpages-$VERSION.tar.bz2\"; sudo tar xjv -C /usr/local/share/man -f \"git-manpages-$VERSION.tar.bz2\"; rm git-manpages-$VERSION.tar.bz2

Almost every git user knows about adding a `.gitignore` file to root of the project to control the visibility of files and folders. Because this file is in the repository; this configuration also applies to remote repositories of the project. But it’s not the only way. I’m going to tell you how to get Git to ignore files on a per computer and per repository basis. These could be better choices in some circumstances. There are three types of exclude files; from highest to lowest order of precedence they are: ##Per Project: .gitignore file in the repository This is the usual way of adding an ignore file. Call it `.gitignore` and save it to the root of your project to apply to all the files (you can add different `.gitignore` files in subdirectories where they have lesser scope). It is a part of the repository, so it will need to be `git-add`ed and committed for each change. This is useful for repositories that are passed around with others who may not have a per computer exclude file, or when there are project specific files that need to be taken into account. Even easier if you have per computer file, you can copy it straight in to your project with a simple `cp ~/.gitignore .gitignore` and edit to handle your specific requirements. ##Per Repository: in .git/info/excludes You can exclude files on a per repository basis by editing the `.git/info/excludes` file in your repository. (Why it takes it from this location rather than `.git/config` I don’t know: add it to the list of git annoyances). These exclusions (or inclusions, you can override the higher level exclusions by prepending `!` to lines that you want to include) are not shared with the working directory, so they only apply to that particular repository, and are not shared with any remotes. This is useful when you have particular requirements because of your workflow or machine setup. ##Per Computer: through settings in ~/.gitconfig There should already be a `.gitconfig` file in your home directory. This is where the global setting for your git installation are stored; such as the user’s name and email address. Within this you can set a path to an excludes file that will apply to all git repositories on the computer in the same way as the name and email defined in this file apply to all repositories. For example: Most of what I do is in [Xcode](http://developer.apple.com/tools/xcode/ "Apple's Xcode site") so I have the following ~/.gitignore file # Mac OS X *.DS_Store # Xcode *.pbxuser *.mode1v3 *.mode2v3 *.perspectivev3 *.xcuserstate project.xcworkspace/ xcuserdata/ # Generated files build/ *.[oa] *.pyc # Backup files *~.nib And in my `.gitconfig` file under the `[core]` section I’ve added the path to this file for the `excludesfile` key.

[core]     excludesfile = ~/.gitignore