#filter-branch

Anonymizing a GIT Repository

Asume you have worked with others on a repository and want to make it public afterwards. Or want to pull request a nice feature you created. But names should be removed for privacy resons.

At university, students create great results during the semester. But some prefer not to be named, if a repository is published. Here is how to get this done.

Steps (for Windows 8.1 environment)

Make an export  from current repository and import it to a new one (that will be public)

Filter in all branches the names and e-mail-adresses

Remove references to old commits

Push it to your new repository remote URL (see our result here)

1. Make an export

First create an empty folder (/empty/repository) and run an git init. The rest is easy with git fast-export. See official docs.

git fast-export --all | (cd /empty/repository && git fast-import)

Note: It offers an option --anonymize, but this removes not only names, but all tag names and commit messages  are cryptified as well. Too much in most cases.

2. Filter out names and e-mail adresses

This is a tough one. Support exists with git filter-branch --env-filter .

Note: For Windows the parameter for environment filtering needs double quotes “ “ instead of ‘‘. Ironically, the shell syntax inside is then evaluated by a cygwin console by git. So rewriting the Linux syntax to windows fails. The following works for overwriting all names and e-mail-adresses:

git filter-branch -f --env-filter "export GIT_COMMITTER_NAME=Anonymized; export [email protected]; export GIT_AUTHOR_NAME=Anonymized; export [email protected]" -- --all

Pay attention to the space before the -- after the exports. git is here very sensitive.

Done? Not completely. In my case, students committed with messages that contained their student id, like “s12123 added file router.ts” and so on. I wanted to remove the student IDs. This can be done by filtering the commit messages:

git filter-branch -f --msg-filter "sed -e 's/s[0-9]\+/someone/g'" -- --all

This sed command for the shell replaces all strings starting with s, followed by numbers, with the string someone.

3. Remove references to old commits

git keeps backups of the old branches pointers and this allows easy access to the names. To remove this, git update-ref helps here, but it is painful to do it for all backed up references manually. Make it a batch process:

git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d

Note: Windows has no xargs. To let this work, install a lightwight version of cygwin for your windows. I used GOW.

4. Push it to the new repositorys remote url

Do it as always.

Compact Batch version with Bonus

I summed the steps all up in one batch line. And I added the --tag-name-filter in order to let git fix the tags automatically. Because git creates new commits with different hashes for the filtering, tags need to be corrected, too.

git filter-branch -f --env-filter "export GIT_COMMITTER_NAME=Anonymized; export [email protected]; export GIT_AUTHOR_NAME=Anonymized; export [email protected]"^ --msg-filter "sed -e 's/s[0-9]\+/someone/g'"^ --tag-name-filter cat^ -- --all git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d

Note: pay attention to the ^ Windows escape character. It prevents interpretation of newline.

DONE.

Giant Undo Button

A friend once described git to me as "a giant undo button." It keeps track of all the changes you make to your file structure so that you can go back to its state at any time later.

Undo the Undo-ability

Unfortunately, a few days ago I let some large images (background images and test images) for my web app into my file structure. They were tracked by git, and now my repository is too large. To undo that would involve undoing many other things, unless it is done carefully. A large repository can stop the webserver we are using from working. It also slows down your git pushes and pulls, and is generally an inefficient way to work. Today I re-wrote history to remove these images from my repository. A few of the commands involved in this process:

git branch (to make any changes undo-able)

git whatchanged

git log -Sjpg (shows only commits that affect filenames with jpg)

在使用git的过程中，有时候可能会不小心将错误的author和commiter的信息记录到repo中，或者有时需要修改以前的author和commiter的陈旧信息。这时候可以用如下方法

在执行下面操作前，一定记住要备份git clone，因为git filter-branch是会rewrite所有的历史信息的，如果出错，以前的记录可能会丢失。

git filter-branch --commit-filter ' if [ "$GIT_AUTHOR_EMAIL" = "author_name@author_email_domain" ]; then GIT_AUTHOR_NAME="author_name"; GIT_AUTHOR_EMAIL="author_name@author_email_domain"; git commit-tree "$@"; else git commit-tree "$@"; fi' HEAD