SonicChicken

tl:dr; Use the command line, Luke!

I forked the Dash2 project on my personal github a while back, and haven't updated it since. I could simply delete my fork and re-create it at this point, but only because I haven't got any commit history on my fork. The more general solution is to get my fork up-to-date with the original project. Github has a pretty nice web front end, but keeping a fork updated isn't a clean and simple process via the GUI. So time for me to see how well hub handles this...

First thing to do is get the fork from github onto my local system:

$ git clone dashv2

Check my remotes:

$ cd dashv2 $ git remote -v

The only remotes listed were my own private fork, so add a new remote pointing to the original project. In keeping with the instructions for configuring a GitHub remote, I'm using upstream as the label:

$ git remote add upstream https://github.com/CDLUC3/dashv2.git $ git remote -v origin {elided} upstream https://github.com/CDLUC3/dashv2.git (fetch) upstream https://github.com/CDLUC3/dashv2.git (push)

Okay, now that's done, it's time to sync my fork with the main project:

$ git fetch upstream $ git branch -va (make sure branches and remotes look sane!) $ git checkout master

At this point, if you have local changes on the master branch, stash them via git stash.

$ git rebase upstream/master $ git push origin master $ git stash pop (if you stashed changes earlier)

The master branch isn't much used (yay!), so other branches need to be updated. Because I'm working with multiple remotes (my own fork on github, plus the original project), I can't simply git checkout branch -- I need to specify the remote as well:

$ git checkout -b development --track upstream/development $ git status On branch development Your branch is up-to-date with 'upstream/development'. nothing to commit, working tree clean $ git rebase upstream/development $ git push origin development

If you have more than 1-2 branches, the steps above are likely to get botched. So here's a little bash loop. Just drop your branch names in the for statement, and assuming you are using origin and upstream for your remotes, you can simply copy/paste the commands:

Over the next few days I'll be trying out hub -- a command line utility to simplify working with github repos via git. On my Mac, this was a super-easy install via homebrew:

brew install hub

Then just add an alias to ~/.bashrc:

alias git=hub

I also set it up to use https by issuing the following bash command:

git config --global hub.protocol https

I've been having trouble adding a second host to my CFEngine network. It looks like out-of-the-box network security in CentOS (which is the O/S running on my policy server) is preventing the new Ubuntu host from connecting and downloading policy.

I checked to see if iptables was running on the policy server:

# /etc/init.d/iptables status (yep, plenty of host firewall rules being enforced!) # chkconfig iptables && echo "iptables set to run" iptables set to run

So I either need to poke a hole in the firewall, or let CFEngine manage this for me (later)! To get the Ubuntu host up and running quickly, I simply turned off iptables:

# /etc/init.d/iptables stop iptables: Setting chains to policy ACCEPT: filter [ OK ] iptables: Flushing firewall rules: [ OK ] iptables: Unloading modules: [ OK ] # chkconfig iptables off

Then, because I've been burned by SELinux multiple times, I changed that from enforcing to permissive:

# sestatus SELinux status: enabled SELinuxfs mount: /selinux Current mode: enforcing Mode from config file: enforcing Policy version: 24 Policy from config file: targeted # edit /etc/selinux/config (change "SELINUX=enforcing" to: SELINUX=permissive

Now reboot the policy server, check iptables and sestatus to make sure they're downgraded.

# shutdown -r now (reboot...) # chkconfig iptables || echo "iptables NOT set to run" iptables NOT set to run # /etc/init.d/iptables status iptables: Firewall is not running. # sestatus SELinux status: enabled SELinuxfs mount: /selinux Current mode: permissive Mode from config file: permissive Policy version: 24 Policy from config file: targeted

And finally, bootstrap the Ubuntu host to the policy server:

/var/cfengine/bin/cf-agent -KI --bootstrap my.policy.server (LOTS of output here...)

Hurrah, my second CFEngine system is online!

Next steps will include:

Make sure my git-daemon is running (via CFEngine).

Setting up a CFEngine policy to poke a hole in iptables to allow policy updates, then resuming iptables (also via CFEngine).

Checking SELinux logs to see if it would be blocking anything important.

Hopefully start writing up my branch naming, testing, and merging procedures for policy files.

References for iptables and SELinux:

https://docs.cfengine.com/latest/guide-installation-and-configuration-general-installation-installation-community.html

tl;dr: Implementing CFEngine policy server, storing policy in a local git repo.

Background

If you maintain one or more Unix boxes, you can benefit from a configuration management system. Want to make sure the root password is changed, on all your systems, to the same new password, securely and automatically? Piece of cake! Want to change an option in all of your sshd config files? No problem! Want to update OpenSSL so that it's not vulnerable to HeartBleed? Totally doable!

I've made several attempts over the past few years to implement a configuration management (CM) system on my home network. I know it's a long- (and even medium-) term win once implemented, but I always seem to run up against one issue or another which derails my progress. I think the underlying problem for me is that I want my network to be "perfect" as soon as I begin implementing CM; instead, what I should focus on is converging on a well-managed network. After all, today's best practice might very well become tomorrow's flawed implementation, so "perfection" is a moving target.

When I first started looking at Open-source CM systems (10 years ago?), I chose CFEngine for the following reasons:

It's mature -- CFEngine has been around for decades(!). It's not quite as old as Unix itself, but it's definitely in the tried-and-true range.

Few dependencies -- You don't need to install a bunch of other software in order to get CFEngine up and running.

Good theoretical underpinning -- Promise Theory underlies everything in CFEngine. All the systems taking part in a configuration are cooperating (rather than coerced) to achieve a desired state, or at least move toward the desired state.

Pre-installation

Here's the (short) list of prerequisites I have before installing CFEngine on my network.

I'll be using an old desktop for my policy server, (Pentium 4 @ 2.8 GHz with 2 GiB RAM). I've set up my DHCP server to give this box a static IP address. There is (was?) a bug in CFEngine that would cause the policy server's IP address to be effectively hard-coded into the network configuration upon the first connection. (My Google-fu is not strong, not finding a reference to it right now.) So I want to give my policy server a well-known, "permanent" IP address.

I installed CentOS version 6 on this system, because it doesn't have a 64-bit processor. (CentOS 7 doesn't seem to play nice with an old 32-bit CPU.) Probably any Linux, BSD, or other Unix variant would work for this. I chose CentOS to gain some more experience with it, and add some heterogeniety to my network.

During installation, I created a user account and added it to sudoers.

After this, just make sure that wget is available:

$ which wget

You should see output like this:

/usr/bin/wget

Installation

Okay, ready to rock! Here's the 1-line command to install CFEngine:

$ wget -O- http://cfengine.package-repos.s3.amazonaws.com/quickinstall/quick-install-cfengine-community.sh | sudo bash

That should get everything installed. You'll probably see a message: Complete! Ready to bootstrap... DON'T DO IT YET!

(Here is a more detailed CFEngine installation tutorial, if needed).

Local git repo setup

I want to have my policy files under revision control, so that I can keep track of changes, undo things that break my network, and hopefully test changes before they cause breakage! I'm using git just to get more experience with it. I'll sort of follow the CFEngine VCS guide. But they use github as a repository server--I don't have a paid github account, and I don't want my policy files to be world-readable, so I'll set up a private git repository on my network.

Git isn't installed yet, and I don't have CFEngine running to do that for me, so I'll install it manually:

sudo yum install git

I set up git-daemon (git repository server) on a separate box, without SELinux enabled (currently). The git server offers read-only access -- I'll use ssh to control write access to the server/repo. The git server is running under my non-privileged account on that box, which means I can manipulate the repository and edit the policy files therein, without having to su/sudo constantly.

ssh [email protected] $ mkdir -p $HOME/git_depot/cfe-home-master.git $ cd $HOME/git_depot/cfe-home-master.git $ git init --bare

The trailing '.git' in the directory name is conventionally used to indicate a "bare" repository, i.e. it doesn't contain a working copy. Shared repositories are normally bare, regardless of how they're served/shared: nfs, git-daemon, file server, http, etc. Note that the filename doesn't turn this into a bare repo, it's merely a naming convention.

Now let's turn the freshly-installed out-of-the-box policy files into a git repo, then push that up to the new repository:

(on the new policy server) $ sudo bash # cd /var/cfengine/masterfiles/ # git init # echo cf_promises_release_id >>.gitignore # echo cf_promises_validated >>.gitignore # git remote add origin ssh://[email protected]/home/me/git_depot/cfe-home-master.git/ # git push -ff origin master

Okay, at this point there are two repositories: The one on the policy server (which has a working directory, but no origin), and the remote/bare repo (which I want to become the "authoritative" one). See Version Control with Git, 2009, pp. 187-8, "Make Your Own origin Remote", for background.

(back on the git server...) $ pwd /home/me/git_depot/cfe-home-master.git $ touch git-daemon-export-ok $ git daemon --reuseaddr --verbose --base-path=$HOME/git_depot -- $HOME/git_depot/cfe-home-master.git

Yes, I am running git-daemon interactively. No, this is not perfect -- see the Background section. But it'll do for now, and I can fix it (with CFEngine!) later. First, let's test the git-daemon. On the policy server, do an "ls-remote":

# git ls-remote git://my.git.server/cfe-home-master.git/ b0b723fa8c254f5a923c101a271a38383e4c5cf4 HEAD b0b723fa8c254f5a923c101a271a38383e4c5cf4 refs/heads/master

Works! You should also see some output on the interactive git-daemon server.

Now let's point the masterfiles/ directory on the policy server to the new git-daemon. We just need to update the policy server's remote:

# edit .git/config ### (remove the '[remote "origin"]' section, including 'url=...' and 'fetch=...' lines) # git remote add origin git://my.git.server/cfe-home-master.git/ ### test! # git remote update # git branch -a * master remotes/origin/master

Success! Moving on...

Bootstrapping the policy server

Remember the Complete! Ready to bootstrap... message from earlier? Time to get that done! We're still root on the policy server:

# /var/cfengine/bin/cf-agent --bootstrap my.policy.server

You'll see a bit of output, ending with:

Bootstrap to '(ip.address)' completed successfully!

Congratulations! Your policy server is up and running, and you're now well on your way to a more perfect network.

Review, next steps

This post is getting pretty long, but I've still got plenty to do. Take a quick glance at this post and you'll notice that the bulk of it has to do with setting up the git server. If a Version Control System (VCS) server was already available (as my dhcp server was), this would have been considerably shorter! Getting CFEngine installed and bootstrapped is really only 2 commands. So if you're willing to hold off on version control for your policy files, you can get this going pretty quickly and easily.

Next step will be adding another system to my configuration. Once you have a real network, (more than just one box), adding more systems is just a matter of "more of the same".

Relevant software & versions:

[email protected]

[email protected]

[email protected]

I wonder if this message came up earlier, and I just missed it in all the other output?

$ npm install -g yo generator-angular npm WARN peerDependencies The peer dependency generator-karma@>=0.9.0 included from generator-angular will no npm WARN peerDependencies longer be automatically installed to fulfill the peerDependency npm WARN peerDependencies in npm 3+. Your application will need to depend on it explicitly. npm WARN deprecated [email protected]: the module is now available as 'css-select' npm WARN deprecated [email protected]: the module is now available as 'css-what'

tl;dr

Do not use sudo npm install -g! Instead, use npm-g_without_sudo.

Still getting EPEERINVALID warnings? Install "missing" peers using npm install --save-dev peer1 peer2...

yeoman was frustrating the hell out of me! Or possibly it was npm (specifically, npm install -g) and homebrew not playing nice. But I think I've fixed everything, and I wanted to make sure I can re-do this if when necessary. (Also wouldn't hurt if I saved somebody the 2+ days I've spent banging my head against the keyboard trying to deal with this!)

So some background... I was trying to set up yeoman for an AngularJS project that I'm working on. Yeoman looks amazing, setting up the application framework and testing, and probably some other goodies that I haven't had time to discover yet. But I kept running into errors and warnings, wanting certain dependencies to be installed globally. I foolishly tried sudo npm install -g some-dependency and cost myself about a day of effort to fix my system. If only I had seen npm-g_without_sudo a little earlier, sigh! But now you are aware of it, and hopefully won't follow in my earlier footsteps, especially if you're on a Mac.

After I had npm and friends installed properly, I was able to make a lot of headway with yeoman. Setting up a new AngularJS app with mkdir newapp && cd newapp && yo angular worked almost flawlessly. I got a warning or two about overwriting package.json, but for these just choose 'overwrite' and you'll be fine. The warnings that were problematic were:

npm WARN EPEERINVALID ...

These started showing up later on, during unit testing. I thought perhaps my npm correction might have fixed this, so I tried updating the npm modules for the project:

$ npm install npm WARN EPEERINVALID [email protected] requires a peer of karma@^0.13.0 || >= 0.14.0-rc.0 but none was installed. npm WARN EPEERINVALID [email protected] requires a peer of jasmine-core@* but none was installed. npm WARN EPEERINVALID [email protected] requires a peer of karma@>=0.9 but none was installed. npm WARN EPEERINVALID [email protected] requires a peer of phantomjs@>=1.9 but none was installed.

Still no luck! I also tried installing the missing peers as globals, but that just took more time (and disk space) and didn't solve the problem.

I finally got everything working by adding the "missing" peers as dev dependencies, via npm install --save-dev jasmine-core karma phantomjs