Linuxus Noobius

Lately, I've had a lot of tasks which required managing disk space on a Linux system running Ubuntu 12.04. Here are some commands I have been using that have helped me achieve those tasks:

Disk Summary

df -h

Total size of current directory + subdirectories.

du -ch | grep total

List all subfolders and their sizes, sorted in descending order

du -h --max-depth=1 | sort -nr

Credit: All content here was taken from http://www.computerhope.com/unix/utar.htm#03 - I have just organized it into a small, easy to follow excerpt.

TAR

Without compression

tar -cvf home.tar home/

In the above example command the system would create a tar file named home.tar containing the home directory and place that file in the current directory.

With compression

tar -cjvf test.tbz home/

Adding the j option to the tar command enables tar to compress files and directories using bzip. In the above example the home directory and all its subdirectories are added to the compressed test.tbz file.

UNTAR

Tip: There is no "untar" Linux and Unix command. The tar command is used to create and extract tar files.

Not really a "pro" tip, but I'll use the title for consistency sake. Been wanting to make a shortcut to /var/www in my /home/myusername directory for some time now so that when I log in from my FTP account, I can directly jump there. Just figured out how to do it. Simple as can be:

ln -s /var/www

Going to start with this as it seems to be a fundamental concept central to figuring the rest of this stuff out. And also because it’s a fairly simple concept to grasp.

If you view the files in a directory by typing ls -l, you will see a sequence before the file name as shown below:

root@servername:/var/www# ls -l total 16 drwxr-xr-x  3 root root   4096 2012-01-24 00:55 . drwxr-xr-x 15 root root   4096 2012-01-23 04:36 .. -rw-r--r--  1 root root    200 2012-01-23 05:25 index.html drwxr-xr-x  4 root developers 4096 2012-01-24 00:55 websiteurl.com

Now that may seem like a lot of information, but it’s really not. Let’s break it down:

On the first line, you can see that I am logged in as root on my server and inside the folder /var/www. Now at this prompt, I typed ls -l to view the contents of this directory in detail. I was then shown the contents. The first two rows are just . and .. which we can ignore for now (just know that they go back to the root folder and parent folder respectively). 

So that leaves us with two things in this directory, index.html and websiteurl.com - One is a file and one is a directory. Let’s have a look at them broken up:

  So you can see that directory listing is displayed in 7 columns. 

Permissions - This is broken up into 4 parts. On the index.html file the permissions column has the following entry: “-rw-r--r--”. This is essentially a 10 character string. The first character position is ‘-’ for index.html and ‘d’ for websiteurl.com -- This is because the first character tells you if this is a file or a directory. When it is a file, it is left blank, and displays a ‘d’ in case of directory.

The next three characters will tell you what permissions the user (or owner) of the file has. They are depicted by rwx in that order. So, in this case we have ‘rw-’ on index.html which means that the owner has read and write access but not execute.

After the user’s permissions come the three characters which tell you what permissions the “group” assigned to that file or directory has. In the case of index.html we see that the group has ‘r--’ which means that they only have read access and cannot write or execute.

The last three characters tell you what rights others (or everyone) has. In the case of index.html again we see that it is only read access ‘r--’.

Directories - Tells you how many directories are within this particular folder.

Owner - Displays the name of the owner of the file.

Group - Displays the name of the group assigned to this file or directory.

Size - Displays the size of the file or folder.

Date - Date and time of the last update made to said file or folder.

Directory or file - Filename or directory name.

So once we know this information, the jargon that is displayed when you type ls -l doesn’t seem all that intimidating anymore. Now we can jump into ‘modifying’ these permissions.

There are three commands that we’ll discuss here for all things permissions related. They are chmod, chown and chgrp.

chmod - Modifies permissions

chown - Changes the owner of the file

chgrp - Changes the group of the file

Simple? I thought so. You can check out the different arguments you can pass with these commands by typing ‘man chmod’ to see the manual. I am going to use my favorite one in the examples below, which is -R. This basically means “recursively apply this command to all the files and sub-folders within this directory”.

So, for this example, I went to my /var/www folder and created a file called test.txt by typing “vi test.txt” and then inserted the text “This is a test file” and saved.

This is what the file looks like when I type ls -l:

-rw-r--r--  1 root root     16 2012-01-24 06:31 test.txt

Now I will change the permissions for this file. I want to give:

Owner - Read, write, execute

Group - Read, execute

Others - Read only

To do this, I will use:

chmod ug=+x (Give ‘User’ i.e. owner and ‘Group’ execute permission)

Now when we do ls -l, our test.txt file is as such:

-rwxr-xr--  1 root root     16 2012-01-24 06:31 test.txt

More more details on chmod and it’s literally hundreds of possibly uses, I highly recommend this tutorial. (http://catcode.com/teachmod/chmod_cmd.html)

chown is far simpler to use. Simply type “chown username file-or-directory-name”. This will change the User (owner) of that file.

chgrp is similar in use. Simply type “chgrp groupname file-or-directory-name”. This will change the Group of that file.

FTP access is something that is one of the most common requirements. I wanted to set it up so that I could:

Restrict FTP accounts to certain folders and not let them roam around the entire directory structure.

Make sure that files that were being uploaded through FTP were being assigned the appropriate permissions -- in my case -rwxr-xr-x (If you don’t know what this is, check out this post on groups and permissions).

Have the ability to have a master FTP account which is allowed to browse all folders.

1) Installation 

I decided to go with vsftpd for a couple of reasons. The biggest of course being that my friend Nestea recommended it. That pretty much holds enough weight to justify the selection right there, but it was also the most recommended one at trusted sources such as /r/linux and Serverfault. 

To install, simply execute: 

sudo apt-get install vsftpd

Before we move on to configuration, always make a backup copy of the original config:

sudo cp /etc/vsftpd.conf /etc/vsftpd.conf.original

Now edit the /etc/vsftpd.conf file to customize the following:

2) Disable anonymous login, enable local users to login and give them write permissions: 

# No anonymous login anonymous_enable=NO # Let local users login # If you connect from the internet with local users, you should enable TLS/SSL/FTPS local_enable=YES # Write permissions write_enable=YES

3) Restrict all users to their home directory, unless specified otherwise in the list: 

# Just some users are "free": chroot_local_user=YES chroot_list_enable=YES # Create the file /etc/vsftpd.chroot_list with a list of the "free" users.

4) Create a group for FTP Logins and user accounts with access to a particular directories:  

sudo groupadd FTPLogins sudo useradd -d /var/www/mydomain.com/public_html -g FTPLogins FTPJim

This creates an account with the username FTPJim and adds it to the group FTPLogins. FTPJim will be restricted to the root of mydomain.com and it’s subfolders.

5) Assign permissions the files uploaded via FTP will possess:

When a file is uploaded, by default it will contain the permissions "-rw-------" which of course is no good for us. So in order to rectify this, we once again edit our /etc/vsftpd.conf file and make the following modifications:

 file_open_mode=0644 local_umask=022

You can play with the values (use octal representations) but this will essentially get you to have the permissions -rw-r--r-- which is a start.

6) Apply configurations and go!

Whenever you don't know what arguments a command has or doesn't have, what the proper syntax is or even what that function actually does, just remember "Man is man's best friend".

A concept that I've finally grasped after several times going to my friend Nestea and asking him a question only to find that the problem could've easily been solved by typing "man commandname" and reading the manual for that command.

So for example, if you type "man ssh" it will tell you everything you need to know about ssh, public keys, private keys, authentication -- everything.

Every now and again I'll post a simple little bit of information that dramatically improved my quality of life when navigating through the Linux world. I will call these "Protips". These will almost always be provided to me by my friend "Nestea".

So this is something my friend Nestea used to tell me often -- learn vim. I never believed him when he would tell me that if you master this tool, you will find yourself flying through code editing.

Turns out he was right... again.

It can seem like a daunting task at first, (especially for IDE lovers like myself) but once you get the hang of the basics, you get used to it. Here's a good place to start:

http://www.lagmonster.org/docs/vi.html

In the interest of full disclosure, I do want to let you know that I have gotten a bit of a head start thanks to a friend of mine, whom we shall call "Nestea" because he plays Zerg in Starcraft 2 and is currently slogging it out in the Silver leagues (as am I). I will often post notes here from my conversations with him where his awesomeness will make insanely difficult problems seem trivial. So in-case you wonder how a Linux n00b such as myself figured out such a complex problem / scenario, well, now you know. It was Nestea.

I have a VPS box which I got from Linode.com -- Just a quick side note, we have used Linode exclusively for the last year and a half now for our own business use and recommended them to dozens of clients. We have nothing but awesome things to say about them each time. If you are in the market for VPS hosting, these guys will give you excellent service at an excellent price.

Back to my current setup, I have a VPS box running Ubuntu 10.04. I setup the server myself from scratch using the documentation found here:

Getting started - Specific to Linode, might not be much help otherwise.

LAMP guide - Install and setup Apache, MySQL & PHP

I have my domain registered at name.com where I changed the nameservers to point to Linode:

NS1.LINODE.COM & NS2.LINODE.COM

A brief introduction on why this space exists.

I am fairly familiar with computers, programming and such (having an undergraduate degree in Computer Science). However, over the years, the one thing that I never learned (properly) was Linux -- more specifically, the art of being able to admin a Linux based server.

I work in web applications development, on the business side, but always like to have enough technical knowledge to understand the inner workings. Thus my 2012 resolution -- to learn how to Linux (did I say that right?)

Here is an accurate description of my current state of Linux knowledge: