Skip to content

Lesson 3: File/directory permissions and more on navigating Unix file system

Quick review

In the previous lesson, we learned about limitations on the tasks that we can perform in the various spaces on Biowulf. We also learned to view pertinent user account information on the Biowulf user dashboard. Finally, we introduced ourselves to navigating directories and listing directory content. The following commands were introduced.

  • pwd (to check the present working directory - ie. the directory that we are currently in)
  • ls (to list directory content)
  • mkdir (to make a new directory)
  • cd (to change between directories)

Lesson objectives

After this lesson, we should

  • Be familiar with copying content from one directory to another
  • Know how to use the ls command with different options to view relevant information regarding files and directories
  • Understand file and directory permissions and be able to modify these
  • Know how to remove files

Unix commands that we will visit in this lesson

  • cp (to copy files or directory)
  • ls (to list directory content)
  • chmod (to change file and directory permission)
  • mkdir (to make new directory)
  • cd (to change directory)
  • rm (to remove content)

Logging into Biowulf

Before getting started, log into Biowulf using either the student account or your personal account. Open the Command Prompt (Windows) or Terminal (Mac) and use the ssh command to connect. Remember that the password does not appear as we are typing it.

ssh username@biowulf.nih.gov

Change into your data directory

Remember to work in your data directory. Take a moment to change into your data directory using the cd command with the arguments below

  • Argument: the path of the directory you want to change into (here it is /data/username, an absolute path because it is starting at the root)
cd /data/username

Use pwd to confirm that you have successfully changed into your data folder.

pwd

/data/username

Copying of files or directories

There is a folder called unix_on_biowulf_2023_documents in the /data/classes/BTEP folder. We can see it using the ls command with the options and arguments below

  • Option: -l for long and detailed view
  • Argument: directory that we want to list contents of (ie. /data/classes/BTEP)

Note: if a directory is not specified with ls, we will see the contents of the present working directory

ls -l /data/classes/BTEP

Copy a folder

The first task for today is to copy the unix_on_biowulf_2023_documents folder to your data directory. We use the cp command with the following options and arguments.

  • Option: -r to copy recursively (ie. the folder and all of its contents)
  • Argument: the path of the folder that we like make a copy of (ie. /data/classes/BTEP/unix_on_biowulf_2023_documents)
  • Argument: where to copy the folder to (ie. in the present working directory or ".")
cp -r /data/classes/BTEP/unix_on_biowulf_2023_documents .

Use ls -l to see if the unix_on_biowulf_2023_documents has been successfuly copied. If it has then change into this folder using cd and then ls -l to see the content of this directory.

ls -l


cd unix_on_biowulf_2023_documents

ls -l

total 130
drwxr-x---. 2 wuz8 wuz8  4096 Jan 12 17:40 SRR1553606_fastqc
drwxr-x---. 5 wuz8 wuz8  4096 Jan 12 17:40 unix_on_biowulf_2023
-rwxr-x---. 1 wuz8 wuz8 41734 Jan 12 17:40 unix_on_biowulf_2023.zip

Next, change into the unix_on_biowulf_2023 folder.

cd unix_on_biowulf_2023

Then list the contents.

ls -l

total 257
-rwxr-x---. 1 wuz8 wuz8  31666 Jan 12 17:47 counts.csv
-rwxr-x---. 1 wuz8 wuz8 104473 Jan 12 17:47 results.csv
-rwxr-x---. 1 wuz8 wuz8     84 Jan 12 17:47 text_1.txt

Copy a file

Earlier, we used cp with the -r option to recursively copy the unix_on_biowulf_2023_documents directory and all of its contents to the data directory. Suppose we want to make a copy of just one file (the counts.csv file) in the unix_on_biowulf_2023 subfolder of unix_on_biowulf_2023_documents, how would we do this? We could use the cp command with the following arguments.

  • Argument: File to make a copy of (ie. counts.csv)
  • Argument: Name of the copy (ie. counts_copy.csv)
cp counts.csv counts_copy.csv

Supposed that we want to make a copy of text_1.txt and call it text_1_copy.txt, we can use the cp command again.

cp text_1.txt text_1_copy.txt

Now, if we wanted to make a copy of counts.csv and place it one directory up in the unix_on_biowulf_2023_documents folder then we can use the command below, where "../" represents go back one directory.

cp counts.csv ../counts_copy_1.csv

Again, "../" denotes one directory back, so the following ls command will list the content of the unix_on_biowulf_2023_documents folder and indeed we see counts_copy_1.csv. 

ls ../

counts_copy_1.csv  SRR1553606_fastqc  unix_on_biowulf_2023  
unix_on_biowulf_2023.zip

File and directory permissions

When we use ls -l we are listing contents of a directory in the long view and are able to see additional details about a file or directory such as permissions, file size, and date and time of last modification (Figure 1).

Figure 1: Explanation of the results from ls -l.

The column "drwr-x---" in Figure 1 tells us the permission (ie. who can read - r, write - w, or execute - x contents of the file or directory), which is an important aspect of work in Unix systems like Biowulf. Figure 2 gives a breakdown of the information provided in the permission block.

Figure 2: The permissions are divided into three chunks of "rwx", corresponding to read, write, and execution privileges of the file/directory user or owner, others in the group, and everyone else. If the permission begins with d, then we are looking at a directory. If the permission begins with "-", then we are looking at file. Source: UF Research Computing Training

Modifying permissions

The command for modifying permissions is chmod. If we append --help to chmod, then we can see how to use it.

chmod --help

Usage: chmod [OPTION]... MODE[,MODE]... FILE...
  or:  chmod [OPTION]... OCTAL-MODE FILE...
  or:  chmod [OPTION]... --reference=RFILE FILE...
Change the mode of each FILE to MODE.
With --reference, change the mode of each FILE to that of RFILE.

  -c, --changes   like verbose but report only when a change is made
  -f, --silent, --quiet  suppress most error messages
  -v, --verbose          output a diagnostic for every file processed
      --no-preserve-root  do not treat '/' specially (the default)
      --preserve-root    fail to operate recursively on '/'
      --reference=RFILE  use RFILE's mode instead of MODE values
  -R, --recursive        change files and directories recursively
      --help     display this help and exit
      --version  output version information and exit

Each MODE is of the form 
'[ugoa]*([-+=]([rwxXst]*|[ugo]))+|[-+=][0-7]+'.

GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
For complete documentation, run: info coreutils 'chmod invocation'

To use chmod, we need to be aware that

  • u is user or owner
  • g is group
  • o is others
  • "-" is used to remove a permission
  • "+" is used to add a permission
  • "=" sets permission

We can also numerically set permissions where

  • 0: No permission
  • 1: Execute permission
  • 2: Write permission
  • 3: Execute and write permission (1+2=3)
  • 4: Read permission
  • 5: Read and execute permission (1+4=5)
  • 6: Read and write permission (2+4=6)
  • 7: All permission (1+2+4=7)

Now, change back up one directory to the unix_on_biowulf_2023_documents folder. In Unix ".." denotes one directory up. 

cd ..

If we removed the user's read privilege of the unix_on_biowulf_2023 folder and then list the contents of this folder, a "permission denied" message will appear. In the chmod command below, we use the following options and arguments

  • Option: u denotes user or owner
  • Option: - denotes to remove
  • Option: r denotes read permission (thus, u-r denotes remove read permission from user or owner)
  • Argument: file or folder that we like to modify the permission for (ie. the folder unix_on_biowulf_2023)
chmod u-r unix_on_biowulf_2023

ls -l unix_on_biowulf_2023

ls: cannot open directory unix_on_biowulf_2023: Permission denied

The error permission denied appears because the read or r permission for the owner has been removed from unix_on_biowulf_2023.

ls -l

total 258
-rwxr-x---. 1 wuz8 wuz8 31666 Jan 21 12:26 counts_copy_1.csv
drwxr-x---. 2 wuz8 wuz8  4096 Jan 21 12:25 SRR1553606_fastqc
d-wxr-x---. 2 wuz8 wuz8  4096 Jan 21 12:26 unix_on_biowulf_2023
-rwxr-x---. 1 wuz8 wuz8 41734 Jan 21 12:25 unix_on_biowulf_2023.zip

To add the user read permission back we will do the following (ie. replace "-" with "+").

chmod u+r unix_on_biowulf_2023

Listing the content of our data directory, we will see that the read permission has been added back for the user for the unix_on_biowulf_2023 folder.

ls -l

total 258
-rwxr-x---. 1 wuz8 wuz8 31666 Jan 21 12:26 counts_copy_1.csv
drwxr-x---. 2 wuz8 wuz8  4096 Jan 21 12:25 SRR1553606_fastqc
drwxr-x---. 2 wuz8 wuz8  4096 Jan 21 12:26 unix_on_biowulf_2023
-rwxr-x---. 1 wuz8 wuz8 41734 Jan 21 12:25 unix_on_biowulf_2023.zip

We are now able to list the contents of the unix_on_biowulf_2023 directory because we have just reassigned the user the read permission.

ls unix_on_biowulf_2023

counts_copy.csv  counts.csv  results.csv  text_1_copy.txt  text_1.txt

Let's use the numbering scheme to add read permission for others to the unix_on_biowulf_2023 folder. The options and arguments for the chmod command below is as follows

  • Option: the first number assigns user or owner permissions (here 7 is used to assign read, write, and execute permission)
  • Option: the second number assigns group permissions (here 5 is used to assign read and execute permission)
  • Option: the third number assigns other permissions (here 4 is used to assign read permission)
  • Argument: name of the file or folder that we like to modify permissions for (ie. unix_on_biowulf_2023)
chmod 754 unix_on_biowulf_2023

List the content of our unix_on_biowulf_2023_documents folder to see the changes.

ls -l

total 258
-rwxr-x---. 1 wuz8 wuz8 31666 Jan 25 10:56 counts_copy_1.csv
drwxr-x---. 2 wuz8 wuz8  4096 Jan 25 10:52 SRR1553606_fastqc
drwxr-xr--. 2 wuz8 wuz8  4096 Jan 25 10:56 unix_on_biowulf_2023
-rwxr-x---. 1 wuz8 wuz8 41734 Jan 25 10:52 unix_on_biowulf_2023.zip

Notice that while we added read permission to others for the unix_on_biowulf_2023 folder, the contents inside the folder do not have read permission for others.

ls -l unix_on_biowulf_2023

total 385
-rwxr-x---. 1 wuz8 wuz8  31666 Jan 25 10:56 counts_copy.csv
-rwxr-x---. 1 wuz8 wuz8  31666 Jan 25 10:52 counts.csv
-rwxr-x---. 1 wuz8 wuz8 104473 Jan 25 10:52 results.csv
-rwxr-x---. 1 wuz8 wuz8     84 Jan 25 10:56 text_1_copy.txt
-rwxr-x---. 1 wuz8 wuz8     84 Jan 25 10:52 text_1.txt

Note that chmod has a recursive mode, denoted by the option -R. For a given directory, -R will change permission for the directory as well its contents.

Let's use the -R option to add read permission to others for the unix_on_biowulf_2023 folder and its contents. The options and arguments for the chmod command below are

  • Option: -R to change permissions recursively
  • Option: o to change permission for others
  • Option: "+" to add permission
  • Option: r denotes read permission (thus, o+r denotes add read permission for others)
  • Argument: File or folder to modify permission for (ie. the unix_on_biowulf_2023 folder)
chmod -R o+r unix_on_biowulf_2023

Listing the content of the unix_on_biowulf_2023_documents folder, we see that the read permission for unix_on_biowulf_2023 has been added for the others.

ls -l

total 258
-rwxr-x---. 1 wuz8 wuz8 31666 Jan 30 22:52 counts_copy_1.csv
drwxr-x---. 2 wuz8 wuz8  4096 Jan 30 22:48 SRR1553606_fastqc
drwxr-xr--. 2 wuz8 wuz8  4096 Jan 30 22:52 unix_on_biowulf_2023
-rwxr-x---. 1 wuz8 wuz8 41734 Jan 30 22:48 unix_on_biowulf_2023.zip

Listing the contents of the unix_on_biowulf_2023 folder, we see that others has read permission for everything inside it.

ls -l unix_on_biowulf_2023

total 385
-rwxr-xr--. 1 wuz8 wuz8  31666 Jan 23 21:05 counts_copy.csv
-rwxr-xr--. 1 wuz8 wuz8  31666 Jan 23 21:05 counts.csv
-rwxr-xr--. 1 wuz8 wuz8 104473 Jan 23 21:05 results.csv
-rwxr-xr--. 1 wuz8 wuz8     84 Jan 23 21:06 text_1_copy.txt
-rwxr-xr--. 1 wuz8 wuz8     84 Jan 23 21:05 text_1.txt

Other commands that might come in handy are chown and chgrp.

  • chown (to change owner of a file)
  • chgroup (to change group ownership of a file)

More on the ls command and viewing directory content

Let's stay in the /data/username/unix_on_biowulf_2023_documents folder for this exercise (change into if not in this directory already).

Note that the size of our content are listed as bytes. We can get a more human readable form of the size by appending the -h option to ls -l. Again we can use ls --help to find out about the -h flag. Of interest is that options for Unix commands can also be written in the long form preceded by "--" (ie. -h is the same as --human-readable).

ls --help

-h, --human-readable  with -l, print sizes in human readable format
                        (e.g., 1K 234M 2G)

Appending the -h option, we see that the content sizes in unix_on_biowulf_2023 are now expressed as kilobytes (K).

ls -lh unix_on_biowulf_2023/

total 385K
-rwxrwx---. 1 wuz8 wuz8  31K Jan 21 12:26 counts_copy.csv
-rwxrwx---. 1 wuz8 wuz8  31K Jan 21 12:25 counts.csv
-rwxrwx---. 1 wuz8 wuz8 103K Jan 21 12:25 results.csv
-rwxrwx---. 1 wuz8 wuz8   84 Jan 21 12:26 text_1_copy.txt
-rwxrwx---. 1 wuz8 wuz8   84 Jan 21 12:25 text_1.txt

Speaking of file sizes, we can use the checkquota command to check disk usage. This should give the same result as that shown in the user dashboard.

checkquota

Mount       Used      Quota   Perrcent    Files    Limit  Percent
/data:  478.8 GB     2.0 TB     23.38%     1763 31457280    0.01%
/home:    2.0 GB    16.0 GB     12.44%    14788      n/a    0.00%

Deleting files

Let's change into the unix_on_biowulf_2023 folder.

cd unix_on_biowulf_2023

Then list the content

ls

counts_copy.csv counts.csv results.csv text_1_copy.txt text_1.txt

Let's remove counts_copy.csv and text_1_copy.txt using the rm command with the following argument.

  • Argument: file that we like to delete (ie. counts_copy.csv and text_1_copy.txt)
rm counts_copy.csv

rm text_1_copy.txt

Listing the contents of unix_on_biowulf_2023, we see that counts_copy.csv and text_1_copy.txt was removed.

ls

counts.csv  results.csv  text_1.txt

To remove a directory, we can use rm -r and this will remove the directory and everything in it recursively. If we have an empty directory, we can use rmdir. We will talk more about the rm command in the next class.