Unix/Linux cheatsheet and simple bash

Don't type the #comments (everything after the hashtag is commentary).

Common commands to check your account and environment

date #show date and time on the system (if it's wrong, that could cause problems)

history #shows the history of everything you've typed into the terminal

history > history.txt #saves your history to a text file, overwriting it if it already exists

history >> history.txt #saves your history to a text file, adding to the end (appending) if it already exists instead of overwriting it

ip a #prints your internet IP addresses (ethernet and wifi)

ls #shows you all the filenames in your directory

ls -la #shows you all the information about the files, not just the filenames

pwd #print current working directory; default is /home/username

which #shows you where a command lives (commands are programs so this helps troubleshoot if a command doesn't work)

which ls #example showing you where the ls command lives on your system

whoami #prints your current login username

[control+C] #this cancels or quits whatever is happening, good if you freeze

------------------------------------------------------------------------

Common commands to work with files and directories

mkdir workshop #create a directory (folder) called "workshop"

touch bar #create an empty file called "bar"

touch bar.sh #create an empty bash file called "bar.sh"

ln -s ~kessnerd/bacteria.fasta #creates a symbolic (-s) link from your own directory to the bacteria.fasta file in kessnerd's directory. Symbolic links are saves just like files. The system treats it like a file. 

cd workshop #change directory and go into directory "workshop", assuming that "workshop" is a folder in your current directory

cd ./workshop #same as above, change directory and go into directory "workshop", assuming that "workshop" is a folder in your current directory. The dot slash indicates your current directory.

cd ../ #backs out of the directory you're in, goes back one folder from your current directory. The double dot indicates the previous directory (the one before yours).

cd ~ #go back to your home directory (relative path)

cd /home/yourusername #go back to your home directory (absolute path)

cp hello.txt hello2.txt #copies the first file and gives it the name you indicate (hello2.txt)  

cp -r workshop tania #copy directory "workshop" to directory "tania"

mv testfile testfile2  #renames file (because it's not really being moved)

mv testfile /../../common/myname/testfile
#moves my testfile from /home/myname to common/myname
#on my institution's HPC, the home directory doesn't have much space
#but the common directory has plenty of space

------------------------------------------------------------------------

Read and search files

wc bacteria.fasta #word count of your file

head bacteria.fasta #shows the first 10 lines of that file

tail bacteria.fasta #shows the last 10 lines of that file

cat bacteria.fasta #shows you what is in your file (could be lots of text so try running wc first!)

less bacteria.fasta #like cat but allows you to go back and forth

grep '>NC' bacteria.fasta #searches the text '>NC' in the file bacteria.fasta and outputs the line that contains that text pattern. The single quotes indicate the string pattern you want (taken more literally than double quotes). If you use "double" quotes, it will be evaluated more liberally and could be evaluated as a command instead. Most times it doesn't matter if you use 'single' or "double" quotes.

grep -i '>NC' bacteria.fasta #case insensitive grep search

------------------------------------------------------------------------

Delete things (use with caution!)

rm bar.sh #permanently deletes the file "bar.sh" 

rm -r workshop #permanently deletes the directory "workshop"

------------------------------------------------------------------------

Starting a bash script, making it run

touch mycode.sh #first make an empty file "mycode.sh"

nano mycode.sh #opens the file "mycode.sh" so you can edit it from the terminal

echo ....Hello Tania!.... #use echo function to print "....Hello Tania!...." when the bash script is run

echo $PATH  #print the full path information (very long)

echo $HOME #prints home environment variable, e.g. /home/gonzalez

[control+O] #from nano saves the file

[control+X] #from nano exits the file

chmod +x mycode.sh #makes the bash script "mycode.sh" executable so it can be run

./mycode.sh #runs the bash script, assuming the script is in your current directory (./)

------------------------------------------------------------------------

Bash coding references

Simple tutorials: http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html

Note that bash is very picky about white spaces, so if something isn't working, check if you added or omitted spaces.

Example:

if [ "tania" = "tania"] #will produce an error
if [ "tania" = "tania" ] #will evaluate as TRUE

The difference is the space before the ending square bracket. Bash likes the square brackets to have spaces before and after.

Other interesting projects:

• How to send email from Linux command line

------------------------------------------------------------------------

Other references

• Vi cheatsheet
• Submitting jobs to the server
• qrun = submit a job
• qstat = check the status
• PBS tutorial from UCSF
• Different ways to run two commands (consecutively, in parallel, conditionally, etc)
• 
   

==================================================

End of the cheatsheet. The following is notes for specialized uses.

==================================================

Free software to remotely log into (ssh into) a Linux server

I am assuming you have login access to the Linux server.

• PuTTY - the old and established way to ssh into a Linux server. Only Windows compatible. Not my favorite.
• Windows terminal (cmd.exe) - Windows 10 onward has better support for ssh. You no longer need PuTTY or other software.
• MobaXterm (Home Edition is free) - my recommend method. The interface is very user friendly.

To remote log into a server, use the following syntax, general example:

ssh [username]@[serveraddress] -p [portnumber]

Made up specific examples:

ssh gonzalez@192.168.0.2 -p 555

ssh gonzalez@someserver.university.edu -p 9

The first example shows how to log user "gonzalez" into a server with a local IP address and port number 555. The second example shows how to log into a server with a website address and port number 9.

The "-p portnumber" is optional if the server uses the default port number (22), but the port is usually changed to a non-default port for server security. Use whichever port the server owner tells you to use.

------------------------------------------------------------------------

Load R on the Cedars HPC cluster (6/2021)

module load R/4.1.0 #load R 4.1.0 instead of using the older default R version

R --version #check the R version

R -e 'install.packages("microbenchmark")'   #install R packages through the terminal

------------------------------------------------------------------------

Log into a server and request X11 forwarding for plotting, plus verbose settings

ssh -X -v username@csclprd3-s009v.csmc.edu  ## for example

The -v results in more information. It can be useful for debugging purposes.

------------------------------------------------------------------------

Windows Subsystem for Linux (WSL)

Windows 10 has a Linux subsystem, meaning you can run Linux within Windows. It's not the same as dual boot. Suggested distribution: Ubuntu.

How to navigate to the C drive or a Dropbox folder using the subsystem:

cd /mnt/c/   #this is like "cd c:" but for the WSL

cd /mnt/c/Users/Gonzalez/Dropbox/ 

How to run commands in the WSL through RStudio:

system('bash -c "ls /"')  #shows the directory

system('bash -c "ls /mnt/c/Users/Gonzalez/Dropbox"')  #shows my Dropbox directory

system('bash -c plink1 --help')    #shows the help screen for plink, a Linux bioinformatics tool

More useful configurations:

https://pythonise.com/series/windows-subsystem-for-linux/configuring-windows-subsystem-for-linux

------------------------------------------------------------------------

Accessing connected hard drives

cd ~

cd ../../media/

ls #identify your user name

cd [yourusername]

ls #check for your external storage drive name and cd into it

------------------------------------------------------------------------

Create a symbolic link to Dropbox on WSL

cd ~ 

ln -s /mnt/c/Users/Gonzalez/Dropbox /db   #makes a symbolic link named "/db"

cd /db   #goes to the Dropbox folder using the symbolic link

ls -l   #show contents to make sure it worked

To remove the symbolic link:

cd ~

rm /db  #deletes the symbolic link "/db" & cannot be undone!

------------------------------------------------------------------------

Run WSL programs from the Windows terminal

Assuming you have WSL and a Linux distribution (such as Ubuntu) installed already, you can run Linux programs from the Windows terminal. See for example this program: https://mafft.cbrc.jp/alignment/software/ubuntu_on_windows.html

Look at section "Running MAFFT from outside of Ubuntu".

Open the Windows PowerShell (cmd) window and add "wsl.exe" before any commands you would type into the Ubuntu window, example:

 C:\Users\Gonzalez> wsl.exe plink --version

------------------------------------------------------------------------

Create a virtual environment for python

Install the software:

pip install virtualenv
python3 -m pip --upgrade virtualenv #optional if you already had virtualenv

 

Identify any existing virtual environments:

locate -b '\activate' | grep "/home" #finds all virtual environments in /home

Create a directory to store virtual environments:

mkfir environments #keep things neat

Create, enter, and leave a virtual environment called "RNAseq":

python3 -m venv ./environments/RNAseq #create
source ./environments/RNAseq #enter
pip list #shows all packages inside this environment
deactivate #quit the environment (go back to the regular Linux shell)

Last updated October 21, 2023