Showing posts with label Programming. Show all posts
Showing posts with label Programming. Show all posts

May 11, 2024

R code to identify the FDR=0.05 equivalent P-value (for plotting purposes)

For Manhattan plots and other plots where you're plotting -log10(P-values) or untransformed P-values, you sometimes want to draw line to identify when FDR=0.05 is reached. But where do you draw the line? Below is R code that identifies the nearest P-value to FDR=0.05. You can then use variable FDR5.equiv.P to create a line on your Manhattan plot.

Code assumes you have a data frame with two columns, "Pval" and "FDR".

December 14, 2022

R code: drop duplicate values in a delimited column

Sometimes I have columns with duplicate values that I want to collapse into unique values. For example: comma-separated values, semicolon separated values, etc. See the "Gene Names" column below.



August 24, 2022

R programming lesson #1: load data, subset, and write to a new file

See also: "How to get started with R programming"

Purpose: 

Learn to prepare an Excel spreadsheet for data import, re-save as a .csv file, load into R, look at the data, subset to significant genes, and write the smaller spreadsheet to a new .csv file.

August 6, 2022

R code: Install code for my favorite R packages for bioinformatics

I am updating my R version and re-installing a bunch of things. Here is the code so I can copy/paste more easily in the future.

April 21, 2022

R code: Reading single spreadsheets and merging them into a multi-sheet Excel file

This R code example shows how to automatically compile multiple csv spreadsheets (or csv-formatted simple Excel spreadsheets) into one multi-sheet Excel spreadsheet. Personally, I use this to combine datasets from Ingenuity Pathways Analysis for supplemental files. I can do it manually but I always worry that I'll accidentally open the wrong analysis file or mis-name the tab. 

This way, I have a record of what files became what named sheets. I can also quickly re-compile a new Excel results file if I rerun the analysis by just changing the filenames in this code.

Packages

I prefer gdata for reading Excel sheets (better support for special characters) and openxlsx for writing the multi-sheet Excel file. 

Although gdata requires Perl installed on your computer, it does not require any knowledge of the Perl programming language. Use Strawberry Perl for Windows. Linux already has Perl.

The openxlsx package does not require any additional installations.
#install.packages("gdata")
#install.packages("openxlsx")

library(gdata)  ## for read.xls() function; requires perl
library(openxlsx)

October 19, 2021

R Markdown: new file template

This is the template I use to make my R Markdown files. This adds the date run, a table of contents, and the session information (R packages loaded and their versions, operating system, etc). The indentation in the title section must be actual tabs, not individual spaces, or you'll get an error. The back quote character ` is from the [~] key, not the ["] key. My filenames always include the date also! See this tutorial for other ways to edit formatting.

May 14, 2021

R code snippets from Tania (in progress)

Little things I re-use often.

Timestamp code


timenow = function() {
time1 = format(Sys.time(),"%Y-%m-%d") #makes a date stamp
time2 = format(Sys.time(),"%H%M%p") #makes a time stamp
x = paste0(time1,"-",time2)
return(x)
} #/end function timenow

timenow()

Example output: 

"2021-05-14-0825AM"


July 11, 2020

How to create publication quality images (high resolution images)

Journals typically request that images for publication be at 300 dpi resolution for photos (e.g. microscope images), 600 dpi for images, and 1200 dpi for line art. PowerPoint and Excel default settings don't meet these requirements. 

Use these tips to get high resolution images:

September 7, 2019

How to get started with SQL

SQL (Standard Query Language) is a programming language used for database filtering and editing. It is used in places like hospitals and libraries. Useful links below.

Tutorial with an online practice environment:
https://www.tutorialrepublic.com/sql-tutorial/sql-get-started.php
https://www.tutorialrepublic.com/codelab.php?topic=sql&file=select-all

Step-by-step tutorial with quizzes:
https://www.w3schools.com/sql/default.asp

If you know R programming, you can also pass SQL queries through R:
https://db.rstudio.com/getting-started/database-queries/

January 31, 2019

R programming lesson #0: How to get started

R is a programming language popular for statistics and bioinformatics.

Table of Contents

Installation

  • Install R, the free behind-the-scenes code that allows your computer to run R scripts
  • Install RStudio, the user interface software
    • Get free RStudio Desktop ("RStudio IDE" open source section). Install this after you install R (not before).
    • Ignore the RStudio Desktop Pro, RStudio Server, or commercial versions. You don't need those.
    • Default settings are ok
    • RStudio is semi-optional. If you are a programming beginner, you do need it. If you have a programming background and know how to run terminal commands, you don't really need it but it's still a nice user interface. 
  • Once you have both installed, open RStudio and it should automatically detect your version of R language (see red arrow). It will also show your default working directory (see orange arrow).

RStudio window showing R version 4.5.3 and working directory C drive
RStudio window showing R version 4.5.3 and working directory C:/

 

Simple practice

  • Open RStudio and try replicating the console commands below. Note that integer lists are written as beginning:ending separated by a colon (e.g. 1:10 means 1,2,3,4,5,6,7,8,9,10).
  • To assign values to variables, R allows either an equal sign or an arrow:
    • x <- 1:10
    • x = 1:10
    • These two things both assign an integer list of 1 to 10 to the variable x.
  • Once you have assigned the values to x, you can now create a new variable y which uses the values of x.
    • y <- x^2
    • This creates variable y with a list of values 1,4,9,16,25,36,49,64,81,100.
  • With two number variables of equal length like our example x and y above, you can create a scatter plot.
    • plot(x,y)
  • The plot function also allows some customization such as selecting the point colors.
    • plot(x,y,col="red")
    • Note that "red" must be in quotes because it is a value, not a variable. You are assigning the value "red" to the function's variable col. This would also work:
    • mycolor = "blue"
    • plot(x,y,col=mycolor)
    • In this case you don't need the quotes since you are passing the value of variable mycolor to variable col. That value is "blue", which R understands as one its named colors.
  • You can also print values to the console using either the variable name or the print() function.
    •  x
    • print(x)
    • print("Hello world!") 
  • If your variable values contain strings (meaning text values), then you can also combine text values with function cat() using commas to separate each string value.
    • mycolor="blue"
    • cat("The value of the variable mycolor is: ", mycolor)  




November 25, 2018

Unix/Linux cheatsheet and simple bash

Don't type the #comments (everything after the hashtag is commentary).

Common commands to check your account and environment

date #show date and time on the system (if it's wrong, that could cause problems)

history #shows the history of everything you've typed into the terminal

history > history.txt #saves your history to a text file, overwriting it if it already exists

history >> history.txt #saves your history to a text file, adding to the end (appending) if it already exists instead of overwriting it

ip a #prints your internet IP addresses (ethernet and wifi)

ls #shows you all the filenames in your directory

ls -la #shows you all the information about the files, not just the filenames

pwd #print current working directory; default is /home/username

which #shows you where a command lives (commands are programs so this helps troubleshoot if a command doesn't work)

which ls #example showing you where the ls command lives on your system

whoami #prints your current login username

[control+C] #this cancels or quits whatever is happening, good if you freeze

March 21, 2017

Python: Starting Python for the First Time (Windows & Linux)

Python is a programming language that is popular as a first programming language. This "getting started" tutorial is aimed at Windows users, with suggestions for Linux users.

Install Python and related tools

Install one of these distributions of Python, pick only one:
  • Just pick ONE of these. Installing multiple will create different versions of Python on your computer and that will cause conflicts later.
  • Anaconda
    • If drive space isn't an issue (you'll need 5-6 GB), get this. It is the most common distribution used by beginners and advanced users.
    • Comes with a common packages for life sciences pre-installed (e.g. pandas, numpy, matplotlib)
    • Makes creating different Python environments easy
    • Pick this if you are attending an Intro to Python workshop
  • Miniconda
    • Lighter version of Anaconda. It comes with less pre-installed programs. I like it.
    • Won't have "Anaconda Navigator" window, so you will need to control Python environments through the terminal (might be frustrating for beginners)
    • Pick this if you already know how to program, have other programming software on your computer, and don't want the full Anaconda installation to create duplicates of Jupyter Lab and other tools.
  • Python.org
    • If drive space is very limited, download this basic version
    • Any additional packages will need to be installed separately. Beware, this requires more steps and is less beginner friendly. 
  • During installation, select click the box to add whichever Python version to your PATH environment variable. You want this! It allows you to run Python scripts from the terminal later. 
Install a text editor for programmers (lightweight software), pick one or more: 
  • The default text application (e.g. Notepad.exe for Windows) will work because programs are just text files with special file extensions, but text editors specifically for programmers will make your life easier by color-coding the programming language syntax. This helps you notice when you forget a closing parenthesis or add an extra apostrophe.
  • Notepad++ (Windows)
  • Sublime Text (Windows, Linux, macOS) 
  • Geany (Linux) 
Install more complex IDE software (highly recommended for beginners), pick one or more: 
  • Jupyter Lab is popular and great for multi-script projects. 
    • From Windows terminal (cmd.exe):
      python -m pip install jupyterlab
    • From Python terminal:
      pip install jupyterlab
  • Spyder comes bundled with Anaconda and is my favorite IDE for Python. It is less complicated the Jupyter Lab.
  • RStudio Desktop now supports Python but I don't recommend it if you are learning Python for the first time. It is useful if you plan to use both R and Python languages together in the future. 
Install pip and use it to install Python packages

Open Jupyter Lab and start writing basic Python code

On Windows with Anaconda, open "Anaconda Navigator" 
  • Click "Launch" on the box for "Jypyter Lab" or "Jupyter Notebook".

On Windows with Miniconda, open software "Anaconda Powershell Prompt"
  • Type into the terminal and press [ENTER] button:
    jupyter lab

Jupyter Lab will launch
Whichever method used, the computer will launch Jupyter Lab on a web browser.

Create a new Jupyter notebook file with...
File: New: Notebook

Navigate to the new notebook tab (Untitled.ipynb) and type code into the gray text book. For example:
x = 5
y = 10
x + y 
 
To run the cell (the block of code), either press the play triangle icon or use keyboard shortcut [SHIFT] + [ENTER]

You should see the output of your code and a new cell (gray text box for code):
15

Type and press [SHIFT] + [ENTER]:
z = x*x

Your code will run and you will get a new cell. However, it doesn't look like it ran (no output). Your code assigned a value to variable z, but it did not instruct Python to print the output.

To print the output, try one of these:
z
print(z)

Either of these will print the output (the value) of variable z to your window.

Note that function print() cannot have a space between print and the open parenthesis. The function needs to be attached to the input inside the parentheses.

Try this and output:
print(x+y)
print("Hello world")
z = "Beautiful"
print("Hello",z)


Note that you just replaced the value of variable z so now it has a string value ("Beautiful") instead of a numerical value (x*x or 25). Variable values can be replaced. When writing code, keep track of what assigns values to your variables and when you are rewriting them.

You can also see the data types of your variables, for example:
print(x, type(x))
print(y, type(y))
print(z, type(z))
 
Remember to close every parenthesis that you open, or you will get an error.
'int' = integer
'str' = string (the programming name for "text" data)

There are different type of number datatypes. For example, change y to a number with decimals:
y = y+0.01
print(y, type(y))

'float' = number with decimals

Math expressions with comments after the hashtag (#):
2 + 1 # addition
2 - 1 # subtraction
2*3  # multiplication
18 / 2 # division
2**3 # exponent

Adding comments is very helpful to take notes on your code. Write #comments regularly.

Conditional statements, if else code blocks, and loops

Relational operators:

x = 100

y = 2 

x < y  # less than

x > y  # greater than

x  <= y # less than or equal to

x >= y # greater than or equal to

x == y # double equal sign evaluates for equivalency (is x equal to y?)

x != y # is x NOT EQUAL to y?


Create ifelse statements.

x = 15

y = 10

if x > y: #Note the use of the colon to indicate the end of the if statement to be evaluated

print(str(x)+ " is greater than " +str(y))  #Code to run if statement is true

else: #What to run if the conditional statement is not true

print(str(x)+ "is " +str(y)+ "or less")


Note that Python requires correct indentation of lines for ifelse statements. Not all programming languages enforce this, but Python is strict about it.

The colons after the if conditional statement and after else are important to tell Python what to do. Python will evaluate if the conditional statement is true or not, then decide which code block to run.

Create a while loop. It repeatedly runs a loop until the conditional statement is no longer true.

iteration=0

while iteration < 5:

print("Iteration:", iteration)

iteration += 1 #increases variable 'iteration' by 1 each time the while loop is run


Be careful with while loops. If you don't create a way for it to be interrupted, you can manually stop the code with the square stop icon.

Combine conditional statements and decide if either needs to be true, or if both need to be true.

x=2 

y=15

a=5

b=6

is_true = x>y or a<b

print(is_true)

is_true = x>y and a<b

print(is_true)


Copying and aliasing comparisons:

a=[2,3,4]
b=a #tell computer to use the same value from a for b
print(a is b)
b=[2,3,4]  #write object b with values [2,3,4], same values as a but it's a different object
print(a is b) 
a==b #compare values of the variables

True
False
True

Lists, dictionaries

Ranges:

for i in range(5):

print(i)

0

1

2

3

4

for i in range(25,30):

    print(i)

25
26
27
28
29

Pre-defined lists and using list indexes to pull out values:

my_list = [1, True, 3, 'Vegas', 5]

print(my_list[3])

Vegas

Because Python is zero-indexed (lists begin with zero), the list item 3 is actually the 4th item because Python counts 0,1,2,3,4.

Strings can also be lists of characters:

word = "Python"

for letter in word:

    print(letter)

p

y

t

h

o

n


Use loops to add values to lists:

Long way:

squares = []
for i in range(5):
    squares.append(i**2)
print(squares)

[0, 1, 4, 9, 16]


Concise way:

squares = [i**2 for i in range(5)]
print(squares)


Slice lists (get list subsets):

fruits = ["apple", "banana", "kiwi", "guava", "cherry"]

for fruit in fruits[1:-1]: 
    print(fruit)

banana
kiwi
guava

The code was told where to start (at index 1, which is the second value since Python starts with 0), and where to end (at index -1, which is the second-to-last spot because Python can go backwards).


Dictionaries store key-value pairs:

student = {
    "name": "Jose Soandso",
    "age": 20,
    "major": "Math"
}

## see the dictionary
print(student)

## pull a specific value
student["age"]

## print in loop
for key, value in student.items():
    print(f"{key}: {value}") #f-string formatting


Functions


How to write a simple Python program inside a text file, then run it on the Windows terminal

R programming lesson #2: merging pdf files

Use R package "pdftools" to merge separate pdf into one pdf file. You will never need to use sketchy websites or pay for software ...