December 20, 2020

Setting up the Ubuntu 20.04 LTS server, desktop, and RStudio server at home

See updated notes for Ubuntu 22.04 LTS. Below are my archived notes for Ubuntu 20.04 LTS.

Why? I want to set up a computer on my home network dedicated to running code. You may want to do this if:

  • You want to run code that takes a long time to complete. Let it run on the server computer so you can use your main computer for other things.
  • Your main computer is low on hard drive space. Upload all the data files to the server and run your analysis on the server.

Who? Programmers comfortable with Linux who need to run code on a separate computer they don't mind reformatting. You don't need to be an expert Linux user. "Comfortable with Linux" can just mean comfortable searching the internet to find out what to type into a terminal. DON'T do this on your main computer. This will completely erase and rewrite the hard drive.

What? Ubuntu server 20.04.2 LTS with Ubuntu Studio desktop, RStudio server and R 4.2.0 (or the latest versions). [Updated instructions for Ubuntu server 22.04 LTS here.]

Cost? All the software is free. For hardware, you need the following:

  • "Main" computer
    • To download files, prepare the USB drives, check that the server works, and search the internet for help
    • I used my regular personal computer
  • Second computer to become the server - this will be erased/overwritten
    • Save anything you want to keep somewhere else, including data and software licenses. Everything will be erased during the setup process.
    • I used an old desktop computer. They sell special server computers (more expensive due to faster processors and better RAM), but I strongly suggest using an old desktop you already have for a first time project. 
  • Empty 4+ GB USB drive
    • This will be erased/overwritten to make bootable USB drives to erase & install things on the server computer
    • Afterward, you can reformat this back to a regular USB flash drive for file storage
  • Ethernet cable and wired access to the internet for the server (at least for the setup steps)
  • Keyboard, mouse, and monitor (at least for the setup steps)
    • Alternatively: I used a cheap no brand VGA switch so I could use the same monitor for both computers. I also used a tiny $9 keyboard+touchpad combo for the server computer, but you can use a Linux-compatible switch, or directly connect your regular mouse and keyboard to avoid buying more.

How?


Table of contents


Prepare an executable USB drive to install the Ubuntu server

  • Detailed instructions with screenshots [WikiHow]
  • Download an .iso file of the Ubuntu server, e.g. Ubuntu Server 20.04.2 LTS
    • Select "Option 2: Manual install"
    • Download the .iso file anywhere (main computer or USB flash drive ok)
    • LTS = long-term support, good so you don't need to upgrade as often
    • .iso files are packaged files that can be used to make installation CD-ROMs or executable USB drive
  • Get a USB drive with at least 4 GB that you're ok erasing completely. This will become the executable USB.
    • If you downloaded the .iso file to a USB flash drive, then you need a second USB flash drive that you are ok erasing. You can't use the same USB flash drive.
      • First USB=storage of the downloaded .iso file, does not need to be erased. 
      • Second USB=to be erased and rewritten with the .iso file contents.
  • Using Rufus (freeware), the .iso file, and your USB drive that you are ok erasing completely, make an executable USB drive to install the Ubuntu server.
    • Run Rufus on your main computer, with your USB drive plugged in
    • Install the .iso file onto the USB that you want to make executable. Use default settings, plus one addition from advanced settings: check the box for "Add fixes for old BIOSes (extra partition, align, etc.)"
    • The final executable USB can used during computer start up to run the .iso data, but is safe to plug into a computer that is already turned on (won't run the .iso data)

Install the server on a computer you are ok overwriting

  • Use an old desktop that you are ok erasing completely
    • Beware: you shouldn't have any files or software you want to keep on it!
    • You will lose the Windows or iOS operating systems, too!
  • Optional (recommended if donating the computer): erase the hard drive with DBAN software
    • Download the DBAN .iso file (free) and create an executable USB drive using Rufus
    • Optional step because installing the Ubuntu server overwrites the hard drive anyway
    • But using DBAN overwrites the hard drive several times and ensures your original data can't be recovered
  • Reboot the computer
    • Disable "fast restart" if you have Windows 8 or later
    • Starting with Windows 8, computers have a "fast restart" mode that skips your opportunity to enter the BIOS menu when you start it again.
  • Enter the BIOS menu by holding down the F2 or F12 key as soon as you press the power button. Keep holding it for several seconds until the BIOS menu appears.
    • Search the internet for your computer's instructions. It might be the F1 key or another key.
    • May require rebooting the computer several times to get it right
    • If you see the Windows screen, shut down and try again
    • Timing is key! Your opportunity to enter the BIOS is a really short window at the very beginning of start up
  • Edit the BIOS settings to boot from USB first, before booting from the hard drive
    • Turn off BIOS safety settings that prevent booting from USB, if any
    • Turn off Windows secure boot
    • Change the boot order to boot from USB before anything else (hard drive, CD, etc)
    • Usually this is the most difficult part because it is very computer-specific
    • Don't quit at this step! The rest is easier!
  • Shut down the computer completely after adjusting BIOS settings
  • Connect the computer to the internet before the Ubuntu setup process
  • Plug in the executable USB drive that you created with Rufus and the Ubuntu server .iso installation file
  • Turn on the computer
    • It should boot from the USB and start the Ubuntu server installation.
    • If not, try holding F2 or F12 when turning on the computer. You may need to adjust more BIOS settings until it works.
  • Follow installation instructions. Default settings are fine if this computer is meant to be a dedicated server. Below is what I did, with [x] meaning I checked that box.
    • Language: English
    • Network connection - connect to ethernet (you can add wifi later)
    • [x] Use entire disk, [x] set disk up at LVM group
    • Accept default storage configurations
    • Select a username and server name (make each one word, no spaces)
    • [x] Install OpenSSH - this is needed to connect from another computer
    • Featured server snaps: keep all defaults and add "wormhole" (useful to move files)
  • Remove the executable USB drive once the installation is complete. 
    • If you don't, the computer will restart and boot from the USB again. You will get a message to "remove installation media" (remove the USB). Do so and hit [Enter] to continue.
    • Keep the installation drive nearby until you install a desktop (next steps) and make sure everything works. If something goes wrong, you can re-install.
  • After all is done, you can reformat (erase) the executable USB with your main computer to use it like a normal USB storage drive again.
    • It is safe to plug in when the computer is already turned on. It was created to execute on start up and will not execute when you plug it in any other time.
    • Just in case, if your main computer is confused and asks if you want to import contents or run the ISO, click "Cancel" or "No". If your main computer asks if you want to reformat the USB drive, say "OK" or "Yes".

Set up a desktop (GUI) for the server

  • By default, the Ubuntu server just has the command line interface. Set up a desktop to make life easier. 

  • Install tasksel, then open it to see what it can install. Check boxes with the [spacebar] and move to "OK" using the [tab] key. Use Ctrl+Z to exit tasksel.
    • sudo apt-get update
    • sudo apt-get upgrade
    • sudo apt-get install tasksel
    • tasksel

  • Install a GUI (a desktop style) from the command line using tasksel. You only need one GUI.

    • Ubuntu Studio desktop is recommended if hard drive space isn't an issue. It comes with audio, video, and graphical editing software. Good if you want to use the server computer as an extra desktop.
      • sudo tasksel install ubuntustudio-desktop
      • During the installation of jackd2 (an audio server installed with Ubuntu Studio), you will get this message: "Enable realtime process priority?" I said no.

    • MATE desktop is fine also. It comes with useful software and takes up less space than Ubuntu Studio.
      • sudo tasksel install ubuntu-mate

    • MATE core desktop is a barebones version of MATE. Useful if you want to save hard drive space and only use the computer as a server. It works for that, but I hated it whenever I had to log into the server computer. For notekeeping purposes only, this is the command for it:
      • sudo tasksel install ubuntu-mate-core

    • Lubuntu desktop is the lightest desktop (smallest file size). When I installed it, I lost access to the terminal window via the GUI or keyboard shortcuts. I had to reinstall the server using the executable USB drive. Not recommended!

  • Select a display manager that works with the GUI you want
    • gdm3 - default for Ubuntu 20.04 LTS with MATE
    • lightdm - lightweight, default for Ubuntu 20.04.2 LTS with Ubuntu Studio

  • Enter the GUI and celebrate when the all text terminal turns into a colorful desktop with ONE of the following commands to start the desktop software (don't run both, pick one):
    • sudo service lightdm start
    • sudo service gdm3 start 

  • Log in with your username and password, then check the desktop environment.
    • Make sure you can open a terminal window
    • Make sure you can access the internet. 
    • Connect to wifi at this point if desired.

  • Prevent the server from going to sleep
    • Settings > Power > Automatic Suspend > Off
    • This keeps the server accessible 

  • Make a file of system specifications
    • sudo lshw -html > server_specs.html
    • This creates an .html file with information on your hard drive, memory, graphics and sound cards, wifi card, etc.

  • Exit the GUI (not recommended unless you're replacing it). This removes the desktop and takes you back to a terminal only interface. You don't need to do this to access the terminal. This is just for reference:
    • sudo service <DisplayManager> stop
    • where <DisplayManager> is either lightdm or gdm3

Set up a static IP address for the server computer

What is a static IP and why is it useful? It is an IP address that does not change. Having a static IP address is useful for a server computer (or printer) so that other computers have a known address to interact with it. Setting up a static IP is accomplished on a private network by telling the internet router to always assign the same local IP address to a specific MAC address (the physical hardware identification). Reserving an IP address for the server computer makes certain things easier, such as accessing the server via ssh or RStudio server web interface.

There are two IP addresses for internet-connected equipment:
  • Local IP address - assigned by your internet router to everything that connects to it, usually starts with 192.168.0.x, or 192.168.1.x, where the number x varies. This local address changes as things get disconnected and re-connected, but you can reserve specific local IP addresses for specific equipment MAC addresses for free. 

  • Public IP address - assigned by your internet service provider, find yours at whatismyIP.com. This is what websites see when you visit them. Unless you pay for premium internet service, your public IP is not static.

For the purposes of a private home Ubuntu server, you only need a static local IP address. It doesn't matter if your public IP address changes. 

  • Identify the MAC addresses for the Ubuntu server computer with this terminal command:
    • ip a

  • MAC addresses look like letter and number pairs in this format, A0:A0:A0:A0:A0:A0 
    • Ethernet and wifi have different MAC addresses
    • Write down both and then see which one is connected to the internet router

  • Log into your internet router's admin website to set up a static IP address ("reserve an IP address") for the Ubuntu server
    • This requires administrative access to the internet router. Ask whoever set up the internet for help, or check the back of your internet router for default login information.
    • Here are default router addresses, usernames, and passwords
    • Instructions to reserve IP addresses vary by manufacturer
    • Search "how to reserve IP address printer __[your router model]__" and follow those instructions for the Ubuntu server instead. Printers are the most common equipment that needs a static IP, so printers will have the most help pages on this topic.
    • Check under "LAN Setup" and find a section called "Address Reservation"

Log into the server via ssh from the main computer

  • On the server computer, turn on ssh access and check ssh server status (should be active)
    • sudo systemctl start ssh
    • sudo systemctl status ssh

  • On the server computer, ssh into the "localhost" (not an IP address, do not replace) and then exit
    • ssh localhost
    • Say "yes" when asked if you want to add the localhost to a list of known hosts
    • exit

  • On the main computer, open a terminal.
    • On Windows, you can open a terminal like this:
      • WindowsKey + R, then type "cmd"
      • Or search "cmd" and click Command Prompt
    • Alternatively, use MobaXterm software (free version) or PuTTY (freeware) to interact with the server. 

  • On the main computer, open a terminal and type something like: 
    • ssh user@192.168.0.10 
      • Replace "user" with your server username
      • Replace "192.168.0.10" with your specific Ubuntu server's local static IP address
      • This example assumes the default ssh port (port 22). 
    • ssh user@192.168.0.10 -p 23
      • This specifies port 23. See the later section on increasing server security, where I provide instructions on how to change the ssh port number. For now, continue with default settings and check that everything works. 
    • If using MobaXterm or PuTTY, fill in the boxes for username, IP, and port number.

  • First time message will look something like this: "The authenticity of host '[192.168.0.10]:22 ([192.168.0.10]:22)' can't be established. ECDSA key fingerprint is...."
    • Say "yes", you want to continue connecting. 

  • The terminal should ask for server login information

  • SECURITY NOTE: the server does not yet have a firewall installed. I suggest continuing with defaults and checking that everything works before enabling the firewall (instructions for that are at the end). 
    • If you are taking a break, you can turn off the ssh server with this:
      • sudo systemctl stop ssh
    • Then turn it back on when you come back:
      • sudo systemctl start ssh
    • Or just turn off the server computer until you are ready to add additional security

Troubleshoot ssh issues

  • https://likegeeks.com/ssh-connection-refused/
  • https://www.webservertalk.com/troubleshoot-ssh-connection-refused
  • https://ubuntu.com/server/docs/service-openssh
  • Check the server IP address to verify that you are attempting to ssh to the correct IP
    • ip route
  • Check that the server computer has internet access at all
    • ping 8.8.8.8 -c2 
    • This sends two packets to check for a connection. Note: there is no space between c and 2
    • Check for 0% packet loss which means the internet works
    • If there is packet loss, then troubleshoot your internet connection
  • Check that the server computer DNS is rerouting internet traffic correctly
    • ping google.com -c2
    • Check for 0% packet loss which means the DNS is working correctly AND you have internet
    • If the ping to 8.8.8.8 works, but the ping to google.com fails, then you have a DNS resolution error. This happened to me after rebooting my original Ubuntu server 20.04 LTS computer and I ended up reinstalling the whole server because nothing seemed to fix it. If it happens to you, try rebooting the internet router! I did not try that.
  • If you have a firewall enabled on the server computer (instructions for that later), make sure you opened the ssh port.
    • sudo ufw allow ssh #enables default port 22
    • sudo ufw allow 25/tcp #where 25 should be replaced with whatever port you are using for ssh if not the default port 22
  • Disable the server firewall if enabled - does that allow you to ssh into the server computer? If so, you will need to edit the firewall rules.
    • sudo ufw disable
  • Check that your main computer's firewall allows internet access to whatever program you are using to ssh into the server, examples:
    • Windows command prompt (cmd.exe)
    • Windows Subsystem Linux (wsl.exe)
    • PuTTY - a free ssh client that you may want to try if the Windows command prompt isn't working and you don't have Windows Subsystem Linux
  • Create a second server account with less privileges and attempt to ssh from that account
    • otheruser@serverIP #assumes default port 22
    • otheruser@serverIP -p 25 #tries port 25 (replace "25" with your ssh port number) 
  • Remove OpenSSH and reinstall. Don't type the #comments
    • sudo apt-get purge openssh-server
    • sudo apt-get autoremove #cleans up dependencies not used by anything else
    • sudo apt-get autoclean #cleans up more leftover files
    • sudo apt-get update 
    • sudo apt-get upgrade
    • sudp apt install openssh-server #reinstall
    • sudo systemctl start ssh #start the ssh server
    • sudo systemctl status ssh #check status
  • If using your main account, try permitting root login temporarily. Not recommended as a long-term solution since it makes the server less secure. 
    • sudo nano /etc/ssh/sshd_config
    • Type this line without any hashtag (#) in front:
      • PermitRootLogin yes
    • [Ctrl+O], [Ctlr+X] to save and exit
    • sudo systemctl restart ssh
    • Set up a firewall rule limiting IP addresses to the ssh port if you want to keep this setting.
  • Reboot the internet router if nothing else works. When I had ssh issues, rebooting the internet router fixed these error messages:
    • "Resource temporarily unavailable" from Windows Subsystem Linux
    • "Connection timed out" from Windows 10 command prompt

Install R 4.2.0 or later (6/2022 update)

My instructions below are for Ubuntu Server 20.04.2 LTS, modified from the official R instructions [CRAN] and some from here to remove the old R version [thinkr.fr]. Don't type the #comments. 
Don't skip installing the development packages. If you only install r-base, you will have problems with missing dependencies when you try installing R packages in the future.

sudo apt-get purge r-base* r-recommended r-cran-* #remove old pre-installed R version

sudo apt autoremove #remove dependencies not used by anything else

sudo apt update 

sudo add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/" 

 This step tell Ubuntu where to download the newest R version.
Note, "focal" is for Ubuntu 20.x versions, but you may need a different version. 

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9

If you get a "malformed line" error, you may have a typo or forgot the final slash at the end of focal-cran40/
To fix, open the "sources.list" file and scroll to the end:
sudo nano /etc/apt/sources.list
Remove or edit the malformed line, save with Ctrl+O, then exit with Ctrl+X.


sudo apt install r-base r-base-core #install new R version

sudo apt install r-recommended r-base-dev #install related things 

sudo apt install libxml2-dev libcurl4-gnutls-dev #development packages

sudo apt install libssl-dev #more development package

sudo apt-get install libmagick++-dev  ## optional, required for R package magick for graphics 

sudo apt install libcairo2-dev ## optional, required for R package svglite to save .svg images

R --version #check for R 4.2.0 (or whichever version is the most current at this time). If you have an older version, start over from the purge line. 

R #enter the R shell

 Type any R code you want in the R shell.

          q()  #exit R shell. Can also use Ctrl+Z to exit.


Keep watch for "ERROR: configuration failed" and "non-zero exit status" errors when installing R packages. Search the internet for those errors, the R package name, and "Ubuntu" to identify which additional development packages you need to install from the Linux terminal.

After this, I found that Ubuntu automatically updated my version of R when I ran regular maintenance updates with:

sudo apt-get update
sudo apt-get upgrade
sudo reboot #this restarts your computer so save any files you need!

Install and login to the RStudio server

Installation instructions for Ubuntu 20.04 LTS on 3/9/2021 (see link below for latest filename):

https://rstudio.com/products/rstudio/download-server/debian-ubuntu/

sudo apt-get install gdebi-core

wget https://download2.rstudio.org/server/bionic/amd64/rstudio-server-1.4.1106-amd64.deb

sudo gdebi rstudio-server-1.4.1106-amd64.deb

sudo ufw allow 8787/tcp #optional, skip if you haven't enabled firewall yet

Updated installation instructions for Ubuntu 20.04.2 LTS on 9/28/2021

Check the RStudio server version:

rstudio-server version

First time setup for each user: ssh into the Ubuntu server from your main computer and add the server IP to your list of known hosts by answering yes to the question, "Are you sure you want to continue connecting (yes/no)?" You MUST be able to successfully ssh into the Ubuntu server before you can use the RStudio server.

From the main computer (not the server computer), either through the Windows 10 command prompt or the Windows Subsystem Linux terminal or PuTTY freeware:
ssh user@serverIP #assuming the using default port 22
ssh user@serverIP -p 23 #example if ssh uses port 23

Log into the RStudio server from your main computer using a web browser, the server IP, and port 8787 [more info]:

http://<your server's IP>:8787
The login is your Ubuntu username and password.

Troubleshooting: if you can ssh into the Ubuntu server, but get a browser error saying the RStudio server website "took too long to respond", try this:

sudo systemctl status rstudio-server #check if the server thinks RStudio is working
    Ctrl+Z keyboard shortcut (not terminal code!) to close out systemctl 

sudo systemctl start rstudio-server #only needed if status said it was inactive

sudo ufw allow 8787 #open up the port again if you have a firewall

Try to log in again with: http://<your server's IP>:8787

Make sure you are not using "https://"

Install Perl, Python and JupyterLab

  • No need to install Perl and Python! They are already installed with Ubuntu Server 20.04.2 LTS
  • Check the versions with 
    • python3 --version  #two hyphens
    • python2 --version 
    • perl --version
  • Install pip if needed for python
  • Upgrade pip
    • sudo apt update
    • python3 pip -m install --upgrade pip
  • To exit either terminal if you're stuck, use Ctrl+Z or quit()
    • ">>>" means you are in the Python shell
  • Install JupyterLab (a Python GUI):
    • pip3 install jupyterlab
  • Detailed video instructions for installing Jupyter for a server [YouTube 2019]
  • Install Spyder (a Python GUI):
    • sudo apt install spyder

Add security to the server (firewall rules with ufw and iptables)

Reading:
Change the ssh port from the default 22 to something else (pick a number, e.g. 23). 
sudo nano /etc/ssh/sshd_config
    • Change the line that says "#Port 22". Remove the hashtag and change the number.
    • Don't use these ports for ssh. They are used for something else already:
      • 22 (default ssh port, often targeted by bots and hackers)
      • 53 (DNS server port for rerouting internet traffic)
      • 80 (internet http port)
      • 443 (internet https port)
      • 8787 (RStudio server port)
      • 8888 (Jupyterlab server port)
    • To save changes: Ctrl+O
    • To exit the text editor: Ctrl+X

Reload the sshd service to apply changes:

sudo systemctl reload sshd


Enable the firewall and allow traffic through specific ports:

sudo ufw enable #turns on the firewall
sudo ufw allow www #allow internet access
sudo ufw allow 53/tcp #allow DNS queries to route internet traffic
sudo ufw allow 80/tcp #allows http access
sudo ufw allow 443/tcp #allows https access
sudo ufw allow 8787/tcp #RStudio server port

Open up the port you are using for ssh access, but only allow access from your local home network IP addresses. Example with port 25:

sudo ufw allow from 192.168.0.0/24 to any port 25
sudo ufw allow from 192.168.1.0/24 to any port 25

Replace "25" with whichever port you are using for ssh access. The default port is 22.

Replace the IP address with your home network range if needed, though 192.168.0.x and 192.168.1.x are common patterns. This is not the same as your public IP address visible to people on the internet. The "/24" at the end is the same as saying the last number can vary.

DON'T do the "ufw allow from" version of the terminal command for the internet ports or you may lose internet access.

Reload the firewall to apply the rules:

sudo ufw reload #apply the rules 

View which ports are open:

sudo ufw status verbose #more details
sudo ufw status numbered #supplies a numbered list

Delete any rules you don't want using the line <number> (without the greater/less than signs):

sudo ufw delete <number> 

I suggest deleting any lines for port 22, which may have been opened automatically if at any point you ran "sudo ufw enable ssh".


To delete all ufw rules and start over:

sudo ufw reset
sudo ufw enable #turns on the firewall again

 

Install Dropbox

Manually download the latest .deb file (either 32-bit or 64-bit depending on your computer) from here: https://www.dropbox.com/install-linux

Don't follow the instructions to install from the terminal. It will partially work, but the installation code will freeze and you won't get a complete installation (my experience with Ubuntu 20.04.2 on 3/29/2021). 

Instead, right-click the .deb file from the Downloads folder and select "Open with Software Install". You won't see this option if you right-click from the browser's download list. [More details on how to use .deb files.]

Sign in and select your preference settings.  

Check the status with:

dropbox status

After rebooting, Dropbox sometimes needs to be restarted manually. You can do this remotely from a terminal using:

dropbox start

To set up Dropbox to start automatically upon login, use this command:

dropbox autostart y

See more Dropbox terminal commands from the official source or with this command:

dropbox


Make your whole hard drive space available (8/2021)

Optional section when getting started! I did this only several months later when Dropbox surprisingly told me I was out of space, when I knew the drive shouldn't be full yet. 

Technical details: Your workspace inside the Ubuntu 20 server is a "logical volume" inside ext4-formatted partition sda3. Ubuntu automatically limits the aliquoted disk space to less than your total hard drive space.
 
Layman explanation: Ubuntu automatically limits the space available to your user account so that there is space available in case you add other users later. If you're the only user, though, you don't need this limitation. You can tell Ubuntu to give you all the space.

Check your storage allocations (doesn't change anything, just informational):
 
df -h #see file sizes in human readable format

cfdisk #shows all partitions on the hard drive and space used

Note: "sda" prefixes refer to your internal storage such as your hard drive, "sdb" prefixes refer to external storage such as USB drives.

du -sh ~/Downloads #see how much space your Downloads folder uses

du -sh ~/Dropbox #see how much space your Dropbox folder uses

df -h ~/Dropbox #see how much space is available, used, and free in this folder. Importantly, this also tells you where this folder is located on the hard drive.
 
sudo lsblk #shows virtual volume breakdown and which partition contains it

In my case, my Drobox folder was located in path "dev/mapper/ubuntu--vg-ubuntu--lv" (notice 2 hyphens, 1 hyphen, then 2 hyphens) in partition sda3. Write down the exact name of the virtual volume path because you will need it later! My system told me that 200 GB were available, even though my hard drive is 1 TB. 
 
Next, here are instructions that you should only follow after backing up your data (beware possible data loss if you have typos or mistakes):

I resized the logical volume (lv) to give it more space by following the instructions here, which I can confirm worked for Ubuntu server 20.04.2 on August 2021: https://slice2.com/2020/12/05/howto-easily-resize-the-default-lvm-volume-on-ubuntu-18-04/

Briefly, I first ran this test (-t) to see if I could resize the logical volume without errors:

sudo lvresize -t -v -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv
 
I checked the output and it told me no errors were found.

This is the start of real changes to your system (so make sure your data is backed up): I repeated the command  above with "-t" removed to run it for real, not as a test. 

After running, I checked the file system type (expect ext4) and size:
 
df -h -T | grep vg
 
Since ext4 was correct, I proceeded with the final resizing step:
 
sudo resize2fs -p /dev/mapper/ubuntu--vg-ubuntu--lv
 
Check that it worked by checking the file system size again: 

df -h -T |grep vg

Restart the server and you are done with this risky section.

Install a Brother network printer using an IP address (6/2022 update)

Network printer = a printer connected to the internet. The printer is not connected to the computer via a USB cable or Ethernet cable.

  • Before starting, assign a static IP to your printer. This requires logging into the internet router admin page.
     
  • Install these dependencies on your computer before installing the printer drivers:
    • sudo apt install lib32zl #optional maybe
    • sudo apt install  lib32ncurses6 #replaces lib32ncurses5

  • Look up your specific printer on the Brother website, e.g. MFC-5460CN

  • Install the printer drivers [instructionsinstructions from Brother]
    • Download the "Driver Install Tool" file, e.g. "linux-brprinter-installer-2.2.3-1.gz", to the Downloads folder
    • These instructions helped me set up a network printer with 32bit drivers on a 64bit computer
    • In a terminal to install the printer drivers:
      cd ~/Downloads && gunzip -v ~/Downloads/linux-brprinter*
      sudo bash ~/Downloads/linux-brprinter*
       
    • Move some files to install the scanner files:
      sudo ln -sf /usr/lib64/libbrscandec*.so* /usr/lib
      sudo mkdir -p /usr/lib/sane
      sudo ln -sf /usr/lib64/sane/libsane-brother*.so* /usr/lib/sane
    • Add your username to the scanner user group, e.g. gonzalez:
      sudo usermod -a -G scanner gonzalez
    • If you get an error after a question about a proxy server, use the alternative method below.

  • Install the printer drivers -- alternative method if the above doesn't work. Installing the two .deb printer drivers separately.
    • From the Brother printer page, download the LPR printer driver and CUPSwrapper printer drivers separately (both .deb files). Install the LPR driver first, then the CUPSwrapper.
    • sudo mkdir /var/spool/lpd/
    • sudo mkdir /var/spool/lpd/mfc5460cn/
      • Replace the last step with whichever directory is missing. You'll see it as an error if you try to run the step below.
    • sudo dpkg -i --force-all mfc5460cnlpr-1.0.1-1.i386.deb
      • Replace the .deb filename with your specific LPR driver filename.
    • sudo dpkg -i --force-all mfc5460cncupswrapper-1.0.1-1.i386.deb
      • Replace the .deb filename with your specific CUPSwrapper driver filename.

  • You should see the printer installed (under "System > Printers"). Restart the computer if not.

  • The default printer settings assume you are connected by a USB cable. For a network printer, you need to change the device URI value. The default device URI value = usb:/dev/usb/lp0
  • If you have a static IP for your printer, change the device URI to specify the static IP address.
    • Go to "Printers" and right-click your device
    • Edit printer properties and find the device URI
    • Instructions: https://www.cups.org/doc/network.html
    • Example static IP = 192.168.0.3
    • The corresponding URI value = socket://192.168.0.3:9100

  • Print a test page.


Maintenance

Things to do occasionally to make things keep running well. Several of these things can be done through another computer logged in through ssh. After restarting, you may need physical access to the server or the internet router.
  • Identify your user account
    • whoami #returns your account name
  • Identify logged in users
    • w
  • Download & install updates (on the server computer or remotely)
    • sudo apt update
    • sudo apt upgrade
  • Restart if the ssh log in message says: "*** System restart required ***"
    • sudo reboot
    • You can do this remotely by ssh
    • You may need physical access to the server afterward if there are any issues with internet, firewall rules, or ssh. The most likely issue is that your need to check the IP address again if you didn't assign the server a static IP on the internet router yet.
  • Check your server IP address
    • ip a
    • enp2s0 = ethernet
    • wlp3s0 = wifi
  • Check firewall rules
    • sudo ufw status
  • Restart Dropbox and check Dropbox status. I need to restart it often on Ubuntu server 20.04.03. It disconnects after a few hours of inactivity.
    • dropbox status
    • dropbox start
  • RStudio Server check status, start if inactive, restart if it is active but not working well
    • systemctl status rstudio-server
    • systemctl start rstudio-server
    • systemctl restart rstudio-server
  • RStudio Server - "took too long to respond" browser error (on the server computer or remotely)
    • sudo ufw allow 8787 #reopen the 8787 port for the RStudio Server
    • If that doesn't work, check your server computer's internet access.
    • Check the RStudio Server link from the server computer itself (through a browser)
      • http://[your server IP address]:8787
      • http://192.168.0.100:8787 (example pattern)
    • Check that you can ssh into the server
  • Check status of the ssh service (on the server computer's terminal)
    • systemctl status ssh.service #check if it's active
    • systemctl restart ssh.service #restart if you're having issues
    • sudo service ssh restart #try restarting another way
    • sudo ufw status numbered #check which ports are open
      • sudo ufw allow <your ssh port> #reopen the ssh port if needed
  • Check that the server computer has internet access
    • sudo lshw -class network #shows network details
    • ping google.com -c2
      • Wait for results from the 2 packets. If the ping statistics say 0% packet loss, then you have internet working properly.
    • ping 8.8.8.8 -c2
      • Wait for results from the 2 packets. If the ping statistics say 0% packet loss, then you are connected to internet even if the DNS isn't working
      • If 100% packet loss, then you're not connected to internet (not a DNS problem)
    • If either ping fails and says "Temporary failure in name resolution" then the DNS is not routing internet traffic correctly.
    • Turn off the internet router for 30 seconds, then back on. This can fix several problems including router issues that affect every device, as well as IP assignment problems that don't affect server internet but prevent remote access through ssh or the RStudio browser url.
    • Reboot the computer.
    • Connect to ethernet and, if you get internet back, then download & install updates.
    • If you have tried everything and nothing works, wait 1 day to see if the problem fixes itself. If not, you may need to reinstall the server.
  • Check Linux kernel currently loaded. The kernel is the skeleton beneath your Ubuntu system. It is important for basic things like loading Ubuntu on startup and hardware compatibility.
    • uname -a
    • If you buy new hardware (e.g. wifi adapter), check that it is compatible with your Linux kernel.
  • Check Linux kernels available to you. [More info
    • dpkg --list | grep linux-image
    • That's two hypens before list
    • It's good to have a few old versions in case an update causes startup problems. You can power down the computer, then power it on and hold the F2 key to bring up the GRUB boot loader. If you can boot using an older kernel, you might be able to resolve problems with the new kernel without reinstalling your whole system.
  • Search available software by keywords, for example:
    • sudo apt search "video editor"
    • This will output a list of all software you have with this function
  • 2/8/2022 issue with "initramfs" noted after a routine apt update and apt upgrade. Rebooting stopped with this error: "[end Kernel panic -  not syncing: VFS: unable to mount fs on unknown block c(0,0)]"
    • Solved on Ubuntu server 20.04.03 LTS using the following commands (adapted from here). I didn't need to mount any partitions beforehand because I was not using a live CD. I accessed the GRUB boot loader (screenshot examples) and booted using an older kernel (5.4.0-97-lowlatency), then used the terminal to fix the newer kernel's initramfs:
      • sudo update-initramfs -u -k 5.4.0-99-lowlatency
        • Change the underlined part to suit your needs
        • The underlined part was my newest Linux kernel that failed to load. That kernel was listed a few lines above the original "Kernel panic" error message. If you missed that information, identify your newest kernel using: 
          dpkg --list | grep linux-image
        • Your newest Linux kernel name may be different from mine!
      • sudo update-grub
      • sudo reboot

Good luck!

  • The key to success is following instructions, trial and error, checking for typos, and using the internet to find solutions for problems you'll encounter along the way
  • Patience is more important than being an expert Linux user or programmer
  • Break it up into a few days and stop to celebrate each completed step
  • You can do this as a beginner! It seems intimidating, but it's not that bad

If you get stuck after a terminal command, use Ctrl+Z

  • This is like the "Esc" key for Linux
  • Ctrl+Z will exit you out of any other shells (the python shell, the R shell) or file editing (vim, nano) that you might find yourself stuck in
  • Ctrl+Z stops processes and brings you back to the main terminal

Originally Ubuntu server 20.04 LTS (published December 20, 2020), but major edits after installing Ubuntu server 20.04.2 (March 2021). Edited occasionally whenever I want to document down anything else I've found useful.  Last edited 2/9/2022 for Ubuntu 20.04 LTS, but see my newer post for Ubuntu 22.04 LTS.

2 comments:

  1. "I had been struggling to set up a dedicated server at home for running my data analysis tasks without slowing down my main computer. This guide was exactly what I needed! It clearly walked me through everything—from preparing a bootable USB for Ubuntu Server installation to setting up RStudio Server, R, and even adding essential tools like JupyterLab and Python.

    The section on securing the server with firewall rules and setting up a static IP address was especially helpful for someone like me, who is comfortable with Linux but not an expert. I was also able to configure a desktop environment for better usability and make the server accessible only within my home network.

    For those starting fresh, I’d highly recommend checking out this Ubuntu 22.04 LTS setup guide https://docs.vultr.com/how-to-install-mysql-on-ubuntu-24-04 as well. It provides updated insights if you’re working with a newer version of Ubuntu.

    Thanks to this guide, I now have a fully functional server that helps me run long scripts and store large datasets without worrying about performance or disk space on my main computer!"

    ReplyDelete
    Replies
    1. You're welcome! I am glad it was helpful for you. When I set up my home server, I had to check a bunch of different websites to figure out how to set up various parts (e.g. firewall and static IP, RStudio server, Dropbox, my printer). I organized all my notes here so it would be easier next time.

      When I updated to Ubuntu 22.04 LTS, I copy/pasted these instructions and edited them here: https://sciencetania.blogspot.com/2022/09/setting-up-ubuntu-2204-lts-server.html
      This link was at the top of the post but I added it to the end so it is easier to find.

      Delete

Bookmarks: single cell RNA-seq tutorials and tools

These are my bookmarks for single cell transcriptomics resources and tutorials. scRNA-seq introductions How to make R obj...