Data Carpentry for Biologists

Data in Tables

Assignment

Learning Objectives

Following this assignment students should be able to:

  • install and load an R package
  • understand the data manipulation functions of dplyr
  • execute a simple import and analyze data scenario

Reading

Readings

Optional Resources:

Lecture Notes

Setup

install.packages(c('dplyr', 'readr', 'tidyr'))
download.file("https://ndownloader.figshare.com/files/2292172",
              "surveys.csv")
download.file("https://ndownloader.figshare.com/files/3299474",
              "plots.csv")
download.file("https://ndownloader.figshare.com/files/3299483",
              "species.csv")
download.file("https://www.datacarpentry.org/semester-biology/data/shrub-volume-data.csv",
              "shrub-volume-data.csv")
download.file("https://www.datacarpentry.org/semester-biology/data/penguins.csv",
              "penguins.csv")

Lecture Notes

Place this code at the start of the assignment to load all the required packages.

library(dplyr)

Exercises

  1. Shrub Volume Data Basics Select (5 pts)

    Dr. Morales is interested in studying the factors controlling the size and carbon storage of shrubs. She has conducted an experiment looking at the effect of three different treatments on shrub volume at four different locations. She has placed the data file on the web for you to download:

    If the file shrub-volume-data.csv is not already in your working directory (it probably is if you’re taking this class using Posit Cloud) then download it into your working directory.

    Get familiar with the data by importing it using read_csv() and use dplyr to complete the following tasks.

    1. Select the data from the length column (using select).
    2. Select the data from the site and experiment columns (using select).
    3. Select the data from the experiment, length, width, and height columns
    Expected outputs for Shrub Volume Data Basics Select

  2. Penguin Data Basics Select (5 pts)

    If the file penguins.csv is not already in your working directory then download it into your working directory.

    Load the data using read_csv() and use dplyr to complete the following tasks.

    1. Select the data from the species column.
    2. Select the data from the island and year.
    3. Select the data from the species, bill_length_mm, bill_depth_mm, and body_mass_g columns.
    Expected outputs for Penguin Data Basics Select

  3. Shrub Volume Data Basics Mutate (5 pts)

    Dr. Morales is interested in studying the factors controlling the size and carbon storage of shrubs. She has conducted an experiment looking at the effect of three different treatments on shrub volume at four different locations. She has placed the data file on the web for you to download:

    If the file shrub-volume-data.csv is not already in your working directory (it probably is if you’re taking this class using Posit Cloud) then download it into your working directory.

    Get familiar with the data by importing it using read_csv() and use dplyr to complete the following tasks.

    1. Sort the data by length (using arrange).
    2. Add a new column named area containing the area of the shrub (length x width) (using mutate).
    3. Add a new column named volume containing the volume of the shrub (length x width x height) (using mutate).
    Expected outputs for Shrub Volume Data Basics Mutate

  4. Penguin Data Basics Mutate (5 pts)

    If the file penguins.csv is not already in your working directory then download it into your working directory.

    Load the data using read_csv() and use dplyr to complete the following tasks.

    1. Add a new column named bill_ratio containing the ratio of the penguin’s bill length to its bill depth (bill length / bill depth).
    2. Sort the data by body mass.
    Expected outputs for Penguin Data Basics Mutate

  5. Shrub Volume Data Basics Filter (5 pts)

    Dr. Morales is interested in studying the factors controlling the size and carbon storage of shrubs. She has conducted an experiment looking at the effect of three different treatments on shrub volume at four different locations. She has placed the data file on the web for you to download:

    If the file shrub-volume-data.csv is not already in your working directory (it probably is if you’re taking this class using Posit Cloud) then download it into your working directory.

    Get familiar with the data by importing it using read_csv() and use dplyr to complete the following tasks.

    1. Filter the data to include only plants with heights greater than 5 (using filter).
    2. Filter the data to include only plants with heights greater than 4 and widths greater than 2 (using , or & to include two conditions).
    3. Filter the data to include only plants from Experiment 1 or Experiment 3 (using %in%).
    Expected outputs for Shrub Volume Data Basics Filter

  6. Penguin Data Basics Filter (5 pts)

    If the file penguins.csv is not already in your working directory then download it into your working directory.

    Load the data using read_csv() and use dplyr to complete the following tasks.

    1. Filter the data to include only penguins with flippers longer than 210 mm.
    2. Filter the data to include only penguins with flippers longer than 210 mm and body masses greater than 5000 g.
    3. Filter the data to include only penguins from the Dream or Torgersen islands.
    Expected outputs for Penguin Data Basics Filter

  7. Shrub Volume Data Basics Filter NA (5 pts)

    Dr. Morales is interested in studying the factors controlling the size and carbon storage of shrubs. She has conducted an experiment looking at the effect of three different treatments on shrub volume at four different locations. She has placed the data file on the web for you to download:

    If the file shrub-volume-data.csv is not already in your working directory (it probably is if you’re taking this class using Posit Cloud) then download it into your working directory.

    Get familiar with the data by importing it using read_csv() and use dplyr to complete the following tasks.

    1. Remove rows with null values in the height column (using drop_na).
    2. Remove rows with null values in the height and width columns (using drop_na).
    Expected outputs for Shrub Volume Data Basics Filter NA

  8. Penguin Data Basics Filter NA (5 pts)

    If the file penguins.csv is not already in your working directory then download it into your working directory.

    Load the data using read_csv() and use dplyr to complete the following tasks.

    1. Remove rows with null values in the body_mass_g column.
    2. Remove rows with null values in the sex column.
    3. Create a new data frame called penguin_masses that includes all of the original data with no null body masses and a new column containing the body mass in kilograms (body mass in grams / 1000).
    Expected outputs for Penguin Data Basics Filter NA

  9. Code Shuffle (5 pts)

    We are interested in understanding the monthly variation in precipitation in Gainesville, FL. We’ll use some data from the NOAA National Climatic Data Center. Each row of the data is a year (from 1961-2013) and each column is a month (January - December).

    Rearrange the following program so that it:

    • Loads the readr package
    • Imports the data from the web into a data frame
    • Calculates the mean precipitation (ppt) in each month across years
    • Plots the monthly averages as a simple line plot

    Finally, add a comment above the code that describes what it does. The comment character in R is #.

    It’s OK if you don’t know exactly how the details of the program work at this point, you just need to figure out the right order of the lines based on when variables are defined and when they are used.

    plot(monthly_mean_ppt, type = "l", xlab = "Month", ylab = "Mean Precipitation")
monthly_mean_ppt <- colMeans(ppt_data)
library(readr)
ppt_data <- read_csv("https://datacarpentry.org/semester-biology/data/gainesville-precip.csv", col_names = FALSE)
    Expected outputs for Code Shuffle

  10. Bird Banding (10 pts)

    The number of birds banded at a series of sampling sites has been counted by your field crew and entered into the following vector. Counts are entered in order and sites are numbered starting at one. Cut and paste the vector into your assignment and then answer the following questions by using code and printing the result to the screen. Some R functions that will come in handy include length(), max(), min(), sum(), and mean().

    number_of_birds <- c(28, 32, 1, 0, 10, 22, 30, 19, 145, 27, 
36, 25, 9, 38, 21, 12, 122, 87, 36, 3, 0, 5, 55, 62, 98, 32, 
900, 33, 14, 39, 56, 81, 29, 38, 1, 0, 143, 37, 98, 77, 92, 
83, 34, 98, 40, 45, 51, 17, 22, 37, 48, 38, 91, 73, 54, 46,
102, 273, 600, 10, 11)
    1. How many sites are there?
    2. How many birds were counted at site 42?
    3. What is the total number of birds counted across all of the sites?
    4. What is the smallest number of birds counted?
    5. What is the largest number of birds counted?
    6. What is the average number of birds seen at a site?
    7. How many birds were counted at the last site? Have the computer choose the last site automatically in some way, not by manually entering its position. Do you know a function that will give you the position of the last value? (since positions start at 1 the position of the last value in a vector is the same as its length).
    Expected outputs for Bird Banding

  11. Portal Data Manipulation (15 pts)

    If the file surveys.csv is not already in your working directory then download a copy.

    Load the file into R using read_csv().

    Do not use pipes for this exercise.

    1. Use select() and arrange() to create a new data frame with just the year, month, day, and plot_id columns with the rows sorted by plot_id.
    2. Use mutate(), select(), and drop_na() to create a new data frame with the year, species_id, and weight in kilograms of each individual, with no null weights. The weight in the table is given in grams so you will need to create a new column for weight in kilograms by dividing the weight column by 1000.
    3. Use filter() and select() to get the year, month, day, and species_id columns for all of the rows in the data frame where species_id is SH.
    4. Use select(), filter(), and arrange() to produce a data frame with plot_id, species_id, weight, and hindfoot_length, where the species is "PB" or "PP" and weight is less than 40. Exclude NA values for both weight and hindfoot_length. Sort the table by weight.
    Expected outputs for Portal Data Manipulation

  12. Portal Data Manipulation Pipes (20 pts)

    If the file surveys.csv is not already in your working directory then download a copy.

    Load the file into R using read_csv().

    Use pipes (|>) to combine the following operations to manipulate the data.

    1. Use select() and arrange() to create a new data frame with just the year, month, day, species_id, and plot_id columns with the rows sorted by species_id.
    2. Use mutate(), select(), and drop_na() to create a new data frame with the year, species_id, and hindfoot_length in cm of each individual, with no null hindfoot lengths. The hindfoot length in the table is given in mm so you will need to create a new column for hindfoot length in cm by dividing the hindfoot_length column by 10.
    3. Use filter() and select() to get the year, month, day, and species_id columns for all of the rows in the data frame where species_id is "OT".
    4. Use select(), filter(), and arrange() to produce a data frame with plot_id, species_id, weight, and hindfoot_length, where the species is "DM" or "DS" and hindfoot_length is greater than 35. Exclude NA values for both weight and hindfoot_length. Sort the result by hindfoot_length.
    Expected outputs for Portal Data Manipulation Pipes

  13. Check That Your Code Runs (10 pts)

    Sometimes you think you’re code runs, but it only actually works because of something else you did previously. To make sure it actually runs you should save your work and then run it in a clean environment.

    Follow these steps in RStudio to make sure your code really runs:

    1. Restart R (see above) by clicking Session in the menu bar and selecting Restart R:

    Screenshot showing clicking session from the menu bar and selecting Restart R

    2. If the Environment tab isn’t empty click on the broom icon to clear it:

    Screenshot showing the Environment tab with the cursor hovering over the broom icon

    The Environment tab should now say “Environment Is Empty”:

    Screenshot showing the Environment tab with only the words Environment Is Empty

    3. Rerun your entire homework assignment using “Source with Echo” to make sure it runs from start to finish and produces the expected results.

    Screenshot showing the RStudio Source with Echo item hovered in the Source dropdown

    4. Make sure that you saved your code with the name assignment somewhere in the file name. You should see the file in the Files tab and the name of the file should be black (not red with an * in the tab at the top of the text editor):

    Screenshot showing the Files tab with the cursor hovering over the assignment file

    Screenshot showing the file name in the editor tab and it is black and there is no *

    5. Make sure that your code will run on other computers

    • No setwd() (use RStudio Projects instead)
    • Use / not \ for paths
    Expected outputs for Check That Your Code Runs

  14. Portal Data Challenge (Challenge - optional)

    If the file surveys.csv is not already in your working directory then download a copy.

    Develop a data manipulation pipeline for the Portal surveys table that produces a table of data for only the three Dipodomys species (DM, DO, DS). The species IDs should be presented as lower case, not upper case. The table should contain information on the date, the species ID, the weight and hindfoot length. The data should not include null values for either weight or hindfoot length. The table should be sorted first by the species (so that each species is grouped together) and then by weight, with the largest weights at the top.

    Expected outputs for Portal Data Challenge

Assignment submission & checklist