Data Carpentry for Biologists

Grouping & Joining Data

Assignment

Learning Objectives

Following this assignment students should be able to:

  • install and load an R package
  • understand the data manipulation functions of dplyr
  • execute a simple import and analyze data scenario

Reading

Readings

Optional Resources:

Lecture Notes

Setup

install.packages(c('dplyr', 'readr', 'tidyr'))
download.file("https://ndownloader.figshare.com/files/2292172",
              "surveys.csv")
download.file("https://ndownloader.figshare.com/files/3299474",
              "plots.csv")
download.file("https://ndownloader.figshare.com/files/3299483",
              "species.csv")
download.file("https://www.datacarpentry.org/semester-biology/data/shrub-volume-data.csv",
              "shrub-volume-data.csv")
download.file("https://www.datacarpentry.org/semester-biology/data/shrub-volume-experiments.csv",
              "shrub-volume-experiments.csv")
download.file("https://www.datacarpentry.org/semester-biology/data/shrub-volume-sites.csv",
              "shrub-volume-sites.csv")
download.file("https://datacarpentry.org/semester-biology/data/mbaiki_measures.csv",
              "mbaiki_measures.csv")
download.file("https://datacarpentry.org/semester-biology/data/mbaiki_trees.csv",
              "mbaiki_trees.csv")
download.file("https://datacarpentry.org/semester-biology/data/mbaiki_species.csv",
              "mbaiki_species.csv")

Lecture Notes

Place this code at the start of the assignment to load all the required packages.

library(dplyr)

Exercises

  1. Portal Data Aggregation (10 pts)

    If the file surveys.csv is not already in your working directory download it.

    Load surveys.csv into R using read_csv().

    1. Use the group_by() and summarize() functions to get a count of the number of individuals in each species ID.
    2. Use the group_by() and summarize() functions to get a count of the number of individuals in each species ID in each year.
    3. Use the filter(), group_by(), and summarize() functions to get the mean mass of species DO in each year.
    Expected outputs for Portal Data Aggregation

  2. Shrub Volume Aggregation (10 pts)

    Dr. Morales wants some summary data of the plants at her sites and for her experiments. If the file shrub-volume-data.csv is not already in your work space download it. Load the data using read_csv.

    1. Use drop_na, group_by and summarize to calculate and print the average height of a plant in each experiment, while ignoring plants that have a height of NA when calculating the average.
    2. Use drop_na, group_by and summarize to determine the maximum height of a plant at each site, while ignoring plants that have a height of NA when calculating the maximum.
    3. Use mutate, drop_na, group_by and summarize to determine the average volume (volume = length * width * height) of a plant in each site, while ignoring plants that have volumes of NA when calculating the average.
    4. Use mutate, drop_na, group_by and summarize to determine both the average volume and the number of plants in each experiment, while ignoring plants that have volumes of NA when calculating the average and when counting plants.
    Expected outputs for Shrub Volume Aggregation

  3. Fix the Code (10 pts)

    This is a follow-up to Shrub Volume Aggregation. If you don’t already have the shrub volume data in your working directory download it.

    The following code is supposed to import the shrub volume data and calculate the average shrub volume for each site and, separately, for each experiment.

    read_csv("shrub-volume-data.csv")
shrub_data |>
  mutate(volume = length * width * height) |>
  group_by(site) |>
  summarize(mean_volume = max(volume))
shrub_data |>
  mutate(volume = length * width * height)
  group_by(experiment) |>
  summarize(mean_volume = mean(volume))
    1. Fix the errors in the code so that it does what it’s supposed to
    2. Add a comment to the top of the code explaining what it does
    Expected outputs for Fix the Code

  4. Portal Data Joins (10 pts)

    If surveys.csv, species.csv, and plots.csv are not available in your workspace download them:

    Load them into R using read_csv().

    1. Use inner_join() to create a table that contains the information from both the surveys table and the species table.
    2. Use inner_join() twice to create a table that contains the information from all three tables.
    3. Use inner_join() and filter() to get a data frame with the information from the surveys and plots tables where the plot_type is Control.
    Expected outputs for Portal Data Joins

  5. Shrub Volume Join (15 pts)

    Dr. Morales has data in three tables in the files:

    a. shrub-volume-data.csv with data on shrub dimensions b. shrub-volume-experiments.csv with data on experimental manipulations c. shrub-volume-sites.csv with data on different sites

    If the files aren’t available in your work space use the links above to download them. Load the data using read_csv.

    1. Use inner_join to combine the experiments data with the shrub volume data.
    2. Combine the sites data with both the data on shrub volume and the data on experiments to produce a single data frame that contains all of the data.
    Expected outputs for Shrub Volume Join

  6. Portal Data dplyr Review (15 pts)

    If surveys.csv, species.csv, and plots.csv are not available in your workspace download them:

    Load them into R using read_csv().

    We want to do an analysis comparing the size of individuals on the Control plots to the Long-term Krat Exclosures. Create a data frame with the year, genus, species, weight and plot_type for all cases where the plot type is either Control or Long-term Krat Exclosure. Only include cases where Taxa is Rodent. Remove any records where the weight is missing.

    Expected outputs for Portal Data dplyr Review

  7. Extracting vectors from data frames (10 pts)

    Using the Portal data surveys table (download a copy if it’s not in your working directory):

    1. Use $ to extract the weight column into a vector
    2. Use [] to extract the month column into a vector
    3. Extract the hindfoot_length column into a vector using pull() and use this vector to calculate the mean hindfoot length ignoring null values
    Expected outputs for Extracting vectors from data frames

  8. Building data frames from vectors (10 pts)

    You have data on the length, width, and height of 10 individuals of the yew Taxus baccata stored in the following vectors:

    length <- c(2.2, 2.1, 2.7, 3.0, 3.1, 2.5, 1.9, 1.1, 3.5, 2.9)
width <- c(1.3, 2.2, 1.5, 4.5, 3.1, NA, 1.8, 0.5, 2.0, 2.7)
height <- c(9.6, 7.6, 2.2, 1.5, 4.0, 3.0, 4.5, 2.3, 7.5, 3.2)

    Make a data frame that contains these three vectors as columns along with a genus column containing the name Taxus on all rows and a species column containing the word baccata on all rows.

    Expected outputs for Building data frames from vectors

  9. Check That Your Code Runs (10 pts)

    Sometimes you think you’re code runs, but it only actually works because of something else you did previously. To make sure it actually runs you should save your work and then run it in a clean environment.

    Follow these steps in RStudio to make sure your code really runs:

    1. Restart R (see above) by clicking Session in the menu bar and selecting Restart R:

    Screenshot showing clicking session from the menu bar and selecting Restart R

    2. If the Environment tab isn’t empty click on the broom icon to clear it:

    Screenshot showing the Environment tab with the cursor hovering over the broom icon

    The Environment tab should now say “Environment Is Empty”:

    Screenshot showing the Environment tab with only the words Environment Is Empty

    3. Rerun your entire homework assignment using “Source with Echo” to make sure it runs from start to finish and produces the expected results.

    Screenshot showing the RStudio Source with Echo item hovered in the Source dropdown

    4. Make sure that you saved your code with the name assignment somewhere in the file name. You should see the file in the Files tab and the name of the file should be black (not red with an * in the tab at the top of the text editor):

    Screenshot showing the Files tab with the cursor hovering over the assignment file

    Screenshot showing the file name in the editor tab and it is black and there is no *

    5. Make sure that your code will run on other computers

    • No setwd() (use RStudio Projects instead)
    • Use / not \ for paths
    Expected outputs for Check That Your Code Runs

  10. M'Baïki Data Challenge (Challenge - optional)

    A long-term study near M’Baïki in the Central African Republic has been monitoring tropical forest recovery from disturbance for 40 years.

    Use the data on yearly tree measurements (in mbaiki_measures.csv), information on the individual trees (in mbaiki_trees.csv), and the names of the species in the forest (in mbaiki_species.csv) to answer the following questions (if isn’t in your working directory download it).

    1. Create a new data frame that contains the following information for each unique tree (each tree has a unique id_tree): The id_tree, the net growth (total change in diameter from the first year a tree is measured to the last year a tree is measured), the time period of sampling in years (number of years between the first and last measurement), and the growth rate (the net growth divided by the time period of sampling). Only include observations while the tree was alive in these calculations.
    2. Starting with the data frame you created in (1) create a new data frame that contains the following information on the average growth rate of trees in each species in each subplot: The ID of the subplot, the scientific name of the species, the mean growth rate of all of the trees of that species in that subplot, and the sample size used to estimate the mean (i.e., the number of trees of that species in that subplot). Make sure the resulting data frame is not grouped.

    Find out more about this dataset by accessing the full dataset or reading the associated paper: Bénédet, F., Gourlet-Fleury, S., Allah-Barem, F. et al. 2024. 40 years of forest dynamics and tree demography in an intact tropical forest at M’Baïki in central Africa. Sci Data 11, 734.

    Expected outputs for M'Baïki Data Challenge

Assignment submission & checklist