fastqcr: An R Package Facilitating Quality Controls of Sequencing Data for Large Numbers of Samples

Introduction

High throughput sequencing data can contain hundreds of millions of sequences (also known as reads).

The raw sequencing reads may contain PCR primers, adaptors, low quality bases, duplicates and other contaminants coming from the experimental protocols. As these may affect the results of downstream analysis, it’s essential to perform some quality control (QC) checks to ensure that the raw data looks good and there are no problems in your data.

The FastQC tool, written by Simon Andrews at the Babraham Institute, is the most widely used tool to perform quality control for high throughput sequence data. To learn more about the FastQC tool, see this Video Tuorial.

It produces, for each sample, an html report and a ‘zip’ file, which contains a file called fastqc_data.txt and summary.txt.

If you have hundreds of samples, you’re not going to open up each HTML page. You need some way of looking at these data in aggregate.

Therefore, we developed the fastqcr R package, which contains helper functions to easily and automatically parse, aggregate and analyze FastQC reports for large numbers of samples.

Additionally, the fastqcr package provides a convenient solution for building a multi-QC report and a one-sample FastQC report with the result interpretations. The online documentation is available at: http://www.sthda.com/english/rpkgs/fastqcr/.

Examples of QC reports, generated automatically by the fastqcr R package, include:

• Multi-QC report for multiple samples
• One sample QC report (+ interpretation)
• One sample QC report (no interpretation)

Contents:

Installation and loading fastqcr

• fastqcr can be installed from CRAN as follow:

install.packages("fastqcr")

• Or, install the latest version from GitHub:

if(!require(devtools)) install.packages("devtools")
devtools::install_github("kassambara/fastqcr")

• Load fastqcr:

library("fastqcr")

Quick Start

library(fastqcr)

# Aggregating Multiple FastQC Reports into a Data Frame 
#%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

# Demo QC directory containing zipped FASTQC reports
qc.dir <- system.file("fastqc_results", package = "fastqcr")
qc <- qc_aggregate(qc.dir)
qc

# Inspecting QC Problems
#%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

# See which modules failed in the most samples
qc_fails(qc, "module")
# Or, see which samples failed the most
qc_fails(qc, "sample")

# Building Multi QC Reports
#%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
qc_report(qc.dir, result.file = "multi-qc-report" )

# Building One-Sample QC Reports (+ Interpretation)
#%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
qc.file <- system.file("fastqc_results", "S1_fastqc.zip", package = "fastqcr")
qc_report(qc.file, result.file = "one-sample-report",
          interpret = TRUE)

Main Functions

1) Installing and Running FastQC

• fastqc_install(): Install the latest version of FastQC tool on Unix systems (MAC OSX and Linux)

• fastqc(): Run the FastQC tool from R.

2) Aggregating and Summarizing Multiple FastQC Reports

• qc <- qc_aggregate(): Aggregate multiple FastQC reports into a data frame.

• summary(qc): Generates a summary of qc_aggregate.

• qc_stats(qc): General statistics of FastQC reports.

3) Inspecting Problems

• qc_fails(qc): Displays samples or modules that failed.

• qc_warns(qc): Displays samples or modules that warned.

• qc_problems(qc): Union of qc_fails() and qc_warns(). Display which samples or modules that failed or warned.

4) Importing and Plotting FastQC Reports

• qc_read(): Read FastQC data into R.

• qc_plot(qc): Plot FastQC data

5) Building One-Sample and Multi-QC Reports

• qc_report(): Create an HTML file containing FastQC reports of one or multiple files. Inputs can be either a directory containing multiple FastQC reports or a single sample FastQC report.

6) Others

• qc_unzip(): Unzip all zipped files in the qc.dir directory.
  

Installing FastQC from R

You can install automatically the FastQC tool from R as follow:

fastqc_install()

Running FastQC from R

The supported file formats by FastQC include:

• FASTQ
• gzip compressed FASTQ

Suppose that your working directory is organized as follow:

• home
  • Documents
    • FASTQ

where, FASTQ is the directory containing your FASTQ files, for which you want to perform the quality control check.

To run FastQC from R, type this:

fastqc(fq.dir = "~/Documents/FASTQ", # FASTQ files directory
       qc.dir = "~/Documents/FASTQC", # Results direcory
       threads = 4                    # Number of threads
       )

FastQC Reports

For each sample, FastQC performs a series of tests called analysis modules.

These modules include:

• Basic Statistics,
• Per base sequence quality,
• Per tile sequence quality
• Per sequence quality scores,
• Per base sequence content,
• Per sequence GC content,
• Per base N content,
• Sequence Length Distribution,
• Sequence Duplication Levels,
• Overrepresented sequences,
• Adapter Content
• Kmer content

The interpretation of these modules are provided in the official documentation of the FastQC tool.

Aggregating Reports

Here, we provide an R function qc_aggregate() to walk the FastQC result directory, find all the FASTQC zipped output folders, read the fastqc_data.txt and the summary.txt files, and aggregate the information into a data frame.

The fastqc_data.txt file contains the raw data and statistics while the summary.txt file summarizes which tests have been passed.

In the example below, we’ll use a demo FastQC output directory available in the fastqcr package.

library(fastqcr)
# Demo QC dir
qc.dir <- system.file("fastqc_results", package = "fastqcr")
qc.dir

## [1] "/Library/Frameworks/R.framework/Versions/3.3/Resources/library/fastqcr/fastqc_results"

# List of files in the directory
list.files(qc.dir)

## [1] "S1_fastqc.zip""S2_fastqc.zip""S3_fastqc.zip""S4_fastqc.zip""S5_fastqc.zip"

The demo QC directory contains five zipped folders corresponding to the FastQC output for 5 samples.

Aggregating FastQC reports:

qc <- qc_aggregate(qc.dir)
qc

The aggregated report looks like this:

Column names:

• sample: sample names
• module: fastqc modules
• status: fastqc module status for each sample
• tot.seq: total sequences (i.e.: the number of reads)
• seq.length: sequence length
• pct.gc: percentage of GC content
• pct.dup: percentage of duplicate reads

Once you have the aggregated data you can use the dplyr package to easily inspect modules that failed or warned in samples. For example, the following R code shows samples with warnings and/or failures:

library(dplyr)
qc %>%
  select(sample, module, status) %>%    
  filter(status %in% c("WARN", "FAIL")) %>%
  arrange(sample)

Summarizing Reports

We start by presenting a summary and general statistics of the aggregated data.

# Summary of qc
summary(qc)

qc_stats(qc)

Inspecting Problems

Once you’ve got this aggregated data, it’s easy to figure out what (if anything) is wrong with your data.

1) R functions. You can inspect problems per either modules or samples using the following R functions:

• qc_fails(qc): Displays samples or modules that failed.
• qc_warns(qc): Displays samples or modules that warned.
• qc_problems(qc): Union of qc_fails() and qc_warns(). Display which samples or modules that failed or warned.

2) Input data: aggregated data from qc_aggregate()

3) Output data: Returns samples or FastQC modules with failures or warnings. By default, these functions return a compact output format. If you want a stretched format, specify the argument compact = FALSE.

The format and the interpretation of the outputs depend on the additional argument element, which value is one of c(“sample”, “module”).

• If element = “sample” (default), results are samples with failed and/or warned modules. The results contain the following columns:
  • sample (sample names),
  • nb_problems (the number of modules with problems),
  • module (the name of modules with problems).
• If element = “module”, results are modules that failed and/or warned in the most samples. The results contain the following columns:
  • module (the name of module with problems),
  • nb_problems (the number of samples with problems),
  • sample (the name of samples with problems)

# See which module failed in the most samples
qc_fails(qc, "module")

# See which module warned in the most samples
qc_warns(qc, "module")

# See which modules failed or warned.
qc_problems(qc, "module")

qc_problems(qc, "module", compact = FALSE)

qc_problems(qc, "module",  name = "Per sequence GC content")

qc_problems(qc, "module",  name = "GC content")

# See which samples had one or more failed modules
qc_fails(qc, "sample")

# See which samples had one or more module with failure or warning
qc_problems(qc, "sample", compact = FALSE)

qc_problems(qc, "sample", name = "S1")

Building an HTML Report

The function qc_report() can be used to build a report of FastQC outputs. It creates an HTML file containing FastQC reports of one or multiple samples.

Inputs can be either a directory containing multiple FastQC reports or a single sample FastQC report.

# Demo QC Directory
qc.dir <- system.file("fastqc_results", package = "fastqcr")
qc.dir

## [1] "/Library/Frameworks/R.framework/Versions/3.3/Resources/library/fastqcr/fastqc_results"

# Build a report
qc_report(qc.dir, result.file = "~/Desktop/multi-qc-result",
          experiment = "Exome sequencing of colon cancer cell lines")

 qc.file <- system.file("fastqc_results", "S1_fastqc.zip", package = "fastqcr")
qc.file

## [1] "/Library/Frameworks/R.framework/Versions/3.3/Resources/library/fastqcr/fastqc_results/S1_fastqc.zip"

 qc_report(qc.file, result.file = "one-sample-report-with-interpretation",
   interpret = TRUE)

 qc_report(qc.file, result.file = "one-sample-report",
   interpret = FALSE)

Importing and Plotting a FastQC QC Report

We’ll visualize the output for sample 1:

# Demo file
qc.file <- system.file("fastqc_results", "S1_fastqc.zip",  package = "fastqcr")
qc.file

## [1] "/Library/Frameworks/R.framework/Versions/3.3/Resources/library/fastqcr/fastqc_results/S1_fastqc.zip"

We start by reading the output using the function qc_read(), which returns a list of tibbles containing the data for specified modules:

# Read all modules
qc <- qc_read(qc.file)
# Elements contained in the qc object
names(qc)

##  [1] "summary""basic_statistics""per_base_sequence_quality""per_tile_sequence_quality"    
##  [5] "per_sequence_quality_scores""per_base_sequence_content""per_sequence_gc_content""per_base_n_content"           
##  [9] "sequence_length_distribution""sequence_duplication_levels""overrepresented_sequences""adapter_content"              
## [13] "kmer_content""total_deduplicated_percentage"

The function qc_plot() is used to visualized the data of a specified module. Allowed values for the argument modules include one or the combination of:

• “Summary”,
• “Basic Statistics”,
• “Per base sequence quality”,
• “Per sequence quality scores”,
• “Per base sequence content”,
• “Per sequence GC content”,
• “Per base N content”,
• “Sequence Length Distribution”,
• “Sequence Duplication Levels”,
• “Overrepresented sequences”,
• “Adapter Content”

qc_plot(qc, "Per sequence GC content")

qc_plot(qc, "Per base sequence quality")

qc_plot(qc, "Per sequence quality scores")

qc_plot(qc, "Per base sequence content")

qc_plot(qc, "Sequence duplication levels")

fastqcr

Interpreting FastQC Reports

• Summary shows a summary of the modules which were tested, and the status of the test results:
  • normal results (PASS),
  • slightly abnormal (WARN: warning)
  • or very unusual (FAIL: failure).

Some experiments may be expected to produce libraries which are biased in particular ways. You should treat the summary evaluations therefore as pointers to where you should concentrate your attention and understand why your library may not look normal.

qc_plot(qc, "summary")

• Basic statistics shows basic data metrics such as:
  • Total sequences: the number of reads (total sequences),
  • Sequence length: the length of reads (minimum - maximum)
  • %GC: GC content

qc_plot(qc, "Basic statistics")

• Per base sequence quality plot depicts the quality scores across all bases at each position in the reads. The background color delimits 3 different zones: very good quality (green), reasonable quality (orange) and poor quality (red). A good sample will have qualities all above 28:

qc_plot(qc, "Per base sequence quality")

fastqcr

Problems:

Common reasons for problems:

• Per sequence quality scores plot shows the frequencies of quality scores in a sample. It allows you to see if a subset of your sequences have low quality values. If the reads are of good quality, the peak on the plot should be shifted to the right as far as possible (quality > 27).

qc_plot(qc, "Per sequence quality scores")

fastqcr

Problems:

Common reasons for problems:

• Per base sequence content shows the four nucleotides’ proportions for each position. In a random library you expect no nucleotide bias and the lines should be almost parallel with each other. In a good sequence composition, the difference between A and T, or G and C is < 10% in any position.

qc_plot(qc, "Per base sequence content")

fastqcr

Problems:

Common reasons for problems:

• Per sequence GC content plot displays GC distribution over all sequences. In a random library you expect a roughly normal GC content distribution. An unusually sharped or shifted distribution could indicate a contamination or some systematic biases:

qc_plot(qc, "Per sequence GC content")

fastqcr

• Per base N content. If a sequencer is unable to make a base call with sufficient confidence then it will normally substitute an N rather than a conventional base call. This module plots out the percentage of base calls at each position for which an N was called.

qc_plot(qc, "Per base N content")

fastqcr

Problems:

Common reasons for problems:

• Sequence length distribution module reports if all sequences have the same length or not. For some sequencing platforms it is entirely normal to have different read lengths so warnings here can be ignored. In many cases this will produce a simple graph showing a peak only at one size. This module will raise an error if any of the sequences have zero length.

qc_plot(qc, "Sequence length distribution")

fastqcr

• Sequence duplication levels. This module counts the degree of duplication for every sequence in a library and creates a plot showing the relative number of sequences with different degrees of duplication. A high level of duplication is more likely to indicate some kind of enrichment bias (eg PCR over amplification).

qc_plot(qc, "Sequence duplication levels")

fastqcr

Problems:

Common reasons for problems:

• Overrepresented sequences section gives information about primer or adaptor contaminations. Finding that a single sequence is very overrepresented in the set either means that it is highly biologically significant, or indicates that the library is contaminated, or not as diverse as you expected. This module lists all of the sequence which make up more than 0.1% of the total.

qc_plot(qc, "Overrepresented sequences")

fastqcr

Problems:

Common reasons for problems:

• Adapter content module checks the presence of read-through adapter sequences. It is useful to know if your library contains a significant amount of adapter in order to be able to assess whether you need to adapter trim or not.

qc_plot(qc, "Adapter content")

fastqcr

Problems:

• K-mer content

qc_plot(qc, "Kmer content")

fastqcr

Useful Links

• FastQC report for a good Illumina dataset
• FastQC report for a bad Illumina dataset
• Online documentation for each FastQC report

Infos

This analysis has been performed using R software (ver. 3.3.2).