Single-cell RNA sequencing (scRNA-seq) has revolutionized the fields of biology and medicine by enabling exploration of the transcriptomic profiles of individual cells.

This post aims to show how to use ScType to annotate cells in R code.

This post can be divided into two main parts:

1) Clustering Tutorial, which covers selecting cells (quality control), normalizing the data, identifying highly variable features, scaling the data, performing PCA, clustering the cells, and visualizing the clusters using UMAP.

2) Cell Annotation using ScType.

1. Clustering Tutorial

Download the example dataset and code

Run git clone or click this link to download.

git clone https://github.com/wcshin-git/Cell_Annotation_with_ScType

Import the required packages

lapply(c("dplyr","Seurat","HGNChelper","openxlsx"), library, character.only = TRUE)

If any of the packages are missing, please install them first.

Load the example dataset

pbmc.data <- Read10X(data.dir = "filtered_gene_bc_matrices/hg19") 
# (32738, 2700)

we will analyze the dataset of Peripheral Blood Mononuclear Cells (PBMC) freely available from 10X Genomics. There are 2,700 single cells that were sequenced on the Illumina NextSeq 500.

Let’s examine pbmc.data by looking at a few genes in the first 30 cells.

pbmc.data[c("CD3D", "TCL1A", "MS4A1"), 1:30]

3 x 30 sparse Matrix of class "dgCMatrix"
                                                                   
CD3D  4 . 10 . . 1 2 3 1 . . 2 7 1 . . 1 3 . 2  3 . . . . . 3 4 1 5
TCL1A . .  . . . . . . 1 . . . . . . . . . . .  . 1 . . . . . . . .
MS4A1 . 6  . . . . . . 1 1 1 . . . . . . . . . 36 1 2 . . 2 . . . .

The . values in the matrix represent 0s. Since most values in an scRNA-seq matrix are 0, Seurat uses a sparse-matrix representation whenever possible.

Setup the Seurat Object

pbmc <- CreateSeuratObject(counts = pbmc.data, project = "pbmc3k", min.cells = 3, min.features = 200)
# Retain genes expressed in at least 3 cells and cells with at least 200 detected genes.
# (13714, 2700)

The object serves as a container that contains both data (like the count matrix) and analysis (like PCA, or clustering results) for a single-cell dataset.

Quality Control

# percentage of counts originating from a set of mitochondrial genes
pbmc[["percent.mt"]] <- PercentageFeatureSet(pbmc, pattern = "^MT-")

Low-quality / dying cells often exhibit extensive mitochondrial contamination.

# Visualize QC metrics as a violin plot
VlnPlot(pbmc, features = c("nFeature_RNA", "nCount_RNA", "percent.mt"), ncol = 3)

nFeature_RNA represents the number of unique genes (features) detected in each cell. nCount_RNA represents the total number of RNA molecules detected in each cell.

# Cell filtering
pbmc <- subset(pbmc, subset = nFeature_RNA > 200 & nFeature_RNA < 2500 & percent.mt < 5)
# (13714, 2638)

Low-quality cells or empty droplets will often have very few genes. Cell doublets or multiplets may exhibit an aberrantly high gene count.

Normalize data

pbmc <- NormalizeData(pbmc, normalization.method = "LogNormalize", scale.factor = 10000)

“LogNormalize” normalizes the feature expression measurements for each cell by the total expression, multiplies this by a scale factor, and log-transforms the result.

Identification of highly variable features

pbmc <- FindVariableFeatures(pbmc, selection.method = "vst", nfeatures = 2000)

Focusing on these genes in downstream analysis helps to highlight biological signal in single-cell datasets.

Scaling the data

pbmc <- ScaleData(pbmc, features = rownames(pbmc))

Shifts the expression of each gene, so that the mean expression across cells is 0.

Scales the expression of each gene, so that the variance across cells is 1.

Run PCA

pbmc <- RunPCA(pbmc, features = VariableFeatures(object = pbmc))

Perform PCA on the scaled data for dimensional reduction.

# Check number of PC components
ElbowPlot(pbmc, ndims=20)

We will select 10 PCs for downstream analysis, based on Elbow plot.

Cluster and visualize

pbmc <- FindNeighbors(pbmc, dims = 1:10)
pbmc <- FindClusters(pbmc, resolution = 0.5)
pbmc <- RunUMAP(pbmc, dims = 1:10)
DimPlot(pbmc, label = TRUE, reduction = "umap")

The resolution parameter in FindClusters() sets the ‘granularity’ of the downstream clustering, with increased values leading to a greater number of clusters.

2. Cell Annotation

Now, let’s annotate the clusters with known cell types using ScType.

Load required functions

# load gene set preparation function
source("https://raw.githubusercontent.com/IanevskiAleksandr/sc-type/master/R/gene_sets_prepare.R")
# load cell type annotation function
source("https://raw.githubusercontent.com/IanevskiAleksandr/sc-type/master/R/sctype_score_.R")

Prepare marker gene sets

# DB file
db_ <- "https://raw.githubusercontent.com/IanevskiAleksandr/sc-type/master/ScTypeDB_full.xlsx";
tissue <- "Immune system" # e.g. Immune system,Pancreas,Liver,Eye,Kidney,Brain,Lung,Adrenal,Heart,Intestine,Muscle,Placenta,Spleen,Stomach,Thymus 

# prepare gene sets
gs_list <- gene_sets_prepare(db_, tissue)

Run ScType

# check Seurat object version (scRNA-seq matrix extracted differently in Seurat v4/v5)
seurat_package_v5 <- isFALSE('counts' %in% names(attributes(pbmc[["RNA"]])));
print(sprintf("Seurat object %s is used", ifelse(seurat_package_v5, "v5", "v4")))

# extract scaled scRNA-seq matrix
scRNAseqData_scaled <- if (seurat_package_v5) as.matrix(pbmc[["RNA"]]$scale.data) else as.matrix(pbmc[["RNA"]]@scale.data)

# run ScType
es.max <- sctype_score(scRNAseqData = scRNAseqData_scaled, scaled = TRUE, gs = gs_list$gs_positive, gs2 = gs_list$gs_negative)

The score is calculated through the following process:

1) Z-score Standardization

For each gene (row), the expression values across multiple cells (columns) are standardized using the z-score.

2) Calculating and Applying Marker Sensitivity

Genes that appear as markers for multiple cell types are given scores closer to 0, while those that are more specific to a single cell type receive scores closer to 1. By multiplying the z-scored expression matrix by these sensitivity scores, we effectively weight each gene according to how important it is for distinguishing that particular cell type.

3) Cell-Type Scoring via Positive/Negative Markers

For a given cell, the expression values (already multiplied by the gene-specificity scores) of the positive marker genes for a particular cell type are summed. To reduce biases due to varying numbers of marker genes, the total is divided by the square root of the number of marker genes. Negative markers for that cell type are also summed, but with a negative sign, and added in. The final value represents the score of that cell for that specific cell type.

Assigning Cell Types to Clusters

# merge by cluster
cL_resutls <- do.call("rbind", lapply(unique(pbmc@meta.data$seurat_clusters), function(cl){
    es.max.cl = sort(rowSums(es.max[ ,rownames(pbmc@meta.data[pbmc@meta.data$seurat_clusters==cl, ])]), decreasing = !0)
    head(data.frame(cluster = cl, type = names(es.max.cl), scores = es.max.cl, ncells = sum(pbmc@meta.data$seurat_clusters==cl)), 10)
}))
sctype_scores <- cL_resutls %>% group_by(cluster) %>% top_n(n = 1, wt = scores)  

# set low-confident (low ScType score) clusters to "unknown"
sctype_scores$type[as.numeric(as.character(sctype_scores$scores)) < sctype_scores$ncells/4] <- "Unknown"

We sum the ScType scores of the cells within each cluster and select the cell type with the highest score for each cluster.

Visualization

pbmc@meta.data$sctype_classification = ""
for(j in unique(sctype_scores$cluster)){
  cl_type = sctype_scores[sctype_scores$cluster==j,]; 
  pbmc@meta.data$sctype_classification[pbmc@meta.data$seurat_clusters == j] = as.character(cl_type$type[1])
}

DimPlot(pbmc, reduction = "umap", label = TRUE, repel = TRUE, group.by = 'sctype_classification')     

References

[1] Jovic, Dragomirka, et al. “Single‐cell RNA sequencing technologies and applications: A brief overview.” Clinical and translational medicine 12.3 (2022).

[2] Ianevski, Aleksandr, Anil K. Giri, and Tero Aittokallio. “Fully-automated and ultra-fast cell-type identification using specific marker combinations from single-cell transcriptomic data.” Nature communications 13.1 (2022)

[3] https://github.com/IanevskiAleksandr/sc-type

[4] https://satijalab.org/seurat/articles/pbmc3k_tutorial#determine-the-dimensionality-of-the-dataset

[5] http://youtube.com/watch?v=huxkc2GH4lk

Note: This is highly based on Lil’Log and Ho et al. 2020.

Forward Diffusion Process

In a nutshell, diffusion models generate a sample in the dataset distribution from noise by learning the reverse process of the forward process consisting of injecting small noise into the images of the dataset.

We define a Markov chain of diffusion steps to slowly add small amount of Gaussian noise to a sample \(\mathbf{x}_0\) in \(T\) steps, producing a sequence of noisy samples \(\mathbf{x}_{1}, \ldots, \mathbf{x}_T\).

Definition: Forward Diffusion Process

where \(\mathbf{x}_0\) is a data point sampled from a real data distribution \(q(\mathbf{x}_0)\) and \(\left\{\beta_{t} \in(0,1)\right\}_{t=1}^{T}\) is a variance schedule.

Usually, we can afford a larger update step when the sample gets noisier, so \(\beta_{1}<\beta_{2}<\cdots<\beta_{T}\). Ho et al. 2020 set the forward process variances to constants increasing linearly from \(\beta_{1}=10^{-4}\) to \(\beta_{T}=0.02\).

Property 1

where \(\alpha_{t}=1-\beta_{t}\) and \(\bar{\alpha}_{t}=\prod_{i=1}^{t} \alpha_{i}\).

Proof)

Eventually when \(T \rightarrow \infty, \mathbf{x}_{T}\) is equivalent to an isotropic Gaussian distribution.

Reverse Diffusion Process

The main idea is, “If we can reverse the above process and sample from \(q\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}\right)\), we will be able to recreate the true sample from a Gaussian noise input, \({\mathbf{x}_{T} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})}\).”

However, we cannot easily estimate \(q\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}\right)\) because it needs to use the entire dataset. Therefore, we need to learn a model \(p_\theta\) to approximate these conditional probabilities!

Definition: Reverse Diffusion Process

Reverse Diffusion Process is defined as a Markov chain starting at \(p_{\theta}(\mathbf{x}_{T})=\mathcal{N}\left(\mathbf{x}_{T} ; \mathbf{0}, \mathbf{I}\right)\):

Note that if \(\beta_t\) is small enough, \(q\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}\right)\) is also Gaussian. Therefore, we define \(p_{\theta}\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}\right)\) as a Gaussian distribution.

The overview of the forward and reverse process is illustrated in Fig. 1.

Fig. 1. Forward and reverse diffusion process. (Source: Lil’Log which is based on Ho et al. 2020)

Property 2

The reverse conditional probability \(q(\mathbf{x}_{t-1}\vert\mathbf{x}_t)\) is tractable when conditioned on \(\mathbf{x}_0\).

where \(\tilde{\mathbf{\mu}}_{t}\left(\mathbf{x}_{t}, \mathbf{x}_{0}\right)=\frac{\sqrt{\alpha_{t}}\left(1-\bar{\alpha}_{t-1}\right)}{1-\bar{\alpha}_{t}} \mathbf{x}_{t}+\frac{\sqrt{\bar{\alpha}_{t-1}} \beta_{t}}{1-\bar{\alpha}_{t}} \mathbf{x}_{0} = \frac{1}{\sqrt{\alpha_{t}}}\left(\mathbf{x}_{t}-\frac{\beta_{t}}{\sqrt{1-\bar{\alpha}_{t}}} \mathbf{\epsilon}_{t}\right)\) and \(\tilde{\beta}_{t}=\frac{1-\bar{\alpha}_{t-1}}{1-\bar{\alpha}_{t}} \cdot \beta_{t}\).

Proof)

\(* \text{ Gaussian PDF}: f(\mathbf{x}|\mathbf{\mu}, \mathbf{\Sigma}) = \operatorname{det}(2 \pi \mathbf{\Sigma})^{-\frac{1}{2}} \exp \left(-\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^{\top} \mathbf{\Sigma}^{-1}(\mathbf{x}-\mathbf{\mu})\right)\) \(\begin{aligned} &q\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}, \mathbf{x}_{0}\right) =q\left(\mathbf{x}_{t} \mid \mathbf{x}_{t-1}, \mathbf{x}_{0}\right) \frac{q\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{0}\right)}{q\left(\mathbf{x}_{t} \mid \mathbf{x}_{0}\right)}=q\left(\mathbf{x}_{t} \mid \mathbf{x}_{t-1}\right) \frac{q\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{0}\right)}{q\left(\mathbf{x}_{t} \mid \mathbf{x}_{0}\right)} \because \text{Markov} \\ & \propto \exp \left(-\frac{1}{2}\left(\frac{\left(\mathbf{x}_{t}-\sqrt{\alpha_{t}} \mathbf{x}_{t-1}\right)^{2}}{\beta_{t}}+\frac{\left(\mathbf{x}_{t-1}-\sqrt{\bar{\alpha}_{t-1}} \mathbf{x}_{0}\right)^{2}}{1-\bar{\alpha}_{t-1}}-\frac{\left(\mathbf{x}_{t}-\sqrt{\bar{\alpha}_{t}} \mathbf{x}_{0}\right)^{2}}{1-\bar{\alpha}_{t}}\right)\right) \\ &=\exp \left(-\frac{1}{2}\left(\left(\frac{\alpha_{t}}{\beta_{t}}+\frac{1}{1-\bar{\alpha}_{t-1}}\right) \mathbf{x}_{t-1}^{2}-\left(\frac{2 \sqrt{\alpha_{t}}}{\beta_{t}} \mathbf{x}_{t}+\frac{2 \sqrt{\bar{\alpha}_{t-1}}}{1-\bar{\alpha}_{t-1}} \mathbf{x}_{0}\right) \mathbf{x}_{t-1}+C\left(\mathbf{x}_{t}, \mathbf{x}_{0}\right)\right)\right) \end{aligned}\) where \(C\left(\mathbf{x}_{t}, \mathbf{x}_{0}\right)\) is a function not involving \(\mathbf{x}_{t-1}\).

Learning Objective

Goal: We want to minimize the negative log-likelihood.

In other words, we can achieve the goal by minimizing \(L_{\mathrm{VLB}}\)!

Remark 1: \(L_{\mathrm{VLB}}\)

We can convert \(L_{\mathrm{VLB}}\) to be analytically computable.

proof)

Definition: \(L_T\), \(L_{t-1}\), and \(L_0\)

1) \(L_T\)
\(\quad \bullet\) From Prop.1, \(q\left(\mathbf{x}_{T} \mid \mathbf{x}_{0}\right) \rightarrow \mathcal{N}\left(\mathbf{x}_{T} ; \mathbf{0}, \mathbf{I}\right)\) when \(T \rightarrow \infty\).
\(\quad \bullet\) We assume that \(p_{\theta}(\mathbf{x}_{T})=\mathcal{N}\left(\mathbf{x}_{T} ; \mathbf{0}, \mathbf{I}\right)\).
\(\quad \bullet\) \(L_T\) is constant and can be ignored during training.

2) \(L_{t-1}\)
\(\quad \bullet\) This term measures the difference between \(q\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}, \mathbf{x}_{0}\right)\) and \(p_{\theta}\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}\right)\).
\(\quad \bullet\) How do we optimize this term? (Explained below)

3) \(L_0\)
\(\quad \bullet\) This term reconstruct the original image from the slightly noised image.
\(\quad \bullet\) This term is optimized by MSE loss: \(\left\|\mathbf{x}_{0}-\mathbf{\mu}_{\theta}\left(\mathbf{x}_{1}, 1\right)\right\|^{2}\)

How to optimize \(L_{t-1}\)

\(L_{t-1} =D_{\mathrm{KL}}\left(q\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}, \mathbf{x}_{0}\right) \| p_{\theta}\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}\right)\right)\)
\(q\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}, \mathbf{x}_{0}\right)=\mathcal{N}\left(\mathbf{x}_{t-1} ; \tilde{\mathbf{\mu}}_{t}\left(\mathbf{x}_{t}, \mathbf{x}_{0}\right), \tilde{\beta}_{t} \mathbf{I}\right) = \mathcal{N}\left(\mathbf{x}_{t-1} ; \frac{1}{\sqrt{\alpha_{t}}}\left(\mathbf{x}_{t}-\frac{\beta_{t}}{\sqrt{1-\bar{\alpha}_{t}}} \mathbf{\epsilon}_{t}\right), \frac{1-\bar{\alpha}_{t-1}}{1-\bar{\alpha}_{t}} \beta_{t}\mathbf{I}\right)\)
\(p_{\theta}\left(\mathbf{x}_{t-1} \mid \mathbf{x}_{t}\right)=\mathcal{N}\left(\mathbf{x}_{t-1} ; \mathbf{\mu}_{\theta}\left(\mathbf{x}_{t}, t\right), \mathbf{\Sigma}_{\theta}\left(\mathbf{x}_{t}, t\right)\right)\)

Let us set \(\mathbf{\mu}_{\theta}\left(\mathbf{x}_{t}, t\right)=\frac{1}{\sqrt{\alpha_{t}}}\left(\mathbf{x}_{t}-\frac{\beta_{t}}{\sqrt{1-\bar{\alpha}_{t}}} \mathbf{\epsilon}_{\theta}(\mathbf{x}_{t}, t)\right)\) and \(\mathbf{\Sigma}_{\theta}\left(\mathbf{x}_{t}, t\right) = \sigma_{t}^{2}\mathbf{I}\).
We have two options for \(\sigma_{t}^{2}\): \(\sigma_{t}^{2}=\beta_{t}\) and \(\sigma_{t}^{2}=\frac{1-\bar{\alpha}_{t-1}}{1-\bar{\alpha}_{t}} \beta_{t}\).
According to Ho et al. 2020, both had similar results experimentally.

\(* D_{K L}(p \| q)=\frac{1}{2}\left[\log \frac{\left|\Sigma_{q}\right|}{\left|\Sigma_{p}\right|}-k+\left(\mathbf{\mu}_{p}-\mathbf{\mu}_{q}\right)^{T} \Sigma_{q}^{-1}\left(\mathbf{\mu}_{p}-\mathbf{\mu}_{q}\right)+\operatorname{tr}\left\{\Sigma_{q}^{-1} \Sigma_{p}\right\}\right]\) \(\begin{aligned} L_{t-1} &\propto \frac{1}{2\sigma_{t}^{2}}\left\|\tilde{\mathbf{\mu}}_{t}\left(\mathbf{x}_{t}, \mathbf{x}_{0}\right)-\mathbf{\mu}_{\theta}\left(\mathbf{x}_{t}, t\right)\right\|^{2}\\ &=\frac{1}{2\sigma_{t}^{2}}\left\|\frac{1}{\sqrt{\alpha_{t}}}\left(\mathbf{x}_{t}-\frac{\beta_{t}}{\sqrt{1-\bar{\alpha}_{t}}} \mathbf{\epsilon}_{t}\right)-\frac{1}{\sqrt{\alpha_{t}}}\left(\mathbf{x}_{t}-\frac{\beta_{t}}{\sqrt{1-\bar{\alpha}_{t}}} \mathbf{\epsilon}_{\theta}(\mathbf{x}_{t}, t)\right)\right\|^{2}\\ &=\frac{\beta_{t}^{2}}{2\sigma_{t}^{2}\alpha_{t}\left(1-\bar{\alpha}_{t}\right)}\left\|\mathbf{\epsilon}_{t}-\mathbf{\epsilon}_{\theta}\left(\mathbf{x}_{t}, t\right)\right\|^{2}\\ &=\frac{\beta_{t}^{2}}{2\sigma_{t}^{2}\alpha_{t}\left(1-\bar{\alpha}_{t}\right)}\left\|\mathbf{\epsilon}_{t}-\mathbf{\epsilon}_{\theta}\left(\sqrt{\bar{\alpha}_{t}} \mathbf{x}_{0}+\sqrt{1-\bar{\alpha}_{t}} \mathbf{\epsilon}_{t}, t\right)\right\|^{2} \end{aligned}\)

Empirically, Ho et al. 2020 found that training the diffusion model works better with a simplified objective that ignores the weighting term:

Training and Sampling Algorithm

Fig. 2. Training and sampling algorithm. (Source: Ho et al. 2020)

Generated Samples

Fig. 3. Unconditional CIFAR10 progressive generation. (Source: Ho et al. 2020)

Lecture Note Version

You can download the lecture note version!

Fig. 4. Example slide 1

Fig. 5. Example slide 2

References

[1] Weng, Lilian. (Jul 2021). What are diffusion models? Lil’Log. https://lilianweng.github.io/posts/2021-07-11-diffusion-models/.

[2] Jonathan Ho et al. “Denoising diffusion probabilistic models.” arxiv Preprint arxiv:2006.11239 (2020).

Citation

Shin, Woncheol. (April 2022). Introduction to Diffusion Models Shin’s Blog. https://wcshin-git.github.io/2022/04/18/Intro_Diffusion.html.

Or

@article{shin2022intro,
  title   = "Introduction to Diffusion Models",
  author  = "Shin, Woncheol",
  journal = "wcshin-git.github.io",
  year    = "2022",
  month   = "April",
  url     = "https://wcshin-git.github.io/2022/04/18/Intro_Diffusion.html"
}