HOW TO Part 2:: How to test models using qpAdm - qpAdm 101

qpAdm is a statistical tool for studying the ancestry of populations with histories that involve admixture between two or more source populations. I hope this tutorial helps people interested in doing own research as qpAdm validated models which are constructed well provide a high standard of evidence to the model.

qpAdm is used to validate models that are fed into it by the user. The details that need to passed to the qpAdm program are as follows.

1. Target population
2. List of 2 or more source populations 
3. List of Right populations or Right Pops.
4. The populations in 1 & 2 are together called Left Populations or Left Pops and the first population in this list is considered as target population by qpAdm.
5. The first population among the right pops has to be a basal population (Outgroup) and usually an african population like Mbuti, ShumLaka or Mota etc is chosen for this purpose.

A standard example of a qpAdm model is 

 Target population (Target) = source population 1 (Source 1) + source population 2 (Source 2) 

The qpAdm output will contain a p-value (also called tail probability or tailprob), admixture coefficients x & y for Source1 and Source2 respectively such that x+y = 1 (or 100%) and standard errors for those coefficients. 

 A successful model will have 

1. A high p-value, and all models above a given threshold are to be accepted as valid. The common threshold used in published pop genomics papers is 0.05. 
2. Low standard errors in the admixture coefficients.
3. Positive admixture co-efficients.

Steps to run qpfstats

1. Install Admixtools on the linux system. The step wise how to for that is provided in this tutorial. also download the eigentrat files from the Harvard Lab database as described in the linked tutorial.
2. Create Fstats output file for use as input into qpAdm. For that
1. Create a new folder in your linux system and create 2 new text files. Name them
1. parqpfstat.txt (parameter file)
2. lista.txt (Write the label names of each population that you will use in the qpAdm for multiple models, one per line. Max 20-23 populations)
2. run the qpfstats program in terminal while being inside this newly created folder.

qpfstats -p parqpfstat.txt >qpfstatlog.txt

Let the qpfstats program finish running. This step takes the most time so be patient.

This is how the output file looks. Refer the video  below to gain more clarity

Steps to run qpAdm

1. Create 3 new text files in same folder. Click on the files below to view example. Name them
1. parqpadm.txt (parameter file)
2. left.txt (list of left populations, 1 population per line, name has to match label in eigenstrat .ind file)
3. right.txt (list of right populations, 1 population per line, name has to match label in eigenstrat .ind file)
2. Run the qpAdm program from terminal while being inside the folder which contains these files.

qpAdm -p parqpadm > outputfile.txt

Refer the below video for clarity

Output file will contain all the necessary details for model evaluation.

How to read the qpAdm output file

In the above video, I tried to model Irula as GanjDareh_N + Onge by specifying a few random right populations.

The output file is here. Please open it as I will be referring to it.

fixed pat  wt  dof     chisq       tail prob
       00  0    4    19.593   0.000600701   0.423  0.577 
       01  1    5   293.832             0   1.000  0.000 
       10  1    5   142.822   4.49515e-29   0.000  1.000 

The first line is our model GanjDareh + Onge and its a fail as p-value (tail prob) < 0.05.

The 2nd and 3rd lines are the submodels with 1 source only, Ganj Dareh only and Onge only respectively, both of which fail.

To know why the model fail, we refer to the generated Dstats, also known as gendstats in the output file. These are fstats which compare the simulated model we proposed to the actual target sample (Irula in our case), and big Z scores (above 3 or below -3) can tell us why our model failed.

gendstat:             Mbuti.DG Russia_MLBA_Sintashta -2.496
gendstat:             Mbuti.DG    Russia_HG_Karelia -3.280
gendstat:             Mbuti.DG     Russia_HG_Tyumen -2.328
gendstat:             Mbuti.DG   Turkey_N_published -0.628
gendstat:             Mbuti.DG          Tarim_EMBA1 -3.415
gendstat: Russia_MLBA_Sintashta    Russia_HG_Karelia -1.909
gendstat: Russia_MLBA_Sintashta     Russia_HG_Tyumen -0.876
gendstat: Russia_MLBA_Sintashta   Turkey_N_published 2.493
gendstat: Russia_MLBA_Sintashta          Tarim_EMBA1 -2.024
gendstat:    Russia_HG_Karelia     Russia_HG_Tyumen 0.709
gendstat:    Russia_HG_Karelia   Turkey_N_published 2.953
gendstat:    Russia_HG_Karelia          Tarim_EMBA1  0.100
gendstat:     Russia_HG_Tyumen   Turkey_N_published 1.973
gendstat:     Russia_HG_Tyumen          Tarim_EMBA1 -0.795
gendstat:   Turkey_N_published          Tarim_EMBA1 -3.118

For example, in this case the highest Z score gendstat tells us that there is too less Tarim_EMBA in the model as compared to Mbuti.

In this manner, you can tweak the model and add more or better sources so that the tail prob eventually becomes > 0.05.

The admixture weights are here (the 2 numbers on the right)

 00  0     4    19.593     0.000600701     0.423     0.577 

The standard errors to the admixture weights are here.

std. errors:     0.032     0.032 

PRO TIPS

1. Make a backup copy of the .ind file first. Then you can rename the labels in the 3rd column of each sample in the .ind file so it gives you flexibility of choosing a subset of samples from a label. DO NOT ALTER THE ORDER OF THE SAMPLES IN THE .ind FILE.

2. Put as many samples in the lista file for qpfstats as you think you will need for your project. Max limit in my opinion is 20-25. Once fstatsa.txt is generated, qpAdm run is lightning fast as it does not have to calculate fstats again.

3. always use allsnps: YES in qpfstats parameter file. No need to mention it again in qpAdm parameter file.

4. In the reich database annotation file, check the captured number of snps of the sample you are using. Please avoid any sample with <50k SNPs captured. ie low coverage samples.

5. Bad right population selection will make bad models pass. Choose a comprehensive list of right populations. Max 13-15.

6. Same population cant be there in both left and right.

See how i resolved the Irula model by adding Tarim_EMBA to left as source and therefore removing it from right.

Common Errors and Solutions:

1. fatalx: unable to allocate 324xxxxxxx units for item - This is a RAM issue, the geno file is larger than the RAM size. Increase the Virtualbox RAM for the guest to above 4gb.

2. zero popsize, Aborted (core dumped) - Check for spellings of your labels in the lista file, or left and right files. Correct spelling and ensure that the population exists in the .ind list or the qpfstat output.

Ask queries in comments section. Thanks.

Edit:
Suggesting Reading to know the limitations of qpAdm and best practices in qpAdm modeling.

Éadaoin Harney, Nick Patterson, David Reich, John Wakeley, Assessing the performance of qpAdm: a statistical tool for studying population admixture, Genetics, Volume 217, Issue 4, April 2021, iyaa045, https://doi.org/10.1093/genetics/iyaa045

Also refer

HOW TO Part 1 :: How to install Admixtools & Plink