Skip to content

BIRDSgroup/MultiPopPred

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

80 Commits
Figures
Figures
 
 
Reproducibility
Reproducibility
 
 
Scripts
Scripts
 
 
LICENSE
LICENSE
 
 
MPP_master.py
MPP_master.py
 
 
README.md
README.md
 
 
environment.yml
environment.yml
 
 
requirements.txt
requirements.txt
 
 

Repository files navigation

MultiPopPred

This is the official repository of the manuscript "MultiPopPred: A Trans-Ethnic Disease Risk Prediction Method, and its Application to the South Asian Population" by Ritwiz Kamal and Manikandan Narayanan.

License Preamble

Copyright 2024 BIRDS Group, IIT Madras

MultiPopPred is a free software: you can redistribute it and modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

MultiPopPred is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Please take a look at the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with MultiPopPred. If not, see https://www.gnu.org/licenses/.

Section 1: MultiPopPred - Getting Started

  • The MultiPopPred Github repository can be downloaded using 
    git clone https://github.com/BIRDSgroup/MultiPopPred.git
  • Example data for running the scripts is made available here: Link to example data.
  • Hardware Requirements
    • MultiPopPred requires only a standard computer with enough RAM and CPU-cores to support the in-memory operations.
  • Software Requirements
    • OS Requirements
      • MultiPopPred is supported for MacOS and Linux. We have tested MultiPopPred on the following systems:
        • Linux: Ubuntu 18.04, 24.04 and CentOS 7.9.2009
        • MacOS: Ventura 13.5
    • Python Dependencies
      • Basic Python packages required to use MultiPopPred are available in requirements.txt. Typical install time for each package on a "normal" desktop computer is ~2-5 minutes.
      • Optional: The complete conda environment can be reproduced using environment.yml. 
        conda env create -f environment.yml
    • PLINK2 must be downloaded and kept in the working directory.
  • Additional information such as Supplementary Data files associated with the manuscript are provided in Supplementary Data .

Section 2: Using MultiPopPred

Running via Command Line

A unified MultiPopPred master python (.py) file including all versions of the method is now available for use. Below we demonstrate a sample usage of for our default version (MPP-PRS+) along with details on all optional and mandatory arguments. Other versions can be used in a similar manner depending on their specific input requirements.

bash
python MPP_master.py \
  --version MPP-PRS+ \
  --chr 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22 \
  --trait_type continuous \
  --bfileTrain /data/target/train_chr \
  --phenoTrain /data/target/train_pheno.txt \
  --covTrain /data/target/train_cov.txt \
  --auxPops EUR,EAS,AFR \
  --prsAux /data/aux/EUR_prs_chr,/data/aux/EAS_prs_chr,/data/aux/AFR_prs_chr \
  --validate True \
  --bfileVal /data/target/val_chr \
  --phenoVal /data/target/val_pheno.txt \
  --covVal /data/target/val_cov.txt \
  --L1 0.001,0.01,0.1,1,10,1000,10000 \
  --out /data/output/mpp_results \
  --Nthreads all \
  --verbose False

File Format Specifications

The pipeline relies on several tab-separated text files. Note: For chromosome-specific files (genotypes, PRS, SS, Covariates), you only provide the prefix to the script, and the pipeline automatically appends the chromosome number and the extension (e.g., prefix1.txt, prefix1.bed).

  1. Summary Statistics (SS) & Polygenic Risk Score (PRS) Files Whether you are providing Auxiliary PRS (--prsAux), Auxiliary Summary Statistics (--ssAux), Target PRS (--prsTar), or Target Summary Statistics (--ssTar), the input .txt files must be tab-separated and contain the following column headers:
CHR SNP POS A1 A2 BETA
1 rs12345 10000 A G 0.052
1 rs67890 10500 C T -0.014

Note 1: The pipeline filters SNPs by intersecting the SNP column across the target genotypes and auxiliary datasets.

Note 2: The BETA column should be populated with logOR values in case of binary traits.

Note 3: It is assumed that the GWAS summary statistics and PRS input files have been through allele harmonization. If not, the allele_harmonization.R script could be used for the same as a pre-prcessing step.

  1. Phenotype Files (--phenoTrain, --phenoVal) Must be a tab-separated file containing three columns explicitly named FID,IID,Pheno.
  • For continuous traits: Float/integer values.
  • For binary traits: Must be coded as 0 (controls) and 1 (cases).
  1. Covariate Files (--covTrain, --covVal) (Optional) Must be a tab-separated file. The script ignores the first two columns (FID and IID in PLINK formats) and reads all subsequent columns as covariates starting from index 2.

Command-Line Arguments

General & System Settings

  • --version <str> (Default: MPP-PRS+) Specify which version of the MultiPopPred model to use. Choices: MPP-PRS+, MPP-PRS, MPP-PRS-TarSS, MPP-GWAS, MPP-GWAS-TarSS, MPP-GWAS-Admix.
  • --chr <str> (Default: 1,2,3,...,22) Comma-separated integers specifying which chromosomes to process.
  • --trait_type <str> Specify whether the phenotype trait is binary or continuous.
  • --out <str> (Required) Absolute or relative path to the output directory where results will be saved.
  • --Nthreads <str> (Default: all) Number of threads to use for parallelizing across chromosomes. Acceptable inputs are an integer or 'all'.
  • --seed <int> (Default: 123) Seed value to ensure reproducibility across runs.
  • --verbose <bool> (Default: True) Set to True or False to enable or disable detailed console output. False is advised when using parallel processing to avoid clutter in outputs.

Target Population Training Files

  • --bfileTrain <str> Prefix path to the target population's training genotype data in PLINK binary format (e.g., /path/tar_train_chr). The script will append the chromosome number and .bed/.bim/.fam extensions.
  • --bfileExtLD <str> Prefix path to the target population's external LD reference file. Required for TarSS model versions.
  • --phenoTrain <str> Path to the target population's ground-truth training phenotype file.
  • --covTrain <str> (Optional) Path to the target population's training set covariates.
  • --ssTar <str> Path to the target population's summary statistics file (.txt). Required for version MPP-GWAS-TarSS.
  • --prsTar <str> Path to the target population's single ancestry PRS file (.txt). Required for version MPP-PRS-TarSS.

Auxiliary Population Files

  • --auxPops <str> Comma-separated list of auxiliary population identifiers (e.g., EUR,EAS,AMR,AFR).
  • --ssAux <str> Comma-separated prefix paths to the auxiliary populations' summary statistics files. Must match the order provided in --auxPops.
  • --prsAux <str> Comma-separated prefix paths to the auxiliary populations' single ancestry PRS files. Must match the order provided in --auxPops.
  • --AdmixWeights <str> Path to the file containing admixture proportions. Required exclusively for the MPP-GWAS-Admix version.
  • --Neff <str> Comma-separated effective sample sizes for auxiliary populations, followed by the target population's effective sample size at the very end.

Validation Settings

  • --validate <bool> (Default: True) Enable or disable the validation step to tune hyperparameters.
  • --bfileVal <str> Prefix path to the validation target genotype data in PLINK format.
  • --phenoVal <str> Path to the validation ground-truth phenotype file.
  • --covVal <str> (Optional) Path to the validation dataset covariates.

Optimization & Penalties (L-BFGS)

  • --L1 <str> (Default: 0.0001,0.001,0.01,0.1,1,10,100,1000,10000) Comma-separated list of L1 penalty floats to test during validation cross-checking. If --validate False, you must provide a single float.
  • --L2 <str> (Default: 0.0001,0.001,0.01,0.1,0.5,0.9,0.99) Comma-separated list of L2 penalty floats (utilized in TarSS versions). If --validate False, you must provide a single float.
  • --smoothing <float> (Default: 0.1) Nesterov smoothing parameter used for calculating the L1 penalty derivative.
  • --max_iter <int> (Default: 10000) Maximum number of iterations allowed for the L-BFGS optimizer.
  • --tol <float> (Default: 0.0001) Tolerance threshold for L-BFGS convergence.
  • --max_fun <int> (Default: 100000) Maximum number of function evaluations for L-BFGS.

Running Jupyter Notebooks

The basic code for all five versions of MultiPopPred has been provided as Jupyter Notebooks (.ipynb files). To run these notebooks, one must have Jupyter installed on their systems which can easily be done following the instructions over here. Once Jupyter has been installed properly, Jupyter Notebook can be initialized as follows:

(a) For initializing Jupyter Notebook on local system

jupyter-notebook

(b) For initializing Jupyter Notebook on remote server

Step 1: From terminal 1 enter the command

jupyter-notebook --no-browser

Step 2: From terminal 2 login to remote server using

ssh -L 8080:localhost:xxxx username@hostname

(Here xxxx comes from the port number that you get after Step 1)

Step 3: From your browser, go to http://localhost:8080/ and enter the token (Obtained after Step 1) for logging into jupyter notebook.

Input Requirements

This section describes the input requirements for the five versions of MultiPopPred - MPP-PRS+, MPP-PRS, MPP-GWAS, MPP-GWAS-TarSS, MPP-GWAS-Admixture.

MPP-PRS+

The code for MPP-PRS+ is available at MPP-PRS+.ipynb

MPP-PRS+ requires the following inputs:

  1. Outputs (from single-ancestry PRS analysis) from Lassosum-TrueLD for each auxiliary population (and optionally the target population). (A detailed account of the code, input and output requirements for Lassosum-TrueLD are provided later in this section)

      CHR	SNP	        POS	        A1 A2 BETA
  22	rs5747999	17075353.0	A  C  0.0004
  22	rs1807512	17221495.0	C  T  -0.0009

  2. Target Population's Genotype files for training, validation and testing datasets in PLINK's bed/bim/fam format.

    2.1. The .bed file contains the binary encoded genotypes

    2.2. The .bim file contains information on SNPs in the following format

    22 rs5747999 0 17075353 A C
22 rs1807512 0 17221495 C T

    2.3. The .fam file contains information on samples/individuals in the following format

    1001 1001 0 0 0 -9
1002 1002 0 0 0 -9

  3. Target Population's Phenotype files for training, validation and testing datasets in the following format

    FID   IID   Pheno
1001  1001  -0.5708
1002  1002  -0.5035

  4. A L1 penalty value.

Lassosum-TrueLD

The code for Lassosum-TrueLD is available at Lassosum_TrueLD.ipynb

Lassosum-TrueLD works on a single population/anestry at a time and requires the following inputs:

  1. The population in consideration's Genotype files for training, validation and testing datasets in PLINK's bed/bim/fam format.

    1.1. The .bed file contains the binary encoded genotypes.

    1.2. The .bim file contains information on SNPs in the same format as depicted in point 2.2 under MPP-PRS+ input requirements.

    1.3. The .fam file contains information on samples/individuals in the same format as depicted in point 2.3 under MPP-PRS+ input requirements.

  2. The population in consideration's Phenotype files for training, validation and testing datasets in the same format as depicted in point 3 under MPP-PRS+ input requirements.

  3. A L1 penalty value.

MPP-PRS

The code for MPP-PRS is available at MPP-PRS.ipynb

MPP-PRS requires the following inputs:

  1. Outputs from Lassosum-ExtLD (or Lassosum2 (Zhang et al., Nat. Comms., 2024)) for each auxiliary population (and optionally the target population). The expected input format for MPP-PRS is the same as MPP-PRS+. (A detailed account of the code, input and output requirements for Lassosum-ExtLD are provided later in this section)

  2. Target Population's Genotype files for training, validation and testing datasets in PLINK's bed/bim/fam format.

    2.1. The .bed file contains the binary encoded genotypes

    2.2. The .bim file contains information on SNPs in the same format as depicted in point 2.2 under MPP-PRS+ input requirements.

    2.3. The .fam file contains information on samples/individuals in the same format as depicted in point 2.3 under MPP-PRS+ input requirements.

  3. Target Population's Phenotype files for training, validation and testing datasets in the same format as depicted in point 3 under MPP-PRS+ input requirements.

  4. A L1 penalty value.

Lassosum-ExtLD

The code for Lassosum-ExtLD (or Lassosum2) can be found at the Github repository of PROSPER (Zhang et al., Nat. Comms., 2024).

Lassosum-ExtLD works on a single population/anestry at a time and requires the following inputs (A more detailed account of Lassosum2's code, input and output requirements can be found on PROSPER's Github):

  1. The population in consideration's GWAS summary statistics

  2. The population in consideration's Genotype files for training, validation and testing datasets in PLINK's bed/bim/fam format.

    2.1. The .bed file contains the binary encoded genotypes.

    2.2. The .bim file contains information on SNPs in the same format as depicted in point 2.2 under MPP-PRS+ input requirements.

    2.3. The .fam file contains information on samples/individuals in the same format as depicted in point 2.3 under MPP-PRS+ input requirements.

  3. The population in consideration's Phenotype files for training, validation and testing datasets merged into a single file in the following format:

  4. The population in consideration's external LD reference panel which can be obtained from PROSPER's Github.

MPP-GWAS

The code for MPP-GWAS can be found at MPP-GWAS.ipynb

MPP-GWAS requires the following inputs:

  1. GWAS Summary Statistics for each auxiliary population (and optionally the target population) in the same format as MPP-PRS+.

  2. Target Population's Genotype files for training, validation and testing datasets in PLINK's bed/bim/fam format.

    2.1. The .bed file contains the binary encoded genotypes

    2.2. The .bim file contains information on SNPs

    2.3. The .fam file contains information on samples/individuals

  3. Target Population's Phenotype files for training, validation and testing datasets

  4. A L1 penalty value.

MPP-GWAS-TarSS

The code for MPP-GWAS-TarSS can be found at MPP-GWAS-TarSS.ipynb

MPP-GWAS-TarSS requires the following inputs:

  1. GWAS Summary Statistics for each auxiliary population as well as the target population in the same format as MPP-PRS+.

  2. Target Population's external LD reference panel in PLINK's bed/bim/fam format. The reference panel can be obtained from an appropriate source such as 1000 Genomes. The computation of LD happens during runtime of the code.

  3. L1 and L2 penalty values.

MPP-GWAS-Admixture

The code for MPP-GWAS-Admixture can be found at MPP-GWAS-Admixture.ipynb

MPP-GWAS-Admixture requires the following inputs:

  1. GWAS Summary Statistics for each auxiliary population (and optionally the target population) in the same format as MPP-PRS+.

  2. Target Population's Genotype files for training, validation and testing datasets in PLINK's bed/bim/fam format.

    2.1. The .bed file contains the binary encoded genotypes

    2.2. The .bim file contains information on SNPs

    2.3. The .fam file contains information on samples/individuals

  3. Target Population's Phenotype files for training, validation and testing datasets.

  4. The admixture proportions for individuals in the target population in the following format (minus the header): (The admixture proportions can be obtained with the help of the tool Admixture)

    AUX1.PROP   AUX2.PROP   AUX3.PROP   AUX4.PROP
0.663683    0.000010    0.243133    0.093174
0.726047    0.000010    0.000010    0.273933
0.512244    0.000010    0.000010    0.487736
0.616460    0.021437    0.248714    0.113389

  5. A L1 penalty value.

MultiPopPred Output

All five versions of MultiPopPred produce output in the following format:

CHR	SNP	        POS	        A1	BETA
22	rs5747999	17075353.0	A	0.008025369947048135
22	rs1807512	17221495.0	C	0.0011822995349338257

Here BETA represents the improved effect size estimates of SNPs present in the SNP column, for the target population as obtained by MultiPopPred. The PRS can then be computed with these improved target betas using either the evaluate function provided in the jupyter notebooks or using PLINK's --score flag.

Running MultiPopPred with example data

To run MultiPopPred on the provided example data, the following steps need to be followed:

  1. Download example_data_updated.zip (~66 Mb) from the link given above and decompress it using 
    unzip example_data_updated.zip
  2. Download MultiPopPred master scripts MPP_master.py.
  3. Run the script using the following template command (here shown for MPP-PRS+, with similar template commands provided in the Expected Outputs directory of the example_data_updated.zip file)
python MPP_master.py \
--version MPP-PRS+ \
--chr 22 \
--trait_type continuous \
--bfileTrain /example_data/Genotypes/target_training_geno_chr \
--phenoTrain /example_data/Phenotypes/pheno_tar_sas_train_truepheno.txt \
--auxPops EUR,EAS,AMR,AFR \
--prsAux /example_data/Lassosum_PRSfiles/lassosum_trueLD_eur_chr,/example_data/Lassosum_PRSfiles/lassosum_trueLD_eas_chr,/example_data/Lassosum_PRSfiles/lassosum_trueLD_amr_chr,/example_data/Lassosum_PRSfiles/lassosum_trueLD_afr_chr \
--validate True \
--bfileVal /example_data/Genotypes/target_validation_geno_chr \
--phenoVal /example_data/Phenotypes/pheno_tar_sas_val_truepheno.txt \
--L1 0.001,0.01,0.1,1,10,1000,10000 \
--out /example_data/ExpectedOutputs/mpp_prs_plus \
--Nthreads all \
--verbose True

Typical expected runtime on a "normal" desktop computer is ~1-3 seconds for each of the five versions of MultiPopPred applied to the provided example data, for a given single hyperparameter configuration.

Section 4: Data Availability and Reproducibility

Data Availability

Simulated Genotype-Phenotype data generated in this work is available through the following links:

Reproducibility - Level 1: Reproducing Figures from Final Results

Reproducibility - Level 2: Reproducing Final Results from Raw Outputs

  • Data for reproducing the final results from raw outputs are available in ./.../Results_MultiPopPred/ and ./.../Results_SOTAmethods/ at Simulated_Genotype_Phenotype_Data.
  • Code for reproducing the final results from raw outputs is available at Reproducbility_Level2.ipynb.
    • The code depicts reproducbility for a few sample datasets corresponding to some simulation configurations.
    • Instructions to extend the same to other/all simulation configurations is included within the code.

Reproducibility - Level 3: Reproducing Raw Outputs from Input Genotype-Phenotype Data

Reproducibility - Level 4: Reproducing Input Genotype-Phenotype Data

  • Data for reproducing input genotype-phenotype data is available at Reproducibility_Level4.tar.gz
  • Script for reproducing input genotype-phenotype data is available at simulate_phenotypes.R
  • The code depicts reproducbility for a few sample datasets corresponding to some simulation configurations.
  • Instructions to extend the same to other/all simulation configurations is included within the code.

Section 5: MultiPopPred - five versions

The code for five versions of MultiPopPred, as illustrated in the figure below, are provided in Scripts.

Methodology Overview

Section 6: Support

For any queries regarding MultiPopPred, please contact Ritwiz Kamal (ritwiz@cse.iitm.ac.in) and/or Manikandan Narayanan (nmanik@cse.iitm.ac.in)

Section 7: Citation

Kamal, R. and Narayanan, M., 2025. MultiPopPred: A Trans-Ethnic Disease Risk Prediction Method, and its Application to the South Asian Population. bioRxiv, pp.2024-11. https://doi.org/10.1101/2024.11.26.625410

⬆️ Back to top

About

MultiPopPred Code

Resources

Readme

License

LGPL-2.1 license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages